Saturday, 22 June 2013

nginx proxy_pass configuration, complexity, settings, issues, solutions

Ideally when you set these parameters for proxy_pass, its good enough.

location / {
               proxy_pass        http://localhost:8080;
               proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_connect_timeout      30;
                proxy_send_timeout         30;
                proxy_read_timeout         600;

                proxy_buffer_size          4k;
                proxy_buffers              4 16k;
                proxy_busy_buffers_size    64k;
                proxy_temp_file_write_size 64k;

How to pass the remote address to back-end server while using nginx

In case of proxy_pass, there is a complexity, when back-end server will try to access the requested IP address it will return either or may be the local subnet IP where nginx is deployed, because nginx is proxy server and it overrides the information of requested IP address. So the solution is to set an extra parameters in request header at the time of making proxy.
statement "proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;" meant for that only.

How to get the remote address in your back-end server while using "X-Forwarded-For"

Story is not yet over, Not now you have to do something at your back-end server to extract the requested IP from request header.

In case of back-end is Apache it is little simple, you just need to install a module
$ sudo apt-get install libapache2-mod-rpaf
And configure the file /etc/apache2/mods-available/rpaf.conf

<IfModule mod_rpaf.c>
RPAFenable On
RPAFsethostname On

But in case of back-end as Tomcat, it is little complex, 
you will never get it via request.getRemoteAddr(); So write a global API to access the remote address, like this,

public static String getRemoteAddress(HttpServletRequest request){
String ip = request.getHeader("X-Forwarded-For");
if(ip==null || "".equals(ip)){
return ip;

So if it is found in "X-Forwarded-For" as a request headers, it will return or else it will get from request.getRemoteAddr(). This kind of programming is good because tomorrow if you plan to use Apache proxying using AJP protocol, then you don't need to make any back-end change, in case of AJP, it will get you remote address directly from request object, and in case of other proxying, it will get you from "X-Forwarded-For" header.

Some other configurations points

1. proxy_connect_timeout directive assigns a timeout for the connection to the upstream server(or back-end server). It's default value is 60s.

This is not the time until the server returns the pages, that is the proxy_read_timeout statement. If your upstream server is up, but hanging (e.g. it does not have enough threads to process your request so it puts you in the pool of connections to deal with later), then this statement will not help as the connection to the server has been made. 

So in case you ever get proxy_connect_timeout at nginx, check your back-end connection limit.

2. proxy_read_timeout - this is very very important, default value is 60s.
This directive sets the read timeout for the response of the proxied server. It determines how long nginx will wait to get the response to a request. The timeout is established not for entire response, but only between two operations of reading.

In contrast to proxy_connect_timeout, this timeout will catch a server that puts you in it's connection pool but does not respond to you with anything beyond that, then proxy_read_timeout will come in picture. Be careful though not to set this too low, as your proxy server might take a longer time to respond to requests on purpose (e.g. when serving you a report page that takes some time to compute). 

You can also set different proxy_read_timeout which could be higer value like 10minutes for certain location.

location /admin/reports/ {
// other proxy_pass settings
proxy_read_timeout  600;

location / {
// other proxy_pass settings
proxy_read_timeout  30;

3. proxy_send_timeout - default value is 60s
This directive assigns timeout with the transfer of request to the upstream server. Timeout is established not on entire transfer of request, but only between two write operations. If after this time the upstream server will not take new data, then nginx is shutdown the connection.