upstream with keepalive has the results almost comparable with tomcat, so yes keepalive between nginx and tomcat really does the trick. :)

Unfortunately we cannot use the upstream as explained in my previous post, unless upstream can offer something like below

map $http_void $header_based_url{

default "no_http_header";
#These Key => URL's are currently stored in a flat file
key1 server_instance_1_url_host;
key2 server_instance_2_url_host;
......
}

upstream http_backend {

server $header_based_url_host;

keepalive 100;

}


location /xxx {
proxy_pass http_backend;
}