1.2.4 Nginx reverse proxy stalls on larger response size

421 views
Skip to first unread message

LetsPlay

unread,
Feb 2, 2012, 12:56:04 AM2/2/12
to play-framework
I'm so excited about building Java web apps with Play but have to
humbly admit to falling at the first hurdle. I suspect an Nginx
config problem or Nginx/Play interaction.

Direct Method
===========
If I install 1.2.4 or the latest 1.x via git, create a "play new
myApp", and change the http.port to 9027 in the conf to avoid a local
9000 conflict, I can get the welcome page fine in a browser with
http://localhost:9027. Awesome. Case 1. Tick.

Indirect Method
============
But if I:

(A) hook up a web domain www.myApp.com (changed from the actual) to
point to my static IP address, which then NATs to my box via a modem/
router, and
(B) install nginx as a front end web server (i.e. default apache
removed), configuring it solely as a reverse proxy to localhost:9027,

then http://www.myApp.com should serve up the myApp welcome page, but
it just stalls and times out completely (Case 2 doh).

Now, if I hack myApp/app/views/Application/index.html to become just a
short static html page including a short css file, no problem, both
methods deliver deliver the page fine (Case 3). mmm, maybe this is
related to response sizes...

But, if I lengthen out the total response data size by adding comment
lines to the css file, the indirect method (www.myApp.com) fails (Case
4). Case 2 hangs and delivers nothing to the browser, Case 4
immediately returns the html page (visible via right-click "view
source" and the page title) but nginx seems to get caught up waiting
for the css file, and the page does not render until (possibly) some
time out is reached. When it does render, it does not use the css
font (presumably it abandons the css request).

So I delete enough of the comment padding from the css file, and the
indirect method works. Adding lines causes hanging/timeout. argh

It works again for the additional comment lines when I add
"proxy_buffering on;" in the nginx location block. But only up to a
point (Case 5), beyond which (Case 6) it fails.

I've hacked myApp/app/controller/Application.java to log request and
response details (not included here). The nginx debug logs are
voluminous (and over my head) but I'll provide them if needed. Access
logs are below.

Can anyone help me? Advice, guidance, ideas? ... encouragement (!) :)

Cheers

Details:
System: Ubuntu 11.10, Play 1.2.4 unzipped and installed, Nginx 1.0.5
via apt-get
Network: www.myApp.com points to a static IP, which is a modem/router
with LAN address 192.168.1.1 wired to my box 192.168.1.13. The router
uses the NAT rule "forward port 80 to 192.168.1.13" (my box)

Note that local access (http://localhost:9027) works fine in all cases
Case 1: WORKS vanilla "play new myApp" works fine at http://localhost:9027
Case 2: FAIL vanilla "play new myApp" times out at http://www.myApp.com,
nothing received by browser
Case 3: WORKS hacked index.html with short toy css, works via www.myApp.com
Case 4: FAIL Case 3 with lengthened css, long delay via www.myApp.com,
css not used
Case 5: WORKS Case 4 but with nginx "proxy_buffering off;", works via
www.myApp.com
Case 6: FAIL Case 5 with an extra css comment line, long delay via
www.myApp.com, css not used

nginx.conf
========

user www-data;
worker_processes 4;
pid /var/run/nginx.pid;

events
{
worker_connections 768;
}

http
{
include /etc/nginx/mime.types;
default_type application/octet-stream;

access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log debug;

server
{
listen 80;
access_log /var/log/nginx/myApp.access.log;
error_log /var/log/nginx/myApp.error.log debug;
location /
{
proxy_set_header X-Forwarded-Host $host;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For
$proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:9027;
# proxy_buffering off;
}
}
}

index.html (471 bytes)
================

<!DOCTYPE html>

<html>
<head>
<title>Your application is ready !</title>
<meta charset="utf-8">
<link rel="stylesheet" media="screen" href="/public/
stylesheets/manual.css">
</head>
<body>

<h1>Your new application is ready!</h1>
<p>
Congratulation, you've just created a new play
application. This page will help you in the few next steps.
</p>

</body>
</html>

myApp/public/stylesheets/manual.css (size in comments)
=============================


html {
overflow-y: scroll;
}

html, body {
background: #F7F7F7;
font-family: 'Lucida Grande', Helvetica, Arial, sans-serif;
font-size: 14px;
line-height: 1.5em !important;
}

html {
border-top: 5px solid #67A927;
}

a {
color: #000;
}

/** 01--0000000000000000000000000000000000000000000000000 **/
/** 02--0000000000000000000000000000000000000000000000000 **/
/** 03--0000000000000000000000000000000000000000000000000 **/
/** 04--0000000000000000000000000000000000000000000000000 **/
/** 05--0000000000000000000000000000000000000000000000000 **/
/** 06--0000000000000000000000000000000000000000000000000 **/
/** 07--0000000000000000000000000000000000000000000000000 **/
/** 08--0000000000000000000000000000000000000000000000000 **/
/** 09--0000000000000000000000000000000000000000000000000 **/
/** 10--0000000000000000000000000000000000000000000000000 **/
/** 11--0000000000000000000000000000000000000000000000000 **/
/** 12--0000000000000000000000000000000000000000000000000 **/
/** 13--0000000000000000000000000000000000000000000000000 **/
/** 14--this-works--0\ CASE 3 /00000= 1137 bytes to EOL = **/
/** 15--this-fails--0\ CASE 4 /00000000000= 1199 to EOL = **/
/** 16--this-works-with-buffering-off--000= 1261 to EOL = **/
/** 17--works-buffering-off--0000000000000000000000000000 **/
/** 18--works-buffering-off--00\ CASE 5 /0= 1385 to EOL = **/
/** 19--fails-buffering-off--00\ CASE 6 /0= 1447 to EOL = **/


Access logs
==========

Case 2
----------
192.168.1.1 - - [02/Feb/2012:12:59:02 +0800] "GET / HTTP/1.1" 200 5749
"-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like
Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/
535.>2"
192.168.1.1 - - [02/Feb/2012:12:59:20 +0800] "-" 400 0 "-" "-"
127.0.0.1 - - [02/Feb/2012:13:00:01 +0800] "GET / HTTP/1.0" 200 5749
"-" "-"
192.168.1.1 - - [02/Feb/2012:13:01:02 +0800] "GET / HTTP/1.1" 200 5749
"-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like
Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/
535.>2"

Case 3
----------
192.168.1.1 - - [02/Feb/2012:13:15:36 +0800] "GET / HTTP/1.1" 200 471
"-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like
Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/
535. >2"
192.168.1.1 - - [02/Feb/2012:13:15:36 +0800] "GET /public/stylesheets/
manual.css HTTP/1.1" 200 1137 "http://www.myApp.com/" "Mozilla/5.0
(X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Ubuntu/
11.10 >Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/535.2"
192.168.1.1 - - [02/Feb/2012:13:15:36 +0800] "GET /favicon.ico HTTP/
1.1" 404 5392 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2
(KHTML, like Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/
15.0.874.106 >Safari/535.2"
192.168.1.1 - - [02/Feb/2012:13:15:50 +0800] "-" 400 0 "-" "-"

Case 4
----------
192.168.1.1 - - [02/Feb/2012:13:19:16 +0800] "GET / HTTP/1.1" 200 471
"-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like
Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/
535. >2"
192.168.1.1 - - [02/Feb/2012:13:19:16 +0800] "GET /public/stylesheets/
manual.css HTTP/1.1" 200 1199 "http://www.myApp.com/" "Mozilla/5.0
(X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Ubuntu/
11.10 >Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/535.2"
192.168.1.1 - - [02/Feb/2012:13:19:29 +0800] "-" 400 0 "-" "-"
127.0.0.1 - - [02/Feb/2012:13:20:01 +0800] "GET / HTTP/1.0" 200 471
"-" "-"
192.168.1.1 - - [02/Feb/2012:13:21:16 +0800] "GET /public/stylesheets/
manual.css HTTP/1.1" 200 1199 "http://www.myApp.com/" "Mozilla/5.0
(X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Ubuntu/
11.10 >Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/535.2"
192.168.1.1 - - [02/Feb/2012:13:23:17 +0800] "GET /favicon.ico HTTP/
1.1" 404 5392 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2
(KHTML, like Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/
15.0.874.106 >Safari/535.2"

Case 5
----------
192.168.1.1 - - [02/Feb/2012:13:29:22 +0800] "GET / HTTP/1.1" 200 471
"-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like
Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/
535. >2"
192.168.1.1 - - [02/Feb/2012:13:29:22 +0800] "GET /public/stylesheets/
manual.css HTTP/1.1" 200 1385 "http://www.myApp.com/" "Mozilla/5.0
(X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Ubuntu/
11.10 >Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/535.2"
192.168.1.1 - - [02/Feb/2012:13:29:23 +0800] "GET /favicon.ico HTTP/
1.1" 404 5392 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2
(KHTML, like Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/
15.0.874.106 >Safari/535.2"

Case 6
----------
192.168.1.1 - - [02/Feb/2012:13:29:22 +0800] "GET / HTTP/1.1" 200 471
"-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like
Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/
535. >2"
192.168.1.1 - - [02/Feb/2012:13:29:22 +0800] "GET /public/stylesheets/
manual.css HTTP/1.1" 200 1385 "http://www.myApp.com/" "Mozilla/5.0
(X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Ubuntu/
11.10 >Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/535.2"
192.168.1.1 - - [02/Feb/2012:13:29:23 +0800] "GET /favicon.ico HTTP/
1.1" 404 5392 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2
(KHTML, like Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/
15.0.874.106 >Safari/535.2"
192.168.1.1 - - [02/Feb/2012:13:31:08 +0800] "GET / HTTP/1.1" 200 471
"-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like
Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/
535. >2"
192.168.1.1 - - [02/Feb/2012:13:31:08 +0800] "GET /public/stylesheets/
manual.css HTTP/1.1" 200 1447 "http://www.myApp.com/" "Mozilla/5.0
(X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Ubuntu/
11.10 >Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/535.2"
192.168.1.1 - - [02/Feb/2012:13:31:26 +0800] "-" 400 0 "-" "-"
192.168.1.1 - - [02/Feb/2012:13:33:08 +0800] "GET /favicon.ico HTTP/
1.1" 404 5392 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2
(KHTML, like Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/
15.0.874.106 >Safari/535.2"

R. Rajesh Jeba Anbiah

unread,
Feb 3, 2012, 12:33:32 AM2/3/12
to play-framework
On Feb 2, 10:56 am, LetsPlay <expiscor.scien...@gmail.com> wrote:
> I'm so excited about building Java web apps with Play but have to
> humbly admit to falling at the first hurdle.  I suspect an Nginx
> config problem or Nginx/Play interaction.

<snip>

You may better crosscheck with CloudFoundry's nginx config
http://support.cloudfoundry.com/entries/20443366-put-cf-behind-nginx#post_20495698

FWIW, CloudFoundry's default setup includes nginx and we used CF
for our alpha Play!/Java app.

LetsPlay

unread,
Feb 8, 2012, 9:34:20 PM2/8/12
to play-framework
Anyone? How to debug this situation?

Thanks R. Rajesh Jeba Anbiah, I tried the CloudFoundry
simple.nginx.conf (adapted, below), same result. One thought is that
this is a system problem. I'm running reasonably standard Ubuntu
11.10 x86_64.

user www-data;
worker_processes 1;

error_log /var/log/nginx/cf.error.log;
pid /var/run/nginx.pid;

events {
worker_connections 1024;
use epoll;
# multi_accept on;
}

http {
include /etc/nginx/mime.types;
access_log /var/log/nginx/cf.access.log;

log_format main '$host - [$time_local] '
'"$request" $status $bytes_sent '
'"$http_referer" "$http_user_agent" '
'$remote_addr response_time:$upstream_response_time';

default_type text/html;

sendfile on;
tcp_nopush on;
tcp_nodelay on;

keepalive_timeout 75 20;

gzip off;
gzip_min_length 1250;
gzip_buffers 16 8k;
gzip_comp_level 2;
gzip_proxied any;
gzip_types text/plain text/css application/javascript
application/x-javascript text/xml application/xml application/xml+rss
text/javascript;
gzip_vary on;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";

client_max_body_size 256M;

upstream play {
server 127.0.0.1:9027;
}

server {
listen *:80;
server_name _;

access_log /var/log/nginx/play.access.log main;
error_log /var/log/nginx/play.error.log;
server_name_in_redirect off;

location / {
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real_IP $remote_addr;
proxy_set_header X-Forwarded_For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded_Proto
http;
proxy_redirect off;
proxy_connect_timeout 10;
proxy_send_timeout 30;
proxy_read_timeout 30;
proxy_pass http://play;
}
}
}

On Feb 3, 1:33 pm, "R. Rajesh Jeba Anbiah"
<ng4rrjanb...@rediffmail.com> wrote:
> On Feb 2, 10:56 am, LetsPlay <expiscor.scien...@gmail.com> wrote:
>
> > I'm so excited about building Java web apps with Play but have to
> > humbly admit to falling at the first hurdle.  I suspect anNginx
> > config problem orNginx/Play interaction.
>
>    <snip>
>
>    You may better crosscheck with CloudFoundry'snginxconfighttp://support.cloudfoundry.com/entries/20443366-put-cf-behind-nginx#...
>
>    FWIW, CloudFoundry's default setup includesnginxand we used CF

Ben McCann

unread,
Feb 9, 2012, 1:04:34 AM2/9/12
to play-fr...@googlegroups.com
I'm using nginx with Play 2 on Ubuntu 11.10 x86-64 and it's been working well (I did need to turn proxy_buffering off to get it to work with Play 2).  I was using Play 1 before and don't remember encountering any problems then either.  Does changing your proxy_buffer_size change the point at which you encounter problems?

R. Rajesh Jeba Anbiah

unread,
Feb 10, 2012, 12:00:45 AM2/10/12
to play-framework
On Feb 9, 7:34 am, LetsPlay <expiscor.scien...@gmail.com> wrote:
> Anyone? How to debug this situation?
>
> Thanks R. Rajesh Jeba Anbiah, I tried the CloudFoundry
> simple.nginx.conf (adapted, below), same result.  One thought is that
> this is a system problem.  I'm running reasonably standard Ubuntu
> 11.10 x86_64.
<snip>

I'm not an expert on nginx. You may try nginx forum for better
support. But, over the top of my head: please check by enabling gzip
and also check if any processes (antivirus like) are mangling

LetsPlay

unread,
Feb 12, 2012, 9:22:29 PM2/12/12
to play-framework
For info, I'm calling on the folks at the nginx forum for further
help:
http://forum.nginx.org/read.php?2,222240,222326#msg-222326

On Feb 10, 1:00 pm, "R. Rajesh Jeba Anbiah"
<ng4rrjanb...@rediffmail.com> wrote:
> On Feb 9, 7:34 am, LetsPlay <expiscor.scien...@gmail.com> wrote:> Anyone? How to debug this situation?
>
> > Thanks R. Rajesh Jeba Anbiah, I tried the CloudFoundry
> > simple.nginx.conf (adapted, below), same result.  One thought is that
> > this is a system problem.  I'm running reasonably standard Ubuntu
> > 11.10 x86_64.
>
>    <snip>
>
> I'm not an expert onnginx. You may trynginxforum for better
Reply all
Reply to author
Forward
0 new messages