Urgent: upstream timed out (110: Connection timed out) while reading response header from upstream

[sserdyuk]

Hi,

I am using Nginx+ Passenger 2.2.4 and getting these when user activity
picks up. It all works fine under light use.

2009/07/23 01:09:57 [error] 2916#0: *108371 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client: 155.69.195.0, server: xxxxx.com, request: "POST /session HTTP/
1.1", upstream: "unix:/tmp/passenger.2864/master/helper_server.sock:",
host: "cornell.xxxxx.com", referrer: "http://cornell.xxxxx.com/login"

Users experience: a minute long delay on a random operation, then get
a "Gateway timeout" error.

Here are more log errors and also passenger-status traces:
http://pastie.org/556047.txt?key=zti81f2ilkww5i9ggrfwq

This problem is really bad, because it fails exactly when the service
is used the most. Please advise asap.

Sergei.

[Hongli Lai]

Try increasing the size of your socket backlogs. This is a system setting.

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

[sserdyuk]

Is it /proc/sys/net/core/netdev_max_backlog or /proc/sys/net/ipv4/
tcp_max_syn_backlog ?
It still very strange, as it is not THAT much of the load. Static
files are served just fine. Also, it seems that the issue is always
with POST requests.

On Jul 23, 5:12 am, Hongli Lai <hon...@phusion.nl> wrote:

> E-mail: i...@phusion.nl

[sserdyuk]

FYI. The problem is currently "resolved" by killing all Passenger
Rails processes that are older then two hours.
It would be nice to have an appropriate solution for it eventually.

On Jul 23, 5:12 am, Hongli Lai <hon...@phusion.nl> wrote:

> E-mail: i...@phusion.nl

[Ed]

How about setting your PassengerMaxRequests to something like 500
http://www.modrails.com/documentation/Users%20guide.html#PassengerMaxRequests

The maximum number of requests an application instance will process. After serving that many requests, the application instance will be shut down and Phusion Passenger will restart it.