Passenger+Nginx - Resource Management Problem? 502 Bad Gateway

[jdpace]

We just move to a new infrastructure for our site 2 weeks ago. Over
2-3 days our memory usage climbs on our 2gb app servers (we have 3)
and starts using swap. After that the server will start returning 502
bad gateway error messages.

I've limited the max pool size to 5 since our app gets very large.
However before our app goes down all the way response time starts
growing also. When I check passenger-status I see that it has
correctly set our max pool size but when I check passenger-memory-
stats there may be as many as 8 rails instances not counting the
spawning processes.

I've also stopped Nginx, verified that passenger is not running, and
then ran passenger-memory-stats only to find that the rails instances
are still running.

Is there some known issue with passenger being unable to kill rails
processes?

Below is a link to some command outputs I ran this morning. The top
half on 'kinetoscope' server are on a server that was returning 502
errors. The bottom half 'lightbulb' shows the rails processes still
running after stopping Nginx.

http://gist.github.com/115513

Thanks for the help,
Jared

[Hongli Lai]

Phusion Passenger gracefully tells Rails processes to shut down, but
it doesn't forcefully kill them. If your Rails processes are, for some
reason, frozen, then they will stay there until you kill them
manually.

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

[Fred]

I had this same problem many time, just out of random with different
rails applications.

It seems to be a compatibility problem with new nginx versions such as
nginx-0.7.59

When roll back to nginx-0.6.36 everything works fine.