passenger_max_requests missing - workaround

[loop]

Hello,

I have a rails (2.0.2) application, I used to run it via nginx/fastCGI
but am very happy with passenger + nginx (I like the easier
configuration and the graceful restarts above all).

Only, I think I could really use passenger_max_requests when it
becomes available, because after a few (~100) requests my application
starts to slow down really bad (some requests start to take 20
seconds, while I am under 1 when processes are fresh).

Anyway, I found a workaround that works very well, but I had to look
at the source because I didn't seem to find anything specific in the
docs:

Have your rails process send itself a USR1 signal if the limit has
been reached. I have something like this in application_controller:

after_filter :expire_process

def expire_process
$served_requests = ($served_requests || 0) + 1
if $served_requests >= REQUEST_LIMIT_FOR_AUTO_SHUTDOWN
logger.debug "Process #{ Process.pid } reached its limit after #
{ $served_requests }, sending :USR1"
Process.kill :USR1, Process.pid
end
end

REQUEST_LIMIT_FOR_AUTO_SHUTDOWN is obviously configured in my
environment.rb

Hope someone may find this useful!

[jonb]

Thanks for that tip! Coincidentally I've been working with an Nginx/
Passenger deployment today and I was hoping to use the
PassengerMaxRequests directive as a line of defense against any
unknown memory leaks in a production site. I was just about to switch
back to Apache just to get that directive, when I came upon your
post.

I took a slightly different approach with sending the USR1 signal,
instead using a cron job run every 15 minutes to check for processes
that are over a certain request limit. I also added some code to
forceably kill any processes that are wedged (those that don't recycle
themselves in a reasonable amount of time). It hasn't been 100%
tested, but here's the script if anyone finds it useful:

#!/usr/local/REE/bin/ruby

# Path to the passenger-status binary
PASSENGER_STATUS='/usr/local/REE/bin/passenger-status'
# Recycle processes after this many requests
MAX_REQUEST_COUNT=2000

class PidCollection < Hash
def any_expired?
!expired.empty?
end

def expired
self.map { |k,v| v[:processed] >= MAX_REQUEST_COUNT ? k :
nil }.compact
end
end

# Turns these into seconds:
# 0h 5m
# 3d 5h 3m
# 3m 5s
def parse_uptime(time)
sec = 0
time.strip.split(/ +/).each do |part|
unit = part[0..-2].to_i
case part[-1..-1]
when 'm' :
unit *= 60
when 'h' :
unit *= 3600
when 'd' :
unit *= (24*3600)
end
sec += unit
end
sec
end

def get_status
pids = PidCollection.new
`#{PASSENGER_STATUS}`.each_line do |line|
if line =~ /^ *PID/ then
parts = line.strip.split(/ +/)
pids[parts[1].to_i] = { :sessions => parts[3], :processed =>
parts[5].to_i, :uptime => parse_uptime(line.match(/Uptime:.*$/).to_s
[7..-1]) }
end
end
pids
end

def main
status = get_status
status.each do |k, v|
puts "#{k}: #{v.inspect}"
end
return unless status.any_expired?
status.expired.each do |pid|
puts "sending -USR1 to #{pid}"
Process.kill :USR1, pid
end
sleep 10
if get_status.any_expired? then
# Hmm, let's give the expired ones some more time to finish their
requests
sleep 65
get_status.expired.each do |pid|
puts "killing -9 #{pid}"
# Don't care if this fails
Process.kill(9, pid) rescue nil
end
end
end

main