process hangs - (Errno::ESPIPE) "Illegal seek"

[Dan Manges]

I've had a few Rails processes raise (Errno::ESPIPE) "Illegal seek" on
a file upload action, and when they do, they start eating up 100% cpu
and stop responding. Sometimes they're still in the passenger-status
list, sometimes not.

I'm using Passenger 2.0.2 with Rails 2.1.0

I received this backtrace from the Rails exception notifier plugin. It
looks like the exception is when flushing the response to the client.
Does anybody know why it might be happening?

[RAILS_ROOT]/vendor/rails/actionpack/lib/action_controller/
cgi_process.rb:217:in `flush'
[RAILS_ROOT]/vendor/rails/actionpack/lib/action_controller/
cgi_process.rb:217:in `out'
[RAILS_ROOT]/vendor/rails/actionpack/lib/action_controller/
dispatcher.rb:149:in `handle_request'
[RAILS_ROOT]/vendor/rails/actionpack/lib/action_controller/
dispatcher.rb:107:in `dispatch'
[RAILS_ROOT]/vendor/rails/actionpack/lib/action_controller/
dispatcher.rb:104:in `synchronize'
[RAILS_ROOT]/vendor/rails/actionpack/lib/action_controller/
dispatcher.rb:104:in `dispatch'
[RAILS_ROOT]/vendor/rails/actionpack/lib/action_controller/
dispatcher.rb:120:in `dispatch_cgi'
[RAILS_ROOT]/vendor/rails/actionpack/lib/action_controller/
dispatcher.rb:35:in `dispatch'
/usr/lib64/ruby/gems/1.8/gems/passenger-2.0.2/lib/passenger/railz/
request_handler.rb:38:in `process_request'
/usr/lib64/ruby/gems/1.8/gems/passenger-2.0.2/lib/passenger/
abstract_request_handler.rb:163:in `main_loop'
/usr/lib64/ruby/gems/1.8/gems/passenger-2.0.2/lib/passenger/railz/
application_spawner.rb:307:in `start_request_handler'
/usr/lib64/ruby/gems/1.8/gems/passenger-2.0.2/lib/passenger/railz/
application_spawner.rb:276:in `handle_spawn_application'

Thanks,
Dan Manges

[Hongli Lai]

Dan Manges wrote:
> I've had a few Rails processes raise (Errno::ESPIPE) "Illegal seek" on
> a file upload action, and when they do, they start eating up 100% cpu
> and stop responding. Sometimes they're still in the passenger-status
> list, sometimes not.
>
> I'm using Passenger 2.0.2 with Rails 2.1.0
>
> I received this backtrace from the Rails exception notifier plugin. It
> looks like the exception is when flushing the response to the client.
> Does anybody know why it might be happening?

Which version of Ruby are you using? And are you capable of reliably
reproducing this problem?

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

[Dan Manges]

On Aug 11, 10:16 am, Hongli Lai <hon...@phusion.nl> wrote:
> Which version of Ruby are you using? And are you capable of reliably
> reproducing this problem?

I'm using Ruby 1.8.5 from the yum package for CentOS. So far I haven't
been able to reproduce the problem, but it happens a couple times a
day if you have a suggestion for something to try.

$ ruby -v
ruby 1.8.5 (2006-08-25) [x86_64-linux]

$ uname -r
2.6.18-92.1.6.el5

$ yum list | grep ruby.x86_64
ruby.x86_64 1.8.5-5.el5_2.3
installed

Thanks,
Dan

[Hongli Lai]

Dan Manges wrote:
> On Aug 11, 10:16 am, Hongli Lai <hon...@phusion.nl> wrote:
>> Which version of Ruby are you using? And are you capable of reliably
>> reproducing this problem?
>
> I'm using Ruby 1.8.5 from the yum package for CentOS. So far I haven't
> been able to reproduce the problem, but it happens a couple times a
> day if you have a suggestion for something to try.

You might want to give Ruby 1.8.6-p286 a try. The problem that you're
experiencing might be a Ruby bug.

[amos]

I just saw this too with:

ruby 1.8.6 (2008-08-11 patchlevel 287) [x86_64-linux]
passenger 2.0.3
centos 5.1

On Aug 11, 8:49 am, Hongli Lai <hon...@phusion.nl> wrote:
> Dan Manges wrote:
> > On Aug 11, 10:16 am, Hongli Lai <hon...@phusion.nl> wrote:
> >> Which version of Ruby are you using? And are you capable of reliably
> >> reproducing this problem?
>
> > I'm using Ruby 1.8.5 from the yum package for CentOS. So far I haven't
> > been able to reproduce the problem, but it happens a couple times a
> > day if you have a suggestion for something to try.
>
> You might want to give Ruby 1.8.6-p286 a try. The problem that you're
> experiencing might be a Ruby bug.
>
> --
> Phusion | The Computer Science Company
>
> Web:http://www.phusion.nl/

> E-mail: i...@phusion.nl

[amos]

This is actually a serious problem. After a while our upload servers
have at least 4 processes that hang in this state and chew up the
CPUs. Has anyone else seen this? Here is the repeated output of
strace:

lseek(9, -2, SEEK_CUR) = -1 ESPIPE (Illegal seek)
lseek(9, -2, SEEK_CUR) = -1 ESPIPE (Illegal seek)
....