Randomly PassengerHelperAgent cause 100% CPU usage and passenger-status shows application spawning in process

[Dongdong Zhou]

Hi,

My environment is as follows:

Passenger            4.0.5

Ruby                    2.0.0p195

Apache                2.2.15

CentOS                6.4

And the configuration in Apache for Passenger is:

LoadModule passenger_module /usr/local/rvm/gems/ruby-2.0.0-p195/gems/passenger-4.0.5/libout/apache2/mod_passenger.so

<IfModule mod_passenger.c>

     PassengerRoot /usr/local/rvm/gems/ruby-2.0.0-p195/gems/passenger-4.0.5

     PassengerRuby /usr/local/rvm/wrappers/ruby-2.0.0-p195/ruby

     PassengerMaxPoolSize 20

     PassengerMinInstances 0

     PassengerPoolIdleTime 60

     PassengerSpawnMethod smart

</IfModule>

Randomly, PassengerHelperAgent will cause 100% CPU and when I do passenger-status, it seems it stuck at application spawning in process. When I do a restart for Apache, there will be a new PassengerHelperAgent process but the old PassengerHelperAgent process is not killed. Normally they should.

Can someone help me with issue please? Thanks.

[Hongli Lai]

Can you show the output of 'ps aux' and 'passenger-status' when the
problem is occurring? Can you also run 'sudo strace -f -p
<PID_OF_HELPER_AGENT>' and 'sudo crash-watch --dump
<PID_OF_HELPER_AGENT>' (https://github.com/FooBarWidget/crash-watch)
and attach all the output?

> --
> You received this message because you are subscribed to the Google Groups
> "Phusion Passenger Discussions" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to phusion-passen...@googlegroups.com.
> To post to this group, send email to phusion-...@googlegroups.com.
> Visit this group at
> http://groups.google.com/group/phusion-passenger?hl=en-US.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
Phusion | Ruby & Rails deployment, scaling and tuning solutions

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

[Dongdong Zhou]

Hi,

Please see the attached. When I do strace, it shows this constantly in std error:

[pid  1964] write(2, "[ 2013-06-07 23:09:25.4738 426/7"..., 86) = -1 EPIPE (Broken pipe)

[pid  1964] --- SIGPIPE (Broken pipe) @ 0 (0) ---

Thank you for your help.

[Hongli Lai]

Can you also attach the output of `sudo lsof -p <PID_OF_HELPER_AGENT> -nP`?

[Hongli Lai]

And could you attach all your Apache configuration files?

[Dongdong Zhou]

Hi,

Please see the attached.

Thanks

[Hongli Lai]

It looks like all Passenger processes' standard error are redirected
to a pipe, which at some point as been closed. Any writes to standard
error fails so the Passenger processes retry infinitely.

I'd like to figure out why standard error is a pipe in the first
please. Could you attach your entire Apache configuration directory?
That's probably /etc/apache2.

[Dongdong Zhou]

Please see the attached.

The apache error_log is piped to an external log splitting script.

[Hongli Lai]

Alright. The problem should be fixed in commit c150c02. Could you try
out git master?

[Dongdong Zhou]

Would the master be suitable for production environment?

You received this message because you are subscribed to a topic in the Google Groups "Phusion Passenger Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/phusion-passenger/5zBobD5GaF0/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to phusion-passen...@googlegroups.com.

[Hongli Lai]

Yes. No new features have been added. The master branch is currently
in the bug fix only phase.

[Dongdong Zhou]

Have used the master branch yesterday and so far so good. Looks fine now.

Thank you for your help.

[Hongli Lai]

Great. Thanks for all the feedback.