Unable to start the Phusion Passenger logging agent: it seems to have been killed with signal SIGKILL during startup

[pavithra....@gmail.com]

Hi,

While starting apache, am getting below error: Can someone please help to resolve.

[ 2014-09-29 08:35:08.7694 18178/7f4ed722d700 apache2/Hooks.cpp:1303 ]: Initializing Phusion Passenger...
[ 2014-09-29 08:35:08.7806 18179/7fc2336a7720 agents/Base.cpp:1591 ]: Random seed: 1411994108
[ 2014-09-29 08:35:08.7807 18179/7fc2336a7720 agents/Watchdog/Main.cpp:698 ]: Starting Watchdog...
[ 2014-09-29 08:35:08.7809 18179/7fc2336a7720 agents/Watchdog/Main.cpp:538 ]: Options: { 'analytics_log_user' => 'nobody', 'default_group' => 'puppet', 'default_python' => 'python', 'default_ruby' => '/usr/bin/ruby', 'default_user' => 'puppet', 'log_level' => '3', 'max_pool_size' => '6', 'passenger_root' => '/usr/lib/ruby/gems/1.8/gems/passenger-4.0.52', 'passenger_version' => '4.0.52', 'pool_idle_time' => '300', 'prestart_urls' => 'aHR0cHM6Ly9kZXYtcHVwcGV0Y2EuYm55bWVsbG9uLm5ldC91c3Ivc2hhcmUvcHVwcGV0L3JhY2svcHVwcGV0bWFzdGVyZC9wdWJsaWMA', 'temp_dir' => '/var/log/httpd', 'union_station_gateway_address' => 'gateway.unionstationapp.com', 'union_station_gateway_port' => '443', 'user_switching' => 'true', 'web_server_passenger_version' => '4.0.52', 'web_server_pid' => '18178', 'web_server_type' => 'apache', 'web_server_worker_gid' => '2', 'web_server_worker_uid' => '2' }
[ 2014-09-29 08:35:08.8074 18182/7ff9c2f6b720 agents/Base.cpp:1591 ]: Random seed: 1411994108
[ 2014-09-29 08:35:08.8076 18182/7ff9c2f6b720 agents/HelperAgent/Main.cpp:643 ]: Starting PassengerHelperAgent...
[ 2014-09-29 08:35:08.8156 18182/7ff9c2f6b720 agents/HelperAgent/Main.cpp:650 ]: PassengerHelperAgent online, listening at unix:/var/log/httpd/passenger.1.0.18178/generation-0/request
[ 2014-09-29 08:35:08.8308 18187/7f4b9962c7e0 agents/Base.cpp:1591 ]: Random seed: 1411994108
[ 2014-09-29 08:35:08.8312 18187/7f4b9962c7e0 agents/LoggingAgent/Main.cpp:333 ]: Starting PassengerLoggingAgent...
[Mon Sep 29 08:35:09.859284 2014] [passenger:error] [pid 18178:tid 139976593561344] *** Passenger could not be initialized because of this error: Unable to start the Phusion Passenger watchdog because it encountered the following error during startup: Unable to start the Phusion Passenger logging agent: it seems to have been killed with signal SIGKILL during startup

I could find the PassengerLoggingAgent in below location:
./usr/lib/ruby/gems/1.8/gems/passenger-4.0.52/buildout/agents/PassengerLoggingAgent

Did a Gem install of Passenger. Build ran fine.

Environment:
RHEL 6.4
Passenger Version: 4.0.52
Apache: 2.4.2

httpd.conf:
 LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-4.0.52/buildout/apache2/mod_passenger.so
   <IfModule mod_passenger.c>
     PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-4.0.52
     PassengerDefaultRuby /usr/bin/ruby
     PassengerLogLevel 3
     PassengerUser puppet
     PassengerGroup puppet
     PassengerDefaultUser puppet
     PassengerDefaultGroup puppet
     PassengerFriendlyErrorPages on
     PassengerTempDir /var/log/httpd
     PassengerSpawnMethod direct
     PassengerStartTimeout 500
     PassengerPreStart https://myserver.com/usr/share/puppet/rack/puppetmasterd/public
   </IfModule>

conf.d
PassengerMaxPoolSize 12
# Recycle master processes after they service 1000 requests
PassengerMaxRequests 1000
# Stop processes if they sit idle for 10 minutes
PassengerPoolIdleTime 600

Listen 8140
<VirtualHost *:8140>
    # Make Apache hand off HTTP requests to Puppet earlier, at the cost of
    # interfering with mod_proxy, mod_rewrite, etc. See note below.
    PassengerHighPerformance On

    ServerName myserver.com

    SSLEngine On

    # Only allow high security cryptography. Alter if needed for compatibility.
    SSLProtocol ALL -SSLv2 -SSLv3
    SSLCipherSuite EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
    SSLHonorCipherOrder     on

    SSLCertificateFile      /var/lib/puppet/ssl/certs/myserver.com.pem
    SSLCertificateKeyFile   /var/lib/puppet/ssl/private_keys/myserver.com.pem
    SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
    SSLCACertificateFile    /var/lib/puppet/ssl/ca/ca_crt.pem
    SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
    SSLCARevocationCheck        chain
    SSLVerifyClient         optional
    SSLVerifyDepth          1
    SSLOptions              +StdEnvVars +ExportCertData

    # Apache 2.4 introduces the SSLCARevocationCheck directive and sets it to none
        # which effectively disables CRL checking. If you are using Apache 2.4+ you must
    # specify 'SSLCARevocationCheck chain' to actually use the CRL.

    # These request headers are used to pass the client certificate
    # authentication information on to the puppet master process
    RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

    DocumentRoot /usr/share/puppet/rack/puppetmasterd/public
PassengerAppRoot /usr/share/puppet/rack/puppetmasterd

    <Directory /usr/share/puppet/rack/puppetmasterd/>
      Options None
      AllowOverride all
      # Apply the right behavior depending on Apache version.
      <IfVersion < 2.4>
        Order allow,deny
        Allow from all
      </IfVersion>
      <IfVersion >= 2.4>
        Require all granted
      </IfVersion>
      Options -MultiViews
    </Directory>

    <Directory /usr/share/puppet/rack/puppetmasterd/public >
      Options None
      AllowOverride all
      # Apply the right behavior depending on Apache version.
      <IfVersion < 2.4>
        Order allow,deny
        Allow from all
      </IfVersion>
      <IfVersion >= 2.4>
        Require all granted
      </IfVersion>
      Options -MultiViews
    </Directory>

    ErrorLog /var/log/httpd/myserver.com_ssl_error.log
    CustomLog /var/log/httpd/myserver.com_ssl_access.log combined
</VirtualHost>

[Tinco Andringa]

Hi, it says the logging agent has been killed with SIGKILL, do you
have an idea what could have made that happen?

> --
> You received this message because you are subscribed to the Google Groups
> "Phusion Passenger Discussions" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to phusion-passen...@googlegroups.com.
> To post to this group, send email to phusion-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/phusion-passenger.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/phusion-passenger/aa6285f2-3d97-46ff-bde4-c0bce8af6207%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Phusion | Web application deployment, scaling and monitoring solutions

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

[pavithra....@gmail.com]

no idea. there is no error message on why it is getting killed. And also no other processes running on that server that could kill this process. During startup, does the logging agent try to talk to union_station_gateway and any related failure could kill logging agent? Is there a way to identify if the loggingagent is dumping core ?

[Hongli Lai]

SIGKILL is often caused by the system running out of memory. Is your
system low on memory?

> https://groups.google.com/d/msgid/phusion-passenger/54aaba71-5524-4f80-81fe-4fa5756cbe1a%40googlegroups.com.

>
> For more options, visit https://groups.google.com/d/optout.



--

Phusion | Web Application deployment, scaling, and monitoring solutions

[pavithra....@gmail.com]

Server has approx 6GB free memory.

Found the root cause of the problem. Server user security profile of the user was killing the process as it was not started as clean root. Corrected the security profile. Now process is running fine.

Thanks a lot for your help!