SIGHUP and Could not find rake errors

[jjsb...@gmail.com]

Hi, I'm new to Passenger and can't figure out some errors! The sys admin cloned another prod and set everything up on the server so I'm not familiar with how the setup went.

I have Passenger+Rails+Apache+Capistrano running on Redhat/6/x86_64.

Problem: Website is fine during the week but keeps crashing over the weekend and getting a 500 server error on login attempt. I get the website back up by deploying with Capistrano on the past two Monday mornings when I find the website down.

Error log contains multiple passenger errors about not finding rake-10.3.1 and also a SIGHUP which shuts down passenger. I'm not sure if these two are related. Rake-10.3.1 is installed and I can't figure out why bundler can't find it. I also don't know why SIGHUP happens.

Error log:

App 26864 stdout:

App 26864 stdout:

[ 2016-08-27 13:45:57.9034 21851/7f65444d4700 age/Cor/App/Implementation.cpp:304 ]: Could not spawn process for application /opt/Dashboard/current: An error occurred while starting up the preloader.

  Error ID: 095c456b

  Error details saved to: /tmp/passenger-error-IXS9sE.html

  Message from application: <p>It looks like Bundler could not find a gem. Maybe you didn't install all the gems that this application needs. To install your gems, please run:</p>

  <pre class="commands">bundle install</pre>

<p>If that didn't work, then maybe the problem is that your gems are installed to <code>/home/jjsb/.rvm/gems</code>, while at the same time you set <code>PassengerRuby</code> (Apache) or <code>passenger_ruby</code> (Nginx) to <code>/usr/local/rvm/gems/ruby-2.0.0-p451/wrappers/ruby</code>. Because of the latter, RVM does not load gems from the home directory.</p>

<p>To make RVM load gems from the home directory, you need to set <code>PassengerRuby</code>/<code>passenger_ruby</code> to an RVM wrapper script inside the home directory:</p>

<ol>

  <li>Login as jjsb.</li>

  <li>Enable RVM mixed mode by running:

      <pre class="commands">rvm user gemsets</pre></li>

  <li>Run this to find out what to set <code>PassengerRuby</code>/<code>passenger_ruby</code> to:

      <pre class="commands">/usr/local/rvm/gems/ruby-2.0.0-p451/wrappers/ruby \

/usr/local/rvm/gems/ruby-2.0.0-p451/gems/passenger-5.0.28/bin/passenger-config --detect-ruby</pre></li>

</ol>

<p>If that didn't help either, then maybe your application is being run under a different environment than it's supposed to. Please check the following:</p>

<ol>

  <li>Is this app supposed to be run as the <code>jjsb</code> user?</li>

  <li>Is this app being run on the correct Ruby interpreter? Below you will

      see which Ruby interpreter Phusion Passenger attempted to use.</li>

  <li>Please check whether the correct RVM gemset is being used.</li>

  <li>Sometimes, RVM gemsets may be broken.

      <a href="https://github.com/phusion/passenger/wiki/Resetting-RVM-gemsets">Try resetting them.</a></li>

</ol>

<p>-------- The exception is as follows: -------</p>

Could not find rake-10.3.1 in any of the sources (Bundler::GemNotFound)

<pre>  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler/spec_set.rb:95:in `block in materialize&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler/spec_set.rb:88:in `map!&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler/spec_set.rb:88:in `materialize&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler/definition.rb:140:in `specs&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler/definition.rb:185:in `specs_for&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler/definition.rb:174:in `requested_specs&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler/environment.rb:19:in `requested_specs&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler/runtime.rb:14:in `setup&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler.rb:95:in `setup&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/bundler-1.12.2/lib/bundler/setup.rb:19:in `&lt;top (required)&gt;&#39;

  /usr/local/rvm/rubies/ruby-2.0.0-p451/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:54:in `require&#39;

  /usr/local/rvm/rubies/ruby-2.0.0-p451/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:54:in `require&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/passenger-5.0.28/src/ruby_supportlib/phusion_passenger/loader_shared_helpers.rb:430:in `activate_gem&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/passenger-5.0.28/src/ruby_supportlib/phusion_passenger/loader_shared_helpers.rb:297:in `block in run_load_path_setup_code&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/passenger-5.0.28/src/ruby_supportlib/phusion_passenger/loader_shared_helpers.rb:435:in `running_bundler&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/passenger-5.0.28/src/ruby_supportlib/phusion_passenger/loader_shared_helpers.rb:296:in `run_load_path_setup_code&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/passenger-5.0.28/src/helper-scripts/rack-preloader.rb:100:in `preload_app&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/passenger-5.0.28/src/helper-scripts/rack-preloader.rb:156:in `&lt;module:App&gt;&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/passenger-5.0.28/src/helper-scripts/rack-preloader.rb:30:in `&lt;module:PhusionPassenger&gt;&#39;

  /usr/local/rvm/gems/ruby-2.0.0-p451/gems/passenger-5.0.28/src/helper-scripts/rack-preloader.rb:29:in `&lt;main&gt;&#39;</pre>

[Sun Aug 28 03:22:06 2016] [notice] SIGHUP received.  Attempting to restart

[ 2016-08-28 03:22:07.5611 21851/7f6545dc6700 age/Cor/CoreMain.cpp:532 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)

[ 2016-08-28 03:22:07.5932 21851/7f65468df7e0 age/Cor/CoreMain.cpp:901 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...

[ 2016-08-28 03:22:07.7252 21851/7f65468df7e0 age/Cor/CoreMain.cpp:819 ]: Checking whether to disconnect long-running connections for process 32139, application /opt/Dashboard/current (production)

[ 2016-08-28 03:22:07.8149 21859/7fbb8ffff700 age/Ust/UstRouterMain.cpp:422 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)

[ 2016-08-28 03:22:07.8533 21859/7fbb94c497e0 age/Ust/UstRouterMain.cpp:492 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...

[ 2016-08-28 03:22:08.1095 21851/7f6527fff700 Ser/Server.h:817 ]: [ApiServer] Freed 0 spare client objects

[ 2016-08-28 03:22:08.1397 21851/7f6545344700 Ser/Server.h:817 ]: [ServerThr.2] Freed 128 spare client objects

[ 2016-08-28 03:22:08.1398 21851/7f6545344700 Ser/Server.h:464 ]: [ServerThr.2] Shutdown finished

[ 2016-08-28 03:22:08.1626 21851/7f6527fff700 Ser/Server.h:464 ]: [ApiServer] Shutdown finished

[ 2016-08-28 03:22:08.2264 21859/7fbb8ffff700 Ser/Server.h:464 ]: [UstRouter] Shutdown finished

[ 2016-08-28 03:22:08.2265 21859/7fbb8f5fe700 Ser/Server.h:817 ]: [UstRouterApiServer] Freed 0 spare client objects

[ 2016-08-28 03:22:08.2265 21859/7fbb8f5fe700 Ser/Server.h:464 ]: [UstRouterApiServer] Shutdown finished

[ 2016-08-28 03:22:08.2278 21859/7fbb94c497e0 age/Ust/UstRouterMain.cpp:523 ]: Passenger UstRouter shutdown finished

[ 2016-08-28 03:22:08.2666 21851/7f6545dc6700 Ser/Server.h:817 ]: [ServerThr.1] Freed 128 spare client objects

[ 2016-08-28 03:22:08.2667 21851/7f6545dc6700 Ser/Server.h:464 ]: [ServerThr.1] Shutdown finished

[ 2016-08-28 03:22:08.2967 21851/7f65468df7e0 age/Cor/CoreMain.cpp:819 ]: Checking whether to disconnect long-running connections for process 32139, application /opt/Dashboard/current (production)

[ 2016-08-28 03:22:11.0622 21851/7f65468df7e0 age/Cor/CoreMain.cpp:967 ]: Passenger core shutdown finished

I've been googling for solutions but there's so many possible causes and I don't know where to start.

Thanks for reading! If anyone has any insights I'd really appreciate it!

- Jessica

[akshar jamgaonkar]

Hi,

Found this solution on stackoverflow.

This appears passenger looks in /home/<username>/.gem/ruby as a part of its spawn process. If /home/<username> doesn't exist (like with system users that don't have home directories), you get this error.

here is the link to it. http://stackoverflow.com/questions/35163623/passenger-could-not-spawn-process-for-application

have you tried this solution?

[Daniel Knoppel]

Jessica,

Did you find the official documentation library?

It has a lot of info, like quickstart tutorials: https://www.phusionpassenger.com/library/

- Daniel