moonshine_cijoe crashes

[Matt R]

We've had cijoe disabled for a long time as we couldn't get it working. Some passenger update happened, I think, around early Oct 2011 and broke cijoe.

When apache restarts, we get:

Tue Mar 06 16:00:35 2012] [notice] caught SIGTERM, shutting down

[Tue Mar 06 16:00:38 2012] [notice] Apache/2.2.14 (Ubuntu) Phusion_Passenger/3.0.9 mod_ssl/2.2.14 OpenSSL/0.9.8k configured -- resuming normal operations

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >'

  what():  Invalid argument (22)

[ pid=9453, timestamp=1331049641 ] Process aborted! signo=SIGABRT(6), reason=SI_TKILL, signal sent by PID 9453 with UID 0, backtrace available.

--------------------------------------

[ pid=9453 ] Backtrace with 34 frames:

PassengerHelperAgent[0x4fda2e]

/lib/libpthread.so.0(+0xf8f0)[0x7f9b945ed8f0]

/lib/libc.so.6(gsignal+0x35)[0x7f9b93ae0a75]

/lib/libc.so.6(abort+0x180)[0x7f9b93ae45c0]

/usr/lib/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x115)[0x7f9b943968e5]

/usr/lib/libstdc++.so.6(+0xcad16)[0x7f9b94394d16]

/usr/lib/libstdc++.so.6(+0xc9c09)[0x7f9b94393c09]

/usr/lib/libstdc++.so.6(__gxx_personality_v0+0x261)[0x7f9b943946b1]

/lib/libgcc_s.so.1(+0x108a3)[0x7f9b93e408a3]

/lib/libgcc_s.so.1(_Unwind_Resume+0x68)[0x7f9b93e40968]

PassengerHelperAgent(_ZN5boost5mutex4lockEv+0x6b)[0x48e8b3]

PassengerHelperAgent(_ZN5boost10lock_guardINS_5mutexEEC1ERS1_+0x2a)[0x4c8b18]

PassengerHelperAgent(_ZN9Passenger5Timer5startEv+0x23)[0x492501]

PassengerHelperAgent(_ZN18TimerUpdateHandler18clientDisconnectedERN9Passenger13MessageServer19CommonClientContextERN5boost10shared_ptrINS1_13ClientContextEEE+0x40)[0x4c4b58]

PassengerHelperAgent(_ZN9Passenger13MessageServer29DisconnectEventBroadcastGuardD1Ev+0x91)[0x49dcdd]

PassengerHelperAgent(_ZN9Passenger13MessageServer22clientHandlingMainLoopERNS_14FileDescriptorE+0xc70)[0x49f524]

PassengerHelperAgent(_ZNK5boost4_mfi3mf1IvN9Passenger13MessageServerERNS2_14FileDescriptorEEclEPS3_S5_+0x64)[0x4f28a2]

PassengerHelperAgent(_ZN5boost3_bi5list2INS0_5valueIPN9Passenger13MessageServerEEENS2_INS3_14FileDescriptorEEEEclINS_4_mfi3mf1IvS4_RS7_EENS0_5list0EEEvNS0_4typeIvEERT_RT0_i+0x69)[0x4f0f03]

PassengerHelperAgent(_ZN5boost3_bi6bind_tIvNS_4_mfi3mf1IvN9Passenger13MessageServerERNS4_14FileDescriptorEEENS0_5list2INS0_5valueIPS5_EENSA_IS6_EEEEEclEv+0x3f)[0x4eebf3]

PassengerHelperAgent(_ZN5boost6detail8function26void_function_obj_invoker0INS_3_bi6bind_tIvNS_4_mfi3mf1IvN9Passenger13MessageServerERNS7_14FileDescriptorEEENS3_5list2INS3_5valueIPS8_EENSD_IS9_EEEEEEvE6invokeERNS1_15function_bufferE+0x23)[0x4e9235]

PassengerHelperAgent(_ZNK5boost9function0IvEclEv+0x73)[0x4c9383]

PassengerHelperAgent(_ZN3oxt20dynamic_thread_group11thread_mainERN5boost8functionIFvvEEEPNS0_13thread_handleE+0x3a)[0x494ae6]

PassengerHelperAgent(_ZNK5boost4_mfi3mf2IvN3oxt20dynamic_thread_groupERNS_8functionIFvvEEEPNS3_13thread_handleEEclEPS3_S7_S9_+0x70)[0x4f2560]

PassengerHelperAgent(_ZN5boost3_bi5list3INS0_5valueIPN3oxt20dynamic_thread_groupEEENS2_INS_8functionIFvvEEEEENS2_IPNS4_13thread_handleEEEEclINS_4_mfi3mf2IvS4_RS9_SC_EENS0_5list0EEEvNS0_4typeIvEERT_RT0_i+0x88)[0x4f09ea]

PassengerHelperAgent(_ZN5boost3_bi6bind_tIvNS_4_mfi3mf2IvN3oxt20dynamic_thread_groupERNS_8functionIFvvEEEPNS5_13thread_handleEEENS0_5list3INS0_5valueIPS5_EENSE_IS8_EENSE_ISB_EEEEEclEv+0x3f)[0x4ee6c7]

PassengerHelperAgent(_ZN5boost6detail8function26void_function_obj_invoker0INS_3_bi6bind_tIvNS_4_mfi3mf2IvN3oxt20dynamic_thread_groupERNS_8functionIFvvEEEPNS8_13thread_handleEEENS3_5list3INS3_5valueIPS8_EENSH_ISB_EENSH_ISE_EEEEEEvE6invokeERNS1_15function_bufferE+0x23)[0x4e872f]

PassengerHelperAgent(_ZNK5boost9function0IvEclEv+0x73)[0x4c9383]

PassengerHelperAgent(_ZN3oxt6thread11thread_mainEN5boost8functionIFvvEEENS1_10shared_ptrINS0_11thread_dataEEE+0x64)[0x48fd48]

PassengerHelperAgent(_ZN5boost3_bi5list2INS0_5valueINS_8functionIFvvEEEEENS2_INS_10shared_ptrIN3oxt6thread11thread_dataEEEEEEclIPFvS5_SB_ENS0_5list0EEEvNS0_4typeIvEERT_RT0_i+0x8c)[0x4f5866]

PassengerHelperAgent(_ZN5boost3_bi6bind_tIvPFvNS_8functionIFvvEEENS_10shared_ptrIN3oxt6thread11thread_dataEEEENS0_5list2INS0_5valueIS4_EENSD_IS9_EEEEEclEv+0x3f)[0x4f57d1]

PassengerHelperAgent(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvPFvNS_8functionIFvvEEENS_10shared_ptrIN3oxt6thread11thread_dataEEEENS2_5list2INS2_5valueIS6_EENSF_ISB_EEEEEEE3runEv+0x1e)[0x4f4fe4]

PassengerHelperAgent(thread_proxy+0x6d)[0x506e5f]

/lib/libpthread.so.0(+0x69ca)[0x7f9b945e49ca]

/lib/libc.so.6(clone+0x6d)[0x7f9b93b9370d]

--------------------------------------

[ pid=9453 ] Dumping a more detailed backtrace with crash-watch ('gem install crash-watch' if you don't have it)...

sh: crash-watch: not found

then

[ pid=9456 thr=9906480 file=utils.rb:176 time=2012-03-06 16:02:37.081 ]: *** Exception Errno::ENOMEM in PhusionPassenger::Rack::ApplicationSpawner (Cannot allocate memory - cd #<Proc:0x00000000021fb178@/usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/server.rb:103> && git config cijoe.user 2>&1) (process 9456, thread #<Thread:0x12e5260>):

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/config.rb:19:in ``'

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/config.rb:19:in `to_s'

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/server.rb:96:in `project_path='

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `__send__'

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `set'

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/server.rb:103:in `project_path='

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `__send__'

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `set'

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/server.rb:103:in `project_path='

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `__send__'

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `set'

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/server.rb:103:in `project_path='

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `__send__'

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `set'

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/server.rb:103:in `project_path='

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `__send__'

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `set'

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/server.rb:103:in `project_path='

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `__send__'

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `set'

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/server.rb:103:in `project_path='

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:983:in `__send__'

from config.ru:9

from /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.2/lib/sinatra/base.rb:1273:in `configure'

from config.ru:8

from /usr/lib/ruby/gems/1.8/gems/rack-1.4.1/lib/rack/builder.rb:51:in `instance_eval'

from /usr/lib/ruby/gems/1.8/gems/rack-1.4.1/lib/rack/builder.rb:51:in `initialize'

from config.ru:1:in `new'

from config.ru:1

:public is no longer used to avoid overloading Module#public, use :public_folder instead

from /usr/lib/ruby/gems/1.8/gems/cijoe-0.9.2/lib/cijoe/server.rb:12

Any ideas?

Thanks,

--

Matt

[Lee Jones]

Low free memory may be the problem:

Exception Errno::ENOMEM in PhusionPassenger::Rack::ApplicationSpawner (Cannot allocate memory

How much free memory is available on the server when you try to run it?

Peace,

Lee

--
You received this message because you are subscribed to the Google Groups "Moonshine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/railsmachine-moonshine/-/Hi1axIDd3Q4J.
To post to this group, send email to railsmachin...@googlegroups.com.
To unsubscribe from this group, send email to railsmachine-moon...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/railsmachine-moonshine?hl=en.

[Matt R]

Hi Lee,

I think it's essentially a stack overflow as I see the __send__ -> project_path thing over and over and over before it times out.

It may be related to this bug:

https://github.com/defunkt/cijoe/pull/79/files

Thanks,

--

Matt

On Tuesday, March 6, 2012 2:59:16 PM UTC-5, Lee Jones wrote:

Low free memory may be the problem:

Exception Errno::ENOMEM in PhusionPassenger::Rack::ApplicationSpawner (Cannot allocate memory

How much free memory is available on the server when you try to run it?

Peace,

Lee

To post to this group, send email to railsmachine-moonshine@googlegroups.com.
To unsubscribe from this group, send email to railsmachine-moonshine+unsub...@googlegroups.com.

[Matt R]

I went ahead and forked cijoe to fix that issue, but I don't see a way, in moonshine, to specify a non-rubygems dependency. I have the gem at github.com/Toura/cijoe, but it doesn't appear moonshine will install from github like bundler does. Any ideas?

Thanks,

--

Matt

[Matt R]

For now, I created a gem and placed it within the project:

    gem 'cijoe', {:version => '0.9.3', :source => "#{rails_root.join('vendor', 'plugins', 'moonshine_cijoe', 'templates', 'cijoe-0.9.3.gem')}" }

That did it!

Thanks,

--

Matt

[Josh Nichols]

Hey Matt,

That's definitely the simplest approach for.

I'm not sure of the exact path yet, but I could imagine updating cijoe.config.ru to use bundler. That way, you could get more control over what is loaded by cijoe and whatnot.

- Josh

To view this discussion on the web visit https://groups.google.com/d/msg/railsmachine-moonshine/-/bJpnvd5F6PQJ.
To post to this group, send email to railsmachin...@googlegroups.com.
To unsubscribe from this group, send email to railsmachine-moon...@googlegroups.com.