[jruby-user] jruby-rack-0.9.4 can't find rubygems?

[Rich Manalang]

We have a JRuby (1.2.0) app running under jruby-rack-0.9.4 just fine under a non-root uid.  We've set up monit to monitor Tomcat.  Monit can shutdown tomcat just fine.  However, when starting it up, it's able to startup Tomcat and all of the non-rails web apps, however, for rails apps, we get the following stack trace:

Looks like it's having a hard time finding RubyGems.  Any ideas?  BTW, monit is starting Tomcat back up under the same non-root uid/gid.  We can startup Tomcat manually just fine without Monit.

Rich


Jul 8, 2009 4:13:07 AM org.apache.catalina.core.ApplicationContext log
SEVERE: unable to create shared application instance
org.jruby.rack.RackInitializationException: exit
        from /u01/app/appslab/tomcat/webapps/ROOT/WEB-INF/config/boot.rb:38:in `run'
        from file:/u01/app/appslab/connect/releases/20090702205834/tmp/war/WEB-INF/lib/jruby-rack-0.9.4.jar!/jruby/rack/rails_boot.rb:20:in `run'
        from /u01/app/appslab/tomcat/webapps/ROOT/WEB-INF/config/boot.rb:11:in `boot!'
        from /u01/app/appslab/tomcat/webapps/ROOT/WEB-INF/config/boot.rb:110
        from /u01/app/appslab/tomcat/webapps/ROOT/WEB-INF/config/boot.rb:6:in `require'
        from /u01/app/appslab/tomcat/webapps/ROOT/WEB-INF/config/environment.rb:6
        from /u01/app/appslab/tomcat/webapps/ROOT/WEB-INF/config/environment.rb:26:in `load'
        from file:/u01/app/appslab/connect/releases/20090702205834/tmp/war/WEB-INF/lib/jruby-rack-0.9.4.jar!/jruby/rack/rails.rb:26:in `load_environment'
        from file:/u01/app/appslab/connect/releases/20090702205834/tmp/war/WEB-INF/lib/jruby-rack-0.9.4.jar!/jruby/rack/rails.rb:151:in `new'  
        from <script>:2
        from file:/u01/app/appslab/connect/releases/20090702205834/tmp/war/WEB-INF/lib/jruby-rack-0.9.4.jar!/rack/builder.rb:29:in `instance_eval'
        from file:/u01/app/appslab/connect/releases/20090702205834/tmp/war/WEB-INF/lib/jruby-rack-0.9.4.jar!/rack/builder.rb:29:in `initialize'
        from <script>:2

        at org.jruby.rack.DefaultRackApplicationFactory$4.init(DefaultRackApplicationFactory.java:153)
        at org.jruby.rack.DefaultRackApplicationFactory.getApplication(DefaultRackApplicationFactory.java:50)
        at org.jruby.rack.SharedRackApplicationFactory.init(SharedRackApplicationFactory.java:26)
        at org.jruby.rack.RackServletContextListener.contextInitialized(RackServletContextListener.java:39)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
        at org.apache.catalina.core.StandardContext.start(StandardContext.java:4350)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
        at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:924)
        at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:887)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
        at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1147)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
        at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
        at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
        at org.apache.catalina.core.StandardService.start(StandardService.java:516)

[Charles Oliver Nutter]

It looks like in this configuration it is unable to unpack the war
contents to disk and use them from there. Note the URL:

file:/u01/app/appslab/connect/releases/20090702205834/tmp/war/WEB-INF/lib/jruby-rack-0.9.4.jar!/jruby/rack/rails_boot.rb

which has the boot.rb loading out of a jar file. I suspect the problem
is that since it can't unpack the jar, all the logic in Rails for
loading the app from the filesystem is borking out, and ultimately it
just gives up because it can't find anything.

Can you check user permissions on the location of the WAR files?

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[Rich Manalang]

Nope... that's not it.  The jars have global read perms.  I'm now running Monit under the same user we use to start Tomcat... same issue.  One thing I noticed is that Monit stops and starts the Tomcat services without any sleep time in between.  Tomcat does start fine, but the Rails app errors out as I reported below.  Now that I think about it, I'm not sure that's the real issue since I've stopped the server with Monit, waited a few seconds, then started it back up with Monit and got the same error.  The odd part is that it works fine if we start it up manually.

Anyway, other ideas?

Rich

[Matthew Kanwisher]

Out of curiosity what operating system are you on? I have a similar issue on solaris a bit different but still.

~Matt

[Matt Burke]

Rich Manalang wrote:
> Nope... that's not it. The jars have global read perms. I'm now
> running Monit under the same user we use to start Tomcat... same issue.
> One thing I noticed is that Monit stops and starts the Tomcat services
> without any sleep time in between. Tomcat does start fine, but the
> Rails app errors out as I reported below. Now that I think about it,
> I'm not sure that's the real issue since I've stopped the server with
> Monit, waited a few seconds, then started it back up with Monit and got
> the same error. The odd part is that it works fine if we start it up
> manually.
>
> Anyway, other ideas?

Just a stab in the dark: Is the cd different between when monit starts
tomcat and when you start tomcat? Or, are there different environment
variables set?

Does monit execute the exact same command you type in your shell, or
does it do something different?

--
Matt

[Rich Manalang]

We're on Oracle Enterprise Linux 5 (basically a variant of RHEL5) -- kernel => 2.6.9-55.0.2.6.2.ELsmp

Rich

[Rich Manalang]

I've been trying to figure out the same thing.  The CD shouldn't matter since I can start tomcat (./tomcat/bin/startup.sh) from any directory.  I also looked at the env vars... I ended up adding the env vars that matter into the catalina.sh file... still borked.

Rich

[Charles Oliver Nutter]

On Wed, Jul 8, 2009 at 5:33 PM, Rich Manalang<rich.m...@gmail.com> wrote:
> I've been trying to figure out the same thing.  The CD shouldn't matter
> since I can start tomcat (./tomcat/bin/startup.sh) from any directory.  I
> also looked at the env vars... I ended up adding the env vars that matter
> into the catalina.sh file... still borked.

It certainly seems like it has to be some permission thing. Perhaps
you could add some logging to the rack .rb files to try to get more
information out...I know it has a tendency to swallow most of the
pertinent error information.

That would be a good bug to file too :) it really needs better error
reporting, so we don't do this dance every time someone sneezes.

- Charlie

[Matthew Kanwisher]

I think your experiencing the same problem as me, if the PWD is "/" the parsing code is breaking some of the requires. I'm having this happen in solaris under "svcadm" which is just like monit. I'm rather curios if you can force monit to have a different working path before launching jruby if it will act any different.

http://jira.codehaus.org/browse/JRUBY-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=183119#action_183119

~Matt

[Rich Manalang]

Ok... I figured it out.  Monit's working directory when running as a daemon is "/".  JRuby-Rack/Rails doesn't like that... I bet there's some code that relies on the working directory of the container (I'll file a bug).  To resolve this, I modified ./tomcat/bin/catalina.sh and added the following to the beginning:

cd $CATALINA_HOME

Rich