[jruby-user] Warbler Tomcat Jruby Rack Timeout Problem

[Chuck O.]

I have an app running in tomcat from a war created with warbler running
with the following:

Apache Tomcat/6.0.24
rack-1.2.1
rack-mount-0.6.13
rails-3.0.4

My webxml looks like:
<!DOCTYPE web-app PUBLIC
"-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
"http://java.sun.com/dtd/web-app_2_3.dtd">
<web-app>

<context-param>
<param-name>rails.env</param-name>
<param-value>production</param-value>
</context-param>

<context-param>
<param-name>public.root</param-name>
<param-value>/</param-value>
</context-param>

<context-param>
<param-name>jruby.min.runtimes</param-name>
<param-value>2</param-value>
</context-param>

<context-param>
<param-name>jruby.max.runtimes</param-name>
<param-value>2</param-value>
</context-param>

<context-param>
<param-name>jruby.rack.logging</param-name>
<param-value>slf4j</param-value>
</context-param>

<filter>
<filter-name>RackFilter</filter-name>
<filter-class>org.jruby.rack.RackFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>RackFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

<listener>
<listener-class>org.jruby.rack.rails.RailsServletContextListener</listener-class>
</listener>

</web-app>

My problem is if the tomcat server is getting requests (say simple ones
that just return a string every 5 seconds ~12ms) during the boot process
after my runtimes spin up they service one request and then go into a
wait/sleep/lock state and all subsequent requests timeout. I've tested
these servers under load and they easily handle a hundred concurrent
request, if the server starts up without any requests during runtime
spinup. Following is an example of the first successful request after
spinup followed by timeouts:

Started GET "/myapp/alive" for 127.0.0.1 at Thu Feb 10 17:49:29 -0500
2011
Processing by AliveController#index as
Rendered text template (0.0ms)
Completed 200 OK in 121ms (Views: 25.0ms | ActiveRecord: 12.0ms)
10 Feb 2011 17:50:02 ERROR [http-80-1] Application
Errororg.jruby.rack.RackInitializationException: timeout: all listeners
busy
at
org.jruby.rack.PoolingRackApplicationFactory.getApplication(PoolingRackApplicationFactory.java:84)
at
org.jruby.rack.DefaultRackDispatcher.process(DefaultRackDispatcher.java:28)
at org.jruby.rack.RackFilter.doFilter(RackFilter.java:63)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.InterruptedException
... 15 more

Anyone else ever experience this? Have a solution or workaround?

Any help would be appreciated, Thanks
Chuck

--
Posted via http://www.ruby-forum.com/.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[Chuck O.]

Well, I enabled threadsafe!, set my runtimes to 1 and used the compiled
feature of warbler and fingers crossed everything seems to be working.
Has any one had any issues with threadsafe! in jruby/tomcat? If so what
were the workarounds?

[Lenny Marks]

On Feb 11, 2011, at 9:32 AM, Chuck O. wrote:

> Well, I enabled threadsafe!, set my runtimes to 1 and used the compiled
> feature of warbler and fingers crossed everything seems to be working.
> Has any one had any issues with threadsafe! in jruby/tomcat? If so what
> were the workarounds?

One thing that tripped us up a bit. threadsafe! disables automatic dependency loading so you might find yourself getting uninitialized constant errors in production. Looking back at the documentation now at http://edgeguides.rubyonrails.org/configuring.html , the docs for config.dependency_loading seem to contradict the docs for theadsafe! saying that it can be enabled for threadsafe!.

http://stackoverflow.com/questions/3336925/what-is-the-correct-way-to-load-modules-classes-from-lib-when-using-the-config-t

-lenny

[Nick Griffiths]

Kind of obvious, but easy to forget, is that your application needs to
be threadsafe! Things like:

def big_thing
@big_thing ||= build_big_thing
end

in a multi threaded scenario can cause lots of calls to build_big_thing
and therefore more than one instance of a big_thing floating around your
application.

You might find other libraries are not clear about their thread safety
(we had trouble with Savon) and could cause nasty errors that will only
appear under load.

On Fri, 2011-02-11 at 15:32 +0100, Chuck O. wrote:
> Well, I enabled threadsafe!, set my runtimes to 1 and used the compiled
> feature of warbler and fingers crossed everything seems to be working.
> Has any one had any issues with threadsafe! in jruby/tomcat? If so what
> were the workarounds?
>

---------------------------------------------------------------------

[Mark Thomas]

On Feb 12, 5:32 am, Nick Griffiths <n...@playlouder.com> wrote:
> Kind of obvious, but easy to forget, is that your application needs to
> be threadsafe!  Things like:
>
> def big_thing
>   @big_thing ||= build_big_thing
> end
>
> in a multi threaded scenario can cause lots of calls to build_big_thing
> and therefore more than one instance of a big_thing floating around your
> application.

I'm not sure I understand the thread safety issue here. Instance
variables are not shared across threads, correct? So this would be no
different than

def big_thing
build_big_thing
end

except you can call big_thing.foo and big_thing.bar without building
big_thing twice. This is all in a single thread.

However, if the intent is to share big_thing across threads, it would
have to be stored in a global.

[Nick Griffiths]

Instance variables are shared across threads if you share the instance
across threads. Rails will create a new controller for each request
(afaik), so you are sort of safe in that respect. You aren't safe if
you share any state at all, which is entirely possible in a rails
application.

FYI: try this in a jruby console and notice how many times the "creating
it" string is puts'd. On my box, which is a quad core, I get around
three each time.

~~~

class Holder
def get_it
@it ||= begin
puts "creating it"
"this is a string that is being returned"
end
end
end

instance = Holder.new
(0...6).each { Thread.new { instance.get_it } }

~~~

--
Nick Griffiths
Developer @ Media Service Provider
+44 207 729 4797