Help understanding Passenger 4/5 community Fork/Processes Threads JRuby Sequel DB pools

[Dale Ackerman]

I am confused by the many posts and other documents and posts regarding how passenger works with JRuby (fork process or spawn thread) and how that impacts non rails(ActiveRecord) connection pools.  I have inherited a bunch of ruby (rails/sinatra) application all running on some combination of Apache2 and passenger both rub/jruby.  I am not convinced that the configuration are correct.  So I need to understand the facts regarding DB connection pool sharing (if it does) and the differences in methods used for allocating threads (JRuby JVM) and or forking new OS processes.

stack #1

------------------------------------

Apache2

Passenger 4/5

JRuby

Sinatra

Sequel ORM (jdbc drivers) 

stack #2

------------------------------------

Apache2

Passenger 4/5

MRI Ruby 1.9 - 2.2.2

Rails 3.x - 4.x

ActiveRecord ORM (MSSQL freeTDS drivers and some Postgresql)

I am placing this same question to the sequel-talk group as well.  I hope to find the official answer and get that posted in both projects documentation.

Best Regards

[Hongli Lai]

I am not entirely sure what your question is. But in general,
Passenger treats MRI and JRuby the same. In both cases, Passenger
spawns multiple JRuby processes (so multiple JVMs). If you are using
Passenger Enterprise and enabled multithreading, then Passenger spawns
Ruby threads in each (J)Ruby process. In all cases, the ActiveRecord
connection pool is shared between threads inside the same process
only. The connection pool is never shared between processes.

> --
> You received this message because you are subscribed to the Google Groups
> "Phusion Passenger Discussions" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to phusion-passen...@googlegroups.com.
> To post to this group, send email to phusion-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/phusion-passenger.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/phusion-passenger/b2e25647-fd07-4f84-a1a5-18169caf7df1%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Phusion B.V. | Web Application deployment, scaling, and monitoring solutions

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 63501007 (The Netherlands)

[Dale Ackerman]

Hi Thanks for the reply.

My question came about due to information I had gotten regarding the Sequel gem  It is a ORM like ActiveRecord but the author had stated that Passenger held onto some descriptor or handle and some how shared connection pool between forked processes?  I understand that a pool or anything else for that matter would only be reachable in the context of a threaded application (in same process).  Just so I am clear :

1.) Community version of Passenger will only Fork New Processes  example 5 processes == 5 complete independent versions of the application load managed etc. by Passenger. Even for Jruby.

2.) The forked processes created and managed by Passenger are completely resource independent (Memory, I/O...)

3.) For Community version of Passenger and JRuby(JVM) each request is load balanced across "Processes" again example 5 JRuby processes == 5 concurrent request.  So no advantage to using JRuby? There are not any request threads created in the JRuby (Rails or Sinatra) application?

4.) The Enterprise version of Passenger does allow Threading if the underlying language can take advantage. so example 1 Jruby Process == However many concurrent request threads...? and it is at this point that resources such as connection pools would be shared.

The author of the Sequel ORM gem recommended closing newly initialized connections before rackup completes i.e. run MyApplication

So if I configure my JRuby application to use 10 DB connection in the pool I am wasting my time and resources because each Process can only handle one request/response at a time?

I really appreciate your helping me understand this...  I would prefer to not have to deal with this level of detail.  BTW we are considering the Enterprise version.

Best Regards.

[Hongli Lai]

On Fri, Aug 21, 2015 at 3:41 PM, Dale Ackerman <dale...@gmail.com> wrote:
> Hi Thanks for the reply.
>
> My question came about due to information I had gotten regarding the Sequel
> gem It is a ORM like ActiveRecord but the author had stated that Passenger
> held onto some descriptor or handle and some how shared connection pool
> between forked processes?

I think that person is referring to smart spawning. See:
https://www.phusionpassenger.com/library/indepth/ruby/spawn_methods/#unintentional-file-descriptor-sharing

However, this does not apply at all if you are using JRuby, because
smart spawning is not supported when using JRuby. When using JRuby,
Passenger will always use direct spawning.

> 1.) Community version of Passenger will only Fork New Processes example 5
> processes == 5 complete independent versions of the application load managed
> etc. by Passenger. Even for Jruby.

Yes.

> 2.) The forked processes created and managed by Passenger are completely
> resource independent (Memory, I/O...)

Yes.

> 3.) For Community version of Passenger and JRuby(JVM) each request is load
> balanced across "Processes" again example 5 JRuby processes == 5 concurrent
> request.

Yes.

> So no advantage to using JRuby?

One advantage is better Java integration. Another advantage is that
you can leverage all the JRuby performance improvements over MRI (e.g.
the JIT and the garbage collection), with the exception of
multithreading.

> There are not any request threads
> created in the JRuby (Rails or Sinatra) application?

There is only 1 request thread in each JRuby application process.

> 4.) The Enterprise version of Passenger does allow Threading if the
> underlying language can take advantage. so example 1 Jruby Process ==
> However many concurrent request threads...?

That depends on the configuration:
https://www.phusionpassenger.com/library/config/apache/reference/#passengerthreadcount

> and it is at this point that
> resources such as connection pools would be shared.

Yes, but only within a single process.

> The author of the Sequel ORM gem recommended closing newly initialized
> connections before rackup completes i.e. run MyApplication

That advice is exactly like what is documented at
https://www.phusionpassenger.com/library/indepth/ruby/spawn_methods/#unintentional-file-descriptor-sharing.
However, the Sequel author's advice does not apply at all in your
case, because you are using JRuby, so no smart spawning is supported.

> So if I configure my JRuby application to use 10 DB connection in the pool I
> am wasting my time and resources because each Process can only handle one
> request/response at a time?

Yes. It only makes sense to configure the DB pool size if you have
enough threads running.