[jruby-user] Anything wrong with shelling out to jruby?

[John Joseph Bachir]

I have an application where I would like to shell out to jruby. A very simple script that essentially does this a dozen times serially:

begin

  `rails runner scripts/foo.rb`

rescue

  # .. report error

end

I remember hearing about problems invoking jruby from jruby. Some initial basic tests in my dev environment don't show any problems.

My guess at what the problem is: if jruby is configured to take an application's worth of RAM (say 512M), then the small out script will take that and then each individual sub process will take that as well.

I wonder if in practice this won't matter much, because the OS will move most of the outer instance's memory into VM and just keep it there?

Thanks for any insights or pointers to other info about this!

John

[Keith Bennett]

I don’t know of any problems shelling out to JRuby, but it will require the memory and startup time of starting a whole new JVM for each time you do. I would not be comforted by the fact that the OS might save your first JVM’s RAM to disk and restore it later; that would take time. Is speed not important in your case?

If I were you, I’d look at minimum to shelling out once rather than a dozen times. Why do you want to shell out so many times?

Also, your script looks like it shells out only once, no?

- Keith

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[Ted Han]

There's a section about this in the jruby wiki: https://github.com/jruby/jruby/wiki/Improving-startup-time#running-sub-rubies-in-the-same-jvm

[Keith Bennett]

Ted -

That text mentions Nailgun; is Nailgun use still supported/recommended?  I thought no.

Thanks,

Keith

[John Joseph Bachir]

On Tue, Mar 17, 2015 at 9:07 AM, Keith Bennett <keithr...@gmail.com> wrote:

Is speed not important in your case?

It's not important. These are batch jobs that run nightly. This master script runs a few sub scripts, each of which do a few calculations and then queue up many (more efficient and well-structured) jobs in my background job system (sidekiq). Eventually I'd like to make the work in the sub scripts be done in workers as well, but for the time being I'd like to just keep things as-is and bring all the invocations into one place, and then start refactoring.

If I were you, I’d look at minimum to shelling out once rather than a dozen times.  Why do you want to shell out so many times?

As described above, I have many sub scripts which work and I don't want to refactor them in this first phase.

 

Also, your script looks like it shells out only once, no?

That's an example of the code that I'll be doing about a dozen times in the script.

On Tue, Mar 17, 2015 at 9:10 AM, Ted Han <t...@knowtheory.net> wrote:

There's a section about this in the jruby wiki: https://github.com/jruby/jruby/wiki/Improving-startup-time#running-sub-rubies-in-the-same-jvm

“we will attempt to run it in the same JVM using a new JRuby instance”

Great news, thanks! FYI if others are interested, this feature can be set at the commandline[1]:

    JRUBY_OPTS="-J-Djruby.launch.inproc=true" rails runner scripts/foo.rb

Or within your ruby code[2]:

    # scripts/foo.rb

    require 'jruby'

    JRuby.runtime.instance_config.run_ruby_in_process = true

[1] http://developers.mobilesystem7.com/blog/post/speeding-up-rails-on-virtualbox/

[2] https://github.com/jruby/jruby/wiki/FAQs#how-can-i-increase-the-heapmemory-size-when-launching-a-sub-jruby

[John Joseph Bachir]

Some simple tests at the heroku console show that this feature doesn't help performance at all:

> JRuby.runtime.instance_config.run_ruby_in_process = false

=> false

> Benchmark.measure{`ruby -e"puts 'foo'"`}

=>   0.020000   0.010000  14.980000 ( 17.739000)

> JRuby.runtime.instance_config.run_ruby_in_process = true

=> true

> Benchmark.measure{`ruby -e"puts 'foo'"`}

=>   0.010000   0.000000  14.740000 ( 17.536000)

I wonder if the in-process feature isn't being used for some reason? Any way to inspect this in either the inner or outer process?

[John Joseph Bachir]

I made some more observations. I did an experiment similar to my March 17 email, but this time with `ruby -e"sleep 1-"`, and then I observed my process list to see if a new "java" was created and how much memory it took.

with JRuby.runtime.instance_config.run_ruby_in_process set to true or false, a second java instance was always created. The outer Java instance (running pry) took about 180 megs, and the inner about 80 megs.

Is this behavior expected? Let me know if there is more info I can provide.

John

[John Joseph Bachir]

On Sat, Mar 21, 2015 at 1:46 PM, John Joseph Bachir <johnjose...@gmail.com> wrote:

`ruby -e"sleep 1-"`

whoops, i mean  `ruby -e"sleep 10"`

[John Joseph Bachir]

I also tried invoking pry like so:

  JRUBY_OPTS="-J-Djruby.launch.inproc=true" pry

Same behavior.

[Tim Uckun]

Any word from the Jruby maintainers about this?

It seems to me that it could be a huge win for jruby if a new java instance was not launched both for speed and memory utilization.

[Christian MICHON]

Regarding pry debugging, you should use pry-remote instead of pry. Indeed, you may have launched the jruby job as a service without a console attached to it.

--

Christian

[John Joseph Bachir]

On Mon, Mar 23, 2015 at 3:02 AM, Christian MICHON <christia...@gmail.com> wrote:

Regarding pry debugging, you should use pry-remote instead of pry. Indeed, you may have launched the jruby job as a service without a console attached to it.

Good thinking. I did another test, but am still finding a new JVM being created.

I wrote a script, shell-out-test.rb:

puts "starting script and sleeping for 5 seconds"

sleep 5

puts "shelling out for 10 seconds"

`ruby -e 'sleep 10'`

puts "done shelling out, sleeping another 5 seconds"

sleep 5

 

Then opened two terminals side-by-side. In one I ran:

  watch 'ps aux |grep [j]ava'

In the other I ran my script:

  ruby shell-out-test.rb

And I observed two java processes running while shelling out.

[John Joseph Bachir]

Okay, in the IRC channel headius requested I make a ticket for this:

  https://github.com/jruby/jruby/issues/2754

So I suppose it's considered broken for the time being.