Why not JRuby?

[Dmytrii Nagirniak]

Hi,

I wonder what arguments do you have against JRuby and what experience have you had with it?

My biggest concerns at this moment are:

• Warmup-time (just too slow for any kind of TDD);
• Stacktraces (yes, it's worth mentioning! :) )
• Gems compatibility;
• Just plain fear of Java :)

It sounds like JRuby project itself is pretty active just looking at @headius.

I am really keen to know what others have to say in this regard.

The reason for this question is basically neo4j database. I can't see all its benefits with REST API.

This leaves me with native Java bindings (and this brings me to JRuby).

Cheers,

Dmytrii

http://www.ApproachE.com

[Michael Pearson]

On Wed, Dec 7, 2011 at 12:54 PM, Dmytrii Nagirniak <dna...@gmail.com> wrote:

Hi,

I wonder what arguments do you have against JRuby and what experience have you had with it?

Short version: it will give you pain, but it's sometimes worth it.

 

My biggest concerns at this moment are:

• Warmup-time (just too slow for any kind of TDD);
• Stacktraces (yes, it's worth mentioning! :) )
• Gems compatibility;

You've covered it pretty effectively. 

Used JRuby at my last job. There are workarounds for the warmup time on tests, but we never _quite_ got them working.

We also found that our hour-long cuke run would get in to some weird GC fail-loop where it would OOM no matter how much memory you gave the JVM.

Also also we had weirdness with the JDBC wrapper & ActiveRecord & background tasks. 

We had the problem where when investigating errors we always had to wonder "is this a common problem, or is this a JRuby incompatability?"

There are good reasons to use JRuby over MRI (performance & Java integration). There are too many downsides to make it my 'default' stack.

The reason for this question is basically neo4j database. I can't see all its benefits with REST API.

This leaves me with native Java bindings (and this brings me to JRuby).

See above. If you're married to that stack, that's a great reason to use JRuby.

--
Michael Pearson

[Tim Uckun]

Torquebox http://torquebox.org/ looks like a pretty nice project and
they just put out the 2.0 beta.

As for startup I wonder if nailgun would be of help
http://railstech.com/2010/08/how-to-improve-performance-of-java-using-nailgun-server/

Hopefully I'll get a chance to play with them one day.

[Ben Hoskings]

On 07/12/2011, at 12:54 PM, Dmytrii Nagirniak wrote:

> My biggest concerns at this moment are:
> Warmup-time (just too slow for any kind of TDD);

JRuby isn't designed for TDD or to run in development. It's intended to be an industrial grade ruby for production.

(For what it's worth, I think the startup time of a rails stack on C ruby (charitably, 5 seconds) is still far too slow to effectively test-drive anything.)

- Ben

[Dmytrii Nagirniak]

On 07/12/2011, at 1:51 PM, Ben Hoskings wrote:

On 07/12/2011, at 12:54 PM, Dmytrii Nagirniak wrote:

My biggest concerns at this moment are:

Warmup-time (just too slow for any kind of TDD);

JRuby isn't designed for TDD or to run in development. It's intended to be an industrial grade ruby for production.

I suppose you are not saying that you should run MRI in development and JRuby in production :)

Even if you do, it is just not possible for me due to native Java bindings (neo4j).

(For what it's worth, I think the startup time of a rails stack on C ruby (charitably, 5 seconds) is still far too slow to effectively test-drive anything.)

It is even slower with JRuby (it adds at least 11s to that).

https://gist.github.com/1423288

[Dmytrii Nagirniak]

On 07/12/2011, at 1:28 PM, Tim Uckun wrote:

> Torquebox http://torquebox.org/ looks like a pretty nice project and
> they just put out the 2.0 beta.

JBoss thing scared me away enough :)
But thank. I'll keep it in mind.

> As for startup I wonder if nailgun would be of help
> http://railstech.com/2010/08/how-to-improve-performance-of-java-using-nailgun-server/

Yeah. It may help but not sure whether it's worth the hassle. It improves the JVM startup time a bit, but the warmup time is still too slow.

[Dmytrii Nagirniak]

On 07/12/2011, at 1:09 PM, Michael Pearson wrote:

On Wed, Dec 7, 2011 at 12:54 PM, Dmytrii Nagirniak <dna...@gmail.com> wrote:

Short version: it will give you pain, but it's sometimes worth it.

It feels like this pain is around every corner. Something very small, but there is always "something" :)

Used JRuby at my last job. There are workarounds for the warmup time on tests, but we never _quite_ got them working.

Got the same feeling. I tried could of those things, but didn't feel it's worth the effort.

We also found that our hour-long cuke run would get in to some weird GC fail-loop where it would OOM no matter how much memory you gave the JVM.

Also also we had weirdness with the JDBC wrapper & ActiveRecord & background tasks. 

Can you give some more details on this if you still remember those? Just to be aware of.

Do you think it could have changed since then?

We had the problem where when investigating errors we always had to wonder "is this a common problem, or is this a JRuby incompatibility?"

So far I Google kind of helps with this. I either search for the error or search for the error with JRuby in it.

But I definitely see what you mean.

There are good reasons to use JRuby over MRI (performance & Java integration). There are too many downsides to make it my 'default' stack.

Right. Makes a lot o sense.

I tried to stay within MRI, but the REST API just isn't good enough. So I quit on it.

Hopefully I won't regret this decision in 2 years time :)

[Michael Pearson]

On Wed, Dec 7, 2011 at 2:30 PM, Dmytrii Nagirniak <dna...@gmail.com> wrote:

We also found that our hour-long cuke run would get in to some weird GC fail-loop where it would OOM no matter how much memory you gave the JVM.

Also also we had weirdness with the JDBC wrapper & ActiveRecord & background tasks. 

Can you give some more details on this if you still remember those? Just to be aware of.
Do you think it could have changed since then?

They were specific to the JRuby / AR version we were using. Unless you're working in a legacy stack (JRuby 1.4, Rails 2.3.5) you won't hit them. 

The point was more that strange things can happen that don't happen to the majority of the community. Ruby/Rails' popularity means that issues are identified and fixed or worked around often. You will be reducing your available knowledge pool if you hit a JRuby specific issue, just as if you decided to deploy your stack on a less common operating system.

However, JRuby is pretty mature and has a solid user base, certainly compared to JVM ports of other languages like Python.

--
Michael Pearson

[Chris Berkhout]

> I tried to stay within MRI, but the REST API just isn't good enough. So I
> quit on it.

Not good enough in terms of features?
I guess it's more Java, but extending the API may be an option.

Cheers,
Chris

[Dmytrii Nagirniak]

Yes, features.
There are couple of gems that help with it: neology (not maintained, failing specs), architect4j (ActiveRecord-like, new project, some failing specs).

I even thought to come up with something better. But then I realised it's not that simple.
Those abstractions are leaky and the best bet is to stay as close to the HTTP request/response as you can.

While, with native bindings, you can traverse the graph natively from ruby. And it's is pretty efficient.
With REST you'd have to run traversal queries for that.
It basically means that anything more or less complicated involves a number of HTTP req/resp and wrapping all that with your models.

[Mike Williams]

On 07/12/2011, at 12:54 PM, Dmytrii Nagirniak wrote:

I wonder what arguments do you have against JRuby and what experience have you had with it?

Back when Lonely Planet was starting to use Ruby/Rails, we (well, I) choose to use JRuby, rather than MRI, in production.  Much like your situation, the reason was ease of data-access - we were required to use MS SQL Server, and given the state of the relevant drivers, it was much easier to do so using JDBC.

We seldom used JRuby in development environments, though, mainly because of the VM startup time.  This led to a situation where development and most of the automated testing ran on a slightly different stacks.  Most of the time that worked out okay, but when there were compatibility problems, they were a real pig to track down and fix.  To be fair, many of these problems were caused by differing RDBMS engines (MS-SQL vs SQLite), rather than anything specific to JRuby.

The situation reached breaking point when we started doing a lot of work with XML and HTML, and became heavily dependent on Nokogiri, which produced a constant stream of issues on JRuby.  In the end, LP yanked out MS-SQL in favour of PostgreSQL, and reverted to MRI.

JRuby is a fantastic project, and I have a lot of respect for @headius, @nickseiger, and the other guys.  In retrospect, I wish I'd pushed harder, and earlier, to use a different (and non-proprietary) database, and stuck with MRI.

As Michael P said, once you wander off the conventional path (Rails on MRI, with MySQL or PostgreSQL behind) you're more likely to hit issues that other people haven't ... or at least issues that other people haven't already solved.  Having said that, you're halfway there already, by choosing to use Neo4J, so why not go all out? :-)

-- 

cheers, 

Mike Williams

[Dmytrii Nagirniak]

On 07/12/2011, at 4:40 PM, Mike Williams wrote:

> We seldom used JRuby in development environments, though, mainly because of the VM startup time. This led to a situation where development and most of the automated testing ran on a slightly different stacks. Most of the time that worked out okay, but when there were compatibility problems, they were a real pig to track down and fix. To be fair, many of these problems were caused by differing RDBMS engines (MS-SQL vs SQLite), rather than anything specific to JRuby.

Every time I hear something like this (different environments) I remember a DBA I used to work with.
He said that you should always run the same environment everywhere.

And I agree with him more and more.
That's fine to use different DBs for very simple, basic stuff.
But the minute you want to write SQL, it's better to stick with the target DB.

Anyway, thanks for sharing this. Much appreciated!

> The situation reached breaking point when we started doing a lot of work with XML and HTML, and became heavily dependent on Nokogiri, which produced a constant stream of issues on JRuby. In the end, LP yanked out MS-SQL in favour of PostgreSQL, and reverted to MRI.

So again gems compatibility...
Hmm. Do you know how much it has changed since?

> In retrospect, I wish I'd pushed harder, and earlier, to use a different (and non-proprietary) database, and stuck with MRI.

Good point. Would you consider neo4j proprietary though?

> As Michael P said, once you wander off the conventional path (Rails on MRI, with MySQL or PostgreSQL behind) you're more likely to hit issues that other people haven't ... or at least issues that other people haven't already solved. Having said that, you're halfway there already, by choosing to use Neo4J, so why not go all out? :-)

That's true. I'm definitely going to have some issues there.
One thing that keeps me warm is that neo4j community seems to be pretty active.
And neo4j devs respond amazingly quickly. So fingers crossed :)

[Mike Williams]

On 07/12/2011, at 6:01 PM, Dmytrii Nagirniak wrote:

> Every time I hear something like this (different environments) I remember a DBA I used to work with. He said that you should always run the same environment everywhere.

Indeed. On the other hand, I think the benefits we got from using a lighter-weight stack for development outweighed the disadvantages.

>> The situation reached breaking point when we started doing a lot of work with XML and HTML, and became heavily dependent on Nokogiri, which produced a constant stream of issues on JRuby. In the end, LP yanked out MS-SQL in favour of PostgreSQL, and reverted to MRI.
>
> So again gems compatibility...
> Hmm. Do you know how much it has changed since?

I think it depends entirely on the gems you're using. Anything pure-Ruby is unlikely to cause issues - it's the stuff that requires native bindings that killed us. In some cases, the gem we wanted simply wasn't available for JRuby. This may have improved since, with the work the JRuby guys did last year to support MRI's C-extension API.

In the case of Nokogiri, using it with JRuby required use of FFI, which caused installation problems; sorry, I can't recall the details. The Nokogiri crowd have now dumped the FFI bindings, and implemented a "pure Java" backend. I played with the early releases of that work, and hit too many problems. I'm sure things have improved, but given that it sits on a completely different foundation (Xerces, Xalan, etc, rather than libxml) I very much doubt that it's 100% compatible across platforms.

>> In retrospect, I wish I'd pushed harder, and earlier, to use a different (and non-proprietary) database, and stuck with MRI.

> Good point. Would you consider neo4j proprietary though?

Not at all. If you can easily run it (for free) in your development environment, then that's good enough for me.

--
cheers,
Mike Williams

[Dmytrii Nagirniak]

On 07/12/2011, at 9:13 PM, Mike Williams wrote:

On 07/12/2011, at 6:01 PM, Dmytrii Nagirniak wrote:

Every time I hear something like this (different environments) I remember a DBA I used to work with. He said that you should always run the same environment everywhere.

Indeed.  On the other hand, I think the benefits we got from using a lighter-weight stack for development outweighed the disadvantages.

I definitely feel the benefits of lightweight dev stack (I myself use sqlite during dev).

But I when you need to do a little bit more complex things (geospatial, DB-driven full-text search etc) then it just doesn't work well.

You either have to effectively maintain both dev and prod environments.

But I agree that something as RDBMS can be relatively easy abstracted away.

The situation reached breaking point when we started doing a lot of work with XML and HTML, and became heavily dependent on Nokogiri, which produced a constant stream of issues on JRuby.  In the end, LP yanked out MS-SQL in favour of PostgreSQL, and reverted to MRI.

So again gems compatibility...

Hmm. Do you know how much it has changed since?

I think it depends entirely on the gems you're using.  Anything pure-Ruby is unlikely to cause issues - it's the stuff that requires native bindings that killed us.  In some cases, the gem we wanted simply wasn't available for JRuby.  This may have improved since, with the work the JRuby guys did last year to support MRI's C-extension API.

Thanks a lot for all the details. Appreciate it a lot.

BTW, another JRuby issue for those interested :) (90sec video) http://www.youtube.com/watch?v=z-WWGnsUXf0

[James Healy]

I use MRI in dev and production, but the concurrency options in JRuby
make me curious enough to give it a go every few months.

Each time I try to get MRI 1.9 compatible code running on JRuby I hit
issues that discourage me enough to give up for another few months.

I tried last week and hit two issues [1] and [2]. I tried a few months
ago and hit [3]. It feels like their 1.9 support is still not
production ready.

You're right to mention stacktraces as an issue too. When things go
wrong you often get a 400 line java stack trace with no relevance to
your ruby code. It's traumatising.

All that said, @headius is super responsive so hopefully things will
continue to improve.

James

[1] https://twitter.com/#!/jim_healy/status/143248520256831490
[2] http://jira.codehaus.org/browse/JRUBY-6170.
[3] http://jira.codehaus.org/browse/JRUBY-5967

> --
> You received this message because you are subscribed to the Google Groups
> "Ruby or Rails Oceania" group.
> To post to this group, send email to rails-...@googlegroups.com.
> To unsubscribe from this group, send email to
> rails-oceani...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/rails-oceania?hl=en.

[Michael Pearson]

On Thu, Dec 8, 2011 at 9:51 AM, Dmytrii Nagirniak <dna...@gmail.com> wrote:

On 07/12/2011, at 9:13 PM, Mike Williams wrote:

On 07/12/2011, at 6:01 PM, Dmytrii Nagirniak wrote:

Every time I hear something like this (different environments) I remember a DBA I used to work with. He said that you should always run the same environment everywhere.

Indeed.  On the other hand, I think the benefits we got from using a lighter-weight stack for development outweighed the disadvantages.

I definitely feel the benefits of lightweight dev stack (I myself use sqlite during dev).

But I when you need to do a little bit more complex things (geospatial, DB-driven full-text search etc) then it just doesn't work well.

You either have to effectively maintain both dev and prod environments.

But I agree that something as RDBMS can be relatively easy abstracted away.

 <trollhat>

You CAN, but you probably don't want to, as you'll start to have lots of fun with heisenbugs.

One fun one I had a while ago was sqlite and mysql doing completely different things WRT nested transactions. Spent a while wondering why tests were failing on CI (mysql) but passing on the devs machines (sqlite, usually).

These days I have a policy of software that runs in prod runs in dev and test. Use unicorn/nginx in production? Don't run your dev environment as passenger/apache.

There's certain cases where that can be unreasonable (eg, JRuby warmup time) but for me that's a reason to _not_ use JRuby more than it is to use entirely different interpreters between environments. I have a massive rant forming in my head regarding why your development environment is just as important as your production one, and why you should avoid software licenses that restrict your developers ability to use production software like the plague.

I don't go as far as Ted Dzuiba and say that people shouldn't develop on Macs because you don't ship on Macs, but I will say that developing with JRuby if you're shipping JRuby will prevent a lot of "What? I can't do X in JRuby?".

--
Michael Pearson

[Andrew Grimm]

I occassionally run my work project on other implementations of Ruby
for the fun of it. Hopefully it'll lead to the alternative
implementations being better, and maybe the bug reports will help
improve RubySpec.

About half of my bug reports for Rubinius are the result of running my
work project against it, and the other half are from the Small Eigen
Collider. https://github.com/rubinius/rubinius/issues/created_by/agrimm?sort=created&direction=desc&state=closed&page=1
https://github.com/agrimm/small_eigen_collider/wiki/Bugs-found

If finding and reporting bugs is easy, and workarounds are
time-consuming and waiting for bug fixes takes a while, then it may
make sense to run tests on Ruby implementations you aren't currently
using.

Andrew

[Dmytrii Nagirniak]

On 08/12/2011, at 2:34 PM, Andrew Grimm wrote:

About half of my bug reports for Rubinius are the result of running my
work project against it, and the other half are from the Small Eigen
Collider. https://github.com/rubinius/rubinius/issues/created_by/agrimm?sort=created&direction=desc&state=closed&page=1
https://github.com/agrimm/small_eigen_collider/wiki/Bugs-found

Isn't this one awesome? Eh? :)

https://github.com/rubinius/rubinius/issues/658

[Michael Pearson]

Not bad eh.

https://gist.github.com/1449509 

 -- 

Michael Pearson

[Xavier Shay]

Sorry this thread is kind of old but I wanted to weigh in, hope it's not
too late to be relevant.

We started embedding neo4j inside JRuby but quickly switched to the
stand-a-lone server. Massive stability and corruption issues with the
embedded one the moment we sent even a trifling production load at it.
This was on 1.4 and 1.5rc1, so perhaps things are a little better now.

Embedded is useful for once off batch loads to get you started though.
Protip for this: use the java classes directly, don't try to use the
neo4j ruby gem abstraction (use it to give you access to the java
classes though: require it, just don't use any of its ruby classes.)
This way you can copy all the examples from the manual and not have to
worry about Weird Stuff.

If you have complicated traversals and gremlin doesn't work for you,
write a java plugin. (Sorry.)

Neology works fine for us even though it's a little cruft. It's a
trivial wrapper on top of HTTP so not much to go wrong.

In general, develop on MRI, deploy to JRuby, run CI over both and you
won't go too far wrong after some initial teething problems. I gave a
talk at the JRuby meetup here last week, it was recorded so hopefully
will be published soon.

Xav

[Dmytrii Nagirniak]

On 23/01/2012, at 4:27 PM, Xavier Shay wrote:

Sorry this thread is kind of old but I wanted to weigh in, hope it's not too late to be relevant.

It's awesome to see your reply here. I wish you'd pop up earlier as it looks like I went through similar pain points :)

We started embedding neo4j inside JRuby but quickly switched to the stand-a-lone server. Massive stability and corruption issues with the embedded one the moment we sent even a trifling production load at it. This was on 1.4 and 1.5rc1, so perhaps things are a little better now.

There are still weird issues sometimes popping up.

Embedded is useful for once off batch loads to get you started though. Protip for this: use the java classes directly, don't try to use the neo4j ruby gem abstraction (use it to give you access to the java classes though: require it, just don't use any of its ruby classes.) This way you can copy all the examples from the manual and not have to worry about Weird Stuff.

Ohh, man. I spent a LOT of time submitting PRs to neo4j.rb, got commit access to the repo an publishing the gem.

But last Friday just gave up on the whole neo4j+JRuby idea.

Great in theory, too much pain in practice. And I didn't even get to production.

Neology works fine for us even though it's a little cruft. It's a trivial wrapper on top of HTTP so not much to go wrong.

I wonder how do you wrap the models/services/logic with it.

It's a bit infortunate to lose all the goodness of Rails with ActiveRecord/DataMapper.

Neo4j.rb does the job there.

I asked about it it http://stackoverflow.com/questions/8335136/which-ruby-rest-api-client-for-neo4j

Then I started my own wrapper (https://github.com/dnagir/morpheus) but then drifted to much from what I thought to do and gave it up.

(Then also gave on on CanCan with embedded neo4j.rb in favour of my own https://github.com/dnagir/allowy).

In general, develop on MRI, deploy to JRuby, run CI over both and you won't go too far wrong after some initial teething problems. I gave a talk at the JRuby meetup here last week, it was recorded so hopefully will be published soon.

I feel like I'd rather deploy on MRI too. Don't have huge traffic and stuff, so "enterprise grade" server isn't a requirement.

I have been waiting for the recording since EngineYard first announce it :) So hope it will be published indeed.

[Xavier Shay]

On 23/01/12 12:10 AM, Dmytrii Nagirniak wrote:
>> Neology works fine for us even though it's a little cruft. It's a
>> trivial wrapper on top of HTTP so not much to go wrong.
>
> I wonder how do you wrap the models/services/logic with it.
> It's a bit infortunate to lose all the goodness of Rails with
> ActiveRecord/DataMapper.
> Neo4j.rb does the job there.

Yeah, we're not doing any of that sort of stuff. Pretty simple ETL typle
stuff.