The last BIG issue: Start up speed

[Scott]

While I'm sure there we other stuff to focus on (read documentation), I thought it might be time to spend a few cycles on trying to speed up the start up process. This is not as much a Ruboto issue as a JRuby issue that is exacerbated by the more limited hardware of mobile devices. I'm looking for some folks who will be willing to dig into the JRuby code to find area that might be tweaked to improve start up.

Here's what I'm finding: We have two lines of code that seem to consume the bulk of the start up time: 

1) Creating the ScriptingContainer

2) The put statement that sets the global for package name

By far, #2 is the one that consumes the most time. 99% of this time is spent initializing the Ruby instance. Three calls are relevant here:

A) initRubyKernel

B) LoadService.require("jruby")

C) bootstrap

A big chunk of C is caused by JRuby reading encoding (jcodings) information into arrays from binary files. RubyString alone cause the loading of 11 binary files. The easy fix here is to dumping those binary files directly into the Java code (as the integer arrays they will become anyway). I have this working for those 11 files. There are another >200, but many of them seem to be used for regional encoding (i.e., only a few used).

Both A & B are really about finding, loading, and parsing various ruby scripts. I think there is a significant overhead with dealing with all the files in the apk jar. This can be dealt with by unpacking some or most files into the file system (files include the standard library, the jcodings bin files, and the joda timezone files). This will, of course, create additional problems (e.g., when to unpack, time to unack). While we might be able to speed up parsing, I'm wondering if we'd have more luck seeing if we could precompile the ruby code loaded at start up.

We might also want to revisit the heap allocation code, playing with the thread priority of the initializing thread, and more.

Anyway, I'm seeing 30-40% reduction in start up time by removing some files and avoiding the loading of those 11 encoding files needed for RubyString. It's been an interesting experiment, but there is a long way to go. Any thoughts or help would be welcomed.

[Robin2]

Is this going in the direction of a special version of JRuby for use on Android?

is that wise? Who will maintain it? (in addition to maintaining and developing Ruboto). What will happen in future if JRuby changes in a direction away from your solution so that it can no longer be converted?

Will it be available for non-Ruboto use?

On a more positive note, would your changes also reduce the startup time for JRuby generally - so they could be incorporated into the standard JRuby code and maintained by the JRuby team?

Will the usual stream of improvements in Android devices (as with all PC stuff) speed things up sufficiently without any effort on your part? Presumably the delay is much less on an Android 4 tablet with lots of memory than on my 2.3.5 phone?

I know the delay is irritating when I am constantly re-loading an app to test my program. But is it really a problem for end-users? If I start an app there is the expected delay but if I switch to another app and then back to my JRuby app it is still "alive" and there is no startup delay.

Design and test the app in Ruboto and then convert it to Mirah! (Since gems don't seem to work well with Ruboto the conversion should be easy). (ok, I'm gone)

...R

[Scott Moyer]

On Jan 29, 2013 12:49 AM, "Robin2" <robin...@gmail.com> wrote:
>
> Is this going in the direction of a special version of JRuby for use on Android?

No

>
> is that wise? Who will maintain it? (in addition to maintaining and developing Ruboto). What will happen in future if JRuby changes in a direction away from your solution so that it can no longer be converted?
>
> Will it be available for non-Ruboto use?

I think any differences would be in line with what we're already doing (i.e., programmatically reconfiguring jruby-jars to meet out needs).

>
> On a more positive note, would your changes also reduce the startup time for JRuby generally - so they could be incorporated into the standard JRuby code and maintained by the JRuby team?

I hope some changes would be accepted.

>
> Will the usual stream of improvements in Android devices (as with all PC stuff) speed things up sufficiently without any effort on your part? Presumably the delay is much less on an Android 4 tablet with lots of memory than on my 2.3.5 phone?

Yes, but I'm not sure it will be enough or happen fast enough. I believe it is worth making some efforts to: A) understand and B) fix the easiest stuff.

>
> I know the delay is irritating when I am constantly re-loading an app to test my program. But is it really a problem for end-users? If I start an app there is the expected delay but if I switch to another app and then back to my JRuby app it is still "alive" and there is no startup delay.

Not sure. I believe that we are outside of the acceptable range on many types of apps. Also, frustrating for developers means limited adoption.

>
> Design and test the app in Ruboto and then convert it to Mirah! (Since gems don't seem to work well with Ruboto the conversion should be easy). (ok, I'm gone)
>

Always an option, but does it have to be?

[Scott Moyer]

I should be clearer:

- I currently see a 9-12 second start up on IRB (on a 4.1, quad core, 1GB RAM device)

- A regular app will see additional start up time based on includes and requires needed for start up

- While start up time is not a huge issue for some apps,  I'd  like to be closer to 3-5 seconds for apps on modern hardware

Other opinions?

[Uwe Kubosch]

Totally agree. Below 4 seconds should be our goal.

--
Uwe Kubosch
http://ruboto.org/

[Robin2]

On my HTC phone "what Matz hath wrought" takes 28 seconds to appear so 9-12 is lightning fast.

On my simple JRuby on Android system my simple camera app takes 17 seconds to start. Is it possible there is dead wood within Ruboto?

...R

[Scott Moyer]

On Tue, Jan 29, 2013 at 8:22 AM, Robin2 <robin...@gmail.com> wrote:

On my HTC phone "what Matz hath wrought" takes 28 seconds to appear so 9-12 is lightning fast.

Yes, so improving start up on newer hardware should get older hardware some significant improvement.

 

On my simple JRuby on Android system my simple camera app takes 17 seconds to start. Is it possible there is dead wood within Ruboto?

There are definitely different choices that will have significant impact (not the same as dead wood :). Understanding the cost of design decisions is important. For example, I've seen some indication of impact to loading JRuby from a separate  app (RubotoCore). I haven't yet quantified it. 

That just made me think of something: Dalvik creates a cache of optimized code for every app (based on my very limited knowledge of what is happening behind the scene). I wonder if we get that optimized code when we load core dynamically, or do we create a separate optimized version, or do we skip this optimization all together.

Anyway, is your simple app available to take a look at? It is always nice to have things to compare to. For instance, part of the win that SL4A gets is from launching JRuby from the command line. That allows them to specify a custom heap (and maybe stack) size.

There is definitely a cost to loading the ruboto scripts. The impact could be reduced by not loading them, improving the loading/interpreting speed, or maybe precompiling them. 

[Robin2]

On Tuesday, January 29, 2013 5:44:38 PM UTC, Scott wrote:

On Tue, Jan 29, 2013 at 8:22 AM, Robin2 <robin...@gmail.com> wrote:

On my HTC phone "what Matz hath wrought" takes 28 seconds to appear so 9-12 is lightning fast.

Yes, so improving start up on newer hardware should get older hardware some significant improvement.

 

That wasn't really the purpose of my comment - I was trying to say you may already be good enough.

Is it worth worrying about Android 2 systems? 

Anyway, is your simple app available to take a look at? .

 

I think that's still a week or two away. I am trying to write documentation :)

And I haven't been able to get JRuby 1.7.x to work yet - only 1.6.8

...R

[ssteidte]

One of the big advantages of Ruboto over other approaches like Rhodes, that precompile everything, is that the Ruby code does _not_ have to be precompiled. This allows to just copy the code that has been developed and verified on a desktop to just copy over to the device and use it. 
From my (limited) experience the most part of the startup time comes from Jruby, not from loading and interpreting the scripts. Of course this depends on the app that is to be executed. If there is a way to pre-compile and pre-verify JRuby itself (dex option or dalvik cache ?) this should do the trick. IBM's J9 Java engine that is somehow similar to Dalvik uses these technologies to make Java acceptable on small devices. I'd volonteer to help with this, but my knowledge of the intrinsics of Android and Dalvik is very limited. Anyway, if there is something I can do to help speed up the start let me know.

[Uwe Kubosch]

On 2013-02-01, at 20:04, ssteidte <admira...@gmail.com> wrote:

> One of the big advantages of Ruboto over other approaches like Rhodes, that precompile everything, is that the Ruby code does _not_ have to be precompiled. This allows to just copy the code that has been developed and verified on a desktop to just copy over to the device and use it.
> From my (limited) experience the most part of the startup time comes from Jruby, not from loading and interpreting the scripts. Of course this depends on the app that is to be executed. If there is a way to pre-compile and pre-verify JRuby itself (dex option or dalvik cache ?) this should do the trick. IBM's J9 Java engine that is somehow similar to Dalvik uses these technologies to make Java acceptable on small devices. I'd volonteer to help with this, but my knowledge of the intrinsics of Android and Dalvik is very limited. Anyway, if there is something I can do to help speed up the start let me know.

Android is still a new platform, so there are not many who have long experience with it. That mean you are in the lead :)

If you could research possibilities for precompiling Ruby code either to the new IR format or to Java source or to Java byte code, that would be very interesting.

Anything you can find on loading Java code faster would make loading JRuby faster and would benefit Ruboto immensely.

Thanks for participating!

[Scott Moyer]

On Feb 1, 2013 11:04 AM, "ssteidte" <admira...@gmail.com> wrote:
>
> One of the big advantages of Ruboto over other approaches like Rhodes, that precompile everything, is that the Ruby code does _not_ have to be precompiled.

Agreed! I'm only talking about compiling the internal scripts that are static to JRuby (jruby, kernal, etc.) or Ruboto (activity, widget etc.). These are (often) loaded at startup and are not edited by the developer. If we can load them fast, we can remove their impact on overall speed of the app.

[ssteidte]

I agree with you. All the stuff that is bundled in Ruboto core should finally be precompiled if possible.

[Scott Moyer]

Here's the current state of my findings...

I see two relatively easy wins:

1) Prevent rubygems from being loaded at start up.

2) Shift default utf codes from files into Java arrays.

Between these two, I think we can cut start up in half. I had thought that shifting files (stdlib, etc.) from the apk/jar to the file system would be a win, but I don't think it is worth it at this time. The reasons: a) can't yet shift most of the start up files to the file system, and b) there is a large hassle in copying files into the file system. This would be a win in reducing heap allocation, but that is not the problem I'm trying to solve a the moment.

More information on making 1 & 2 happen:

1) JRuby currently loads three things from .rb files at start up: jruby, kernel, and rubygems. Each of these accounts for 2-5 seconds at start up (because  it means parsing and loading 5-10 files. Of the three, rubygems is the obvious one for deferring loading (i.e., making sure that it only gets loaded when needed...like normal ruby). It is fairly easy to turn off, but I need to do additional work to make sure it is available when needed. This does save 4-5 seconds at start up.

2) JCodings is the project that handles character encoding formats in JRuby. There are about 100 files that contain the codings. Of these, ~12 get loaded at start up when the RubyString class is initialized. The files are read into a Java structure. We can save 2-4 seconds by shifting this process directly into Java structures. The challenge here is that this will likely take changes to the JCoding project (i.e., we need others to buy in to this approach). JCoding comes into JRuby as a jar. We could strip out and replace a single class, but this is not a long-term solution.

Other improvements are possible, but I suggest that we get these into Ruboto before pursuing more.

[Uwe Kubosch]

On 2013-03-12, at 16:19, Scott Moyer <rscot...@gmail.com> wrote:

> Here's the current state of my findings...
> I see two relatively easy wins:
> 1) Prevent rubygems from being loaded at start up.
> 2) Shift default utf codes from files into Java arrays.
>
> Between these two, I think we can cut start up in half. I had thought that shifting files (stdlib, etc.) from the apk/jar to the file system would be a win, but I don't think it is worth it at this time. The reasons: a) can't yet shift most of the start up files to the file system, and b) there is a large hassle in copying files into the file system. This would be a win in reducing heap allocation, but that is not the problem I'm trying to solve a the moment.

Really, really cool!

> More information on making 1 & 2 happen:
>
> 1) JRuby currently loads three things from .rb files at start up: jruby, kernel, and rubygems. Each of these accounts for 2-5 seconds at start up (because it means parsing and loading 5-10 files. Of the three, rubygems is the obvious one for deferring loading (i.e., making sure that it only gets loaded when needed...like normal ruby). It is fairly easy to turn off, but I need to do additional work to make sure it is available when needed. This does save 4-5 seconds at start up.

For most setups, I _think_ that RubyGems should never be used, and therefore should never be loaded. I am really looking forward to getting this into Ruboto. I would expect this to benefit straight JRuby as well, so I'll help sell this into the main project.

> 2) JCodings is the project that handles character encoding formats in JRuby. There are about 100 files that contain the codings. Of these, ~12 get loaded at start up when the RubyString class is initialized. The files are read into a Java structure. We can save 2-4 seconds by shifting this process directly into Java structures. The challenge here is that this will likely take changes to the JCoding project (i.e., we need others to buy in to this approach). JCoding comes into JRuby as a jar. We could strip out and replace a single class, but this is not a long-term solution.

Adding a step in the build process that reads the files and generates classes should be an easy sell to the JRuby project. Similar build steps already exist. We need to make sure it is easily maintainable, and maybe promise to maintain it :)

> Other improvements are possible, but I suggest that we get these into Ruboto before pursuing more.

YES! One step at a time. Halving startup time would bring it below the 4 seconds goal on my phone.

[Ashiq A.]

Whoa, how exactly did you profile and figure this out? It sounds like you pinpointed two pretty specific targets.

Or is it an educated guess? :)

--Ashiq

[donV]

Are your changes available anywhere for testing, Scott?

[Scott Moyer]

On Wed, Mar 13, 2013 at 11:58 AM, donV <u...@kubosch.no> wrote:

Are your changes available anywhere for testing, Scott?

I've got #1 worked out and it will be easy to roll into our code base (JRubyAdapter). 

The quick way for me to get #2 out for people to try may be a gist with the change to JCodings.

I'll try to get something out today.

[Scott Moyer]

On Tue, Mar 12, 2013 at 5:16 PM, Ashiq A. <alibha...@gmail.com> wrote:

Whoa, how exactly did you profile and figure this out? It sounds like you pinpointed two pretty specific targets.

Or is it an educated guess? :)

I first used android.os.Debug's method tracing capability. That revealed the issue with JCodings. I could couldn't get a good handle on other start up issues (mostly because a lot is happening through loading .rb files), so I ended up doing a large number of useless experiments. Later, when I re-enabled JRuby LoadService timings, it became clear that initializing rubygems was an issue. I was already curious why "require 'rubygems'" was unnecessary in JRuby, so I finally got a chance to follow up on that issue. 

So, there are two helpful tools that I used: Android method tracing and JRuby load service timings. The rest was trial and error.

Good education in JRuby internals or better guessing would have yielded success much faster :)

[Uwe Kubosch]

Awesome! Track against Issue #75 https://github.com/ruboto/ruboto/issues/75

[Scott]

I suspect the next most reasonable win will be to precompile the ruby code loaded at start up (jruby, kernel, ruboto). I'm interested in getting some thoughts from others on how to make this happen. I'm using Charlie's post from alomost three years ago as a starting point:

https://groups.google.com/forum/?fromgroups=#!searchin/ruboto/authorname:charles/ruboto/I6YxCSiKmvw/BAUDlfuiZVsJ

Charlie was compiling the stdlib code (on much older versions of JRuby and Android). Ultimately, the overhead of that many class (one for every method) was too much to pursue at the time. 

I'm hoping we can compile much less code. The command line call that Charlie suggests works fine:

jruby --fast -S jrubyc --sha1 --handles -t <destination> <source>

Generating direct handles for jruby/jruby.rb

Generating direct handles for jruby/kernel20.rb

Generating direct handles for jruby/kernel19.rb

Generating direct handles for jruby/java.rb

Generating direct handles for jruby/path_helper.rb

Generating direct handles for jruby/kernel18.rb

Generating direct handles for jruby/kernel.rb

Generating direct handles for jruby/commands.rb

Generating direct handles for jruby/kernel18/enumerator.rb

Generating direct handles for jruby/java/core_ext.rb

Generating direct handles for jruby/java/java_utilities.rb

Generating direct handles for jruby/java/java_package_module_template.rb

Generating direct handles for jruby/java/java_module.rb

Generating direct handles for jruby/java/java_ext.rb

Generating direct handles for jruby/java/java_ext/org.jruby.ast.rb

Generating direct handles for jruby/java/java_ext/java.util.regex.rb

Generating direct handles for jruby/java/java_ext/java.io.rb

Generating direct handles for jruby/java/java_ext/java.util.rb

Generating direct handles for jruby/java/java_ext/java.lang.rb

Generating direct handles for jruby/java/java_ext/java.net.rb

Generating direct handles for jruby/java/core_ext/object.rb

Generating direct handles for jruby/java/core_ext/module.rb

Generating direct handles for jruby/java/core_ext/kernel.rb

Generating direct handles for jruby/kernel/signal.rb

Generating direct handles for jruby/kernel/jruby/generator.rb

Generating direct handles for jruby/kernel/jruby/process_manager.rb

Generating direct handles for jruby/kernel20/enumerable.rb

Generating direct handles for jruby/kernel19/process.rb

Generating direct handles for jruby/kernel19/gc.rb

Generating direct handles for jruby/kernel19/enumerable.rb

Generating direct handles for jruby/kernel19/rubygems.rb

Generating direct handles for jruby/kernel19/io.rb

Generating direct handles for jruby/kernel19/proc.rb

Generating direct handles for jruby/kernel19/thread.rb

Generating direct handles for jruby/kernel19/enumerator.rb

Generating direct handles for jruby/kernel19/time.rb

Generating direct handles for jruby/kernel19/kernel.rb

Generating direct handles for jruby/kernel19/jruby/type.rb

Generating direct handles for jruby/kernel19/jruby/process_util.rb

Generating direct handles for jruby/kernel19/encoding/converter.rb

Generating direct handles for jruby/bundler/startup.rb

Looks good until the compile which throws a bunch of warnings like:

       [dx] trouble processing:

       [dx] class name (ruby/jit/FILE_FC601E649826D6362DF26D05D515910F4DCD60D2) does not match path (jruby/kernel19.class)

       [dx] ...while parsing jruby/kernel19.class

       [dx] ...while processing jruby/kernel19.class

The compile succeeds, but the resulting fails to install.

I/PackageManager(  411): Running dexopt on: org.ruboto.irb

D/dalvikvm(  411): WAIT_FOR_CONCURRENT_GC blocked 0ms

E/dalvikvm(29653): Invalid type descriptor: 'Lruby$jit$FILE_07DA92714FEE7E964C4ED4BDC4626319676D79C7$method__1$RUBY$\=\{\};'

E/dalvikvm(29653): Trouble with item 9279 @ offset 0x40004

E/dalvikvm(29653): Cross-item verify of section type 0002 failed

E/dalvikvm(29653): ERROR: Byte swap + verify failed

E/dalvikvm(29653): Optimization failed

W/installd(  123): DexInv: --- END '/data/app/org.ruboto.irb-2.apk' --- status=0xff00, process failed

E/installd(  123): dexopt failed on '/data/dalvik-cache/data@app@org.ruboto...@classes.dex' res = 65280

W/PackageManager(  411): Package couldn't be installed in /data/app/org.ruboto.irb-2.apk

I'd love to hear any ideas people have.

[Uwe Kubosch]

I have not tried AOT compilation for JRuby at all, but it has been on the list for a long time. I would think we could gain a lot from it. Maybe contact Charlie directly? I am sure he has many thoughts on the subject.

> --
> You received this message because you are subscribed to the Google Groups "Ruboto (JRuby on Android)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ruboto+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
Uwe Kubosch
u...@kubosch.no
http://kubosch.no/