fun with memory with origin/jvm-support

[Nicholas Clark]

So I can build Rakudo for the JVM no problem on a rather beefy server.
When I tried it on a desktop with 2GB it fails:

$ java -Xbootclasspath/a:.:/home/nick/Perl/rakudo/nqp/install/nqp-runtime.jar:/home/nick/Perl/rakudo/nqp/install/asm-4.1.jar:/home/nick/Perl/rakudo/nqp/install/jline-1.0.jar:rakudo-runtime.jar -cp /home/nick/Perl/rakudo/nqp/install perl6 --setting=NULL --optimize=3 --target=classfile --stagestats --output=CORE.setting.class src/gen/CORE.setting
Stage start : 0.000
Stage parse : 77.657
Stage syntaxcheck: 0.000
Stage ast : 0.000
Stage optimize : 4.934
Stage jast : 37.581
java.lang.OutOfMemoryError: Java heap space
in dump
in <anon>
in dump
in classfile
in <anon>
in compile
in eval
in evalfiles
in command_eval
in command_eval
in command_line
in MAIN
in <anon>
in <anon>


A bit of Googling reveals that the heap space can be altered with -Xmx, and
it seems that I can get the current value like this:

$ java -XshowSettings
VM settings:
Max. Heap Size (Estimated): 592.00M
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM

[snip lots more exciting information]


So, boosting it to a gig works:

$ java -Xmx1024m -Xbootclasspath/a:.:/home/nick/Perl/rakudo/nqp/install/nqp-runtime.jar:/home/nick/Perl/rakudo/nqp/install/asm-4.1.jar:/home/nick/Perl/rakudo/nqp/install/jline-1.0.jar:rakudo-runtime.jar -cp /home/nick/Perl/rakudo/nqp/install perl6 --setting=NULL --optimize=3 --target=classfile --stagestats --output=CORE.setting.class src/gen/CORE.setting
Stage start : 0.000
Stage parse : 78.145
Stage syntaxcheck: 0.001
Stage ast : 0.001
Stage optimize : 6.578
Stage jast : 30.003
Stage classfile : 14.435


(and I can get to "Hello world". Although I admit I haven't figured out
command line arguments yet)

Anyway, the most interesting thing was actually the suggestion in this
answer on stack overflow:

http://stackoverflow.com/questions/37335/how-to-deal-with-java-lang-outofmemoryerror-java-heap-space-error-64mb-heap/186390#186390

Yes, with -Xmx you can configure more memory for you JVM. To be
sure that you don't leak or waste memory. Take a heap dump and use
the Eclipse Memory Analyzer to analyze your memory consumption.


Does that sound familiar enough to someone to be tempting?

Nicholas Clark

[Gerd Pokorra]

Hello,

how did you fix the build of NQP for the JVM?

I have the same problem that you reported in an other email.

> perl ConfigureJVM.pl
> make
...
perl tools/build/gen-cat.pl jvm src/vm/jvm/QAST/Compiler.nqp >
src/stage1/gen/QAST.nqp
java -cp src/vm/jvm/stage0:nqp-runtime.jar:3rdparty/asm/asm-4.1.jar nqp
--bootstrap --module-path=src/stage1 --setting-path=src/stage1 \
--setting=NQPCORE --no-regex-lib --target=classfile \
--output=src/stage1/QAST.class src/stage1/gen/QAST.nqp
java.lang.RuntimeException: java.lang.RuntimeException: Method code too
large!

in <anon>
in compile
in eval
in evalfiles
in command_eval

in command_line
in MAIN
in <anon>

in <anon>make: *** [src/stage1/QAST.class] Fehler 1
[gz016@vdesk1 nqp.new.git]$ java -version
java version "1.7.0_19"
OpenJDK Runtime Environment (fedora-2.3.9.3.fc17-x86_64)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
[gz016@vdesk1 nqp.new.git]$ free
total used free shared buffers
cached
Mem: 32946108 31436504 1509604 0 608828
28459716
-/+ buffers/cache: 2367960 30578148
Swap: 32767996 22960 32745036
[gz016@vdesk1 nqp.new.git]$

It is a rather beefy server. Should I try it with another Java?

Gerd Pokorra

[Nicholas Clark]

On Sat, May 18, 2013 at 05:17:58PM +0200, Gerd Pokorra wrote:
> Hello,
>
> how did you fix the build of NQP for the JVM?
>
> I have the same problem that you reported in an other email.

As I said in the e-mail:

> Am Freitag, den 17.05.2013, 08:28 +0100 schrieb Nicholas Clark:
> > So I can build Rakudo for the JVM no problem on a rather beefy server.
> > When I tried it on a desktop with 2GB it fails:
> >
> > $ java -Xbootclasspath/a:.:/home/nick/Perl/rakudo/nqp/install/nqp-runtime.jar:/home/nick/Perl/rakudo/nqp/install/asm-4.1.jar:/home/nick/Perl/rakudo/nqp/install/jline-1.0.jar:rakudo-runtime.jar -cp /home/nick/Perl/rakudo/nqp/install perl6 --setting=NULL --optimize=3 --target=classfile --stagestats --output=CORE.setting.class src/gen/CORE.setting

> > A bit of Googling reveals that the heap space can be altered with -Xmx, and
> > it seems that I can get the current value like this:
> >
> > $ java -XshowSettings
> > VM settings:
> > Max. Heap Size (Estimated): 592.00M
> > Ergonomics Machine Class: server
> > Using VM: OpenJDK 64-Bit Server VM
> >
> > [snip lots more exciting information]
> >
> >
> > So, boosting it to a gig works:
> >
> > $ java -Xmx1024m -Xbootclasspath/a:.:/home/nick/Perl/rakudo/nqp/install/nqp-runtime.jar:/home/nick/Perl/rakudo/nqp/install/asm-4.1.jar:/home/nick/Perl/rakudo/nqp/install/jline-1.0.jar:rakudo-runtime.jar -cp /home/nick/Perl/rakudo/nqp/install perl6 --setting=NULL --optimize=3 --target=classfile --stagestats --output=CORE.setting.class src/gen/CORE.setting

^^^^^^^^^

Manually re-running that command that failed, adding the -Xmx1024m flag
to set the heap size to 1024 megabytes.


I guess I could have edited the Makefile (or even the thing that generates
the Makefile), but it was easiest to re-run that command by hand, then
re-run make, which carried on from the next step needed.

I don't know if -Xmx1024m is a sensible thing to commit. Particularly as it's
probably the wrong number for a 32 bit system.

Nicholas Clark

[Gerd Pokorra]

Hello,

I get the same error with the -Xmx1024m flag.

> java -Xmx1024m -cp

src/vm/jvm/stage0:nqp-runtime.jar:3rdparty/asm/asm-4.1.jar nqp
--bootstrap --module-path=src/stage1 --setting-path=src/stage1
--setting=NQPCORE --no-regex-lib --target=classfile
--output=src/stage1/QAST.class src/stage1/gen/QAST.nqp
java.lang.RuntimeException: java.lang.RuntimeException: Method code too
large!
in <anon>
in compile
in eval
in evalfiles
in command_eval
in command_line
in MAIN
in <anon>
in <anon>

Gerd Pokorra

[Nicholas Clark]

On Sat, May 18, 2013 at 09:03:35PM +0200, Gerd Pokorra wrote:
> Hello,
>

> I get the same error with the -Xmx1024m flag.
>
> > java -Xmx1024m -cp
> src/vm/jvm/stage0:nqp-runtime.jar:3rdparty/asm/asm-4.1.jar nqp
> --bootstrap --module-path=src/stage1 --setting-path=src/stage1
> --setting=NQPCORE --no-regex-lib --target=classfile
> --output=src/stage1/QAST.class src/stage1/gen/QAST.nqp
> java.lang.RuntimeException: java.lang.RuntimeException: Method code too
> large!
> in <anon>
> in compile
> in eval
> in evalfiles
> in command_eval
> in command_line
> in MAIN
> in <anon>
> in <anon>

Aha. Right. I might have overlooked something else, which was obvious to me,
but not actually obvious. Sorry.

Based on a conversation on #perl6, my NQP is built on the branch
rak-jvm-support, and my Rakudo is from the branch jvm-support

Neither NQP master nor Rakudo nom have the most recent changes merged in.
In particular, NQP master accidentally has a commit merged which breaks
things, IIRC with hat error, but the fix for that commit is not merged in to
master. (But will be after the next compiler release)

So if you're not using the branches rak-jvm-support and jvm-support that
will be the problem.

Also, there's an obsolete branch jvm-support in NQP. You might have that
locally, just to confuse things. If you run `git remote prune origin` that
should clean up. (Or git fetch --prune)

Nicholas Clark

[Gerd Pokorra]

Hello,

after checking out the branch "rak-jvm-support" from the nqp repository
and the branch "jvm-support" from the rakudo repository both build fine
on JVM.


./perl6 --version
This is perl6 version 2013.04-225-g7d2e5b4 built on JVM


Thank you very much for the information Nicholas.

-- Gerd Pokorra