Strange compilation errors on linux platform

[bananos]

We have a pretty heterogeneous team which works with GWT on Mac,
Windows & linux machines.
One of our latest commit crashed the automatic build which runs on
linux box.
The problem is that GWT java source generator fails with
NullPointerException at different places with the same configuration
inputs, therefore it is very hard to reproduce bug or nail it down.

The only regularity we've found is that it fails at some point when
trying to generate client bundles from PNG files.
Here are few examples:

Case #1

[java] Scanning for additional dependencies: file:/home/bear-z/
work/client/Application/src/com/project/client/common/bundles/
CommonResources.java
[java] Computing all possible rebind results for
'com.project.client.common.bundles.CommonResources'
[java] Rebinding
com.project.client.common.bundles.CommonResources
[java] Invoking
com.google.gwt.dev.javac.StandardGeneratorContext@72af7016
[java] [ERROR] Generator
'com.google.gwt.resources.rebind.context.InlineClientBundleGenerator'
threw threw an exception while rebinding
'com.project.client.common.bundles.CommonResources'
[java] java.lang.NullPointerException
[java] at
com.google.gwt.resources.rebind.context.InlineResourceContext.deploy(InlineResourceContext.java:
40)
[java] at
com.google.gwt.resources.rebind.context.AbstractResourceContext.deploy(AbstractResourceContext.java:
97)
[java] at
com.google.gwt.resources.rg.ImageResourceGenerator.maybeDeploy(ImageResourceGenerator.java:
369)
[java] at
com.google.gwt.resources.rg.ImageResourceGenerator.createFields(ImageResourceGenerator.java:
176)
[java] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.createFieldsAndAssignments(AbstractClientBundleGenerator.java:
328)
[java] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.createFieldsAndAssignments(AbstractClientBundleGenerator.java:
385)
[java] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.generate(AbstractClientBundleGenerator.java:
245)
[java] at
com.google.gwt.dev.javac.StandardGeneratorContext.runGenerator(StandardGeneratorContext.java:
418)
[java] at
com.google.gwt.dev.cfg.RuleGenerateWith.realize(RuleGenerateWith.java:
38)
[java] at com.google.gwt.dev.shell.StandardRebindOracle
$Rebinder.tryRebind(StandardRebindOracle.java:108)
[java] at com.google.gwt.dev.shell.StandardRebindOracle
$Rebinder.rebind(StandardRebindOracle.java:54)
[java] at
com.google.gwt.dev.shell.StandardRebindOracle.rebind(StandardRebindOracle.java:
154)
[java] at
com.google.gwt.dev.shell.StandardRebindOracle.rebind(StandardRebindOracle.java:
143)
[java] at com.google.gwt.dev.Precompile
$DistillerRebindPermutationOracle.getAllPossibleRebindAnswers(Precompile.java:
317)
[java] at
com.google.gwt.dev.jdt.WebModeCompilerFrontEnd.doFindAdditionalTypesUsingRebinds(WebModeCompilerFrontEnd.java:
95)
[java] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.process(AbstractCompiler.java:200)
[java] at
org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:444)
[java] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.compile(AbstractCompiler.java:123)
[java] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.compile(AbstractCompiler.java:234)
[java] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.access$200(AbstractCompiler.java:109)
[java] at
com.google.gwt.dev.jdt.AbstractCompiler.compile(AbstractCompiler.java:
522)
[java] at
com.google.gwt.dev.jdt.BasicWebModeCompiler.getCompilationUnitDeclarations(BasicWebModeCompiler.java:
112)
[java] at
com.google.gwt.dev.jdt.WebModeCompilerFrontEnd.getCompilationUnitDeclarations(WebModeCompilerFrontEnd.java:
47)
[java] at
com.google.gwt.dev.jjs.JavaToJavaScriptCompiler.precompile(JavaToJavaScriptCompiler.java:
430)
[java] at
com.google.gwt.dev.jjs.JavaScriptCompiler.precompile(JavaScriptCompiler.java:
32)
[java] at
com.google.gwt.dev.Precompile.precompile(Precompile.java:522)
[java] at
com.google.gwt.dev.Precompile.precompile(Precompile.java:414)
[java] at com.google.gwt.dev.Compiler.run(Compiler.java:201)
[java] at com.google.gwt.dev.Compiler$1.run(Compiler.java:
152)
[java] at
com.google.gwt.dev.CompileTaskRunner.doRun(CompileTaskRunner.java:87)
[java] at
com.google.gwt.dev.CompileTaskRunner.runWithAppropriateLogger(CompileTaskRunner.java:
81)
[java] at com.google.gwt.dev.Compiler.main(Compiler.java:159)

Fails at
com.google.gwt.resources.rebind.context.InlineClientBundleGenerator



Case #2
[java] [ERROR] Errors in 'file:/home/bear-z/work/client/
Application/src/com/project/client/common/bundles/
CommonResources.java'
[java] [ERROR] Line 13: Failed to resolve
'com.project.client.common.bundles.CommonResources' via deferred
binding
[java] Scanning for additional dependencies: file:/home/bear-z/
work/client/Application/src/com/project/client/contacts/bundles/
ContactsResources.java
[java] Computing all possible rebind results for
'com.project.client.contacts.bundles.ContactsResources'
[java] Rebinding
com.project.client.contacts.bundles.ContactsResources
[java] Invoking
com.google.gwt.dev.javac.StandardGeneratorContext@72af7016
[java] [ERROR] Generator
'com.google.gwt.resources.rebind.context.StaticClientBundleGenerator'
threw threw an exception while rebinding
'com.project.client.contacts.bundles.ContactsResources'
[java] java.lang.NullPointerException
[java] at
com.google.gwt.dev.util.Util.computeStrongName(Util.java:166)
[java] at
com.google.gwt.dev.util.Util.computeStrongName(Util.java:141)
[java] at
com.google.gwt.resources.rg.ImageBundleBuilder.assimilate(ImageBundleBuilder.java:
627)
[java] at
com.google.gwt.resources.rg.ImageResourceGenerator.prepare(ImageResourceGenerator.java:
235)
[java] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.initAndPrepare(AbstractClientBundleGenerator.java:
556)
[java] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.initAndPrepare(AbstractClientBundleGenerator.java:
582)
[java] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.generate(AbstractClientBundleGenerator.java:
204)
[java] at
com.google.gwt.dev.javac.StandardGeneratorContext.runGenerator(StandardGeneratorContext.java:
418)
[java] at
com.google.gwt.dev.cfg.RuleGenerateWith.realize(RuleGenerateWith.java:
38)
[java] at com.google.gwt.dev.shell.StandardRebindOracle
$Rebinder.tryRebind(StandardRebindOracle.java:108)
[java] at com.google.gwt.dev.shell.StandardRebindOracle
$Rebinder.rebind(StandardRebindOracle.java:54)
[java] at
com.google.gwt.dev.shell.StandardRebindOracle.rebind(StandardRebindOracle.java:
154)
[java] at
com.google.gwt.dev.shell.StandardRebindOracle.rebind(StandardRebindOracle.java:
143)
[java] at com.google.gwt.dev.Precompile
$DistillerRebindPermutationOracle.getAllPossibleRebindAnswers(Precompile.java:
317)
[java] at
com.google.gwt.dev.jdt.WebModeCompilerFrontEnd.doFindAdditionalTypesUsingRebinds(WebModeCompilerFrontEnd.java:
95)
[java] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.process(AbstractCompiler.java:200)
[java] at
org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:444)
[java] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.compile(AbstractCompiler.java:123)
[java] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.compile(AbstractCompiler.java:234)
[java] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.access$200(AbstractCompiler.java:109)
[java] at
com.google.gwt.dev.jdt.AbstractCompiler.compile(AbstractCompiler.java:
522)
[java] at
com.google.gwt.dev.jdt.BasicWebModeCompiler.getCompilationUnitDeclarations(BasicWebModeCompiler.java:
112)
[java] at
com.google.gwt.dev.jdt.WebModeCompilerFrontEnd.getCompilationUnitDeclarations(WebModeCompilerFrontEnd.java:
47)
[java] at
com.google.gwt.dev.jjs.JavaToJavaScriptCompiler.precompile(JavaToJavaScriptCompiler.java:
430)
[java] at
com.google.gwt.dev.jjs.JavaScriptCompiler.precompile(JavaScriptCompiler.java:
32)
[java] at
com.google.gwt.dev.Precompile.precompile(Precompile.java:522)
[java] at
com.google.gwt.dev.Precompile.precompile(Precompile.java:414)
[java] at com.google.gwt.dev.Compiler.run(Compiler.java:201)
[java] at com.google.gwt.dev.Compiler$1.run(Compiler.java:
152)
[java] at
com.google.gwt.dev.CompileTaskRunner.doRun(CompileTaskRunner.java:87)
[java] at
com.google.gwt.dev.CompileTaskRunner.runWithAppropriateLogger(CompileTaskRunner.java:
81)
[java] at com.google.gwt.dev.Compiler.main(Compiler.java:159)

Fails at
com.google.gwt.resources.rebind.context.StaticClientBundleGenerator

We are using GWT 2.0.4 and tried to compile on
a) fedora 13 x86_64 / Both sun Java 1.6.0_16 & java-1.6.0-
openjdk-1.6.0.0-42.b18.fc13.x86_64
b) Ubuntu 10.04 x86 / Both sun Java 1.6.0_16 & OpenJDK Server VM
(16.0-b13) for linux-x86


I believe that problem is somewhere deep inside Java/GWT internals,
related to low level memory/threads handling because of following
1) it compiles like a charm on Windows & Mac boxes
2) it is not regular, i.e. the same code fails at different places
during compilation


Is there anyone who have experienced something similar on linux? I
would really appreciate any help

[Jeff Chimene]

At first glance, it seems there is a problem with the environment settings.

Have you successfully built other projects on the Linux box?

FWIW, I do 99% of my GWT work on Debian.

[bananos]

Yep, there was no problem for almost a year (we were having Hudson
linux box for deployment from the very start),
and suddenly after small reorganizing of resources (actually only
paths to image resources were changed) -- it crashed

As for the env, I tried to play a bit with gwtc settings, but no
luck.
Currently, our project consists of 2 applications, and here is come
details from ant build:

<jvmarg value="-Xmx1512M"/>
<arg value="-style"/>
<arg value="OBF"/>

<arg value="com.project.Register"/>
<arg value="com.project.Application"/>

Increasing/decreasing Xmx didn't help at all, changing Java from
OpenJDK to Sun java didn't help either

[Jeff Chimene]

Alright. I don't have a good answer for you other than to create an app from scratch and see if it compiles.

--
You received this message because you are subscribed to the Google Groups "Google Web Toolkit" group.
To post to this group, send email to google-we...@googlegroups.com.
To unsubscribe from this group, send email to google-web-tool...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.

[Chris Conroy]

It looks like something about your images/system is causing ImageIO.write to fail. My best guess is that it's returning that there are no available ImageWriters, but the compiler incorrectly does not read the return value.

Any time the compiler dies with an NPE, we consider that a compiler bug. I should have a patch later today to fail on this condition a bit more gracefully. That said, the root cause will still mean your build won't complete.

[paxdei]

I ran into the same problem (compile fails only on linux) with GWT version 2.2.0

With hacking some log statements into the GWT compiler I finally found the reason:

java.io.FileNotFoundException: /XXXXXXX/calendar.year.gif (Too many open files)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:106)
        at java.io.FileInputStream.<init>(FileInputStream.java:66)
        at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
        at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
        at com.google.gwt.dev.util.Util.readURLConnectionAsBytes(Util.java:865)
        at com.google.gwt.dev.util.Util.readURLAsBytes(Util.java:810)
        at com.google.gwt.resources.rebind.context.AbstractResourceContext.deploy(AbstractResourceContext.java:72)
        at com.google.gwt.resources.rg.DataResourceGenerator.createAssignment(DataResourceGenerator.java:59)
        at com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.createFieldsAndAssignments(AbstractClientBundleGenerator.java:394)
        at com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.createFieldsAndAssignments(AbstractClientBundleGenerator.java:439)
        at com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.generate(AbstractClientBundleGenerator.java:287)

So the solution was for me to increase the open file limit in /etc/security/limits.conf from 1024 to 2048 by editing this line:

*               soft    nofile          2048

regards

[Sai Siddarth Muralidharan]

I am running into the same issue now. Any idea how to fix this?

[nik]

Same identical issue, build works on windows, but fails with all sorts of strange compiler errors on RHEL6

Tried forcing encoding via <project.build.sourceEncoding>cp1252</project.build.sourceEncoding>

Tried via export JAVA_TOOL_OPTIONS=-Dfile.encoding=iso-8859-1

Same result, i guess u can fix it by not using gwt...

[Jens]

Same result, i guess u can fix it by not using gwt...

Have you tried increasing the max number of open files on linux as mentioned in this thread? Alternatively try upgrading to GWT 2.8 which fixed a file handle issue, see: https://github.com/gwtproject/gwt/commit/349210a31ec593cb499ac73a80743b89bd8d5aac

-- J.

[nik]

Yup, I do have root access, tried pushing ulimit -n to 2048 and 4096, didn't work (ulimit pushed but compilation still failed in the same fashion)

I tried to push gwt version to 2.8 but it failed to compile even on windows, I can probably fix it some time in the future, just don't have time to worry about it now...

Settled on building this component manually on windows :(