Compiler errors: NullPointerException and IOException

193 views
Skip to first unread message

Jason

unread,
Dec 13, 2011, 8:32:52 PM12/13/11
to Google Web Toolkit, deng...@evernote.com
Folks,

We have a large GWT application with 161 compilation permutations (7
user agent permutations * 23 languages). The compiler requires 2.5GB
of memory to run and compilations have recently started to fail (non-
deterministically) with the following errors (full stack traces at the
end of this message):

[INFO] java.lang.NullPointerException
[INFO] at
com.google.gwt.resources.rebind.context.InlineResourceContext.deploy(InlineResourceContext.java:
40)

[INFO] javax.imageio.IIOException: Can't create output stream!
[INFO] at javax.imageio.ImageIO.write(ImageIO.java:1560)
[INFO] at
com.google.gwt.resources.rg.ImageBundleBuilder.createImageBytes(ImageBundleBuilder.java:
558)

Is there any way that we can reduce the memory footprint of the
compiler? How can we prevent the above compilation errors from
occuring?

Thanks,
-Jason Terk

---

[INFO] java.lang.NullPointerException
[INFO] at
com.google.gwt.resources.rebind.context.InlineResourceContext.deploy(InlineResourceContext.java:
40)
[INFO] at
com.google.gwt.resources.rebind.context.AbstractResourceContext.deploy(AbstractResourceContext.java:
78)
[INFO] at com.google.gwt.resources.rg.ImageResourceGenerator
$ExternalImage.render(ImageResourceGenerator.java:310)
[INFO] at
com.google.gwt.resources.rg.ImageResourceGenerator.renderImageMap(ImageResourceGenerator.java:
662)
[INFO] at
com.google.gwt.resources.rg.ImageResourceGenerator.createFields(ImageResourceGenerator.java:
504)
[INFO] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.createFieldsAndAssignments(AbstractClientBundleGenerator.java:
773)
[INFO] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.createFieldsAndAssignments(AbstractClientBundleGenerator.java:
854)
[INFO] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.generateIncrementally(AbstractClientBundleGenerator.java:
460)
[INFO] at
com.google.gwt.dev.javac.StandardGeneratorContext.runGeneratorIncrementally(StandardGeneratorContext.java:
647)
[INFO] at
com.google.gwt.dev.cfg.RuleGenerateWith.realize(RuleGenerateWith.java:
41)
[INFO] at com.google.gwt.dev.shell.StandardRebindOracle
$Rebinder.rebind(StandardRebindOracle.java:78)
[INFO] at
com.google.gwt.dev.shell.StandardRebindOracle.rebind(StandardRebindOracle.java:
268)
[INFO] at
com.google.gwt.dev.shell.StandardRebindOracle.rebind(StandardRebindOracle.java:
257)
[INFO] at
com.google.gwt.dev.DistillerRebindPermutationOracle.getAllPossibleRebindAnswers(DistillerRebindPermutationOracle.java:
91)
[INFO] at
com.google.gwt.dev.jdt.WebModeCompilerFrontEnd.doFindAdditionalTypesUsingRebinds(WebModeCompilerFrontEnd.java:
96)
[INFO] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.process(AbstractCompiler.java:254)
[INFO] at
org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:444)
[INFO] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.compile(AbstractCompiler.java:173)
[INFO] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.compile(AbstractCompiler.java:288)
[INFO] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.access$400(AbstractCompiler.java:139)
[INFO] at
com.google.gwt.dev.jdt.AbstractCompiler.compile(AbstractCompiler.java:
588)
[INFO] at
com.google.gwt.dev.jdt.BasicWebModeCompiler.getCompilationUnitDeclarations(BasicWebModeCompiler.java:
97)
[INFO] at
com.google.gwt.dev.jdt.WebModeCompilerFrontEnd.getCompilationUnitDeclarations(WebModeCompilerFrontEnd.java:
52)
[INFO] at
com.google.gwt.dev.jjs.JavaToJavaScriptCompiler.precompile(JavaToJavaScriptCompiler.java:
569)
[INFO] at
com.google.gwt.dev.jjs.JavaScriptCompiler.precompile(JavaScriptCompiler.java:
33)
[INFO] at com.google.gwt.dev.Precompile.precompile(Precompile.java:
284)
[INFO] at com.google.gwt.dev.Precompile.precompile(Precompile.java:
233)
[INFO] at com.google.gwt.dev.Precompile.precompile(Precompile.java:
145)
[INFO] at com.google.gwt.dev.Compiler.run(Compiler.java:232)
[INFO] at com.google.gwt.dev.Compiler.run(Compiler.java:198)
[INFO] at com.google.gwt.dev.Compiler$1.run(Compiler.java:170)
[INFO] at
com.google.gwt.dev.CompileTaskRunner.doRun(CompileTaskRunner.java:88)
[INFO] at
com.google.gwt.dev.CompileTaskRunner.runWithAppropriateLogger(CompileTaskRunner.java:
82)
[INFO] at com.google.gwt.dev.Compiler.main(Compiler.java:177)

---

[INFO] javax.imageio.IIOException: Can't create output stream!
[INFO] at javax.imageio.ImageIO.write(ImageIO.java:1560)
[INFO] at
com.google.gwt.resources.rg.ImageBundleBuilder.createImageBytes(ImageBundleBuilder.java:
558)
[INFO] at
com.google.gwt.resources.rg.ImageBundleBuilder.toPng(ImageBundleBuilder.java:
544)
[INFO] at
com.google.gwt.resources.rg.ImageResourceGenerator.reencodeToTempFile(ImageResourceGenerator.java:
641)
[INFO] at
com.google.gwt.resources.rg.ImageResourceGenerator.prepare(ImageResourceGenerator.java:
567)
[INFO] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.initAndPrepare(AbstractClientBundleGenerator.java:
1043)
[INFO] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.initAndPrepare(AbstractClientBundleGenerator.java:
1069)
[INFO] at
com.google.gwt.resources.rebind.context.AbstractClientBundleGenerator.generateIncrementally(AbstractClientBundleGenerator.java:
412)
[INFO] at
com.google.gwt.dev.javac.StandardGeneratorContext.runGeneratorIncrementally(StandardGeneratorContext.java:
647)
[INFO] at
com.google.gwt.dev.cfg.RuleGenerateWith.realize(RuleGenerateWith.java:
41)
[INFO] at com.google.gwt.dev.shell.StandardRebindOracle
$Rebinder.rebind(StandardRebindOracle.java:78)
[INFO] at
com.google.gwt.dev.shell.StandardRebindOracle.rebind(StandardRebindOracle.java:
268)
[INFO] at
com.google.gwt.dev.shell.StandardRebindOracle.rebind(StandardRebindOracle.java:
257)
[INFO] at
com.google.gwt.dev.DistillerRebindPermutationOracle.getAllPossibleRebindAnswers(DistillerRebindPermutationOracle.java:
91)
[INFO] at
com.google.gwt.dev.jdt.WebModeCompilerFrontEnd.doFindAdditionalTypesUsingRebinds(WebModeCompilerFrontEnd.java:
96)
[INFO] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.process(AbstractCompiler.java:254)
[INFO] at
org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:444)
[INFO] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.compile(AbstractCompiler.java:173)
[INFO] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.compile(AbstractCompiler.java:288)
[INFO] at com.google.gwt.dev.jdt.AbstractCompiler$Sandbox
$CompilerImpl.access$400(AbstractCompiler.java:139)
[INFO] at
com.google.gwt.dev.jdt.AbstractCompiler.compile(AbstractCompiler.java:
588)
[INFO] at
com.google.gwt.dev.jdt.BasicWebModeCompiler.getCompilationUnitDeclarations(BasicWebModeCompiler.java:
97)
[INFO] at
com.google.gwt.dev.jdt.WebModeCompilerFrontEnd.getCompilationUnitDeclarations(WebModeCompilerFrontEnd.java:
52)
[INFO] at
com.google.gwt.dev.jjs.JavaToJavaScriptCompiler.precompile(JavaToJavaScriptCompiler.java:
569)
[INFO] at
com.google.gwt.dev.jjs.JavaScriptCompiler.precompile(JavaScriptCompiler.java:
33)
[INFO] at com.google.gwt.dev.Precompile.precompile(Precompile.java:
284)
[INFO] at com.google.gwt.dev.Precompile.precompile(Precompile.java:
233)
[INFO] at com.google.gwt.dev.Precompile.precompile(Precompile.java:
145)
[INFO] at com.google.gwt.dev.Compiler.run(Compiler.java:232)
[INFO] at com.google.gwt.dev.Compiler.run(Compiler.java:198)
[INFO] at com.google.gwt.dev.Compiler$1.run(Compiler.java:170)
[INFO] at
com.google.gwt.dev.CompileTaskRunner.doRun(CompileTaskRunner.java:88)
[INFO] at
com.google.gwt.dev.CompileTaskRunner.runWithAppropriateLogger(CompileTaskRunner.java:
82)
[INFO] at com.google.gwt.dev.Compiler.main(Compiler.java:177)
[INFO] Caused by: javax.imageio.IIOException: Can't create cache file!
[INFO] at javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:
397)
[INFO] at javax.imageio.ImageIO.write(ImageIO.java:1558)
[INFO] ... 33 more
[INFO] Caused by: java.io.IOException: Too many open files
[INFO] at java.io.UnixFileSystem.createFileExclusively(Native Method)
[INFO] at java.io.File.checkAndCreate(File.java:1704)
[INFO] at java.io.File.createTempFile(File.java:1792)
[INFO] at
javax.imageio.stream.FileCacheImageOutputStream.<init>(FileCacheImageOutputStream.java:
71)
[INFO] at
com.sun.imageio.spi.OutputStreamImageOutputStreamSpi.createOutputStreamInstance(OutputStreamImageOutputStreamSpi.java:
50)
[INFO] at javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:
393)
[INFO] ... 34 more

Ed

unread,
Dec 14, 2011, 3:56:43 AM12/14/11
to google-we...@googlegroups.com, deng...@evernote.com
You can control your memory settings in several ways:
+ jvm args. Something like this (using the gwt maven plugin):
<extraJvmArgs>-Xms1G -Xmx1G -Xss1024k</extraJvmArgs>

+ Local workers to control the number of cores it uses. Per default it try to use as much cores as possible. In my experience you have to reduce this number if you have limited ram memory. I my case I had to use it to 1 as otherwise I got strange memory problems. In gwt maven  plugin:
<localWorkers>1</localWorkers>

- Ed

Ed

unread,
Dec 14, 2011, 4:00:54 AM12/14/11
to google-we...@googlegroups.com, deng...@evernote.com
And another thing I forgot:
+ increase the number of open files. Especialy this error sames to show up when using many Image resources that need to be striped (you can find more about this in the forum).
On my linux box I had to change this to 2048 (default is 1024 which seem to be too small).
I changed it on Centos in /etc/security/limits.conf. But google around and you have find how to do it on your linux distribution.

If you tune all these variables well, you should be fine (it takes some time to tune them well).
- Ed

Thomas Broyer

unread,
Dec 14, 2011, 5:41:01 AM12/14/11
to google-we...@googlegroups.com, deng...@evernote.com
Also, have a look at http://code.google.com/p/google-web-toolkit/wiki/DistributedBuilds

But the "too man open files" is a strong indication you need to bump the "ulimit -n".

Also update to the latest GWT version if you can, I know there were a few issues in earlier versions about files that were not closed (but maybe it only affected the DevMode, I can't remember).

Paul Robinson

unread,
Dec 14, 2011, 6:13:04 AM12/14/11
to google-we...@googlegroups.com

On 14/12/11 10:41, Thomas Broyer wrote:
>
> But the "too many open files" is a strong indication you need to bump the "ulimit -n".
>
>
If you have a resource bundle with a large number of images, it seems the gwt compile likes to have them all open at the same time, regardless of whether you actually use them. This means the compile will take longer and you need your ulimit set large, as Thomas said.

The alternative is to only put into your bundle the images you actually need.

Paul

Reply all
Reply to author
Forward
0 new messages