Maven JVM terminated unexpectedly with exit code 134

[Alexander Potapov]

Hello,

Build fails sometimes in Jenkins during Tycho compilation:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGBUS (0x7) at pc=0x00007ffae80661b0, pid=26833, tid=140715262596864
#
# JRE version: 6.0_29-b11
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.4-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libzip.so+0xb1b0]  __int128+0x60
#
# An error report file with more information is saved as:
# /d1/maven/.jenkins/jobs/c.v.s.r.lp/workspace/hs_err_pid26833.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
ERROR: Maven JVM terminated unexpect
edly with exit code 134

When I open this log /d1/maven/.jenkins/jobs/c.v.s.r.lp/workspace/hs_err_pid26833.log

I see something like:

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J  java.util.zip.ZipFile.getEntry(JLjava/lang/String;Z)J
J  java.util.zip.ZipFile.getEntry(Ljava/lang/String;)Ljava/util/zip/ZipEntry;
j  org.eclipse.jdt.internal.compiler.classfmt.ClassFileReader.read(Ljava/util/zip/ZipFile;Ljava/lang/String;Z)Lorg/eclipse/jdt/internal/compiler/classfmt/ClassFileReader;+2
j  org.eclipse.jdt.internal.compiler.classfmt.ClassFileReader.read(Ljava/util/zip/ZipFile;Ljava/lang/String;)Lorg/eclipse/jdt/internal/compiler/classfmt/ClassFileReader;+3
j  org.eclipse.jdt.internal.compiler.batch.ClasspathJar.findClass([CLjava/lang/String;Ljava/lang/String;Z)Lorg/eclipse/jdt/internal/compiler/env/NameEnvironmentAnswer;+15
j  org.eclipse.jdt.internal.compiler.batch.FileSystem.findClass(Ljava/lang/String;[CZ)Lorg/eclipse/jdt/internal/compiler/env/NameEnvironmentAnswer;+129
j  org.eclipse.jdt.internal.compiler.batch.FileSystem.findType([[C)Lorg/eclipse/jdt/internal/compiler/env/NameEnvironmentAnswer;+25
j  org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType([[C)Lorg/eclipse/jdt/internal/compiler/lookup/ReferenceBinding;+5
j  org.eclipse.jdt.internal.compiler.lookup.UnresolvedReferenceBinding.resolve(Lorg/eclipse/jdt/internal/compiler/lookup/LookupEnvironment;Z)Lorg/eclipse/jdt/internal/compiler/lookup/ReferenceBinding;+39
j  org.eclipse.jdt.internal.compiler.lookup.BinaryTypeBinding.resolveType(Lorg/eclipse/jdt/internal/compiler/lookup/TypeBinding;Lorg/eclipse/jdt/internal/compiler/lookup/LookupEnvironment;Z)Lorg/eclipse/jdt/internal/compiler/lookup/Type$
j  org.eclipse.jdt.internal.compiler.lookup.BinaryTypeBinding.getMemberType([C)Lorg/eclipse/jdt/internal/compiler/lookup/ReferenceBinding;+80
j  org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.getTypeFromTypeSignature(Lorg/eclipse/jdt/internal/compiler/lookup/SignatureWrapper;[Lorg/eclipse/jdt/internal/compiler/lookup/TypeVariableBinding;Lorg/eclipse/jdt/internal/c$
j  org.eclipse.jdt.internal.compiler.lookup.BinaryTypeBinding.createFields([Lorg/eclipse/jdt/internal/compiler/env/IBinaryField;J[[[C)V+130
j  org.eclipse.jdt.internal.compiler.lookup.BinaryTypeBinding.cachePartsFrom(Lorg/eclipse/jdt/internal/compiler/env/IBinaryType;Z)V+569
j  org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.createBinaryTypeFrom(Lorg/eclipse/jdt/internal/compiler/env/IBinaryType;Lorg/eclipse/jdt/internal/compiler/lookup/PackageBinding;ZLorg/eclipse/jdt/internal/compiler/env/Acces$
j  org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.createBinaryTypeFrom(Lorg/eclipse/jdt/internal/compiler/env/IBinaryType;Lorg/eclipse/jdt/internal/compiler/lookup/PackageBinding;Lorg/eclipse/jdt/internal/compiler/env/Access$
j  org.eclipse.jdt.internal.compiler.Compiler.accept(Lorg/eclipse/jdt/internal/compiler/env/IBinaryType;Lorg/eclipse/jdt/internal/compiler/lookup/PackageBinding;Lorg/eclipse/jdt/internal/compiler/env/AccessRestriction;)V+43
j  org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(Lorg/eclipse/jdt/internal/compiler/lookup/PackageBinding;[C)Lorg/eclipse/jdt/internal/compiler/lookup/ReferenceBinding;+50
j  org.eclipse.jdt.internal.compiler.lookup.PackageBinding.getTypeOrPackage([C)Lorg/eclipse/jdt/internal/compiler/lookup/Binding;+99
j  org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.findImport([[CI)Lorg/eclipse/jdt/internal/compiler/lookup/Binding;+41
j  org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.findSingleImport([[CIZ)Lorg/eclipse/jdt/internal/compiler/lookup/Binding;+85
j  org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.resolveSingleImport(Lorg/eclipse/jdt/internal/compiler/lookup/ImportBinding;I)Lorg/eclipse/jdt/internal/compiler/lookup/Binding;+18
j  org.eclipse.jdt.internal.compiler.lookup.Scope.getTypeOrPackage([CIZ)Lorg/eclipse/jdt/internal/compiler/lookup/Binding;+973
j  org.eclipse.jdt.internal.compiler.lookup.Scope.getType([C)Lorg/eclipse/jdt/internal/compiler/lookup/TypeBinding;+15
j  org.eclipse.jdt.internal.compiler.ast.ParameterizedSingleTypeReference.internalResolveLeafType(Lorg/eclipse/jdt/internal/compiler/lookup/Scope;Lorg/eclipse/jdt/internal/compiler/lookup/ReferenceBinding;Z)Lorg/eclipse/jdt/internal/com$
j  org.eclipse.jdt.internal.compiler.ast.ParameterizedSingleTypeReference.internalResolveType(Lorg/eclipse/jdt/internal/compiler/lookup/Scope;Lorg/eclipse/jdt/internal/compiler/lookup/ReferenceBinding;Z)Lorg/eclipse/jdt/internal/compile$
j  org.eclipse.jdt.internal.compiler.ast.ParameterizedSingleTypeReference.resolveType(Lorg/eclipse/jdt/internal/compiler/lookup/ClassScope;)Lorg/eclipse/jdt/internal/compiler/lookup/TypeBinding;+4
j  org.eclipse.jdt.internal.compiler.ast.TypeReference.resolveSuperType(Lorg/eclipse/jdt/internal/compiler/lookup/ClassScope;)Lorg/eclipse/jdt/internal/compiler/lookup/TypeBinding;+2
j  org.eclipse.jdt.internal.compiler.lookup.ClassScope.findSupertype(Lorg/eclipse/jdt/internal/compiler/ast/TypeReference;)Lorg/eclipse/jdt/internal/compiler/lookup/ReferenceBinding;+35
j  org.eclipse.jdt.internal.compiler.lookup.ClassScope.connectSuperclass()Z+157
j  org.eclipse.jdt.internal.compiler.lookup.ClassScope.connectTypeHierarchy()V+48
j  org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.connectTypeHierarchy()V+20
j  org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.completeTypeBindings()V+72
j  org.eclipse.jdt.internal.compiler.Compiler.internalBeginToCompile([Lorg/eclipse/jdt/internal/compiler/env/ICompilationUnit;I)V+323
j  org.eclipse.jdt.internal.compiler.Compiler.beginToCompile([Lorg/eclipse/jdt/internal/compiler/env/ICompilationUnit;)V+19
j  org.eclipse.jdt.internal.compiler.Compiler.compile([Lorg/eclipse/jdt/internal/compiler/env/ICompilationUnit;)V+47
j  org.eclipse.jdt.internal.compiler.batch.Main.performCompilation()V+238
j  org.eclipse.jdt.internal.compiler.batch.Main.compile([Ljava/lang/String;)Z+125
j  org.eclipse.tycho.compiler.jdt.JDTCompiler.compileInProcess([Ljava/lang/String;)Ljava/util/List;+98
j  org.eclipse.tycho.compiler.jdt.JDTCompiler.compile(Lorg/codehaus/plexus/compiler/CompilerConfiguration;)Ljava/util/List;+170
j  copied.org.apache.maven.plugin.AbstractCompilerMojo.execute()V+714
j  org.eclipse.tycho.compiler.AbstractOsgiCompilerMojo.execute()V+51
j  org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(Lorg/apache/maven/execution/MavenSession;Lorg/apache/maven/plugin/MojoExecution;)V+101
j  org.apache.maven.lifecycle.internal.MojoExecutor.execute(Lorg/apache/maven/execution/MavenSession;Lorg/apache/maven/plugin/MojoExecution;Lorg/apache/maven/lifecycle/internal/ProjectIndex;Lorg/apache/maven/lifecycle/internal/Dependenc$
j  org.apache.maven.lifecycle.internal.MojoExecutor.execute(Lorg/apache/maven/execution/MavenSession;Lorg/apache/maven/plugin/MojoExecution;Lorg/apache/maven/lifecycle/internal/ProjectIndex;Lorg/apache/maven/lifecycle/internal/Dependenc$
j  org.apache.maven.lifecycle.internal.MojoExecutor.execute(Lorg/apache/maven/execution/MavenSession;Ljava/util/List;Lorg/apache/maven/lifecycle/internal/ProjectIndex;)V+60
j  org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(Lorg/apache/maven/execution/MavenSession;Lorg/apache/maven/execution/MavenSession;Lorg/apache/maven/lifecycle/internal/ReactorContext;Lorg/apache/maven/project/M$
j  org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(Lorg/apache/maven/execution/MavenSession;Lorg/apache/maven/lifecycle/internal/ReactorContext;Lorg/apache/maven/project/MavenProject;Lorg/apache/maven/lifecycle/i$
j  org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(Lorg/apache/maven/execution/MavenSession;Lorg/apache/maven/lifecycle/internal/ReactorContext;Lorg/apache/maven/lifecycle/internal/ProjectBuildList;Ljava/util/Li$
j  org.apache.maven.lifecycle.internal.LifecycleStarter.execute(Lorg/apache/maven/execution/MavenSession;)V+421
j  org.apache.maven.DefaultMaven.doExecute(Lorg/apache/maven/execution/MavenExecutionRequest;)Lorg/apache/maven/execution/MavenExecutionResult;+585
j  org.apache.maven.DefaultMaven.execute(Lorg/apache/maven/execution/MavenExecutionRequest;)Lorg/apache/maven/execution/MavenExecutionResult;+11
j  org.jvnet.hudson.maven3.launcher.Maven3Launcher.main([Ljava/lang/String;)I+98
v  ~StubRoutines::call_stub
j  sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+0
j  sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+87
j  sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+6
j  java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+161
j  org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard([Ljava/lang/String;)V+34
j  org.codehaus.plexus.classworlds.launcher.Launcher.launch([Ljava/lang/String;)V+9
j  org.jvnet.hudson.maven3.agent.Maven3Main.launch([Ljava/lang/String;)I+4
j  hudson.maven.Maven3Builder.call()Lhudson/model/Result;+69
j  hudson.maven.Maven3Builder.call()Ljava/lang/Object;+1
j  hudson.remoting.UserRequest.perform(Lhudson/remoting/Channel;)Lhudson/remoting/UserResponse;+137
j  hudson.remoting.UserRequest.perform(Lhudson/remoting/Channel;)Ljava/io/Serializable;+2
j  hudson.remoting.Request$2.run()V+52
j  hudson.remoting.InterceptingExecutorService$1.call()Ljava/lang/Object;+4
j  java.util.concurrent.FutureTask$Sync.innerRun()V+30
j  java.util.concurrent.FutureTask.run()V+4
j  java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Ljava/lang/Runnable;)V+59
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+28
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

Haven't seen during the build on my local machine.

Any ideas?

Br,

Alexander

[Jesse Glick]

On 09/07/2012 08:15 AM, Alexander Potapov wrote:
> SIGBUS (0x7) at pc=0x00007ffae80661b0, pid=26833, tid=140715262596864

> J java.util.zip.ZipFile.getEntry(JLjava/lang/String;Z)J
> J java.util.zip.ZipFile.getEntry(Ljava/lang/String;)Ljava/util/zip/ZipEntry;
> j org.eclipse.jdt.internal.compiler.classfmt.ClassFileReader.read(Ljava/util/zip/ZipFile;Ljava/lang/String;Z)Lorg/eclipse/jdt/internal/compiler/classfmt/ClassFileReader;+2

This Java code (in JDT) is trying to read classes from a JAR file which has been, or is being, overwritten (probably by the same Maven process) with a newer version since
the JAR was opened. Oracle-derived JVMs use libzip.so/dll (rather than pure Java code) to implement ZipFile, and this library was not designed to handle ZIP files being
replaced “under its feet”. I have repeatedly advocated that ZipFile use a more robust pure Java implementation (leaving libzip for use only by HotSpot loading the
bootclasspath, which is expected to be immutable), but no one in the Java team seems to consider this a priority, so do not expect it to be fixed unless someone pushes it
from OpenJDK. In the meantime the workarounds, insofar as you can control the code immediately involved in the bug, are:

1. Prevent the JAR from being overwritten.

2. Rather than reading the JAR “in-place”, make a temporary copy (File.createTempFile/deleteOnExit) and open that instead.

Note that this bug does not occur on Windows—mandatory file locking means that if the ZipFile is open for reading, any attempt to overwrite (or delete) it will fail with
an IOException or the like.

[Alexander Potapov]

Hello,

Haven't seen this problem when I had one build process. Thus, we cannot take into account the case of one Maven process for sure.

In my setup I increased the number of build executors. By default it is 2, I have 5.

All jobs in my Jenkins are interconnected, they have Maven dependencies specified what is not uncommon. I believe it is something to do with Jenkins. 

It gives possibility to have multiple build processes at the same time on the same machine. And it fails to perform that.

Br,

Alexander.

[Alexander Potapov]

Hello once again, 

Should I report bug about that?

Br,

Alexander

[Jerome Lacoste]

On Monday, September 24, 2012 10:50:39 AM UTC+2, Alexander Potapov wrote:

Hello once again, 

Should I report bug about that?

It's not necessarily because jenkins gives you the ability to schedule multiple builds at the same time, that it is jenkins fault when the builds fail due to a limitation in the build environment. Jenkins is just your scheduler and you should make sure your build process can be run through different executors.

E.g. on your local machine, if you run multiple builds in parallel (e.g. different shells), do you get the same failure ? If yes, then Jenkins is definitively not the cause.

As Jesse wrote, make sure your jobs do not step on each other toes. Make a copy of the to be rewritten jars, use a per job local maven repo, etc.

Jerome