Problems with memory usage on 1.560

[Tobias Hieta]

Hello,

I installed 1.560 the other day and I have seen a few of the following exceptions when building my (rather large) matrix builds:

00:12:32.091 FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected reader termination

00:12:32.130 hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected reader termination

00:12:32.810 at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41)

00:12:32.810 at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34)

00:12:32.810 at hudson.remoting.Request.call(Request.java:174)

00:12:32.810 at hudson.remoting.Channel.call(Channel.java:738)

00:12:32.810 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:168)

00:12:32.810 at com.sun.proxy.$Proxy73.join(Unknown Source)

00:12:32.810 at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:951)

00:12:32.810 at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137)

00:12:32.810 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)

00:12:32.810 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)

00:12:32.810 at org.jenkins_ci.plugins.run_condition.BuildStepRunner$2.run(BuildStepRunner.java:110)

00:12:32.810 at org.jenkins_ci.plugins.run_condition.BuildStepRunner$Fail.conditionalRun(BuildStepRunner.java:154)

00:12:32.810 at org.jenkins_ci.plugins.run_condition.BuildStepRunner.perform(BuildStepRunner.java:105)

00:12:32.810 at org.jenkinsci.plugins.conditionalbuildstep.singlestep.SingleConditionalBuilder.perform(SingleConditionalBuilder.java:108)

00:12:32.810 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)

00:12:32.810 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:740)

00:12:32.810 at hudson.model.Build$BuildExecution.build(Build.java:198)

00:12:32.810 at hudson.model.Build$BuildExecution.doRun(Build.java:159)

00:12:32.810 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:519)

00:12:32.810 at hudson.model.Run.execute(Run.java:1703)

00:12:32.810 at hudson.matrix.MatrixRun.run(MatrixRun.java:146)

00:12:32.810 at hudson.model.ResourceController.execute(ResourceController.java:88)

00:12:32.810 at hudson.model.Executor.run(Executor.java:231)

00:12:32.810 Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected reader termination

00:12:32.810 at hudson.remoting.Request.abort(Request.java:299)

00:12:32.810 at hudson.remoting.Channel.terminate(Channel.java:801)

00:12:32.810 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:76)

00:12:32.810 Caused by: java.io.IOException: Unexpected reader termination

00:12:32.810 ... 1 more

00:12:32.810 Caused by: java.lang.OutOfMemoryError: Java heap space

00:12:32.810 at java.util.Arrays.copyOf(Arrays.java:2271)

00:12:32.810 at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)

00:12:32.810 at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)

00:12:32.810 at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)

00:12:32.810 at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:87)

00:12:32.810 at hudson.remoting.ChunkedInputStream.read(ChunkedInputStream.java:46)

00:12:32.810 at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:88)

00:12:32.810 at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:33)

00:12:32.810 at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)

00:12:32.810 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

I have tried to up the heap space both on the master node and on the slaves. I have currently given the master over 8GB of Heap Space and the Slaves around the same, but this still happens from time to time. First question, is this running out of memory on the master node or on the slaves? Secondly, should a matrix build with ~15 builds really use this much memory? Should I just up the heap space on the master node until it goes away or is there some code that have recently changed the memory usage?

Thanks,

Tobias

[Larry Shatzer, Jr.]

I've run into issues with the Clone Workspace plugin. Once I rolled back to 1.599, it went away. Have you tried to downgrade back to 1.559?

I've noticed that 1.559 uses Remoting 2.37, and 1.560 uses 2.39, and master is currently on 2.40. I've not tried running this with what is on master yet. I've been meaning to ping the IRC channel about this.

Here's my stacktrace:

Archiving workspace
ERROR: Publisher hudson.plugins.cloneworkspace.CloneWorkspacePublisher aborted due to exception
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected reader termination
	at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41)
	at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34)
	at hudson.remoting.Request.call(Request.java:174)
	at hudson.remoting.Channel.call(Channel.java:738)
	at hudson.FilePath.act(FilePath.java:909)
	at hudson.FilePath.act(FilePath.java:893)
	at hudson.FilePath.archive(FilePath.java:423)
	at hudson.FilePath.tar(FilePath.java:2047)
	at hudson.plugins.cloneworkspace.CloneWorkspacePublisher.snapshot(CloneWorkspacePublisher.java:231)
	at hudson.plugins.cloneworkspace.CloneWorkspacePublisher.perform(CloneWorkspacePublisher.java:178)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:740)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:710)
	at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.post2(MavenModuleSetBuild.java:1036)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:659)
	at hudson.model.Run.execute(Run.java:1728)
	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:525)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:231)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected reader termination
	at hudson.remoting.Request.abort(Request.java:299)
	at hudson.remoting.Channel.terminate(Channel.java:801)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:76)
Caused by: java.io.IOException: Unexpected reader termination
	... 1 more
Caused by: java.lang.OutOfMemoryError: Java heap space

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Morten Andersen]

Hi Tobias,

Could it be this issue that you are seing: [JENKINS-22734] OutOfMemory on file transfer

Best Regards

Morten

[Tobias Hieta]

Yes this seems to be exactly my issue. I reverted back to a earlier version of Jenkins and it seems to build fine now.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/QSZu57CmmfA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Tobias Hieta
http://about.me/thieta