Running >100 Pipeline builds can cause OutOfMemoryErrors, leading to Jenkins eventually becoming unresponsive. Sample log snippet below. After I get this issue opened, I'll recreate to produce cleaner logs.
WARNING: RunListener failed
java.lang.OutOfMemoryError: GC overhead limit exceeded
Feb 14, 2017 2:47:04 PM org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService reportProblem
WARNING: Unexpected exception in CPS VM thread: CpsFlowExecution[Owner[PipelinePipelines/pipeline-testcases---parallel-abuse180/198:PipelinePipelines/pipeline-testcases---parallel-abuse180 #198]]
java.lang.OutOfMemoryError: GC overhead limit exceeded
Feb 14, 2017 2:47:03 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:47:10 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:47:20 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:47:27 PM org.jenkinsci.plugins.workflow.job.WorkflowRun finish
INFO: PipelinePipelines/pipeline-testcases---parallel-abuse24 #198 completed: FAILURE
Feb 14, 2017 2:47:28 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:47:30 PM org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService reportProblem
WARNING: Unexpected exception in CPS VM thread: CpsFlowExecution[Owner[PipelinePipelines/pipeline-testcases---parallel-abuse180/198:PipelinePipelines/pipeline-testcases---parallel-abuse180 #198]]
java.lang.NullPointerException
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:41)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:165)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:328)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:240)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:228)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Feb 14, 2017 2:47:35 PM hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.model.Queue$MaintainTask@34463f11 failed
java.lang.OutOfMemoryError: GC overhead limit exceeded
Feb 14, 2017 2:47:46 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:47:55 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:48:00 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:48:06 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:48:14 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:48:21 PM hudson.model.AsyncPeriodicWork doRun
INFO: com.cloudbees.jenkins.plugin.metrics.views.Alerter thread is still running. Execution aborted.
Feb 14, 2017 2:48:24 PM org.jenkinsci.plugins.workflow.flow.FlowExecutionList load
WARNING: null
java.io.IOException: Unable to read /var/lib/jenkins/org.jenkinsci.plugins.workflow.flow.FlowExecutionList.xml
at hudson.XmlFile.read(XmlFile.java:144)
at org.jenkinsci.plugins.workflow.flow.FlowExecutionList.load(FlowExecutionList.java:99)
at org.jenkinsci.plugins.workflow.flow.FlowExecutionList.register(FlowExecutionList.java:112)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:233)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
|