[JIRA] (JENKINS-60715) Fleet plugin throws java.util.concurrent.CancellationException and overprovisions

0 views
Skip to first unread message

alan.l.sparks@microfocus.com (JIRA)

unread,
Jan 9, 2020, 2:35:04 PM1/9/20
to jenkinsc...@googlegroups.com
Alan Sparks created an issue
 
Jenkins / Bug JENKINS-60715
Fleet plugin throws java.util.concurrent.CancellationException and overprovisions
Issue Type: Bug Bug
Assignee: Chad Schmutzer
Components: ec2-fleet-plugin
Created: 2020-01-09 19:34
Environment: jenkins 2.204.1
ec2-fleet 1.17.1
Priority: Major Major
Reporter: Alan Sparks

Using ec2-fleet plugin with autoscaling group.

Too often, with no slaves running, a job will get scheduled and invoke the ec2-fleet.  However, the Jenkins logs show the following traceback, and another instance startup will be started.  With this, I'm left with 2 slaves - the one started when the exception was thrown, plus another.  Is wasting money.  How to avoid triggering this exception?  Why is it happening?

2020-01-09 19:21:14.368+0000 [id=34] SEVERE hudson.slaves.NodeProvisioner#lambda$update$6: Unexpected uncaught exception encountered while processing agent FleetNode-0
java.util.concurrent.CancellationException: Task was cancelled.
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:295)
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:276)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)
at hudson.slaves.NodeProvisioner.lambda$update$6(NodeProvisioner.java:236)
at hudson.model.Queue._withLock(Queue.java:1392)
at hudson.model.Queue.withLock(Queue.java:1269)
at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:223)
at hudson.slaves.NodeProvisioner.access$900(NodeProvisioner.java:63)
at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:817)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:70)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2020-01-09 19:21:14.369+0000 [id=34] INFO c.a.j.e.NoDelayProvisionStrategy#apply: currentDemand 2 availableCapacity 0 (availableExecutors 0 connectingExecutors 0 plannedCapacitySnapshot 0 additionalPlannedCapacity 0)

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

alan.l.sparks@microfocus.com (JIRA)

unread,
Jan 16, 2020, 11:17:07 AM1/16/20
to jenkinsc...@googlegroups.com

alan.l.sparks@microfocus.com (JIRA)

unread,
Jan 16, 2020, 11:19:07 AM1/16/20
to jenkinsc...@googlegroups.com
Alan Sparks commented on Bug JENKINS-60715
 
Re: Fleet plugin throws java.util.concurrent.CancellationException and overprovisions

Any comments on this?  Have tried changing the timeout values for the init and SSH timeout, ,to no avail.  We still frequently get, say, 2 instances launched when one job is queued to run.  Is a significant resource waste.

Please advise what else I can provide, logs or configuration, to help determine where the problem is.

 

alan.l.sparks@microfocus.com (JIRA)

unread,
Jan 20, 2020, 2:36:07 PM1/20/20
to jenkinsc...@googlegroups.com

alan.l.sparks@microfocus.com (JIRA)

unread,
Jan 20, 2020, 2:37:07 PM1/20/20
to jenkinsc...@googlegroups.com

alan.l.sparks@microfocus.com (JIRA)

unread,
Jan 23, 2020, 11:33:04 AM1/23/20
to jenkinsc...@googlegroups.com
 
Re: Fleet plugin throws java.util.concurrent.CancellationException and overprovisions

Was able to get answer via GitHub issue: https://github.com/jenkinsci/ec2-fleet-plugin/issues/167

Was a bug in ec2-fleet plugin, fixed in version 1.17.3 of the plugin.

alan.l.sparks@microfocus.com (JIRA)

unread,
Jan 23, 2020, 11:34:03 AM1/23/20
to jenkinsc...@googlegroups.com
Alan Sparks closed an issue as Fixed
 

Was able to get answer via GitHub issue: https://github.com/jenkinsci/ec2-fleet-plugin/issues/167

Was a bug in ec2-fleet plugin, fixed in version 1.17.3 of the plugin.

Change By: Alan Sparks
Status: Open Closed
Resolution: Fixed
Reply all
Reply to author
Forward
0 new messages