[JIRA] (JENKINS-38942) Stopped EC2 agents not waking up if the number of stopped EC2 agents equals the global Instance Cap

[darin@planetpope.com (JIRA)]

Darin Pope created an issue

Jenkins / JENKINS-38942 Stopped EC2 agents not waking up if the number of stopped EC2 agents equals the global Instance Cap Issue Type: Bug Assignee: Francis Upton Components: ec2-plugin Created: 2016/Oct/12 9:57 PM Environment: ec2-plugin 1.34; aws-java-sdk 1.10.50 Priority: Minor Reporter: Darin Pope Steps to reproduce: set the global Instance Cap to 5 (or some other number) check the box for "Stop/Disconnect on Idle Timeout" so the instances stop instead of terminate set the "Idle termination time" to a low number so you don't have to wait too long after the jobs complete run a few jobs so you have some EC2 agents available to be stopped give some time for the EC2 agents to idle and stop change the global Instance Cap to equal the number of stopped EC2 agents Queue up some jobs to run again. The EC2 agents will not wake up and the following entry is in the log: hudson.plugins.ec2.EC2Cloud#provisionSlaveIfPossible: Cannot provision - no capacity for instances: -1 Variation on the issue: If you change the global Instance Cap to <number of stopped EC2 agents> + 1 and you start running your jobs, the stopped EC2 agents will wake up and the jobs will run on them, but another new EC2 agent will not be created/started. Add Comment

This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)

[timmy.leahy@gmail.com (JIRA)]

Timmy Leahy commented on JENKINS-38942

Re: Stopped EC2 agents not waking up if the number of stopped EC2 agents equals the global Instance Cap I'm also experiencing this. We normally bump up against our configured instance cap when rolling out and testing newly provisioned instances. We ran into some delays and just noticed that our job queue kept growing and wasn't waking up any inactive instances. We just updated our configured instance cap and then it started activating instances. {{Dec 22, 2016 11:48:38 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting to provision slave from template hudson.plugins.ec2.SlaveTemplate@3f079942 needed by excess workload of 1 units of label 'linux_test' Dec 22, 2016 11:48:38 PM hudson.plugins.ec2.EC2Cloud getNewOrExistingAvailableSlave INFO: Cannot provision - no capacity for instances: -1 Dec 22, 2016 11:48:38 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 }}

Add Comment

[timmy.leahy@gmail.com (JIRA)]

Timmy Leahy edited a comment on JENKINS-38942

Re: Stopped EC2 agents not waking up if the number of stopped EC2 agents equals the global Instance Cap

I'm also experiencing this. We normally bump up against our configured instance cap when rolling out and testing newly provisioned instances. We ran into some delays and just noticed that our job queue kept growing and wasn't waking up any inactive instances. We just updated our configured instance cap and then it started activating instances.

here's a sample from the logs { { code:java}

Dec 22, 2016 11:48:38 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting to provision slave from template hudson.plugins.ec2.SlaveTemplate@3f079942 needed by excess workload of 1 units of label 'linux_test' Dec 22, 2016 11:48:38 PM hudson.plugins.ec2.EC2Cloud getNewOrExistingAvailableSlave INFO: Cannot provision - no capacity for instances: -1 Dec 22, 2016 11:48:38 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1

{code } }

Add Comment

[fabrizio.manfredi@gmail.com (JIRA)]

FABRIZIO MANFREDI resolved as Fixed

Jenkins / JENKINS-38942

Stopped EC2 agents not waking up if the number of stopped EC2 agents equals the global Instance Cap

Change By: FABRIZIO MANFREDI Status: Open Resolved Resolution: Fixed Released As: 1.44.1

Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)