Re:Stop EC2 slave instances

[Kevin Fleming (BLOOMBERG/ 731 LEXIN)]

Yes, a Groovy script could iterate over the current list of Jenkins nodes, determine which ones are EC2 slaves, and then delete them. This would terminate the EC2 instances for those slaves.

----- Original Message -----
From: jenkins...@googlegroups.com
To: jenkins...@googlegroups.com
At: Jan 21 2014 12:24:52

Our Jenkins master server runs on an EC2 instance and we stop it twice a day to switch between a small and medium instance to reduce our bills during off hours. Our build slaves are EC2 instances using the Jenkins EC2 plugin. To keep the builds fast we have it configured to only stop the slaves rather than terminate them.

We are finding that when the master machine stops and restarts when an EC2 slave is running, the EC2 instance stays running and the master slave is never able to re-establish a connection with it. It still has the instance ID correct. But, it is unable to connect. This requires manual intervention of terminating the old instances and starting new ones.

Is there some way we can properly stop the EC2 instances using a Jenkins Groovy script before shutting down the Jenkins instance? Hopefully this way the EC2 plugin will correctly re-establish the connection on start-up. We have a Jenkins job which controls the timing of the shutdown process.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[David V]

I would like to stop them, not terminate them. Is there a proper way to stop the instance from a Groovy script?

[Kevin Fleming (BLOOMBERG/ 731 LEXIN)]

Are these slaves running active jobs, or are they idle (from Jenkins point of view)?

[David V]

Presently, they have been idle when the shutdown occurs. I'm working on updating the shutdown process to try to ensure that it waits until all jobs are complete to enforce this.

[Kevin Fleming (BLOOMBERG/ 731 LEXIN)]

Ahh... well that makes sense, but I'm fairly certain that the EC2 plugin doesn't currently offer any way to initiate the 'timeout' process that results in an instance being stopped (as opposed to terminated). It's possible that the existing function that handles timeouts could be called from a Groovy script, but this would require some experimentation.

If you start by building a testing a script that can enumerate the Jenkins nodes that are EC2 slaves, I can help you try to find the next steps.

[David V]

Hey Kevin. Thanks for your help. The following script prints out the EC2 slaves and computers (it does not provide an actual description)

jenkins = Jenkins.instance;

for (slave in jenkins.slaves.findAll({s -> s instanceof hudson.plugins.ec2.EC2OndemandSlave})) {

  println slave

  computer = slave.computer

  println computer

}

The result on my Jenkins server.

hudson.plugins.ec2.EC2OndemandSlave@f39f2278
hudson.plugins.ec2.EC2Computer@1532744
hudson.plugins.ec2.EC2OndemandSlave@d935070e
hudson.plugins.ec2.EC2Computer@13bae09

[David V]

What do you think about exposing the stop() method? I found it in the source on GitHub, but I've only just begun to take a look at the source code for this project.

void stop() {
        try {
            AmazonEC2 ec2 = getCloud().connect();
            StopInstancesRequest request = new StopInstancesRequest(
                    Collections.singletonList(getInstanceId()));
            ec2.stopInstances(request);
            LOGGER.info("EC2 instance stopped: " + getInstanceId());
            toComputer().disconnect(null);
        } catch (AmazonClientException e) {
            Instance i = getInstance(getInstanceId(), getCloud());
            LOGGER.log(Level.WARNING, "Failed to terminate EC2 instance: "+getInstanceId() + " info: "+((i != null)?i:"") , e);
        }
    }