Dynamic Slaves Extension?

[Chris Drake]

Hi all,

Jenkins newbie here. Looking for some advice.

I would like the ability to:

1. As the build queue grows, dynamically create slaves.

2. The slaves are just terminals on the host machine.

3. Slaves source a script before doing any work (preferable to using ~/.bashrc).

4. Slaves are deleted as build queue shrinks.

I have an external batch scheduler (think GridEngine/LSF/Torque/SLURM/etc), and I would like the slave to just be a terminal that can execute batch jobs using the "interactive" option. Obviously, the terminal might wait around a while until the external batch scheduler gets around to executing the job.

I have looked at the Mesos Cloud plugin, and the LSF Cloud plugin. It seems the "cloud" abstraction is useful to me, but those two extensions use JNLP and SSH slaves. I just want the slave to be a terminal, but I don't understand the API well enough.

Can someone please point me in the right direction? Maybe this plugin already exists, and I just can't find it?

Regards,

Chris

[Mark Waite]

I think it depends what you mean by "terminal".  

If by terminal you mean "a Linux or FreeBSD or MacOS or OpenBSD shell evaluating commands", then running one slave with ssh and enabling multiple executors will do what you need.  Each executor can run shell scripts, independent of other executors.  No need to shrink the number of executors, since idle executors are very low overhead.

If you find that you don't have enough compute power on the machine hosting the slave, you can add more machines, each with a slave on the machine, and several executors available.  Then you can assign labels to the slaves and associate jobs with labels so that the jobs run on the type of slave which meets the need of that job.  

If by terminal you mean "a Windows command prompt evaluating commands", then running one slave on your Windows machine and enabling multiple executors will probably do what you need.

If by terminal you mean "a 3270 emulator connected to a mainframe", then I'm not sure there is a readily available solution.  

Thanks,

Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/55fd832f-3692-4285-87e0-45cbac89da1d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Drake]

Yes, I mean a Linux shell evaluating commands.

Looking into the details of using an SSH slave. Thanks for the pointer.