| For reasons I don't understand, my installation produces a number of orphaned docker containers every day or so (although most of the time everything works). There are a number of variants, this issue concerns this one:
The retry continues forever, tying up the Docker swarm. I have to kill and remove the containers manually. I haven't been able to catch the master logs that correspond to this yet, since this happens only occasionally and it's a busy installation, so I don't know why the master doesn't know about the slave it's launching - perhaps a job abort before it is provisioned, or just a job abort at any point. I have the Cloud Connection Timeout and Read Timeout set to 10, and the individual template timeouts set, but of course these do not affect the operation of the JNLP jar. The only option I see from the jar is -noReconnect, which would give up on the first 404 in this case. I don't know with complete certainty that this would work for everyone, so perhaps it would be helpful to make it an option? Outside of this, I am left with having to rig up hacky monitoring and kill-old-containers scripts. |