[JIRA] (JENKINS-17116) graceful job termination

18 views
Skip to first unread message

batmat@batmat.net (JIRA)

unread,
Feb 14, 2019, 6:30:06 AM2/14/19
to jenkinsc...@googlegroups.com
Baptiste Mathus updated an issue
 
Jenkins / Bug JENKINS-17116
graceful job termination
Change By: Baptiste Mathus
Summary: gracefull graceful job termination
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

osmith@sysmocom.de (JIRA)

unread,
May 9, 2019, 5:52:05 AM5/9/19
to jenkinsc...@googlegroups.com
Oliver Smith commented on Bug JENKINS-17116
 
Re: graceful job termination

I have created the following demo script:

#!/bin/sh -ex

trap cleanup "TERM"
set +x

cleanup() {
	echo "Caught signal, cleaning up..."
	exit 1
}

echo "Sleeping..."

while true; do
	sleep 0.1
done

# should not get here due to while true
echo "EOF"

When running in a terminal without jenkins, it catches the signal as expected (e.g. with "pkill -TERM trapscript.sh"):

$ ./trapscript.sh
+ trap cleanup TERM
+ set +x
Sleeping...
Caught signal, cleaning up...

On Jenkins 2.150.2, it does not run the cleanup function:

[TEST_trap_in_jenkins_job] $ /bin/sh -ex /tmp/jenkins5365212366501463498.sh
+ trap cleanup TERM
+ set +x
Sleeping...
Build was aborted
Aborted by Oliver Smith
Terminated
Finished: ABORTED

The server is configured to run all jobs on nodes, so this might be the same problem that Martin d'Anjou pointed out above:
when it runs on a node, the signal is not received (or not sent?).

It would be great if somebody could look into this, thanks!

martin.danjou14@gmail.com (JIRA)

unread,
May 9, 2019, 6:11:11 AM5/9/19
to jenkinsc...@googlegroups.com
Martin d'Anjou commented on Bug JENKINS-17116
 
Re: graceful job termination

Pham Vu Tuan do you know how we could debug the communication between master and agents? It seems like the unix kill signal is not sent or received by the agent.

owen@nerdnetworks.org (JIRA)

unread,
May 9, 2019, 9:35:04 PM5/9/19
to jenkinsc...@googlegroups.com
Owen Mehegan assigned an issue to Unassigned
Change By: Owen Mehegan
Assignee: Robin Jarry

owen@nerdnetworks.org (JIRA)

unread,
May 9, 2019, 9:38:06 PM5/9/19
to jenkinsc...@googlegroups.com

uwe.teichmann@mailbox.org (JIRA)

unread,
Oct 6, 2019, 11:44:04 AM10/6/19
to jenkinsc...@googlegroups.com
Uwe Teichmann commented on Bug JENKINS-17116
 
Re: graceful job termination

I'm using Jenkins 2.198 and JRE  jre-1.8.0-openjdk. I created a free style job based on the script by Oliver Smith. The Build Step defined as:

sudo su - oracle<<eof
./shell_signal_handling
eof

If i cancel the job the process and it's child process get killed correctly, but the console output is:

Gestartet durch Benutzer Uwe Teichmann
Running as SYSTEM
Baue auf dem Agenten „elendil“ (oracle) in Arbeitsbereich /vorlons/jenkins/elendil/workspace/Training/FreeStyle/Shell_Signal_Handling
[Shell_Signal_Handling] $ /bin/bash -xe /tmp/jenkins10950746844547967776.sh
+ sudo su - oracle
Sleeping...
Build wurde abgebrochen
Abgebrochen von Uwe Teichmann
Finished: ABORTED

If i change the Build Step to

sudo su - oracle<<eof
./shell_signal_handling 1>./shell_signal_handling.log 2>&1
eof

the log file contains

Sleeping...
Caught signal, cleaning up...

My conclusion: The behaviour is correct, because:

  • The SIGTERM results in the parent process and his childs to stop and exit correctly.
  • The output of the shell script appears in the console job output as long as the connection is active. When killed, the child processes can't send there output anymore to the parent. To see there output we need the log file.
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

martin.danjou14@gmail.com (JIRA)

unread,
Oct 6, 2019, 5:35:04 PM10/6/19
to jenkinsc...@googlegroups.com
Martin d'Anjou commented on Bug JENKINS-17116
 
Re: graceful job termination

Uwe Teichmann are you running the job on the master or on a remote node? The problem manifests itself when the job runs on a remote node.

uwe.teichmann@mailbox.org (JIRA)

unread,
Oct 7, 2019, 12:33:06 AM10/7/19
to jenkinsc...@googlegroups.com
Uwe Teichmann commented on Bug JENKINS-17116
 
Re: graceful job termination

The test was done on a local machine, where Jenkins Master and Slave communicate via JNLP.

uwe.teichmann@mailbox.org (JIRA)

unread,
Oct 7, 2019, 2:26:06 AM10/7/19
to jenkinsc...@googlegroups.com
 
Re: graceful job termination

At work we use Jenkins 2.138.1. The problem can be reproduced for both freestyle jobs and pipelines. The testcase is performed between two different servers using SLES 12.4, OpenJDP 1.8.0_191-b12.

marat@slonopotamus.org (JIRA)

unread,
Oct 7, 2019, 2:39:15 AM10/7/19
to jenkinsc...@googlegroups.com
Change By: Marat Radchenko
Status: Fixed but Unreleased Resolved

marat@slonopotamus.org (JIRA)

unread,
Oct 7, 2019, 2:39:16 AM10/7/19
to jenkinsc...@googlegroups.com
Change By: Marat Radchenko
Status: In Review Fixed but Unreleased
Resolution: Fixed
Released As: 2.141
Reply all
Reply to author
Forward
0 new messages