Delay in exiting shell

20 views
Skip to first unread message

Gardell, Steven

unread,
May 21, 2017, 7:33:39 AM5/21/17
to jenkins...@googlegroups.com

We are using an imperative pipe with a set of parallel activities. One of them executes a shell on a slave node:

 

       withEnv(["CLIENT=$p4clientName","STAGING=$bldStaging"]) {sh '''#!/bin/sh -x

                date

                <do work>

echo "Return status from qcowbuild workon: $status"

                cp $ROOT/orca/qcowbuild.out $WORKSPACE/qcowbuild.out

                date

                 exit $status

                 '''

                 time=getTime()

                 println "Exit Shell: $time"

       }

 

Frequently, but not 100% consistently, I see a delay of about 40 minutes between the date output of the shell and what I print from groovy.

 

This seems really weird and I certain appreciate expected skepticism here, but I have this:

 

[qcow2] Return status from qcowbuild workon: 0

[qcow2] + cp /ssd-disks/ws/jenkinsbuild/sbxmain/orca/qcowbuild.out /tmp/workspace/sbxMain/qcowbuild.out

[qcow2] + date

[qcow2] Sat May 20 20:23:53 EDT 2017

[qcow2] + exit 0

< logging from parallel tasks>

[Pipeline] [qcow2] echo
[qcow2] Exit Shell: 9:05:28 PM
[Pipeline] [qcow2] }

 

We are currently at Jenkins 2.9 and Groovy pipeline 2.27. Everything is RH Linux 6.5 from a Jenkins master/slave perspective. I don’t think our Jenkins master is very loaded since we only have a handful of active jobs and all of the real work happens on slaves. This slave was also not very loaded at the time.

 

Any suggestions?

 

Any significance to the placement of the “[Pipeline] [qcow2] echo” statement?

 

 

 

Stephen Connolly

unread,
May 21, 2017, 7:36:50 AM5/21/17
to jenkins...@googlegroups.com
On Sun 21 May 2017 at 12:33, Gardell, Steven <sgar...@sonusnet.com> wrote:

We are using an imperative pipe with a set of parallel activities. One of them executes a shell on a slave node:

 

       withEnv(["CLIENT=$p4clientName","STAGING=$bldStaging"]) {sh '''#!/bin/sh -x

                date

                <do work>

echo "Return status from qcowbuild workon: $status"

                cp $ROOT/orca/qcowbuild.out $WORKSPACE/qcowbuild.out

                date

                 exit $status

                 '''

                 time=getTime()

                 println "Exit Shell: $time"

       }

 

Frequently, but not 100% consistently, I see a delay of about 40 minutes between the date output of the shell and what I print from groovy.


So you do know that the shell executes on the build agent but the groovy executes on the master?

If the build agent clocks are skewed w.r.t. the master then that could explain the behaviour 

 

This seems really weird and I certain appreciate expected skepticism here, but I have this:

 

[qcow2] Return status from qcowbuild workon: 0

[qcow2] + cp /ssd-disks/ws/jenkinsbuild/sbxmain/orca/qcowbuild.out /tmp/workspace/sbxMain/qcowbuild.out

[qcow2] + date

[qcow2] Sat May 20 20:23:53 EDT 2017

[qcow2] + exit 0

< logging from parallel tasks>

[Pipeline] [qcow2] echo
[qcow2] Exit Shell: 9:05:28 PM
[Pipeline] [qcow2] }

 

We are currently at Jenkins 2.9 and Groovy pipeline 2.27. Everything is RH Linux 6.5 from a Jenkins master/slave perspective. I don’t think our Jenkins master is very loaded since we only have a handful of active jobs and all of the real work happens on slaves. This slave was also not very loaded at the time.

 

Any suggestions?

 

Any significance to the placement of the “[Pipeline] [qcow2] echo” statement?

 

 

 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/BY2PR0301MB1991310237F1A52A2FE7626CB8FB0%40BY2PR0301MB1991.namprd03.prod.outlook.com.
For more options, visit https://groups.google.com/d/optout.
--
Sent from my phone

Gardell, Steven

unread,
May 21, 2017, 7:47:34 AM5/21/17
to jenkins...@googlegroups.com

Thanks for the quick response! I guess my understanding is probably imperfect. I thought that groovy within a slave execution node block (not shown for brevity) was actually executing on the slave itself. At any rate our environment is 100% NTP. No meaningful clock skew – and this behavior is not 100% consistent. Maybe 50% of the time??

Reply all
Reply to author
Forward
0 new messages