[JIRA] (JENKINS-53158) Pipelines leak OneOffExecutors when triggered on a temporary offline master

400 views
Skip to first unread message

pibeitz@gmail.com (JIRA)

unread,
Aug 21, 2018, 7:13:02 AM8/21/18
to jenkinsc...@googlegroups.com
Pierre Beitz created an issue
 
Jenkins / Bug JENKINS-53158
Pipelines leak OneOffExecutors when triggered on a temporary offline master
Issue Type: Bug Bug
Assignee: Unassigned
Components: core, pipeline
Created: 2018-08-21 11:12
Environment: * Version: 2.121.3
* workflow-aggregator:2.5
* workflow-cps:2.54 'Pipeline: Groovy'
Priority: Major Major
Reporter: Pierre Beitz

When a Pipeline is triggered while the master node has been put temporarily offline, a OneOff (flyweight) executor is leaked.

As I'm unsure whether the issue lies in core or in the pipelines at put both components.

 

Environment

  • Version: 2.121.3
  • workflow-aggregator:2.5
  • workflow-cps:2.54 'Pipeline: Groovy'

 
Steps to reproduce

  • Create a simple Pipeline Job, eg:
node('master') {
    sh 'echo hello'
}
  • Put the master node temporary offline
  • Trigger the Pipeline build
  • Put the master node back online
  • Observe the leaked executor

 

Observations

This executor has the following characteristics:

Executor #-1(-1)
 Active? false
 Likely Stuck? false
 Interrupted? false
 Busy? false
 Owner? hudson.model.Hudson$MasterComputer@228777fd
 Current Work Unit? null
 Causes Of Interruption? []
 Idle Start Milliseconds? 1534847122799
 Progress: -1
 Asynchronous Execution: null
 Executable: null
 executorOwner: hudson.model.Hudson$MasterComputer@228777fd
  tiedJobs: []

Jenkins logs show:

Executor #-1 for master grabbed hudson.model.queue.WorkUnit@4891a81f[work=hello] from queue but master went off-line before the task's worker thread started. No termination trace available.

Pipeline logs show nothing:

Démarré par l'utilisateur admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
Running on Jenkins in /Users/pierrebeitz/cbsupporthome/jenkins-home/workspace/hello
[Pipeline] {
[Pipeline] sh
[hello] Running shell script
+ echo hello
hello
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Workaround

A restart of Jenkins will cleanup the executors.

 

 

 

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

pibeitz@gmail.com (JIRA)

unread,
Aug 21, 2018, 8:11:02 AM8/21/18
to jenkinsc...@googlegroups.com
Pierre Beitz updated an issue
Change By: Pierre Beitz
When a Pipeline is triggered while the master node has been put temporarily offline, a OneOff (flyweight) executor is leaked.

As I'm unsure whether the issue lies in core or in the pipelines at put both components.

 

*Environment*

* Version: 2.121.3
* workflow-aggregator:2.5
* workflow-cps:2.54
'Pipeline: Groovy'

 
*Steps to reproduce*
* Create a simple Pipeline Job, eg:

{code:java}

node('master') {
    sh 'echo hello'
}{code}
* Put the master node temporary offline
* Trigger the Pipeline build
* Put the master node back online
* Observe the leaked executor

 

*Observations*


This executor has the following characteristics:
{code:java}

Executor #-1(-1)
Active? false
Likely Stuck? false
Interrupted? false
Busy? false
Owner? hudson.model.Hudson$MasterComputer@228777fd
Current Work Unit? null
Causes Of Interruption? []
Idle Start Milliseconds? 1534847122799
Progress: -1
Asynchronous Execution: null
Executable: null
executorOwner: hudson.model.Hudson$MasterComputer@228777fd
  tiedJobs: []{code}
Jenkins logs show:
{noformat}
Executor #-1 for master grabbed hudson.model.queue.WorkUnit@4891a81f[work=hello] from queue but master went off-line before the task's worker thread started. No termination trace available.{noformat}
Pipeline logs show nothing:
{noformat}

Démarré par l'utilisateur admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
Running on Jenkins in /Users/pierrebeitz/cbsupporthome/jenkins-home/workspace/hello
[Pipeline] {
[Pipeline] sh
[hello] Running shell script
+ echo hello
hello
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS{noformat}
*Workaround*


A restart of Jenkins will cleanup the executors.

 

 

 

pibeitz@gmail.com (JIRA)

unread,
Aug 21, 2018, 8:11:02 AM8/21/18
to jenkinsc...@googlegroups.com
Pierre Beitz updated an issue
Change By: Pierre Beitz
Environment:
* Version: 2.121.3
* workflow-aggregator:2.5
* workflow-cps:2.54
'Pipeline: Groovy'

andrew.bayer@gmail.com (JIRA)

unread,
Aug 23, 2018, 11:35:01 AM8/23/18
to jenkinsc...@googlegroups.com
Andrew Bayer updated an issue
Change By: Andrew Bayer
Component/s: workflow-durable-task-step-plugin
Component/s: workflow-job-plugin
Component/s: pipeline

rcampbell@cloudbees.com (JIRA)

unread,
Jan 4, 2020, 4:10:02 PM1/4/20
to jenkinsc...@googlegroups.com
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

pibeitz@gmail.com (JIRA)

unread,
Jan 6, 2020, 5:29:02 AM1/6/20
to jenkinsc...@googlegroups.com

Still reproducible:

  • Jenkins 2.204.1
  • workflow-aggregator:2.6
  • workflow-cps:2.78

pibeitz@gmail.com (JIRA)

unread,
Jan 6, 2020, 5:54:03 AM1/6/20
to jenkinsc...@googlegroups.com

pibeitz@gmail.com (JIRA)

unread,
Jan 6, 2020, 5:55:03 AM1/6/20
to jenkinsc...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages