[JIRA] [workflow-plugin] (JENKINS-31314) Running asynchronous code inside a @NonCPS method should fail cleanly

1 view
Skip to first unread message

jglick@cloudbees.com (JIRA)

unread,
Oct 30, 2015, 9:43:03 AM10/30/15
to jenkinsc...@googlegroups.com
Jesse Glick created an issue
 
Jenkins / Bug JENKINS-31314
Running asynchronous code inside a @NonCPS method should fail cleanly
Issue Type: Bug Bug
Assignee: Kohsuke Kawaguchi
Components: workflow-plugin
Created: 30/Oct/15 1:42 PM
Labels: diagnostics groovy
Priority: Major Major
Reporter: Jesse Glick

Currently if you run a step with asynchronous execution (which is to say, almost all of them, since AbstractSynchronousNonBlockingStepExecution counts) inside a method marked @NonCPS, or otherwise not transformed (for example because of JENKINS-26481), you get bizarre results: the return value of the step becomes the return value of the method, regardless of what was supposed to happen in the rest of the method. This is because CpsCallableInvocation is thrown from DSL.invokeMethod and interrupts the method's execution.

This is very confusing behavior. It would be better for CpsCallableInvocation.<init> to somehow detect that it is inside an untransformed method and throw a comprehensible error.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

scm_issue_link@java.net (JIRA)

unread,
Nov 3, 2015, 11:33:01 AM11/3/15
to jenkinsc...@googlegroups.com
SCM/JIRA link daemon commented on Bug JENKINS-31314
 
Re: Running asynchronous code inside a @NonCPS method should fail cleanly

Code changed in jenkins
User: Jesse Glick
Path:
aggregator/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
http://jenkins-ci.org/commit/workflow-plugin/59e6cf394ba79dea4db3dd2b99b7cca891341f16
Log:
Merge pull request #233 from jglick/test-JENKINS-31314

JENKINS-31314 Demonstrating problem in test

Compare: https://github.com/jenkinsci/workflow-plugin/compare/1b83632c5b8d...59e6cf394ba7

scm_issue_link@java.net (JIRA)

unread,
Nov 3, 2015, 11:33:01 AM11/3/15
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Jesse Glick
Path:
aggregator/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java

http://jenkins-ci.org/commit/workflow-plugin/e5d9b321b7490d811cf3db5e4e93630c2f912412
Log:
JENKINS-31314 Demonstrating problem in test.
The sample flow cannot possibly work, yet it runs without errors, just weird behavior.

scm_issue_link@java.net (JIRA)

unread,
Apr 5, 2016, 4:31:35 PM4/5/16
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Jesse Glick
Path:
aggregator/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java


Log:
JENKINS-31314 Demonstrating problem in test.
The sample flow cannot possibly work, yet it runs without errors, just weird behavior.

Originally-Committed-As: e5d9b321b7490d811cf3db5e4e93630c2f912412

scm_issue_link@java.net (JIRA)

unread,
Apr 5, 2016, 4:31:39 PM4/5/16
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Jesse Glick
Path:
aggregator/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java


Log:
Merge pull request #233 from jglick/test-JENKINS-31314

JENKINS-31314 Demonstrating problem in test

Originally-Committed-As: 59e6cf394ba79dea4db3dd2b99b7cca891341f16

thomas.goeppel@gmail.com (JIRA)

unread,
May 2, 2016, 1:50:01 AM5/2/16
to jenkinsc...@googlegroups.com

Thanks for the first explanation of what is goes wrong when I try writing any non-trivial CPS script. Usually I'm prepared to do the following:

1. Define @NonCPS functions at the top level of the script, and only call them from the master node
2. Use data processing in CPS very carefully on any non-master node (e.g. no closures, not even a regexp for string manipulation)
3. Expect any feature of Groovy to fail at any time.

Needless to say, that I have a hard time explaining people how to solve a problem with CPS, even if my solutions demonstrate that it can work. Maybe it would be easier to raise the priority of such issues if CPS didn't work at all.

jglick@cloudbees.com (JIRA)

unread,
Aug 10, 2016, 8:43:01 AM8/10/16
to jenkinsc...@googlegroups.com

only call them from the master node

All Pipeline scripts runs inside the master process. node blocks have no effect on this whatsoever. They only control where nested sh steps and the like connect to.
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

jglick@cloudbees.com (JIRA)

unread,
Aug 29, 2016, 2:24:06 PM8/29/16
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
 
Change By: Jesse Glick
Component/s: workflow-cps-plugin
Component/s: pipeline

ryan.campbell@gmail.com (JIRA)

unread,
Sep 2, 2016, 12:02:01 PM9/2/16
to jenkinsc...@googlegroups.com
recampbell updated an issue
Change By: recampbell
Labels: diagnostics groovy kohsuke-plane-project

jglick@cloudbees.com (JIRA)

unread,
Feb 28, 2018, 11:18:02 AM2/28/18
to jenkinsc...@googlegroups.com
 
Re: Running asynchronous code inside a @NonCPS method should fail cleanly

perhaps just print a warning

This is the most attractive option since any mistakes would not result in a serious regression, just a misleading message. I think this could be accomplished with the help of a new method PrintStream Invoker.getLog() which CpsFlowExecution.start could delegate to owner.getListener().getLogger(), thus making messages appear in the build log (outside the scope of any particular step).
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

andrew.bayer@gmail.com (JIRA)

unread,
Feb 28, 2018, 2:55:03 PM2/28/18
to jenkinsc...@googlegroups.com

Jesse Glick - so, I'm lost. DSL.invokeStep is only happening for actual Step implementations, right? So any other CPS-transformed code never calls Continuable.suspend. And CpsClosure throws CpsCallableInvocation from its call and doCall methods. Given that most of the problem cases are actually CpsClosure instances throwing CpsCallableInvocation in non-CPS contexts, how would any of what you said work there?

jglick@cloudbees.com (JIRA)

unread,
Mar 9, 2018, 5:05:05 PM3/9/18
to jenkinsc...@googlegroups.com

Not sure. Linked to a sketch of doing the detection for the originally suggested case. Definitely there are others, and those might be harder to handle—I am not sure. Maybe you can figure something out from these hints.

jglick@cloudbees.com (JIRA)

unread,
Mar 13, 2018, 4:00:09 PM3/13/18
to jenkinsc...@googlegroups.com
Jesse Glick started work on Bug JENKINS-31314
 
Change By: Jesse Glick
Status: Open In Progress

jglick@cloudbees.com (JIRA)

unread,
Mar 13, 2018, 4:00:09 PM3/13/18
to jenkinsc...@googlegroups.com
Jesse Glick assigned an issue to Jesse Glick
 
Change By: Jesse Glick
Assignee: Jesse Glick

jglick@cloudbees.com (JIRA)

unread,
Mar 13, 2018, 5:13:03 PM3/13/18
to jenkinsc...@googlegroups.com
 
Re: Running asynchronous code inside a @NonCPS method should fail cleanly

Seems to be catching the cases I would want it to catch. For now it is catching too much, and it will be some work to prune the warnings back to legitimate cases.

andrew.bayer@gmail.com (JIRA)

unread,
Mar 13, 2018, 5:32:02 PM3/13/18
to jenkinsc...@googlegroups.com

I’ll take a look tomorrow.

vivek.pandey@gmail.com (JIRA)

unread,
Nov 15, 2018, 5:30:05 PM11/15/18
to jenkinsc...@googlegroups.com
Vivek Pandey updated an issue
 
Change By: Vivek Pandey
Labels: diagnostics groovy kohsuke-plane-project triaged-2018-11
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

yngvark@gmail.com (JIRA)

unread,
Feb 17, 2019, 3:21:05 AM2/17/19
to jenkinsc...@googlegroups.com

jglick@cloudbees.com (JIRA)

unread,
Feb 18, 2019, 1:23:02 PM2/18/19
to jenkinsc...@googlegroups.com

No, I had a prototype but stopped work on this.

jglick@cloudbees.com (JIRA)

unread,
Apr 23, 2019, 9:30:06 PM4/23/19
to jenkinsc...@googlegroups.com
Change By: Jesse Glick
Status: In Progress Review

dnusbaum@cloudbees.com (JIRA)

unread,
Jun 17, 2019, 12:13:06 PM6/17/19
to jenkinsc...@googlegroups.com
Change By: Devin Nusbaum
Status: In Review Fixed but Unreleased
Resolution: Fixed

dnusbaum@cloudbees.com (JIRA)

unread,
Jul 5, 2019, 5:07:03 PM7/5/19
to jenkinsc...@googlegroups.com
 

A fix for this issue was released in Pipeline: Groovy Plugin version 2.71.
Change By: Devin Nusbaum
Status: Fixed but Unreleased Resolved
Released As: workflow-cps 2.71

fwilhelm.wgt@gmail.com (JIRA)

unread,
Jul 12, 2019, 5:09:03 AM7/12/19
to jenkinsc...@googlegroups.com
Florian Wilhelm commented on Bug JENKINS-31314
 
Re: Running asynchronous code inside a @NonCPS method should fail cleanly

Hi Devin Nusbaum, I’m afraid this change broke our Jenkinsfile runner based testing because the warnings will be thrown as an exception there. Reproducer here https://github.com/fwilhe/repro-cps-issue

I know we have to fix this in a proper way, but is there some way to turn it into warnings also on Jenkinsfile runner and not have it throw an exception?

jglick@cloudbees.com (JIRA)

unread,
Jul 12, 2019, 9:11:04 AM7/12/19
to jenkinsc...@googlegroups.com

Florian Wilhelm JFR is being tracked in JENKINS-58407.

sparshev@griddynamics.com (JIRA)

unread,
Nov 1, 2019, 3:24:08 PM11/1/19
to jenkinsc...@googlegroups.com
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

sparshev@griddynamics.com (JIRA)

unread,
Nov 1, 2019, 3:24:08 PM11/1/19
to jenkinsc...@googlegroups.com
Sergei Parshev closed an issue as Fixed
 
Change By: Sergei Parshev
Status: Resolved Closed

sparshev@griddynamics.com (JIRA)

unread,
Nov 1, 2019, 3:25:04 PM11/1/19
to jenkinsc...@googlegroups.com
Sergei Parshev reopened an issue
 

Sorry, closed by mistake
Change By: Sergei Parshev
Resolution: Fixed
Status: Closed Reopened

sparshev@griddynamics.com (JIRA)

unread,
Nov 1, 2019, 3:26:06 PM11/1/19
to jenkinsc...@googlegroups.com
Sergei Parshev updated an issue
Change By: Sergei Parshev
Comment:

jglick@cloudbees.com (JIRA)

unread,
Nov 1, 2019, 3:42:10 PM11/1/19
to jenkinsc...@googlegroups.com
Jesse Glick resolved as Fixed
Change By: Jesse Glick
Status: Reopened Resolved
Resolution: Fixed
Reply all
Reply to author
Forward
0 new messages