[JIRA] (JENKINS-58407) Fail to call a Closure stored as a class attribute/field

[dicomj23@gmail.com (JIRA)]

Dicom J created an issue

Jenkins / JENKINS-58407 Fail to call a Closure stored as a class attribute/field Issue Type: Bug Assignee: Unassigned Components: workflow-cps-plugin Created: 2019-07-09 16:59 Priority: Blocker Reporter: Dicom J simple groovy code, like this class SomeStruct { def aPointerToClosure } node { def st = new SomeStruct() st.aPointerToClosure = { println 'foobar' } st.aPointerToClosure() } and even within groovy library, started yelling the following exception: java.lang.IllegalStateException: expected to call SomeStruct.aPointerToClosure but wound up catching org.jenkinsci.plugins.workflow.cps.CpsClosure2.call; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/ at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.handleMismatch(CpsVmExecutorService.java:117) at The issue's appears as of 2.71 (05 Jul 2019), with 2.70 works properly. [Changelog|https://plugins.jenkins.io/workflow-cps], among other things, states the following: Improvement: Print detailed warnings to the build log when CPS-transformed code is called in a non-CPS context where possible. The warnings link to https://jenkins.io/redirect/pipeline-cps-method-mismatches/ which gives additional context and some examples of how to fix common issues. (JENKINS-31314)   Most likely, this is that "nice improvement"... Whole Stack Trace:   11.906 [id=48] WARNING o.j.p.w.cps.CpsVmExecutorService#reportProblem: Unexpected exception in CPS VM thread: CpsFlowExecution[Owner[job/1:job #1]]  11.906 [id=48] WARNING o.j.p.w.cps.CpsVmExecutorService#reportProblem: Unexpected exception in CPS VM thread: CpsFlowExecution[Owner[job/1:job #1]]java.lang.IllegalStateException: expected to call SomeStruct.aPointerToClosure but wound up catching org.jenkinsci.plugins.workflow.cps.CpsClosure2.call; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/ at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.handleMismatch(CpsVmExecutorService.java:117) at com.cloudbees.groovy.cps.impl.CpsCallableInvocation.checkMismatch(CpsCallableInvocation.java:63) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:94) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:186) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:370) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:93) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:282) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:270) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:66) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[dicomj23@gmail.com (JIRA)]

Dicom J updated an issue

Jenkins / JENKINS-58407 Fail to call a Closure stored as a class attribute/field

Change By: Dicom J

simple groovy code, like this

{code:java}

class SomeStruct {     def aPointerToClosure } node {     def st = new SomeStruct()     st.aPointerToClosure = { println 'foobar' }     st.aPointerToClosure()

} {code}

and even within groovy library, started yelling the following exception:

{code:java} java.lang.IllegalStateException: expected to call SomeStruct.aPointerToClosure but wound up catching org.jenkinsci.plugins.workflow.cps.CpsClosure2.call; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/ at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.handleMismatch(CpsVmExecutorService.java:117) at {code}

The issue's appears as of 2.71 (05 Jul 2019), with 2.70 works properly.

[ [ Changelog ](https://plugins.jenkins.io/workflow-cps) | [ https://plugins.jenkins.io/workflow-cps] ] , among other things, states the following:

Improvement: Print detailed warnings to the build log when CPS-transformed code is called in a non-CPS context where possible. The warnings link to [https://jenkins.io/redirect/pipeline-cps-method-mismatches/] which gives additional context and some examples of how to fix common issues. (JENKINS-31314)   Most likely, this is that "nice improvement"... Whole Stack Trace:

{code:java}   11.906 [id=48] WARNING o.j.p.w.cps.CpsVmExecutorService#reportProblem: Unexpected exception in CPS VM thread: CpsFlowExecution[Owner[job/1:job #1]]  11.906 [id=48] WARNING o.j.p.w.cps.CpsVmExecutorService#reportProblem: Unexpected exception in CPS VM thread: CpsFlowExecution[Owner[job/1:job #1]]java.lang.IllegalStateException: expected to call SomeStruct.aPointerToClosure but wound up catching org.jenkinsci.plugins.workflow.cps.CpsClosure2.call; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/ at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.handleMismatch(CpsVmExecutorService.java:117) at com.cloudbees.groovy.cps.impl.CpsCallableInvocation.checkMismatch(CpsCallableInvocation.java:63) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:94) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:186) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:370) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:93) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:282) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:270) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:66) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) {code}

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J assigned an issue to Devin Nusbaum

Jenkins / JENKINS-58407 Fail to call a Closure stored as a class attribute/field

Change By: Dicom J Assignee: Devin Nusbaum

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field Devin Nusbaum, assigning this to you as you are the one reported in  https://issues.jenkins-ci.org/browse/JENKINS-31314 that most likely this what I consider as a bug (which I think has been introduces as a part of what you think is a fix for something else (JENKINS-31314)) was released in Groovy Plugin version 2.71. Could you please properly address this, and ASAP since it's breaks fundamental things...

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

This code is only supposed to throw an exception when running plugin unit tests, see the code here (FAIL_ON_MISMATCH is set to Main.isUnitTest). In what context did you see the error? I will look into your reported case to understand exactly what is happening.

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum edited a comment on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

This code is only supposed to throw an exception when running plugin unit tests and log a warning (with a stack trace) otherwise , see [the code here|https://github.com/jenkinsci/workflow-cps-plugin/blob/609bc51d4cca65e8c851cdbc05a3f0efd8d60638/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java#L116-L124] ({{FAIL_ON_MISMATCH}} is set to {{Main.isUnitTest}}). In what context did you see the error?

I will look into your reported case to understand exactly what is happening.

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum edited a comment on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

This code is only supposed to throw an exception when running plugin unit tests and log a warning (with a stack trace) otherwise, see [the code here|https://github.com/jenkinsci/workflow-cps-plugin/blob/609bc51d4cca65e8c851cdbc05a3f0efd8d60638/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java#L116-L124] ({{FAIL_ON_MISMATCH}} is set to {{Main.isUnitTest}}). In what context did you see the error?

I will look into your reported case to understand exactly what is happening.

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

When I run your example inside of the sandbox, I get the following error in both 2.70 and 2.71: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use method groovy.lang.GroovyObject invokeMethod java.lang.String java.lang.Object (SomeStruct aPointerToClosure) I don't think it's a good idea to try to whitelist that method, but if I run the Pipeline outside of the sandbox, it works correctly in both cases. In 2.71, a warning is printed, which looks like a false positive picked up by JENKINS-31314 (CC Jesse Glick), but that shouldn't be fatal (outside of unit tests).

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Devin Nusbaum, you can compose a Jenkinsfile with exact code from my example, it's how a I tested, the only difference might be that I'm running this Jenkinsfile with https://github.com/jenkinsci/jenkinsfile-runner But my understanding is that it shouldn't affect in any way. Just make sure you have Groovy Plugin version 2.71. Thanks for such prompt reply

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Ah, that explains it, jenkinsfile-runner actually sets Main.isUnitTest = true when it runs a Jenkinsfile, because it is based on the Jenkins Test Harness, see here. Also see JENKINS-55764.

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Ok, so how do I make jenkinsfile-runner not to set this into true, as I can see it's hardcoded in there ? since runner doc (https://github.com/jenkinsci/jenkinsfile-runner#jenkinsfile-runner) states that it could be used for Use Jenkins in Function-as-a-Service context My goal is to use it as a function and not mixing with testing of Jenkins itself...

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Devin Nusbaum, Can this value Main.isUnitTest = true for runner come and so be controlled from some external configuration or at least environment variable ?

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

For now, there is no way to do that. I'm checking to see if there is some way we can detect that jenkinsfile-runner is being used and use that in addition to Main.isUnitTest so that jenkinsfile-runner does not get the strict behavior.

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Yes I see (it's hard coded), I just tried to make silly suggestions, but seems like you don't need them (have better ones in mind). Also it looks like it might be just an intermediate solution introduced by the following commit https://github.com/jenkinsci/jenkinsfile-runner/commit/f72641df3f31588c750226ea33c4a4178d8ed39a "JENKINS-54424 - Remove Jenkins Test Harness Dependency & replace by local code". Maybe an owner of that code (Oleg Nenashev), had something in mind, but it's never properly followed up since then, never mind... Anyways, thank you very much, seems like I've blindly picked the right person. I'm really impressed about you taking care of it on so short notice...

Add Comment

[o.v.nenashev@gmail.com (JIRA)]

Oleg Nenashev updated an issue

Jenkins / JENKINS-58407

Fail to call a Closure stored as a class attribute/field

Change By: Oleg Nenashev Component/s: jenkinsfile-runner

Add Comment

[o.v.nenashev@gmail.com (JIRA)]

Oleg Nenashev commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field Devin Nusbaum Dicom J Jenkinsfile Runner is in the pre-release state. We can afford any kind of breaking changes there if needed. IMHO no need to apply hacks on the Pipeline side. If you just provide any public API to define whether a strict behavior should be used, I can configure Pipeline in JFR and cut a new release

Add Comment

[o.v.nenashev@gmail.com (JIRA)]

Oleg Nenashev commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

My suggestion is... Disable the strict behavior by default in JFR Introduce an opt-in flag to enable the strict behavior. jenkinsfile Runner test framework may require enabling it by default Fix any regressions which are coming from NOT using isUnitTest = true by default

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Oleg Nenashev, you guys can figure out your self. Just I'd like to mention that I have tried to apply Main.isUnitTest = false in that place Devin Nusbaum pointed out, it helped, so it works (doesn't throw any exceptions) but anyways it prints that warning, not sure it entirely helped and so only this opt-in flag could mitigate such behavior ...

java.lang.IllegalStateException: expected to call SomeStruct.aPointerToClosure but wound up catching org.jenkinsci.plugins.workflow.cps.CpsClosure2.call; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/

But as I said, you are the smart ones here...

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Dicom J There are two problems in this ticket: Your code triggers a warning even though your code itself is not problematic in terms of what JENKINS-31314 is trying to detect, i.e. it is a false positive. What should be a harmless warning printed to the logs is actually causing an exception to be thrown because jenkinsfile-runner activates the flag that gates the strict behavior, breaking your Pipeline. For now I am only concerned about the second point. Even once that is fixed, the warning message will be printed in your logs, it just won't stop the Pipeline from executing.

Add Comment

[o.v.nenashev@gmail.com (JIRA)]

Oleg Nenashev commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

There is a bunch of places where isUnitTest is used apparently: https://github.com/search?q=org%3Ajenkinsci+%22main.isUnitTest%22&type=Code Many of them represent a debug mode (with asserts like in Pipeline CPS), but some actually include the behavior changes. Will need a careful review while removing the flag

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Devin Nusbaum, I can live with that message, as you qualified that code as safe to use  , since it's a fundamental thing (storing this closure to be called a bit further) around which entire pipeline is built...

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Dicom J To be clear, I do not think that code is safe in general, as enabling it inside of the sandbox would require whitelisting method groovy.lang.GroovyObject invokeMethod java.lang.String java.lang.Object, which might allow the sandbox to be bypassed (it is on the sandbox blacklist here), but I am not sure if that is relevant in jenkinsfile-runner's security model. My point is just that as far as what JENKINS-31314 is trying to detect, I do not think your code should be triggering a warning.

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

I don't think it's a good idea to try to whitelist that method It is not; indeed that method is blacklisted. Of course using the sandbox at all in JFR is pointless (provides no actual protection), and so I would hope that JFR disables it. For the case of the sandbox being used, it is a bug in SandboxInterceptor that Jenkins does not recognize what method is actually being called; there are many such bugs (this one would perhaps be in script-security-plugin).

I'm checking to see if there is some way we can detect that jenkinsfile-runner is being used and use that in addition to Main.isUnitTest

Better to just fix JENKINS-55764 already than to introduce hacks into other (supported!) components to work around it. Anyway, the root issue here is that JENKINS-31314 is issuing a false positive for what looks like a fairly straightforward case that did not happen to be covered by any of our existing tests. (We have some involving closure calls, but apparently not this precise variant.) Dicom J to give you some background, the Jenkins Groovy interpreter (essentially) is seeing st.aPointerToClosure() and mistakenly assumes that it is about to call a method named aPointerToClosure defined on the type SomeStruct. In fact the byzantine Groovy call dispatch mechanism winds up calling Closure.call instead. The likely workaround is simple enough—be explicit, rather than relying on exotic syntax: st.aPointerToClosure.call()

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Devin Nusbaum, exactly, and I'm perfectly fine with that. regarding sandboxes and security in general, IMHO, it's almost nobody needs feature which costs a lot, since it breaks a lot of things. As it lames groovy (just opened another fundamental bug https://issues.jenkins-ci.org/browse/JENKINS-58414, most likely related to your case in bullet #1, Jesse Glick I think this one would be interesting for you) what doesn't allow me to do actual programming job on my CICD pipeline, then at the end I'll have nothing to secure (put in that sandbox), what's the point ? the real value of pipelines for me is actual ability to program pipelines, put common part into share library etc, not secure it, as there're a lot of means for that, e.g. private networks with vpn access, etc. I'm not in Jenkins As A Service business, not sure what are actual selling points for sandboxes. But again if simple groovy stuff doesn't work for me then all other 1K plugins and features are useless/irrelevant and I'd rather go and find other means to fulfill my CI/CD requirements...

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Jesse Glick, thanks for explicit workaround, will follow that

Add Comment

[o.v.nenashev@gmail.com (JIRA)]

Oleg Nenashev commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

> Of course using the sandbox at all in JFR is pointless (provides no actual protection), and so I would hope that JFR disables it.  There is an opt-out flag for it ("--no-sandbox" flag)

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

the real value of pipelines for me is actual ability to program pipelines, put common part into share library etc. Other than for the sake of compatibility when migrating an existing corpus of Pipeline script from a Jenkins server, there is no real reason for a Jenkinsfile in JFR to be anything more than node { checkout scm sh 'bash run.sh' } with all the details being handled by some external programs. This is because the purpose of Pipeline script is to script Jenkins—allocate nodes, publish test or analysis results in visual form, trigger builds of other jobs, accept user input, model structure for Blue Ocean, etc.—none of which is possible inside JFR as currently conceived. (Well, if you had bundled a cloud plugin such as kubernetes, you may be able to script node allocation, in which case parallel + podTemplate + container + node would be an alternative idiom to a direct kubectl run.)

not sure what are actual selling points for sandboxes

The only reason I can think of to enable the sandbox in JFR is to help test behavior of scripts which you also plan to run in a Jenkins server. At any rate, JENKINS-58414 does not sound specific to the sandbox.

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Jesse Glick, I have never said that I have enabled sandbox anywhere. Actually I have it disabled, but still from time to time face security (sandbox) related exceptions and I suspect (sorry, I shouldn't do that, it's not my business) that the root cause of all these strange Groovy behaviors are due to the fact that at some point sandbox has been plugged in. maybe my suspicions are wrong, again it's not my business, but as I said I have to deal with it. Regarding JFR, I found it as very useful tool in development of pipelines and shared libraries as: 1. Many people run it locally from docker against theirs local codebases of the same project/product. Everybody has it's own Jenkins running without interfering with others. 2. I personally can run it from command line, actually from my vim's command line, it's how I've got used to do actual programming. 3. I personally, don't like to deal with UIs. Also my/our Jenkinsfile comprises just 1 line of code to call the code from shared library, which loads source code base for target project from scm and then looks for *.groovy cicd related modules within that code base and dynamically loads them, looks for required interface inside, and plug them in, in which way actual pipeline is built dynamically (where closures are heavily used). So I really like that JFR exists, in many other products headless part goes first and UI is just an extension of UX. So many thanks to those who've come up with JFR...

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

I have [the sandbox] disabled, but still from time to time face security (sandbox) related exceptions and I suspect […] that the root cause of all these strange Groovy behaviors are due to the fact that at some point sandbox has been plugged in No. Pipeline script is compiled with either one or two syntax transformers: A CPS transformer, which is always enabled, though it skips methods marked @NonCPS. This permits a build to survive a Jenkins restart. A sandbox transformer, if selected. (Actually it is even a bit more complicated than that: there is a generic sandbox transformer in the script-security plugin which can be applied to any Groovy scripting in Jenkins; and then the workflow-cps plugin includes a hybrid transformer which mixes the CPS transformer with the sandbox transformer.) tl;dr: if you run Pipeline script (including JFR) you are not running plain Groovy, and so there are corner cases where standard language features do not work as expected. Note that, like the sandbox, the CPS transform is pointless for JFR because there is no server to restart; if the JFR JVM dies, your build is dead and that is that. So it would be great to make JFR run your script as plain Groovy. It is not enough to just disable the CPS transformer, though, since the entire execution model of workflow-cps is built to assume this transform. We would need to develop a separate execution engine that just delegates to the default Groovy runtime while offering Pipeline steps and global variables in the binding. Would make JFR faster, smaller, and more usable. There is other stuff that does not add any value in JFR, like the durable-task plugin; it would be simpler and better for sh in JFR to just run the specified command and keep the stdout / stderr file descriptors open using plain old java.lang.Process (or hudson.Proc), like freestyle projects do in hudson.tasks.Shell.

Add Comment

[fwilhelm.wgt@gmail.com (JIRA)]

Florian Wilhelm commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Hi, is there any workaround to set isUnitTest to false in Jenkinsfile runner? We use it for running tests on travis which now all fail due to this situation :-/ Some sort of short term mitigation would be highly appreciated.   Thanks, Florian

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Florian Wilhelm Not as such, but you can almost certainly fix the actual issue (not just reduce it to a warning) by simply avoiding overly magical syntaxes in your Pipeline script to begin with. See discussion above.

Add Comment

[andrew.bayer@gmail.com (JIRA)]

Andrew Bayer assigned an issue to Andrew Bayer

Jenkins / JENKINS-58407

Fail to call a Closure stored as a class attribute/field

Change By: Andrew Bayer Assignee: Devin Nusbaum Andrew Bayer

Add Comment

[andrew.bayer@gmail.com (JIRA)]

Andrew Bayer started work on JENKINS-58407

Change By: Andrew Bayer Status: Open In Progress

Add Comment

[andrew.bayer@gmail.com (JIRA)]

Andrew Bayer updated JENKINS-58407

Jenkins / JENKINS-58407 Fail to call a Closure stored as a class attribute/field

Change By: Andrew Bayer Status: In Progress Review

Add Comment

[andrew.bayer@gmail.com (JIRA)]

Andrew Bayer commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field PRs up.

Add Comment

[andrew.bayer@gmail.com (JIRA)]

Andrew Bayer commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

groovy-cps merged, releasing.

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field

Andrew Bayer, when is this going to be out https://github.com/cloudbees/groovy-cps/commit/93c916d482b63bf5ed46a84d56cb934401be2020  ? As I still get such errors on 2.190.1 (LTS) # Jenkins file def fn() { echo "foobar" } def map = [fn: this.&fn] map.fn.call() java.lang.IllegalStateException: expected to call org.codehaus.groovy.runtime.MethodClosure.call but wound up catching WorkflowScript.fn; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/         at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.handleMismatch(CpsVmExecutorService.java:117)         at com.cloudbees.groovy.cps.impl.CpsCallableInvocation.checkMismatch(CpsCallableInvocation.java:74) but I do believe this is because the fix is not incorporated yet, right ? Or this workaround https://wiki.jenkins.io/display/JENKINS/Pipeline+CPS+method+mismatches#PipelineCPSmethodmismatches-Directinvocationofclosuresstoredinobjectfieldsormaps is irrelevant to my case (a pointer to script method) ?   Thanks in advance!

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J updated an issue

Jenkins / JENKINS-58407

Fail to call a Closure stored as a class attribute/field

Change By: Dicom J Comment: [~abayer], when is this going to be out [https://github.com/cloudbees/groovy-cps/commit/93c916d482b63bf5ed46a84d56cb934401be2020]  ?

As I still get such errors on 2.190.1 (LTS)

{code:java}

# Jenkins file def fn() {     echo "foobar" } def map = [fn: this.&fn] map.fn.call()

{code} {code:java}

java.lang.IllegalStateException: expected to call org.codehaus.groovy.runtime.MethodClosure.call but wound up catching WorkflowScript.fn; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/         at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.handleMismatch(CpsVmExecutorService.java:117)

at com.cloudbees.groovy.cps.impl.CpsCallableInvocation.checkMismatch(CpsCallableInvocation.java:74) {code}

but I do believe this is because the fix is not incorporated yet, right ? Or this workaround [https://wiki.jenkins.io/display/JENKINS/Pipeline+CPS+method+mismatches#PipelineCPSmethodmismatches-Directinvocationofclosuresstoredinobjectfieldsormaps] is irrelevant to my case (a pointer to script method) ?   Thanks in advance!

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field Andrew Bayer, your fix https://github.com/cloudbees/groovy-cps/commit/93c916d482b63bf5ed46a84d56cb934401be2020#diff-1c76c1813ed2b2b2ac87244e44c14641 is incorporated into groovy-cps-1.31, but the latest workflow-cps-2.74 still uses previous groovy-cps-1.30, https://github.com/jenkinsci/workflow-cps-plugin/blob/workflow-cps-2.74/pom.xml#L74 without the fix and so having that described here https://wiki.jenkins.io/display/JENKINS/Pipeline+CPS+method+mismatches#PipelineCPSmethodmismatches-Directinvocationofclosuresstoredinobjectfieldsormaps doesn't help much to solve the issue, as workaround (explicit call) is not in GA. So does anyone know when we may have the next cut of workflow-cps-2.75 with your fix? Thanks for any information.

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum updated JENKINS-58407

A fix for this issue was just released in Pipeline: Groovy Plugin version 2.75.

Jenkins / JENKINS-58407

Fail to call a Closure stored as a class attribute/field

Change By: Devin Nusbaum Status: In Review Resolved Resolution: Fixed Released As: workflow-cps 2.75

Add Comment

[dicomj23@gmail.com (JIRA)]

Dicom J commented on JENKINS-58407

Re: Fail to call a Closure stored as a class attribute/field Thanks, Devin Nusbaum, for very good news, I've been waiting a while for this cut... Thanks a lot all of you, participated in getting it done!

Add Comment