[JIRA] (JENKINS-49904) workflow-cps groovy engine hijacks closure.rehydrate()

11 views
Skip to first unread message

gustaf.lundh@gmail.com (JIRA)

unread,
Mar 4, 2018, 4:49:03 PM3/4/18
to jenkinsc...@googlegroups.com
Gustaf Lundh created an issue
 
Jenkins / Bug JENKINS-49904
workflow-cps groovy engine hijacks closure.rehydrate()
Issue Type: Bug Bug
Assignee: Unassigned
Components: workflow-cps-plugin
Created: 2018-03-04 21:48
Environment: workflow-cps-plugin 2.45, jenkins 2.7.3
Labels: pipeline groovy global-lib
Priority: Major Major
Reporter: Gustaf Lundh

workflow-cps-plugin's CpsScript breaks a fundamental Groovy language feature: closure.rehydrate().
 
In this short snippet of pipeline code we expect the closure to run successfully (which it does in a regular GroovyShell or Jenkins script console) :
 

class TasksSpec implements Serializable {
    TasksSpec() {
    }

    def foo() {
        println this
    }

    def runClosure(closure) {
        def hydratedClosure = closure.rehydrate(this, this, this)
        hydratedClosure.resolveStrategy = Closure.DELEGATE_ONLY
        hydratedClosure()
     }
}

def tasksSpec = new TasksSpec()
tasksSpec.runClosure() {
    foo()
}

 
However, the Groovy Cps implementation seems to totally disregard the fact that we carry out a rehydrate() with the intention to execute the closure in the context of the tasksSpec object. ButiInstead the closure is executed in the context of the WorkflowScript and fails with:
 

Java.lang.NoSuchMethodError: No such DSL method 'foo' found among steps [archive, bat, catchError, checkout, deleteDir, dir, echo, error, fileExists, getContext, git, isUnix, library, libraryResource, load, mail, node, parallel, pwd, readFile, retry, sh, sleep, stash, step, svn, timeout, tool, unarchive, unstash, waitUntil, withContext, withEnv, wrap, writeFile, ws] or symbols [all, always, apiToken, architecture, archiveArtifacts, artifactManager, batchFile, booleanParam, buildButton, buildDiscarder, caseInsensitive, caseSensitive, choice, choiceParam, clock, cloud, command, credentials, cron, crumb, defaultView, demand, disableConcurrentBuilds, downloadSettings, downstream, dumb, envVars, file, fileParam, filePath, fingerprint, frameOptions, freeStyle, freeStyleJob, fromScm, fromSource, git, headRegexFilter, headWildcardFilter, hyperlink, hyperlinkToModels, installSource, jdk, jdkInstaller, jgit, jgitapache, jnlp, jobName, junit, lastDuration, lastFailure, lastGrantedAuthorities, lastStable, lastSuccess, legacy, legacySCM, list, local, location, logRotator, loggedInUsersCanDoAnything, masterBuild, maven, maven3Mojos, mavenErrors, mavenMojos, mavenWarnings, modernSCM, myView, nodeProperties, nonStoredPasswordParam, none, paneStatus, parameters, password, pattern, pipelineTriggers, plainText, plugin, projectNamingStrategy, proxy, queueItemAuthenticator, quietPeriod, run, runParam, schedule, scm, scmRetryCount, search, security, shell, slave, sourceRegexFilter, sourceWildcardFilter, stackTrace, standard, status, string, stringParam, swapSpace, text, textParam, tmpSpace, toolLocation, unsecured, upstream, viewsTabBar, weather, zfs, zip] or globals [currentBuild, env, params]
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:176)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108)
	at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1280)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1174)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:19)
	at WorkflowScript.run(WorkflowScript:23)
	at TasksSpec.runClosure(WorkflowScript:14)
	at WorkflowScript.run(WorkflowScript:21)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:77)
	at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:331)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:82)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:243)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:231)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at java.util.concurrent.Executors$RunnableAdapter.call$$$capture(Executors.java:511)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

 
It seems that if we try to carry out a rehydrate(), the closure ends up being always executed through CpsScripts invokeMethod():

    /**
     * We use DSL here to try invoke the step implementation, if there is Step implementation found it's handled or
     * it's an error.
     *
     * <p>
     * sandbox security execution relies on the assumption that CpsScript.invokeMethod() is safe for sandboxed code.
     * That means we cannot let user-written script override this method, hence the final.
     */
    @Override
    public final Object invokeMethod(String name, Object args) {
        // TODO probably better to call super method and only proceed here incase of MissingMethodException:
        // if global variables are defined by that name, try to call it.
        // the 'call' convention comes from Closure
        GlobalVariable v = GlobalVariable.byName(name, $buildNoException());
        if (v != null) {
            try {
                Object o = v.getValue(this);
                return InvokerHelper.getMetaClass(o).invokeMethod(o, "call", args);
            } catch (Exception x) {
                throw new InvokerInvocationException(x);
            }
        }

        // otherwise try Step impls.
        DSL dsl = (DSL) getBinding().getVariable(STEPS_VAR);
        return dsl.invokeMethod(name,args);
    }

Instead of executing tasksSpec.foo() when hydratedClosure() is executed, invokeMethod() tries to find/execute the call()-method on the (non-existing) global variable foo or if that fails search the pipeline build steps for a match!
 
This has some major annoying consequences when trying to reuse (or design) a groovy framework for building pipelines.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

gustaf.lundh@gmail.com (JIRA)

unread,
Mar 4, 2018, 4:51:02 PM3/4/18
to jenkinsc...@googlegroups.com
Gustaf Lundh updated an issue
Change By: Gustaf Lundh
workflow-cps-plugin's CpsScript breaks a fundamental Groovy language feature: closure.rehydrate().
 
In this short snippet of pipeline code we expect the closure hydratedClosure() to run foo() successfully (which it does in a regular GroovyShell or Jenkins script console) :
 
{code:java}

class TasksSpec implements Serializable {
    TasksSpec() {
    }

    def foo() {
        println this
    }

    def runClosure(closure) {
        def hydratedClosure = closure.rehydrate(this, this, this)
        hydratedClosure.resolveStrategy = Closure.DELEGATE_ONLY
        hydratedClosure()
     }
}

def tasksSpec = new TasksSpec()
tasksSpec.runClosure() {
    foo()
}
{code}

 
However, the Groovy Cps implementation seems to totally disregard the fact that we carry out a rehydrate() with the intention to execute the closure in the context of the tasksSpec object. ButiInstead the closure is executed in the context of the WorkflowScript and fails with:
 
{code:java}
{code}

 
It seems that if we try to carry out a rehydrate(), the closure ends up being always executed through CpsScripts invokeMethod():

{code:java}

    /**
     * We use DSL here to try invoke the step implementation, if there is Step implementation found it's handled or
     * it's an error.
     *
     * <p>
     * sandbox security execution relies on the assumption that CpsScript.invokeMethod() is safe for sandboxed code.
     * That means we cannot let user-written script override this method, hence the final.
     */
    @Override
    public final Object invokeMethod(String name, Object args) {
        // TODO probably better to call super method and only proceed here incase of MissingMethodException:
        // if global variables are defined by that name, try to call it.
        // the 'call' convention comes from Closure
        GlobalVariable v = GlobalVariable.byName(name, $buildNoException());
        if (v != null) {
            try {
                Object o = v.getValue(this);
                return InvokerHelper.getMetaClass(o).invokeMethod(o, "call", args);
            } catch (Exception x) {
                throw new InvokerInvocationException(x);
            }
        }

        // otherwise try Step impls.
        DSL dsl = (DSL) getBinding().getVariable(STEPS_VAR);
        return dsl.invokeMethod(name,args);
    }
{code}

Instead of executing tasksSpec.foo() when hydratedClosure() is executed, invokeMethod() tries to find/execute the call()-method on the (non-existing) global variable foo (!) or if that fails search the pipeline build steps for a match!  

 
This has some major annoying consequences when trying to reuse (or design) a groovy framework for building pipelines.

gustaf.lundh@gmail.com (JIRA)

unread,
Mar 4, 2018, 4:51:02 PM3/4/18
to jenkinsc...@googlegroups.com
Gustaf Lundh updated an issue
workflow-cps-plugin's CpsScript breaks a fundamental Groovy language feature: closure.rehydrate().
 
In this short snippet of pipeline code we expect hydratedClosure() to run foo() successfully (which it does in a regular GroovyShell or Jenkins script console):

 
{code:java}
class TasksSpec implements Serializable {
    TasksSpec() {
    }

    def foo() {
        println this
    }

    def runClosure(closure) {
        def hydratedClosure = closure.rehydrate(this, this, this)
        hydratedClosure.resolveStrategy = Closure.DELEGATE_ONLY
        hydratedClosure()
     }
}

def tasksSpec = new TasksSpec()
tasksSpec.runClosure() {
    foo()
}
{code}
 
However, the Groovy Cps implementation seems to totally disregard the fact that we carry out a rehydrate() with the intention to execute the closure in the context of the tasksSpec object. ButiInstead Instead the closure is executed in the context of the WorkflowScript and fails with:

gustaf.lundh@gmail.com (JIRA)

unread,
Mar 4, 2018, 4:51:02 PM3/4/18
to jenkinsc...@googlegroups.com
Gustaf Lundh updated an issue
workflow-cps-plugin's CpsScript breaks a fundamental Groovy language feature: closure.rehydrate().
 
In this short snippet of pipeline code we expect hydratedClosure() to run foo() successfully (which it does in a regular GroovyShell or Jenkins script console) :
 
{code:java}
class TasksSpec implements Serializable {
    TasksSpec() {
    }

    def foo() {
        println this
    }

    def runClosure(closure) {
        def hydratedClosure = closure.rehydrate(this, this, this)
        hydratedClosure.resolveStrategy = Closure.DELEGATE_ONLY
        hydratedClosure()
     }
}

def tasksSpec = new TasksSpec()
tasksSpec.runClosure() {
    foo()
}
{code}
 
However, the Groovy Cps implementation seems to totally disregard the fact that we carry out a rehydrate() with the intention to execute the closure in the context of the tasksSpec object. ButiInstead the closure is executed in the context of the WorkflowScript and fails with:

gustaf.lundh@gmail.com (JIRA)

unread,
Mar 4, 2018, 4:52:04 PM3/4/18
to jenkinsc...@googlegroups.com
Gustaf Lundh updated an issue
workflow-cps-plugin's CpsScript breaks a fundamental Groovy language feature: closure.rehydrate().
 
In this short snippet of pipeline code we expect hydratedClosure() to run foo() successfully (which it does in a regular GroovyShell or Jenkins script console):
 
{code:java}
class TasksSpec implements Serializable {
    TasksSpec() {
    }

    def foo() {
        println this
    }

    def runClosure(closure) {
        def hydratedClosure = closure.rehydrate(this, this, this)
        hydratedClosure.resolveStrategy = Closure.DELEGATE_ONLY
        hydratedClosure()
     }
}

def tasksSpec = new TasksSpec()
tasksSpec.runClosure() {
    foo()
}
{code}
 
However, the Groovy Cps implementation seems to totally disregard the fact that we carry out a rehydrate() with the intention to execute the closure in the context of the tasksSpec object. Instead the closure is executed in the context of the WorkflowScript and fails with:
Instead of executing running tasksSpec.foo() when hydratedClosure() is executed, invokeMethod() tries to find/execute the call()-method on the (non-existing) global variable foo (!) or if that fails search , continue by searching through the pipeline build steps for a match!  

 
This has some major annoying consequences when trying to reuse (or design) a groovy framework for building pipelines.

gustaf.lundh@gmail.com (JIRA)

unread,
Mar 6, 2018, 5:59:02 AM3/6/18
to jenkinsc...@googlegroups.com
Gustaf Lundh commented on Bug JENKINS-49904
 
Re: workflow-cps groovy engine hijacks closure.rehydrate()

Jesse Glick: Sorry for pinging you, but we are currently building a groovy framework/backend for pipeline jobs. The framework would allow our end users to setup and manage extremely complex pipelines, consisting of 100's of projects (and allow them to this in an reasonable easy way).

Our original vision was very dependent on a working rehydrate() functionality. So before we restart the design work, I would like to know if you think this issue will be fixed within a reasonable timeframe (or if Cloudbees was already aware of the issue). Or perhaps this behaviour is by design?

The Groovy and Cps implementation is still a bit too alien to allow me to fix the issue myself within a sensible timeframe. 

andrew.bayer@gmail.com (JIRA)

unread,
Mar 7, 2018, 12:17:02 PM3/7/18
to jenkinsc...@googlegroups.com

Gustaf Lundh - so from a preliminary investigation, the underlying problem here is something with Closure#clone(). If you just set the delegate on the closure (and the resolveStrategy, it works, but cloning somehow gets things stuck.

gustaf.lundh@gmail.com (JIRA)

unread,
Jun 1, 2018, 5:14:02 AM6/1/18
to jenkinsc...@googlegroups.com
Gustaf Lundh updated an issue
Change By: Gustaf Lundh
Instead of running tasksSpec.foo() when hydratedClosure() is executed, invokeMethod() tries to find/execute the call()-method on the (non-existing) global variable foo (!) or if that fails, continue by searching through the pipeline build steps for a match!  

 
This has some major annoying consequences when trying to reuse (or design) a groovy framework for building pipelines.

gustaf.lundh@gmail.com (JIRA)

unread,
Jun 1, 2018, 5:14:02 AM6/1/18
to jenkinsc...@googlegroups.com

logan.mauzaize@gmail.com (JIRA)

unread,
Sep 14, 2018, 5:41:03 AM9/14/18
to jenkinsc...@googlegroups.com
Logan Mzz commented on Bug JENKINS-49904
 
Re: workflow-cps groovy engine hijacks closure.rehydrate()

I have same needs for same reason: having both a delegation DSL and access to script/definition-time scope.

 

mylib.pipeline { // <- call it 'pipeline' closure, only delegate to 'PipelineDSL'
    init { // <- call it 'pipeline.init' closure, delegate to 'StageInitDSL'
        // delegate == StageInitDSL@xxxxxx
        // owner == pipeline
        // this == WorkflowScript@xxxxxxx
        
        echo 'blablabla...' // <- Not possible because, it's only accessible from 'script' scope
        this.echo 'blablabla' // OK   
    }
}

Solution would be to set init.owner from pipeline.thisObject but it currently doesn't work:

class MyLib {
    def pipeline(declaration) {
        def stages = [init: {}]
        def pipelineDSL = new PipelineDSL()
        def stageDSL    = new StageDSL()
        pipelineDsl.init = {  // it.delegate == declaration
                              // it.owner    == declaration
                              // it.this     == script
            def fn = it.rehydrate(stageDSL, it.thisObject, it.thisObject)
            fn.resolveStrategy = Closure.DELEGATE_FIRST
            stages['init'] = fn  // fn.delegate == stageDSL
                                 // fn.owner    == script
                                 // fn.this     == script
        }
        declaration.delegate = pipelineDSL
        declaration.resolveStrategy = Closure.DELEGATE_ONLY
        declaration()

        stages.each { name, fn -> fn() }
    }
}

def mylib = new MyLib()

mylib.pipeline {
    init {
        delegate  // == declaration/pipeline  <>  script
        owner     // == declaration/pipeline  <>  script
        this      // == script

        echo       //  Unknown property
        mylib      //  Unknown property
        this.echo  //  Ok
        this.mylib //  Ok
    }
}

 

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

andrew.bayer@gmail.com (JIRA)

unread,
Nov 16, 2018, 9:41:03 AM11/16/18
to jenkinsc...@googlegroups.com
Andrew Bayer updated an issue
 
Change By: Andrew Bayer
Labels: complex-cps-code global-lib groovy pipeline triaged-2018-11

benj_boyer@hotmail.com (JIRA)

unread,
Apr 4, 2019, 7:06:02 AM4/4/19
to jenkinsc...@googlegroups.com
Benjamin Boyer commented on Bug JENKINS-49904
 
Re: workflow-cps groovy engine hijacks closure.rehydrate()

Has anyone managed to get through this problem OR is there any fix for it ?

ivan.martinez.rodriguez@ericsson.com (JIRA)

unread,
Jan 9, 2020, 6:27:02 AM1/9/20
to jenkinsc...@googlegroups.com

Any news on this issue? Is there any workaround to rehydrate a Closure without using the rehydrate method from the Closure class itself?

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages