Pipeline Groovy Madness

283 views

Skip to first unread message

Jonathan Hodgson

unread,

Oct 14, 2016, 7:25:36 AM10/14/16

to Jenkins Users

Hi,

I'm trying to get some Groovy code working in the pipeline.

The code used to work fine, somewhere in the endless stream of updates of jenkins and plugins, it stopped.

It used to look like this (I've stripped the bits that do actual work out for brevity)

@NonCPS def isPlatformActive(platform, active_builds)
{
    found = active_builds[platform].find { it.value == true } 
    found
}

def buildAll(build_types, active_builds, plugin_formats)
{
    stage ("Building And Testing") {
        platform_builders = [:]
        something_failed = false
        failed_message = ""
                if (isPlatformActive("OSX", active_builds))
        {
            platform_builders ["Mac"] = {
                node ("XCode")
                {
                    ws (project_workspace) {
                        echo "Substage running on OSX"
                    }
                }
            }
        }
        if (isPlatformActive("Win32", active_builds) || isPlatformActive("Win64", active_builds))
        {
            platform_builders ["Windows"] = { 
                node ("VS2013")
                {
                    ws (project_workspace) {
                        echo "Substage running on Windows"
                    }
                }
            }
        }
   }
}

And it used to work.

But then when I came back to it, I found I was getting an exception

An Error Occured java.io.NotSerializableException: java.util.LinkedHashMap$Entry

So, putting this down to the known issue with Groovy in the pipeline I changed isPlatformActive to a @NonCPS function, which worked find so long as the second if statement was commented out, but not when I uncommented it.

After further experimentation I've found that this works

@NonCPS def isPlatformActive(platform, active_builds)
{
    found = active_builds[platform].find { it.value == true } 
    found
}

def buildAll(build_types, active_builds, plugin_formats)
{
    stage ("Building And Testing") {
        platform_builders = [:]
        something_failed = false
        failed_message = ""
        isPlatformActive("Win32", active_builds) // Just in here to test nothing goes wrong - it doesn't
        if (isPlatformActive("OSX", active_builds))
        {
            platform_builders ["Mac"] = {
                node ("XCode")
                {
                    ws (project_workspace) {
                        echo "Substage running on OSX"
                    }
                }
            }
        }
   }
}

But this doesn't

@NonCPS def isPlatformActive(platform, active_builds)
{
    found = active_builds[platform].find { it.value == true } 
    found
}

def buildAll(build_types, active_builds, plugin_formats)
{
    stage ("Building And Testing") {
        platform_builders = [:]
        something_failed = false
        failed_message = ""
               if (isPlatformActive("OSX", active_builds))
        {
            platform_builders ["Mac"] = {
                node ("XCode")
                {
                    ws (project_workspace) {
                        echo "Substage running on OSX"
                    }
                }
            }
        }        isPlatformActive("Win32", active_builds) // Just in here to test nothing goes wrong - IT DOES
   }
}

I get

java.io.NotSerializableException: java.util.LinkedHashMap$Entry
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
	at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
	at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
	at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
	at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:343)
	at java.util.HashMap.writeObject(HashMap.java:1129)
	at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
	at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
	at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
	at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:343)
	at com.cloudbees.groovy.cps.SerializableScript.writeObject(SerializableScript.java:26)
	at sun.reflect.GeneratedMethodAccessor283.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
	at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
	at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
	at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:343)
	at java.util.HashMap.writeObject(HashMap.java:1129)
	at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
	at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:132)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:429)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:408)
	at org.jenkinsci.plugins.workflow.cps.CpsStepContext$3.onSuccess(CpsStepContext.java:488)
	at org.jenkinsci.plugins.workflow.cps.CpsStepContext$3.onSuccess(CpsStepContext.java:484)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:627)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: an exception which occurred:
	in field delegate
	in field closures
	in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@470056de

All I've done is move that one line, and for the record at the moment it's being called without OSX being active, so nothing inside that first if statement is being executed (though it shouldn't cause any problems even if its was)

I'm confused, and frustrated. Any help greatly appreciated.

Daniel Beck

unread,

Oct 14, 2016, 8:47:24 AM10/14/16

to jenkins...@googlegroups.com

> On 14.10.2016, at 13:25, Jonathan Hodgson <j.r.h...@gmail.com> wrote:
>
> All I've done is move that one line, and for the record at the moment it's being called without OSX being active, so nothing inside that first if statement is being executed (though it shouldn't cause any problems even if its was)
>
> I'm confused, and frustrated. Any help greatly appreciated.

The last statement in a Groovy method is the implicit return value. `find` returns a LinkedHashMap$Entry, it gets returned from the method call, and it breaks due to CPS serialization.

http://groovy-lang.org/semantics.html#_optional_return_keyword

Regarding the regression, likely due to https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Groovy+Plugin#PipelineGroovyPlugin-2.14%28Sep07%2C2016%29

Jonathan Hodgson

unread,

Oct 14, 2016, 11:33:49 AM10/14/16

to Jenkins Users, m...@beckweb.net

On Friday, October 14, 2016 at 1:47:24 PM UTC+1, Daniel Beck wrote:

> On 14.10.2016, at 13:25, Jonathan Hodgson <j.r.h...@gmail.com> wrote:
>
> All I've done is move that one line, and for the record at the moment it's being called without OSX being active, so nothing inside that first if statement is being executed (though it shouldn't cause any problems even if its was)
>
> I'm confused, and frustrated. Any help greatly appreciated.

The last statement in a Groovy method is the implicit return value. `find` returns a LinkedHashMap$Entry, it gets returned from the method call, and it breaks due to CPS serialization.

Thanks Daniel, I'm still finding my feet with Groovy, throw in the CPS issues plus other pipeline job problems (e.g. the lack of working file parameters) and I find myself going round and round until I don't know which way is up, down or sideways.

Changing things to

@NonCPS def isPlatformActive(platform, active_builds)

{

    found = (null != active_builds[platform].find { it.value == true } )

    found

}

Seems to have fixed this issue

Regarding the regression, likely due to https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Groovy+Plugin#PipelineGroovyPlugin-2.14%28Sep07%2C2016%29

Thanks again.

It's the lack of consistency which is driving me nuts, things work in one place, but not another, or they work with one version, but not the next. It makes debugging so hard.

And for the love of God could somebody please implement some useful error messages!?

Things should either work, or not work with a suitable error message.

I don't know how difficult it would be, but a lint style checker that would pick up forbidden constructs in CPS functions would be nice too... but there I'm getting into dreamland I fear.

Brian Ray

unread,

Oct 14, 2016, 12:11:28 PM10/14/16

to Jenkins Users, m...@beckweb.net

Keep an eye on JENKINS-35390.

Though the other day I optimistically tried to use Groovy's AST transform @Immutable on a class ... should've known that would not play nicely with the CPS transforms.

Reply all

Reply to author

Forward

0 new messages