Latest changes to Jenkins Pipeline break serializable on KeySet

[Sverre Moe]

After upgrade to Jenkins 2.22 and the Pipeline plugins 2.x my build script failed

[Pipeline] End of Pipeline

java.io.NotSerializableException: java.util.LinkedHashMap$LinkedKeySet

It failed when I called map.keySet() which did worked before the update.

This was logic executing on a Jenkins flyweight executor before any stage step.

Moving this build logic within a method with @NonCPS worked.

It is a bit annoying that this just suddenly stopped working when I updated our Jenkins production instance.

Any changes that breaks previous functionality should be mentioned. Perhaps it did, but I did not see it. Usually I read the changelog of Jenkins and the plugins before I perform any update.

I am beginning to consider cloning our Jenkins production instance to use in a test environment to test update changes. A few things must be disabled in the build script in regards when running within a test instance, such as git tagging and artifact publish.

[Sverre Moe]

I got a nother problem in production.

It has worked before to use "for (var : list)".

I have used this because "list.each { var -> }" does not work.

Why does not the former still work? Also list.get(i) does not work. Had to use list.getAt(i)

    if (extraHosts != null && !extraHosts.isEmpty()) {
        final def extraHostsList = extraHosts.split(", ")
        for (def host : extraHostsList) {
            envBuildHosts.add(host)
        }
    }

    if (extraHosts != null && !extraHosts.isEmpty()) {
        final def extraHostsList = extraHosts.split(", ")
        for (int i = 0; i < extraHostsList.size(); i++) {
            def host = extraHostsList.getAt(i)
            envBuildHosts.add(host)
        }
    }

java.io.NotSerializableException: java.util.AbstractList$Itr

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)

at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)

at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)

at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)

at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)

at java.util.TreeMap.writeObject(TreeMap.java:2434)

at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)

at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)

at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:132)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:429)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:408)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:356)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:78)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:224)

at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)

at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

[letha...@gmail.com]

Hi all,

Having a similar issue here. I have a pipeline that worked well until I updated both jenkins to 2.22 and all my plugins (unfortunately I did that like a cowboy and don't know which versions I had before).

When I run the build it fails with:

java.io.NotSerializableException: groovy.json.internal.LazyMap

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)

at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)

at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)

at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)

at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)

at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)

at java.util.HashMap.internalWriteEntries(HashMap.java:1777)

at java.util.HashMap.writeObject(HashMap.java:1354)

at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at sun.reflect.GeneratedMethodAccessor498.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)

at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)

at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)

at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)

at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:132)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:429)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:408)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:356)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:78)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236)

at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:224)

at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)

at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: an exception which occurred:

in field locals

in field caller

in field parent

in field caller

in field e

in field program

in field threads

in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@35e68b9c

Finished: FAILURE

I do some JSON parsing with JsonSlurper in a @NonCps function that used to return a Map (and it worked in pre-2.22), that I've changed to return a list (still doesn't work).

Relevant code (edited to remove some complexity) is

@NonCPS
def getRegions() {
  def res = []

  def jsonSlurper = new groovy.json.JsonSlurper()
  def regions = jsonSlurper.parseText(string)

  for (def regionin regions) {
      res.add(region)
  }

  regions = null

  res
}


@NonCPS
def get_stash_region_list(stash_name) {
  def regions = getRegions()

  List<String> stash_list = []

  for(def regionIdx = 0; regionIdx < regions.size(); regionIdx++) {
    def region = regions[regionIdx]

    def full_stash_name = "${stash_name}_${region}"
    stash_list.add(full_stash_name)

    region = null
  }

  regions = null
  stash_list as String[]
}

def unstash_region(stash_name) {
  def stash_list = get_stash_region_list(stash_name)

  for(def stashIdx = 0; stashIdx < stash_list.size(); stashIdx++) {
    unstash stash_list[stashIdx]
  }

  stash_list = null
}

If I hardcode the stash_list in unstash_region my build runs successfully.

stash_list = [
    "ap-southeast-2",
    "us-west-2",
  ]

Strangely when I was messing around with the unstash_region method, I had to authorize method java.util.Map size in the In-script approvals screen for stash_list.size(). That variable is supposed to be an array of string, so I'm not sure what's going on there (hence all the type fiddling in get_stash_region_list, which doesn't seem to have any effect whatsoever).

So is that a bug introduced in a newer version of jenkins/some plugin, is it an intended sideeffect of some feature, or am I doing something wrong (I am a noob in groovy and Jenkins pipelines)?

The problem happens on Jenkins 2.22 and the following plugin versions:

• Pipeline 2.3
  
• Pipeline Graph Analysis Plugin 1.1
  
• Pipeline visualization plugin 0.3.1
  
• Pipeline: API 2.3
  
• Pipeline: Basic Steps 2.1
  
• Pipeline: Build Step 2.2
  
• Pipeline: Groovy 2.17
  
• Pipeline: Input Step 2.1
  
• Pipeline: Job 2.6
  
• Pipeline: Model Definition 0.2
  
• Pipeline: Multibranch 2.8
  
• Pipeline: Nodes and Processes 2.4
  
• Pipeline: REST API Plugin 2.0
  
• Pipeline: SCM Step 2.2
  
• Pipeline: Shared Groovy Libraries 2.3
  
• Pipeline: Stage Step 2.2
  
• Pipeline: Stage View Plugin 2.0
  
• Pipeline: Step API 2.3
  
• Pipeline: Supporting APIs 2.4