ConcurrentModificationException when removing "using (template)"

[Torsten Reinhard]

Hi, 

I´m using Jenkins 1.569 and job-dsl-plugin 1.25 (old versions, I know..)

When trying to migrate my existing code like
job {
 name "${jobName}"
 using("${templateName}")
..
}

into code that doesnt use the "using(${templateName}) anymore, I run into following Exception:

Unreferenced Templates: template-Pipeline-env,template-Shutdown-WebLogic-env,template-Startup-WebLogic-env
FATAL: null
java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922)
        at java.util.HashMap$KeyIterator.next(HashMap.java:956)
        at com.google.common.collect.AbstractMapBasedMultimap$WrappedCollection$WrappedIterator.next(AbstractMapBasedMultimap.java:484)
        at com.google.common.collect.Iterators$7.computeNext(Iterators.java:701)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at com.google.common.collect.Sets.removeAllImpl(Sets.java:1598)
        at com.google.common.collect.Sets.removeAllImpl(Sets.java:1627)
        at com.google.common.collect.AbstractMapBasedMultimap$WrappedSet.removeAll(AbstractMapBasedMultimap.java:614)
        at javaposse.jobdsl.plugin.ExecuteDslScripts.updateTemplates(ExecuteDslScripts.java:226)
        at javaposse.jobdsl.plugin.ExecuteDslScripts.perform(ExecuteDslScripts.java:183)
        at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
        at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:772)
        at hudson.model.Build$BuildExecution.build(Build.java:199)
        at hudson.model.Build$BuildExecution.doRun(Build.java:160)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:535)
        at hudson.model.Run.execute(Run.java:1732)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:88)
        at hudson.model.Executor.run(Executor.java:234)


I than removed the JENKINS_HOME/javaposse.jobdsl.plugin.ExecuteDslScripts.xml file - but now the jobs are not updated - they are 1st deleted, and afterwards re-generated by the new code (without the using(...))

=> Is there a way to migrate the jobs without running into this kind of Exception - and with keeping all the history (builds, ...) of the jobs? 

Thanx, Torsten

[Daniel Spilker]

Ah, I came across that issue when reviewing a related pull request and was actually wandering why it never popped up. I'll try to fix it for the next release.

Daniel

--
You received this message because you are subscribed to the Google Groups "job-dsl-plugin" group.
To unsubscribe from this group and stop receiving emails from it, send an email to job-dsl-plugi...@googlegroups.com.
To post to this group, send email to job-dsl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/job-dsl-plugin/17616196-c39f-4c7a-9a30-d81aacb49566%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Spilker]

BTW, the exception should only occur when running multiple seed jobs in parallel.

[Daniel Spilker]

Meh, the ConcurrentModificationException is harder to fix than I thought.

I created two bug reports for your issues. The first one (https://issues.jenkins-ci.org/browse/JENKINS-27602) for the job deletion/re-creation and the second one for the ConcurrentModificationException (https://issues.jenkins-ci.org/browse/JENKINS-27603). I could reproduce JENKINS-27602, but not JENKINS-27603. Is the ConcurrentModificationException reproduceable?

Daniel

[Torsten Reinhard]

thx for creating the issues - any workaround for the deletion/re-creation problem? Mine is, to rsync the build jobs before - and afterwards restore everything but the (new) config.xml.

JENKINS-27603 happens sometimes, but I actually didn´t try to reproduce it clearly - but you´re right, I´m using multiple Seed Jobs in parallel to generate different jobs/branches for different environments.
In some cases throwing the ConcurrentModificationException was right telling me that there´s an issue in my /jobs code (I generated the same job with two seed jobs by accident).

I´m still working on it - and if I have more details about reproducing this issue, I´ll add it to the JIRA.

regards, Torsten

[Daniel Spilker]

You can configure your seed job to ignore removed job by setting "Action for removed jobs" to "Ignore" as a workaround.

Daniel

To view this discussion on the web visit https://groups.google.com/d/msgid/job-dsl-plugin/26904e06-0ace-4140-8bbd-0b810ac369ad%40googlegroups.com.