Why does Groovy seem to be so broken in jenkins?
308 views
Skip to first unread message
Jonathan Hodgson
Oct 13, 2016, 1:13:58 PM10/13/16
to Jenkins Developers
Hi,
Groovy in Jenkins has some quirks and they're really driving me nuts.
Some of the very neat features of Groovy, such as closures and .each() and .findAll() don't work as advertised. Which is a problem because it's these neat ways of doing things that are always used when you google a problem!
It seems I can get around it sometimes by using a helper function declared as @NonCPS... but I don't have any idea what that means, despite looking at the documentation (ok, I haven't read it exhaustively, but the references to NonCPS I've looked at so far haven't clarified things.
And it doesn't even seem consistent in when things work and when they fail. I guess there must be some pattern to it, but sometimes it seems like I do something in one place, and it works fine, do the same thing somewhere else, and it doesn't work. Often failing quietly so I'm scratching my head trying to work out what I've done wrong, until I start sticking liberal echo statements into things and find that I haven't done anything wrong other than use standard Groovy where Jenkins doesn't support standard Groovy.
My development time woud be much reduced if I could just test groovy code locally in Eclipse.. but since stuff that works correctly there often doesn't work in Jenkins, I'm stuck doing endless test builds. Each one may not take long (at because I comment out what I'm not testing) but when it's minutes rather than seconds, multiply that by a couple of hundred times trying to work out why it's not performing as expected.. you get the idea.
Is there at least a page somewhere that says what works and what doesn't? And if not, why not?
regards
Jon
Jonathan Hodgson
Oct 13, 2016, 1:55:53 PM10/13/16
to Jenkins Developers
Case in point....
From adding a for loop with a fixed index that just echos some text??
I'm currently trying to "debug" a function (I put "debug" in quotes because as far as I'm concerned it's more that I'm trying to avoid the bugs in Jenkin's implementation of Groovy rather than any actuall faults in my code).
Now, bear in mind that this function was previously tested and worked fine on jenkins, this issue seems to have been introduced in an update of jenkins or a plugin, unfortunately I have no idea which one because I was distracted by trying to work around other issues in Jenkins.
Now, to try to isolate the problem, I've cut it down and put in checks, so what's left wouldn't do anything useful if it worked.
def getBuildTypes()
{
if (!fileExists(common_cmake))
{
throw new Exception("${common_cmake} not found")
}
else
{
def matcher = readFile(common_cmake) =~ /(?:\s*#\s*(Generators)\s+(.+?"[^"]+"\s*(?:,\s+.+?"[^"]+"\s*)+))|(?:\s*(CMAKE_CONFIGURATION_TYPES)\s+"(.+?(?:;.+?)*)")/
if (!matcher)
{
throw new Exception("Configuration and generator information not found in ${common_cmake}")
}
num_matches = matcher.size()
def configurations = []
def generators = [:]
echo "${num_matches}"
/*for (def index = 0; index < 2; index++)
{
echo "Checking"
}*/
["generators" : generators, "configurations" : configurations]
}
}Now, the above function works as expected, num_matches turns out to be 2, as it should be.
However, if all I do is uncomment that for loop, then suddenly I get
An Error Occured java.io.NotSerializableException: java.util.regex.Matcher
[Pipeline] End of Pipeline
java.io.NotSerializableException: java.util.regex.Matcher
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:343)
at java.util.HashMap.writeObject(HashMap.java:1129)
at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:343)
at java.util.HashMap.writeObject(HashMap.java:1129)
at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:132)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:429)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:408)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:356)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:78)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:224)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:63)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: an exception which occurred:
in field locals
in field parent
in field parent
in field caller
in field e
in field program
in field thread
in field body
in field step
in field thread
in field this$0
in field returnAddress
in field parent
in field caller
in field parent
in field parent
in field capture
in field def
in field closures
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@1257cff9
From adding a for loop with a fixed index that just echos some text??
Baptiste Mathus
Oct 13, 2016, 1:59:16 PM10/13/16
to Jenkins Developers
Hi, please use the users list if this is to discuss Pipeline usage. Also, don't mix up the Groovy flavour in use with Pipeline, and the one (standard) you can use in the script console, as job steps, and so on.
What you seem to describe is "just" a known issue: https://issues.jenkins-ci.org/browse/JENKINS-26481 (though, granted, an important when you don't know about it).
Cheers
--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/da477cbe-a38a-4b23-93cb-6a1964c0d562%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jonathan Hodgson
Oct 13, 2016, 2:28:41 PM10/13/16
to Jenkins Developers, m...@batmat.net
On Thursday, October 13, 2016 at 6:59:16 PM UTC+1, Baptiste Mathus wrote:
Hi, please use the users list if this is to discuss Pipeline usage.
I'm sorry, since the big selling point of Jenkins 2 was that pipelines are now integral to the Jenkins plan, and since this is not user error I'm talking about, but bugs in the implementation, I thought that the development forum was the correct one.
Also, don't mix up the Groovy flavour in use with Pipeline, and the one (standard) you can use in the script console, as job steps, and so on.
I didn't actually realize there were two, thanks for informing me, but since the pipeline is where I actually have to do my work, that doesn't help me much.
What you seem to describe is "just" a known issue: https://issues.jenkins-ci.org/browse/JENKINS-26481 (though, granted, an important when you don't know about it).
It doesn't seem so to me, some of what I talked about is that known issue, the problem with closures. But the problem I described as a case in point doesn't seem to fall under that umbrella.
I've managed to work around the known issue, but this new problem, I don't even have a clue. When adding a couple of lines of simple and unrelated code cause a regex matcher to suddenly break, I don't even know where to start working out what's going on.
regards
Jon
Daniel Beck
Oct 13, 2016, 4:07:03 PM10/13/16
to jenkin...@googlegroups.com
> On 13.10.2016, at 20:28, Jonathan Hodgson <j.r.h...@gmail.com> wrote:
>
> since this is not user error I'm talking about, but bugs in the implementation, I thought that the development forum was the correct one.
> But the problem I described as a case in point doesn't seem to fall under that umbrella.
Jonathan Hodgson
Oct 14, 2016, 6:04:11 AM10/14/16
to Jenkins Developers, m...@beckweb.net
> But the problem I described as a case in point doesn't seem to fall under that umbrella.
Explanation here: https://github.com/jenkinsci/workflow-cps-plugin/#technical-design
Thanks. that helps explain things.
Though my confusion is increased by the fact that right now I'm having to go through previously tested and working code and move stuff to @NonCPS functions, because at some point in the various updates of jenkins and the pipeline plugins it appears to have broken stuff that was previously working. Things actually seem to be getting worse!
It's a shame that pipeline plugin doesn't at least recognize errors that might be related to this issute and add some useful information to the error message, such as a suggestion to look at that explanation you linked to. As a newbie to Groovy I have wasted so much time looking for errors in my groovy code, when in fact I had written perfectly valid groovy, just the pipeline implementation couldn't handle it.
Daniel Beck
Oct 14, 2016, 6:07:02 AM10/14/16
to Jenkins Developers
> On 14.10.2016, at 12:04, Jonathan Hodgson <j.r.h...@gmail.com> wrote:
>
> at some point in the various updates of jenkins and the pipeline plugins it appears to have broken stuff that was previously working. Things actually seem to be getting worse!
https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Groovy+Plugin#PipelineGroovyPlugin-2.14%28Sep07,2016%29
Although that changelog admittedly does look scary…
Jesse Glick
Oct 14, 2016, 9:00:13 AM10/14/16
to Jenkins Dev
On Fri, Oct 14, 2016 at 6:04 AM, Jonathan Hodgson <j.r.h...@gmail.com> wrote:
> As
> a newbie to Groovy I have wasted so much time looking for errors in my
> groovy code, when in fact I had written perfectly valid groovy
In general, do not go looking for fun and exciting Groovy language
> As
> a newbie to Groovy I have wasted so much time looking for errors in my
> groovy code, when in fact I had written perfectly valid groovy
features. Keep to the basics: `if`/`then`, `for` loops with an index,
local variables, method calls. Any kind of real programming is better
done in external processes.
Reply all
Reply to author
Forward
0 new messages