each method on maps not working in jenkins pipeline groovy?

[Jonathan Hodgson]

Hi,

this has me mystefied.

I am trying to write groovy code for a pipeline build which extracts some information from a config file, and in the process have run into what seems like a bug, but perhaps I'm just not understanding something.

As a stand alone test to check my code I produced the following groovy app

package buildtest
import static groovy.json.JsonOutput.*




class firstGroovy {
static  common_string = '''
if (__common_cmake__)
    return()
endif ()


#Generators OSX = "Xcode", Win32 = "Visual Studio 12 2013", Win64 = "Visual Studio 12 2013 Win64"
SET(CMAKE_CONFIGURATION_TYPES "Debug;Release" CACHE STRING "limited configs" FORCE)


SET ( PREFIX_HEADER "${wdlol_DIR}/compiler_helpers/compiler_helpers.h" )




'''


    static getBuildTypes()
    {
        def file_path = "..."
        def matcher = common_string =~ /(?:\s*#\s*(Generators)\s+(.+?"[^"]+"\s*(?:,\s+.+?"[^"]+"\s*)+))|(?:\s*(CMAKE_CONFIGURATION_TYPES)\s+"(.+?(?:;.+?)*)")/
        if (!matcher)
        {
            throw new Exception("Configuration information not found in ${file_path}")
        }
        def configurations = []
        def generators = [:]                
        for (def index = 0; index < matcher.size(); index++)
        {
            if (matcher[index][1] == "Generators")
            {
                if (generators)
                {
                    throw new Exception("Multiple generator definitions found in ${file_path}")
                }
                def generator_list = matcher[index][2].split(",")
                generator_list.each{
                    def generator = it.split("=")
                    generators[generator[0].trim()] = generator[1].trim()
                }
            }
        }
        println prettyPrint(toJson(generators))
        println "Generators ${generators.size()}"
        println ("each { entry ->")
        generators.each{entry -> println "$entry.key : $entry.value" }
        println "each { entry, key ->"
        generators.each{key, value -> println "$key : $value" }
    }


    static void main(def args)
    {
        println GroovySystem.version
         getBuildTypes()
       
    }  
}

common_string is just the relevant lines taken from the config file, which happens to be named common.cmake

Now if I run that from eclipse on my windows machine (groovy 2.4.3) or on the machine that jenkins is running on (groovy 2.4.7) then the output I get is

{
    "OSX": "\"Xcode\"",
    "Win32": "\"Visual Studio 12 2013\"",
    "Win64": "\"Visual Studio 12 2013 Win64\""
}
Generators 3
each { entry ->
OSX : "Xcode"
Win32 : "Visual Studio 12 2013"
Win64 : "Visual Studio 12 2013 Win64"
each { entry, key ->
OSX : "Xcode"
Win32 : "Visual Studio 12 2013"
Win64 : "Visual Studio 12 2013 Win64"

So exactly what I expect in each case.

However the same code (with the exception that I made GetBuildTypes non static) run as part of a pipeline job, and I get

(I've taken out the "[Pipeline] echo" lines

{
    "OSX": "\"Xcode\"",
    "Win32": "\"Visual Studio 12 2013\"",
    "Win64": "\"Visual Studio 12 2013 Win64\""
}
Generators 3
each { entry ->
OSX : "Xcode"
each { entry, key ->
OSX="Xcode" : null

As you can see, json extracts the correct keys and values out of the map, but neither version of "each" produces what I'd expext.

Any comments? Do I need to be filing a bug?

[Jesse Glick]

You cannot currently call methods like `each` taking a closure from
Pipeline script if the methods are defined in Groovy itself. We are
investigating a fix but it is very difficult.

If you want to iterate a map, try using the helper method

@NonCPS def entries(m) {m.collect {k, v -> [k, v]}}

which “flattens” it to a list of lists. Then you can use C-style
`for`-loops (with a loop index).

[Jonathan Hodgson]

Thanks,

Are these sorts of issues documented anywhere? I lost a day on this, and thought I must be losing my sanity at one point because if you don't understand the inner working of groovy (and I currently don't) then it's a very mysterious looking problem.

[Jesse Glick]

On Wed, Jul 13, 2016 at 11:24 AM, Jonathan Hodgson
<j.r.h...@gmail.com> wrote:
> Are these sorts of issues documented anywhere?

Not really I am afraid. A more complete user guide is needed.

[Liam Newman]

What is JIRA ID for the work to enable using "each" and other standard groovy iterators? 

I would also suggest that this is not a user guide issue but a matter of proper feature specification.  There need to be a clear statement of what parts of groovy we do and do not support.

[Mark Waite]

I disagree that it is "a matter of proper feature specification", or possibly I'm misunderstanding what you mean by "proper feature specification".

The existing groovy pipeline code is a great help, even without a precise specification of the groovy features which are not available.  I believe that the rapid evolution of the pipeline capabilities is valuable to more people (and more valuable) than detailed documentation of a set of slower evolving pipeline capabilities.

Mark Waite

On Fri, Jul 15, 2016 at 3:04 PM Liam Newman <bitwi...@gmail.com> wrote:

What is JIRA ID for the work to enable using "each" and other standard groovy iterators? 

I would also suggest that this is not a user guide issue but a matter of proper feature specification.  There need to be a clear statement of what parts of groovy we do and do not support.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5a58ba28-09be-4fb1-b9ec-404101b2fb38%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Baptiste Mathus]

Le 15 juil. 2016 11:04 PM, "Liam Newman" <bitwi...@gmail.com> a écrit :
>
> What is JIRA ID for the work to enable using "each" and other standard groovy iterators? 

https://issues.jenkins-ci.org/browse/JENKINS-26481