[JIRA] (JENKINS-56567) Build fails when reading more than 125 environment credentials at the same time

9 views
Skip to first unread message

thomas.bessou@hotmail.fr (JIRA)

unread,
Mar 14, 2019, 1:28:02 PM3/14/19
to jenkinsc...@googlegroups.com
Thomas Bessou created an issue
 
Jenkins / Bug JENKINS-56567
Build fails when reading more than 125 environment credentials at the same time
Issue Type: Bug Bug
Assignee: Unassigned
Components: credentials-binding-plugin
Created: 2019-03-14 17:27
Environment: jenkins-2.150.3-1.1
Credentials Binding Plugin v1.18
Labels: map 125
Priority: Blocker Blocker
Reporter: Thomas Bessou

 

Builds fail with the following error when reading more than 125 credentials into the environment in the Jenkinsfile.

This prevents us from deploying our services, hence is very blocking.

 

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: -1: Map expressions can only contain up to 125 entries @ line -1, column -1.
1 error

	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.lambda$doParse$0(CpsGroovyShell.java:135)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:136)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:132)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE

It seems weird to use a data structure that only supports up to 125 values in this context, since performance is not ever going to be blocking there.

 

 

Note that this error does not happen if the credentials are registered in Jenkins and the environment variables are created from string litterals, which is why I beleive the problem comes from this plugin.
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

thomas.bessou@hotmail.fr (JIRA)

unread,
Mar 14, 2019, 1:29:06 PM3/14/19
to jenkinsc...@googlegroups.com
Thomas Bessou assigned an issue to Jesse Glick
Change By: Thomas Bessou
Assignee: Jesse Glick

jglick@cloudbees.com (JIRA)

unread,
Mar 18, 2019, 3:21:02 PM3/18/19
to jenkinsc...@googlegroups.com
Jesse Glick assigned an issue to Andrew Bayer
 

There are no steps to reproduce but from the sound of it I assume this is something specific to Declarative Pipeline, as the credentials-binding plugin itself does nothing special with Groovy.
Change By: Jesse Glick
Component/s: pipeline-model-definition-plugin
Component/s: credentials-binding-plugin
Priority: Blocker Critical
Assignee: Jesse Glick Andrew Bayer

thomas.bessou@hotmail.fr (JIRA)

unread,
Mar 20, 2019, 5:13:39 AM3/20/19
to jenkinsc...@googlegroups.com
Thomas Bessou updated an issue
Change By: Thomas Bessou
 

 Builds fail with the following error when reading more than 125 credentials into the environment in the Jenkinsfile.

This prevents us from deploying our services, hence is very blocking.
 
{code:java}

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: -1: Map expressions can only contain up to 125 entries @ line -1, column -1.
1 error

at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.lambda$doParse$0(CpsGroovyShell.java:135)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:136)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:132)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
{code}

 It seems weird to use a data structure that only supports up to 125 values in this context, since performance is not ever going to be blocking there.

 

  
Note that this error does not happen if the credentials are registered in Jenkins and the environment variables are created from string litterals, which is why I beleive the problem comes from this plugin (credentials-binding) .

dnusbaum@cloudbees.com (JIRA)

unread,
Mar 26, 2019, 2:37:02 PM3/26/19
to jenkinsc...@googlegroups.com
Devin Nusbaum commented on Bug JENKINS-56567
 
Re: Build fails when reading more than 125 environment credentials at the same time

The underlying issue is the same as JENKINS-47363. Here is the commit to groovy-cps that added the error message for map expressions with more than 125 entries. Maybe something could be done in Declarative to change the underlying construct used for environment blocks so that it isn't using Groovy map expressions directly.

thomas.bessou@hotmail.fr (JIRA)

unread,
Apr 14, 2019, 7:46:02 AM4/14/19
to jenkinsc...@googlegroups.com

Maybe Groovy should use the fix that was proposed here since it works around the problem instead of keeping that crazy limitation?

thomas.bessou@hotmail.fr (JIRA)

unread,
Apr 14, 2019, 7:47:03 AM4/14/19
to jenkinsc...@googlegroups.com
Thomas Bessou edited a comment on Bug JENKINS-56567
Maybe Groovy should use the fix that was proposed [here|https://github.com/cloudbees/groovy-cps/pull/78] since it works around the problem instead of keeping that crazy limitation?

Or dig a little more into why there are only 125 entries allowed in that data structure and maybe use another one ?

1petemoore@gmail.com (JIRA)

unread,
Oct 15, 2019, 2:06:02 PM10/15/19
to jenkinsc...@googlegroups.com

In case there is any problems recreating this it's present in scripted pipelines. This code will produce test.txt to reproduce the error

bash-4.2$ python << EOF | tee test.txt
print('node

{\n parallel(\n') for i in range(200): print(' item%s : \{ sh "echo this is item %s" }

,' % (i,i))
print(' )\n}\n')
EOF

copy paste the output from that to jenkins window or run load "test.txt". and you get this error

WorkflowScript: 4: Map expressions can only contain up to 125 entries @ line 4, column 12.
item0 : { sh "echo this is item 0" },
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

1petemoore@gmail.com (JIRA)

unread,
Oct 15, 2019, 2:08:03 PM10/15/19
to jenkinsc...@googlegroups.com

I should add that this is limiting on large projects where you want to make a lot of jobs go parallel. 

1petemoore@gmail.com (JIRA)

unread,
Oct 15, 2019, 2:13:03 PM10/15/19
to jenkinsc...@googlegroups.com
Pete Moore edited a comment on Bug JENKINS-56567
In case there is any problems recreating this it's present in scripted pipelines. This code will produce test.txt to reproduce the error

 {{}}{{ bash-4.2$ python << EOF | tee test.txt }}
{{ print('node {\n parallel(\n') }}
{{ for i in range(200): }}
{{  print('    item%s : \{ sh "echo this is item %s" },' % (i,i)) }}
{{ print(' )\n}\n') }}
{{ EOF }}

copy paste the output from that to jenkins window or run load "test.txt". and you get this error

WorkflowScript: 4: Map expressions can only contain up to 125 entries @ line 4, column 12.
item0 : \{ sh "echo this is item 0" },

1petemoore@gmail.com (JIRA)

unread,
Oct 15, 2019, 2:13:03 PM10/15/19
to jenkinsc...@googlegroups.com

dnusbaum@cloudbees.com (JIRA)

unread,
Oct 15, 2019, 2:29:03 PM10/15/19
to jenkinsc...@googlegroups.com

Pete Moore This limitation only affects map and list literals. In scripted Pipeline, you should be able to adapt your script as a workaround. Instead of creating a parallel step with branches inline like this:

parallel(branch1: { ... }, branch2: { ... })

Create the branches separately like this:

Map branches = []
branches['branch1'] = { ... }
branches['branch2'] = { ... }
parallel(branches)

dnusbaum@cloudbees.com (JIRA)

unread,
Oct 15, 2019, 2:30:02 PM10/15/19
to jenkinsc...@googlegroups.com
Devin Nusbaum edited a comment on Bug JENKINS-56567
[~pete312] This limitation only affects map and list literals. In scripted Pipeline, you should be able to adapt your script as a workaround. Instead of creating a parallel step with branches inline like this:
{code:java}

parallel(branch1: { ... }, branch2: { ... })
{code}

Create the branches separately like this:
{code:java}
Map branches = [ : ]

branches['branch1'] = { ... }
branches['branch2'] = { ... }
parallel(branches)
{code}

1petemoore@gmail.com (JIRA)

unread,
Oct 15, 2019, 4:41:03 PM10/15/19
to jenkinsc...@googlegroups.com

Thank you Devin your workaround is a lot cleaner than my workaround which was this.

parallel(
  group_0 : { 
    parallel( 
      item0 : { sh "echo this is item 0" },
      item1 : { sh "echo this is item 1" },
      ...
    )
  },
  group_1 : { 
    parallel( 
      item200 : { sh "echo this is item 200" },
      item201 : { sh "echo this is item 201" },
      ...
    )
  },
)

Is the limit in Jenkins or groovy? If the limit is in Jenkins can we look forward to a fix?  New users (Ive talked to) like me are pretty stunned by such a small limit.
Reply all
Reply to author
Forward
0 new messages