[JIRA] (JENKINS-56500) Declarative pipeline restricted in code size

Change By:	efo plo
Summary:	Declerative Declarative pipeline restricted in code size

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

bitwiseman@gmail.com (JIRA)

unread,

Oct 22, 2019, 9:27:02 PM10/22/19

to jenkinsc...@googlegroups.com

Liam Newman assigned an issue to Liam Newman

Jenkins /

Change By:	Liam Newman
Assignee:	Andrew Bayer Liam Newman

bitwiseman@gmail.com (JIRA)

unread,

Oct 22, 2019, 9:34:28 PM10/22/19

to jenkinsc...@googlegroups.com

Liam Newman commented on

https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/355

This PR will address this issue for scripts that do not use "def" variables before the "pipeline {}" block.

Your example uses defs so it would still not be fixed. Still, this is progress.

wim.gaethofs@gmail.com (JIRA)

unread,

Oct 23, 2019, 4:24:03 AM10/23/19

to jenkinsc...@googlegroups.com

Wim Gaethofs commented on

Thanks for looking into this issue. I'm glad to hear there is progress in fixing the issue.

bitwiseman@gmail.com (JIRA)

unread,

Nov 4, 2019, 5:24:04 PM11/4/19

to jenkinsc...@googlegroups.com

Liam Newman updated

Jenkins /

Change By:	Liam Newman
Status:	In Progress Review

bitwiseman@gmail.com (JIRA)

unread,

Nov 4, 2019, 5:24:05 PM11/4/19

to jenkinsc...@googlegroups.com

Liam Newman started work on

Change By:	Liam Newman
Status:	Open In Progress

bitwiseman@gmail.com (JIRA)

unread,

Nov 4, 2019, 5:29:02 PM11/4/19

to jenkinsc...@googlegroups.com

Liam Newman commented on

There is partial fix for this in v1.4.0. Due to the extent to which it change how pipelines are executed it is turned off by default. It can be turned on by setting the property {org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true}. As noted, this still works to some extent for pipelines using `def` variables, but not as well.

Please give it a try and provide feedback.

bitwiseman@gmail.com (JIRA)

unread,

Nov 4, 2019, 5:31:05 PM11/4/19

to jenkinsc...@googlegroups.com

Liam Newman edited a comment on

There is partial fix for this in v1.4.0. Due to the extent to which it change how pipelines are executed it is turned off by default. It can be turned on by setting the a JVM property \ (either on the command-line or in Jenkins script console):

{ { org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true }. }}

As noted, this still works to some extent for pipelines using `def` variables, but not as well.

Please give it a try and provide feedback.

bitwiseman@gmail.com (JIRA)

unread,

Nov 4, 2019, 5:32:03 PM11/4/19

to jenkinsc...@googlegroups.com

Liam Newman updated

Jenkins /

Change By:	Liam Newman
Status:	In Review Resolved
Resolution:	Fixed

eplotkin@drivenets.com (JIRA)

unread,

Nov 5, 2019, 2:49:04 AM11/5/19

to jenkinsc...@googlegroups.com

efo plo commented on

Thanks Liam Newman.

With your fix, can the attachment be compiled and run?

bitwiseman@gmail.com (JIRA)

unread,

Nov 5, 2019, 1:41:03 PM11/5/19

to jenkinsc...@googlegroups.com

Liam Newman commented on

efo plo

Yes, even with the "def" variables, it was able to run the example without encountering "method too large".

moglimcgrath@yahoo.com (JIRA)

unread,

Dec 10, 2019, 8:27:03 AM12/10/19

to jenkinsc...@googlegroups.com

Michael McGrath commented on

Liam Newman

Updated plugin, removed any def vars, set JVM property on script console.

I am still encountering "method too large" on a test job.

Reaching this issue on a shared library approach as well, where we have stripped everything out into vars.

Work arounds?

moglimcgrath@yahoo.com (JIRA)

unread,

Dec 10, 2019, 8:29:02 AM12/10/19

to jenkinsc...@googlegroups.com

M McGrath edited a comment on

[~bitwiseman]

Updated plugin, removed any def vars, set JVM property on script console.

I am still encountering "method too large" on a test job.

Reaching this issue on a shared library approach as well, where we have stripped everything out into vars / global methods .

Work arounds?

bitwiseman@gmail.com (JIRA)

unread,

Dec 10, 2019, 8:54:06 AM12/10/19

to jenkinsc...@googlegroups.com

Liam Newman commented on

I'm on vacation for the next week. When I get back I'll add a patch to log when this feature is turned on. Maybe some debug information about what it is doing.
There should not be a need for further workarounds. How big is your pipeline? Would you be willing to share it with me for debugging?

moglimcgrath@yahoo.com (JIRA)

unread,

Dec 17, 2019, 8:38:04 AM12/17/19

to jenkinsc...@googlegroups.com

M McGrath commented on

Liam Newman Still hitting this size limit issue with declarative with everything broken out into shared libraries.

My code is predominantly in /vars. Templated pipelines with each stage broken out into global vars. Jenkinsfile passes pipeline params.

At this point i'm starting to test breaking logic out into classes /scr, and a move to scripted pipelines. Im not fully sure how the jenkins compiles declarative apart from the 64K size limit. We really liked alot of what declarative pipelines offer, but the file size issue is a pain.

Is a move to scripted the right option?

moglimcgrath@yahoo.com (JIRA)

unread,

Dec 17, 2019, 11:15:03 AM12/17/19

to jenkinsc...@googlegroups.com

M McGrath edited a comment on

[~bitwiseman] Still hitting this size limit issue with declarative with everything broken out into shared libraries.

My code is predominantly in /vars. Templated pipelines with each stage broken out into global vars. Jenkinsfile passes pipeline params.

At this point i'm starting to test breaking logic out into classes /scr, and a move to scripted pipelines. Im not fully sure vey techie when it come to how the jenkins compiles declarative apart from the 64K size limit. We really liked alot of what declarative pipelines offer, but the file size issue is a pain.

Is a move to scripted the right option?

moglimcgrath@yahoo.com (JIRA)

unread,

Dec 17, 2019, 11:16:03 AM12/17/19

to jenkinsc...@googlegroups.com

M McGrath edited a comment on

[~bitwiseman] Still hitting this size limit issue with declarative with everything broken out into shared libraries.

My code is predominantly in /vars. Templated pipelines with each stage broken out into global vars. Jenkinsfile passes pipeline params.

At this point i'm starting to test breaking logic out into classes /scr, and a move to scripted pipelines. Im vey techie when it come to how jenkins compiles declarative apart from the 64K size limit. We really liked alot of what declarative pipelines offer, but the file size issue is a pain.

Is a move to scripted the right option?

bitwiseman@gmail.com (JIRA)

unread,

Dec 17, 2019, 6:40:03 PM12/17/19

to jenkinsc...@googlegroups.com

Liam Newman commented on

M McGrath
There are reasons to move to Scripted but this is not one of them. In fact, this issue is a reason not to move to scripted. This error can occur in declarative or scripted, but I have no ability to automatically work around it scripted.

Perhaps we can try this. Take the Jenkinsfile attached to this issue and try to run it on your Jenkins (without the JVM property). Make sure it encounters the error.
Do what you've done before to turn on the fix. Make sure that after that the attached Jenkinsfile does not report the error. That way we can at least be sure that the fix is active on your system.

"Templated pipelines with each stage broken out into global vars." - This doesn't sound like Declarative, but maybe I'm not understanding what you're meaning. Can you at least provide an example of what your pipeline looks like (even if you can show your actual pipeline)?

mbrunton@eidosmontreal.com (JIRA)

unread,

May 5, 2020, 4:15:03 PM5/5/20

to jenkinsc...@googlegroups.com

Matthew Brunton commented on

Liam Newman

We have also been running into the method code too large. I was able to get a workaround by switching to a hybrid scripted/declarative, but have been trying various things to get back to declarative.

I've tried various methods, including having everything under vars, a mixture of vars and src, and playing around with the various classes we have. Eventually I commented out chunks of code until the pipeline stopped giving "method code too large" and then started adding stuff back.

I think I've narrowed it down to the "when {}" stage conditionals. I'm currently using 25 parallel stages (each one has 2 stages, and they are skeletal because of my commented out code) and I've changed all the conditionals to "when { expression

{true}

}". Some of our conditionals use anyOf {} and not {}, but I set everything to true in order to rule out the expressions themselves.

Without the conditionals, there is no "method code too large" error. When I add back the conditionals, the error returns.

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)

bitwiseman@gmail.com (JIRA)

unread,

May 5, 2020, 6:26:02 PM5/5/20

to jenkinsc...@googlegroups.com

Liam Newman updated an issue

Jenkins /

Change By:	Liam Newman

{quote}
There is partial fix for this in v1.4.0. Due to the extent to which it change how pipelines are executed it is turned off by default. It can be turned on by setting a JVM property (either on the command-line or in Jenkins script console):

{{org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true }}

As noted, this still works to some extent for pipelines using `def` variables, but not nearly as well.
This also does not work if the {{

Please give it a try and provide feedback.

{quote}

When working with a declarative pipeline script, we run into an error "Method Code too large".

This seems to happen when the code between 'pipeline{}' is more than a specific size.

I'm creating this issue for this component following the suggestion of Jesse Glick at the issue 37984

I've attached a declarative pipeline script that reproduces the issue.

bitwiseman@gmail.com (JIRA)

unread,

May 5, 2020, 6:33:03 PM5/5/20

to jenkinsc...@googlegroups.com

Liam Newman updated an issue

Jenkins /

Change By:	Liam Newman

{quote}
There is partial fix for this in v1.4.0. Due to the extent to which it change how pipelines are executed it is turned off by default. It can be turned on by setting a JVM property (either on the command-line or in Jenkins script console):

{{org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true }}

As noted, this still works to some extent for pipelines using `def` variables, but not nearly as well.

This also generally does not work if the {{ pipeline}} directive inside a shared library method. If this is a scenario you want, please come join the pipeline authoring SIG and we can discuss.

Please give it a try and provide feedback.
{quote}

When working with a declarative pipeline script, we run into an error "Method Code too large".

This seems to happen when the code between 'pipeline{}' is more than a specific size.

I'm creating this issue for this component following the suggestion of Jesse Glick at the issue 37984

I've attached a declarative pipeline script that reproduces the issue.

bitwiseman@gmail.com (JIRA)

unread,

May 5, 2020, 6:36:05 PM5/5/20

to jenkinsc...@googlegroups.com

Liam Newman updated an issue

Jenkins /

Change By:	Liam Newman

{quote}
There is partial fix for this in pipeline-model-definition-plugin v1.4.0 and later . Due to the extent to which it change how pipelines are executed it is turned off by default. It can be turned on by setting a JVM property (either on the command-line or in Jenkins script console):

{{org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true }}

As noted, this still works best with a Jenkinsfile with {{pipeline}} directive as the only root item in the file.
This workaround also works to some extent for pipelines using `def` variables before the {{pipeline}} directive , but not nearly as well.
This also workaround generally does not NOT work if the {{pipeline}} directive inside a shared library method. If this is a scenario you want, please come join the pipeline authoring SIG and we can discuss.

Please give it a try and provide feedback.
{quote}

When working with a declarative pipeline script, we run into an error "Method Code too large".

This seems to happen when the code between 'pipeline{}' is more than a specific size.

I'm creating this issue for this component following the suggestion of Jesse Glick at the issue 37984

I've attached a declarative pipeline script that reproduces the issue.

bitwiseman@gmail.com (JIRA)

unread,

May 5, 2020, 6:36:05 PM5/5/20

to jenkinsc...@googlegroups.com

Liam Newman commented on

Matthew Brunton

Have you tried the feature flag? (See description for note)

Do you have any def variables before (or after) your pipeline directive in your Jenkinsfile?
Is your pipeline directive inside a shared library method?

mbrunton@eidosmontreal.com (JIRA)

unread,

May 6, 2020, 8:40:02 AM5/6/20

to jenkinsc...@googlegroups.com

Matthew Brunton commented on

Liam Newman

Yes, I have the feature flag turned on. I also got rid of all def variables outside of the pipeline directive, and I even got rid of all script blocks inside just in case.

We do run inside a shared library method, however. Our pipeline is version controlled separately. I see now that this could be the reason why the feature flag workaround isn't working.

mbrunton@eidosmontreal.com (JIRA)

unread,

May 6, 2020, 9:43:03 AM5/6/20

to jenkinsc...@googlegroups.com

Matthew Brunton edited a comment on

[~bitwiseman]

Yes, I have the feature flag turned on. I also got rid of all def variables outside of the pipeline directive, and I even got rid of all script blocks inside just in case.

We do run inside a shared library method, however. Our pipeline is version controlled separately. I see now that this could be the reason why the feature flag workaround isn't working.

I tried copying the pipeline directive into the Jenkinsfile so the only thing outside of the pipeline is the " @ Library ( ) _ " . I still get the error.

mbrunton@eidosmontreal.com (JIRA)

unread,

May 6, 2020, 3:27:04 PM5/6/20

to jenkinsc...@googlegroups.com

Matthew Brunton commented on

I ran some tests with a fresh bare-bones script and a fresh Jenkins, both with and without the feature flag. With the flag on, I do indeed get a few more stages before I hit the error.

This is a typical stage, and I'm able to get 27 in parallel without the flag. With the flag, I can get somewhere between 30 and 35 stages.

 
                                                                def call()
{ 
  pipeline 
  { 
    agent any 
    stages
    { 
      stage('Build') 
      { 
        parallel 
        {

...

stage('Parallel Stage 35')
{ 
    when {expression {true}}
    stages { 
    stage('35-A')
    {
        when 
        { 
            anyOf
            { 
                expression { true } 
                not { expression { true } }
            } 
        } 
        steps
        { 
            testMethod('TEST') 
        } 
    } 
    stage('35-B') 
    { 
        options { timeout(time: 1, unit: 'HOURS') }
        when 
        { 
            anyOf 
            { 
                expression { true } 
                not { expression { true } } 
            } 
        } 
        steps 
        { 
            testMethod('TEST') 
        } 
    } 
    } 
} 
                                                            

The call to testMethod is just another file under vars:

 
                                                                // vars/testMethod.groovy

def call(String testInput)
{    
    def testVariable
}

mbrunton@eidosmontreal.com (JIRA)

unread,

May 6, 2020, 3:34:08 PM5/6/20

to jenkinsc...@googlegroups.com

Matthew Brunton edited a comment on

I ran some tests with a fresh bare-bones script and a fresh Jenkins, both with and without the feature flag. With the flag on, I do indeed get a few more stages before I hit the error.

This is a typical stage, and I'm able to get 27 in parallel without the flag. With the flag, I can get somewhere between 30 and 35 stages.

{code:java}

}{code}

The call to testMethod is just another file under vars:

{code:java}

// vars/testMethod.groovy

def call(String testInput)
{
def testVariable
}

{code}

I get a similar result when I move the pipeline into the Jenkinsfile

bitwiseman@gmail.com (JIRA)

unread,

May 6, 2020, 4:25:03 PM5/6/20

to jenkinsc...@googlegroups.com

Liam Newman commented on

Matthew Brunton
I just found out the shared library method limitation this week, due to help from part out this week.

Would you be willing to share your Jenkinsfile? The last one you mentioned that still gets the error.

It would also be useful to me to see the one before this - where your pipeline is in a shared library method. For this one, I only need the Jenkinsfile and the groovy file with the shared library method containing your pipeline directive.

bitwiseman@gmail.com (JIRA)

unread,

May 6, 2020, 4:28:04 PM5/6/20

to jenkinsc...@googlegroups.com

Liam Newman edited a comment on