[JIRA] (JENKINS-49135) Support for reusable sharing Declarative directives

8 views
Skip to first unread message

tobilarscheid@gmail.com (JIRA)

unread,
Apr 3, 2018, 11:45:02 AM4/3/18
to jenkinsc...@googlegroups.com
Tobias Larscheid commented on New Feature JENKINS-49135
 
Re: Support for reusable sharing Declarative directives

Hi Andrew Bayer, can you elaborate a little more on why it'd be a pain in the ass to support stages? Because I feel like you wouldn't be very happy about JENKINS-50548 then...

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

andrew.bayer@gmail.com (JIRA)

unread,
Apr 3, 2018, 12:09:04 PM4/3/18
to jenkinsc...@googlegroups.com

Tobias Larscheid - parsing hell, largely. Declarative is all a bundle of fun Groovy AST manipulation at its core. Initially, that was just for compile-time validation and for transformation into an intermediate JSON format for use with the Blue Ocean Pipeline Editor, but Pipeline's CPS transformations made a traditional Groovy closure-based DSL interpretation approach too painful after a while, so now we actually parse the pipeline block into an internal representation at compile time, use that internal representation for validation, and then transform that internal representation into the actual runtime model that tells us what to execute when, etc, via some janky Groovy AST manipulation.

Figuring out how to mash together the parsing, the validation, the runtime transformation, and the ability to actually specify parts of the pipeline block elsewhere and consume them as method calls/objects within the pipeline block is something I have not managed to achieve yet. I'm not averse to the idea, but I am wary of making this already overly-complex tooling even more complex, making maintainability even harder, potentially breaking existing things, etc.

Feel free to take a look at the code - https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/master/pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/ModelParser.groovy is probably the place to start looking. That gets called at the time that the Jenkinsfile is compiled - it does the parsing from the raw Groovy AST into the internal representation, calls the validation, and finally transforms the Groovy AST into various constructor calls so that the pipeline block actually ends up returning a https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/master/pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Root.groovy instance to the runtime interpreter for actual execution. It's messy. I apologize.

tobilarscheid@gmail.com (JIRA)

unread,
Apr 3, 2018, 2:04:02 PM4/3/18
to jenkinsc...@googlegroups.com

woops, what a can of worms I opened here  thanks for your detailed explanation, if anything I at least understood it's way more complex than I was hoping. If I find the time I will give the code a look, happy to year that you are not averse to the idea itself. Thank you!

andrew.bayer@gmail.com (JIRA)

unread,
Apr 4, 2018, 11:24:02 AM4/4/18
to jenkinsc...@googlegroups.com

So fwiw, https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/260 may end up being the beginning of a way to kinda do this. I'm doing it for a more complex proprietary use case, and it does require an actual Jenkins plugin to implement a new directive to do the replacement, but I suppose it's theoretically possible to build something off the new extension point there to provide a new directive like, say, stageFromLibrary library: "some-lib@master", method: "someMethodInLib" that would (through some more complex tooling than is available currently) go get and run that method and insert the result into the model before validation and transformation occur... We'll see. This is very early prototype stuff at this point - I threw the PR up mainly because I like to have a PR to point to when I'm doing downstream prototyping work, but I think this would open options for someone to implement tooling to satisfy the requests behind this ticket.

andrew.bayer@gmail.com (JIRA)

unread,
Feb 26, 2019, 2:27:03 PM2/26/19
to jenkinsc...@googlegroups.com
Andrew Bayer stopped work on New Feature JENKINS-49135
 
Change By: Andrew Bayer
Status: In Progress Open
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

dominik_heim@gmx.de (JIRA)

unread,
Sep 10, 2019, 8:16:03 AM9/10/19
to jenkinsc...@googlegroups.com

This would help absolutely to clean up Jenkinsfile and make also common code possible outside of steps within stages. Really looking forward to this. 

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

daniel.geissler@salt-solutions.de (JIRA)

unread,
Nov 14, 2019, 5:12:04 AM11/14/19
to jenkinsc...@googlegroups.com

The linked PR is closed since Nov 2018. Have there been any other ideas or actions on this ticket?

tavin.cole@gmail.com (JIRA)

unread,
Apr 18, 2020, 8:19:06 PM4/18/20
to jenkinsc...@googlegroups.com

Well if you want to mess around with Groovy AST, it does work to write something like this:

pipeline {
    @MyAnnotation options
    ...
}

You can define @MyAnnotation just before calling the pipeline, e.g.:

@org.codehaus.groovy.transform.GroovyASTTransformationClass(['mypkg.MyTransformation'])
@interface MyAnnotation {}

MyTransformation will need to be in a global shared library, loaded automatically or with @Library. Then you have to replace the annotated statement with valid pipeline syntax. If the transformation is written in Groovy, it must be @NonCPS, and I've noticed that any errors don't seem to be logged – Jenkins just goes on and tries to execute the untransformed pipeline, which then fails syntax validation.

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

tavin.cole@gmail.com (JIRA)

unread,
Apr 19, 2020, 2:35:04 PM4/19/20
to jenkinsc...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages