[JIRA] (JENKINS-42079) Allow "local" Shared Library to be loaded from current repo branch

3 views
Skip to first unread message

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 4:54:02 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman created an issue
 
Jenkins / Improvement JENKINS-42079
Allow "local" Shared Library to be loaded from current repo branch
Issue Type: Improvement Improvement
Assignee: Unassigned
Attachments: Screen Shot 2017-02-15 at 1.12.56 PM.png, Screen Shot 2017-02-15 at 1.24.08 PM.png
Components: workflow-cps-global-lib-plugin
Created: 2017/Feb/15 9:53 PM
Priority: Major Major
Reporter: Liam Newman

For context see: https://jenkins.io/blog/2017/02/15/declarative-notifications/
And the linked bugs, in particular JENKINS-41396.

See:
https://github.com/bitwiseman/hermann/tree/issue/jenkins-library/example

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 5:31:01 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman updated an issue
Change By: Liam Newman
For context see: https://jenkins.io/blog/2017/02/15/declarative-notifications/
And the linked bugs, in particular JENKINS-41396.


I've been informed that this is a horrible idea, but I'm going to describe it here just to make sure.


See:
https://github.com/bitwiseman/hermann/tree/issue/jenkins-library/example

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 5:57:02 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman updated an issue
For context see: https://jenkins.io/blog/2017/02/15/declarative-notifications/
And the linked bugs, in particular JENKINS-41396.

I've been informed that this is a horrible idea, but I'm going to describe it here just to make sure.

There have been a number of issues and discussions going around, dealing with how to handle variables and methods in Declarative Pipeline.



See:
https://github.com/bitwiseman/hermann/tree/issue/jenkins-library/example

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 6:01:01 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman updated an issue
Change By: Liam Newman
Component/s: pipeline-model-definition-plugin

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 6:35:01 PM2/15/17
to jenkinsc...@googlegroups.com

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 6:36:01 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman updated an issue
For context see: https://jenkins.io/blog/2017/02/15/declarative-notifications/
And the linked bugs, in particular JENKINS-41396.

I've been informed that this is a horrible idea, but I'm going to describe it here just to make sure.

There have been a number of issues and discussions going around, dealing with how to handle variables and methods in Declarative Pipeline.




See:
https://github.com/bitwiseman/hermann/tree/issue/jenkins-library/example

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 7:12:02 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman updated an issue
For context see: https://jenkins.io/blog/2017/02/15/declarative-notifications/
And the linked bugs, in particular JENKINS-41396.

I've been informed that this is a horrible idea, but I'm going to describe it here just to make sure.

There have been a number of issues and discussions going around, dealing with how to handle define variables and methods in Declarative Pipeline.

Right now, we can define variables and methods outside of the pipeline section but that is not a supported solution and it is not viable long-term.

As noted in JENKINS-41396, we have a supported way to define variables and methods that we can use in Declarative Pipeline via
Shared Libraries.  However, that means I have to spin up a separate repository just to make a helper method that might be used in only one
Pipeline.  

JENKINS-41396 and JENKINS-41335 suggest adding sections to the Declarative Pipeline model to allow these things to defined inside
the same {{Jenkinsfile}}.  This is also problematic: there are implementation issues, and it means that the distinction between the overall flow
(defined in Declarative Pipeline) and the implementation details (defined in methods, variables, and classes) becomes more blurry.

What I'm suggesting, is we reuse the functionality of "Shared libraries" but support loading from a subfolder in the current branch of the currently checked out repository.   

This would have a number of benefits over other solutions, it would:
* reuse existing (and tested) functionality, limiting risk
* not require creating a separate repository
* keep the script code out of the Declarative Pipeline Jenkinsfile
* use the same file/folder structure as Shared Library, getting users familiar with it for when they decide to use Shared Libraries

On the downside, it would:
* require additional files beyond Jenkins file (method and define would be in Jenkinsfile)
* TODO still editing.



See:
https://github.com/bitwiseman/hermann/tree/issue/jenkins-library/example

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 7:15:01 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman updated an issue
For context see: https://jenkins.io/blog/2017/02/15/declarative-notifications/
And the linked bugs, in particular JENKINS-41396.

I've been informed that this is a horrible idea, but I'm going to describe it here just to make sure.

There have been a number of issues and discussions going around, dealing with how to define variables and methods in Declarative Pipeline.

Right now, we can define variables and methods outside of the pipeline section but that is not a supported solution and it is not viable long-term.

As noted in JENKINS-41396, we have a supported way to define variables and methods that we can use in Declarative Pipeline via
Shared Libraries.  However, that means I have to spin up a separate repository just to make a helper method that might be used in only one
Pipeline.  

JENKINS-41396 and JENKINS-41335 suggest adding sections to the Declarative Pipeline model to allow these things to defined inside
the same {{Jenkinsfile}}.  This is also problematic: there are implementation issues, and it means that the distinction between the overall flow
(defined in Declarative Pipeline) and the implementation details (defined in methods, variables, and classes) becomes more blurry.

What I'm suggesting, is we reuse the functionality of "Shared libraries" but support loading from a subfolder in the current branch of the currently checked out repository.   

This would have a number of benefits over other solutions, it would:
* reuse existing (and tested) functionality, limiting risk
* not require creating a separate repository
* keep the script code out of the Declarative Pipeline Jenkinsfile
* use the same file/folder structure as Shared Library, getting users familiar with it for when they decide to use Shared Libraries

On the downside, it would:
* require additional files beyond Jenkins file (method and define would be in Jenkinsfile)
* TODO still editing. would require some changes to pipeline




See:
https://github.com/bitwiseman/hermann/tree/issue/jenkins-library/example

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 7:28:05 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman updated an issue
For context see: https://jenkins.io/blog/2017/02/15/declarative-notifications/
And the linked bugs, in particular JENKINS-41396.

I've been informed that this is a horrible idea, but I'm going to describe it here just to make sure.

There have been a number of issues and discussions going around, dealing with how to define variables and methods in Declarative Pipeline.

Right now, we can define variables and methods outside of the pipeline section but that is not a supported solution and it is not viable long-term.

As noted in JENKINS-41396, we have a supported way to define variables and methods that we can use in Declarative Pipeline via
Shared Libraries.  However, that means I have to spin up a separate repository just to make a helper method that might be used in only one
Pipeline.  

JENKINS-41396 and JENKINS-41335 suggest adding sections to the Declarative Pipeline model to allow these things to defined inside
the same {{Jenkinsfile}}.  This is also problematic: there are implementation issues, and it means that the distinction between the overall flow
(defined in Declarative Pipeline) and the implementation details (defined in methods, variables, and classes) becomes more blurry.

What I'm suggesting, is we reuse the functionality of "Shared libraries" but support loading from a subfolder in the current branch of the currently checked out repository.   

This would have a number of benefits over other solutions, it would:
* reuse existing (and tested) functionality, limiting risk
* not require creating a separate repository
* keep the script code out of the Declarative Pipeline Jenkinsfile
* use the same file/folder structure as Shared Library, getting users familiar with it for when they decide to use Shared Libraries

On the downside, it would:
* require depend on additional folder/ files beyond Jenkins file Jenkinsfile ( {{ method }} and {{ define }} would be in Jenkinsfile)
*
would require some changes to pipeline
workflow-cps-global-lib to support loading from local and from subfolder instead of root folder.
* make users have to understand library folder/file structure even to write simple helper method.

See: We had a IRC chat about this, which I've attached.  

Also attached are images from a Pipeline job that I created as a proof-of-concept.

The code for this POC is in: https://github.com/bitwiseman/hermann/tree/issue/jenkins-library/example

What is not shown/done in the POC:
* loading from a subfolder (not currently supported), subfolder name should default to "jenkins" (maybe?) and be configurable to other values.
* * Worked around by putting the {{vars}} folder in the root - this would not work if the project already had a {{src}} or {{vars}} folder.
* loading from the currently checked out branch/commit (not currently supported)
* * Worked around by hard coding the branch name
* using a {{library local}} directive to tell declarative to load a local library
* * Worked around by adding the configuration to the parent Multibranch pipeline job and loading implicitly.
* Using the already checked-out code
* * Shared library simply clones it's own copy.  For this small repo, this is fine, for larger projects it would not be acceptable.

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 7:29:01 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman updated an issue
For context see: https://jenkins.io/blog/2017/02/15/declarative-notifications/
And the linked bugs, in particular JENKINS-41396.

I've been informed that this is a horrible idea, but I'm going to describe it here just to make sure.

There have been a number of issues and discussions going around, dealing with how to define variables and methods in Declarative Pipeline.

Right now, we can define variables and methods outside of the pipeline section but that is not a supported solution and it is not viable long-term.

As noted in JENKINS-41396, we have a supported way to define variables and methods that we can use in Declarative Pipeline via
Shared Libraries.  However, that means I have to spin up a separate repository just to make a helper method that might be used in only one
Pipeline.  

JENKINS-41396 and JENKINS-41335 suggest adding sections to the Declarative Pipeline model to allow these things to defined inside
the same {{Jenkinsfile}}.  This is also problematic: there are implementation issues, and it means that the distinction between the overall flow
(defined in Declarative Pipeline) and the implementation details (defined in methods, variables, and classes) becomes more blurry.

What I'm suggesting, is we reuse the functionality of "Shared libraries" but support loading from a subfolder in the current branch of the currently checked out repository.   

This would have a number of benefits over other solutions, it would:
* reuse existing (and tested) functionality, limiting risk
* not require creating a separate repository
* keep the script code out of the Declarative Pipeline Jenkinsfile
* use the same file/folder structure as Shared Library, getting users familiar with it for when they decide to use Shared Libraries

On the downside, it would:
* depend on additional folder/files beyond Jenkinsfile ({{method}} and {{define}} would be in Jenkinsfile)
* require some changes to workflow-cps-global-lib to support loading from local and from subfolder instead of root folder.

* make users have to understand library folder/file structure even to write simple helper method.

We had a IRC chat about this, which I've attached.  

Also attached are images from a Pipeline job that I created as a proof-of-concept.
The code for this POC is in:  https://github.com/bitwiseman/hermann/tree/issue/jenkins-library/example

What is not shown/done in the POC:
* loading from a subfolder (not currently supported), subfolder name should default to "jenkins" (maybe?) and be configurable to other values.
* * Worked around by putting the {{vars}} folder in the root - this would not work if the project already had a {{src}} or {{vars}} folder.
* loading from the currently checked out branch/commit (not currently supported)
* * Worked around by hard coding the branch name
* using a {{library local}} directive to tell declarative to load a local library
* * Worked around by adding the configuration to the parent Multibranch pipeline job and loading implicitly.
* Using the already checked-out code
* * Shared library simply clones it's own copy.  For this small repo, this is fine, for larger projects it would not be acceptable.

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 7:39:01 PM2/15/17
to jenkinsc...@googlegroups.com
Liam Newman commented on Improvement JENKINS-42079
 
Re: Allow "local" Shared Library to be loaded from current repo branch

Some have suggested that evaluate or readTrusted would be preferable solutions.
They are fine for loading individual files, but not for more than one. Also, they behave differently from code in shared libraries, meaning users will have learn different behaviors for each.
These also have drawbacks in areas such as testing, support, and documentation. If we guide users towards one consistent way to load things (Libraries) we do not have to explain it multiple times and fixes that help Shared Libraries would generally help local libraries as well.

bitwiseman@gmail.com (JIRA)

unread,
Feb 15, 2017, 7:43:01 PM2/15/17
to jenkinsc...@googlegroups.com

One other note: even if we implement the method or define sections, I believe we should still consider this option. It provides another helpful interim step between on the way to Shared Libraries.

bitwiseman@gmail.com (JIRA)

unread,
Feb 16, 2017, 8:34:01 PM2/16/17
to jenkinsc...@googlegroups.com

jglick@cloudbees.com (JIRA)

unread,
Mar 3, 2017, 4:36:02 PM3/3/17
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
Change By: Jesse Glick
Component/s: workflow-cps-global-lib-plugin

mkobit@gmail.com (JIRA)

unread,
Apr 11, 2018, 1:10:02 PM4/11/18
to jenkinsc...@googlegroups.com

jon.sten@gmail.com (JIRA)

unread,
Apr 12, 2018, 12:59:02 AM4/12/18
to jenkinsc...@googlegroups.com

andretmcarmo@gmail.com (JIRA)

unread,
Dec 11, 2019, 2:43:02 PM12/11/19
to jenkinsc...@googlegroups.com

I would really like this to be implemented. There is a scenario where this makes perfectly sense.

 

Jenkins Shared Libraries are code, and because of that, me and my team we follow best practices. And of them is to have unit tests for our code. Because of this we have a Jenkins pipeline for the Jenkins Shared Library. We need to test against the current version of the code without manually managing versions.

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

andretmcarmo@gmail.com (JIRA)

unread,
Dec 11, 2019, 2:47:03 PM12/11/19
to jenkinsc...@googlegroups.com
André Carmo edited a comment on Improvement JENKINS-42079
Reply all
Reply to author
Forward
0 new messages