[JIRA] (JENKINS-60969) Unwanted job interaction if using global pipeline libraries

15 views
Skip to first unread message

yves.schumann@ti8m.ch (JIRA)

unread,
Feb 5, 2020, 3:39:03 AM2/5/20
to jenkinsc...@googlegroups.com
Yves Schumann created an issue
 
Jenkins / Bug JENKINS-60969
Unwanted job interaction if using global pipeline libraries
Issue Type: Bug Bug
Assignee: Unassigned
Components: core, pipeline
Created: 2020-02-05 08:38
Priority: Minor Minor
Reporter: Yves Schumann

I've spot an issue with unwanted interaction between different, independent jobs.

Here are the details: Our Jenkins instance (2.176.1 LTS) has some global pipeline libraries configured. My task was some kind of structural migration on a bunch of different projects. So I created a corresponding branch on each of these projects, did the change there and pushed it. So all the build jobs started at the same time and surprisingly most of them failed with a strange error during initial jobs steps like this:

...
remote: Compressing objects: 100% (6/6), done.        
remote: Total 8 (delta 0), reused 0 (delta 0)        
error: cannot lock ref 'refs/remotes/origin/bugfix/ESTA-3146-yla': Unable to create '/var/jenkins_home/workspace/rate-build-description-to-json_5@libs/pipeline-helper/.git/refs/remotes/origin/bugfix/ESTA-3146-yla.lock': File exists.

Another git process seems to be running in this repository, e.g.
...

After digging around I figured out, that the pipeline helper libs are cloned on the Jenkins master using a folder named like the job a/o branch, which is suffixed with `_<buildnumber>@libs`. Additionally the folder name is stripped to 37 characters by removing characters from the beginning of the name. This leads to the real cause: Because of the identical branch name on each Git repository, the unique part of the folder names got lost and so the jobs try to use the same folder during their initialization steps respectively the cloning of the pipeline helper libs. As soon as the simultaneous running jobs have a differnt build number, it works as expected.

The branch was named "feature/KIHUB-7882-separate-build-description-to-json" on all modified projects respectively Git repositories and so at the master it looks like this right now:

$ cd /var/jenkins_home/workspace
$ ls -1
...
rate-build-description-to-json_2@libs
rate-build-description-to-json_3@libs
rate-build-description-to-json_4@libs
rate-build-description-to-json_5@libs
rate-build-description-to-json_6@libs
rate-build-description-to-json_7@libs
rate-build-description-to-json_8@libs
rate-build-description-to-json_9@libs
...
$

So for me there are two questions and two possible solutions:
1. Why is the folder name stripped?
2. Why is the folder structure "flat" at this point and not like on the build jobs themself?

I think there should be at least one folder level in between like this pseudo code example:

/var/jenkins_home/workspace/OrgaScannerA/JobA-A/rate-build-description-to-json_3@libs
/var/jenkins_home/workspace/OrgaScannerA/JobA-B/rate-build-description-to-json_3@libs
/var/jenkins_home/workspace/OrgaScannerB/JobB-A/rate-build-description-to-json_3@libs
/var/jenkins_home/workspace/OrgaScannerB/JobB-B/rate-build-description-to-json_3@libs

Or the folder name itself should be unique:

/var/jenkins_home/workspace/OrgaScannerA-JobA-A-rate-build-description-to-json_3@libs
/var/jenkins_home/workspace/OrgaScannerA-JobA-B-rate-build-description-to-json_3@libs
/var/jenkins_home/workspace/OrgaScannerB-JobB-A-rate-build-description-to-json_3@libs
/var/jenkins_home/workspace/OrgaScannerB-JobB-B-rate-build-description-to-json_3@libs
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

soundcracker@gmail.com (JIRA)

unread,
Feb 5, 2020, 4:43:04 AM2/5/20
to jenkinsc...@googlegroups.com
Daniel Estermann assigned an issue to Unassigned
Change By: Daniel Estermann
Assignee: Daniel Estermann

soundcracker@gmail.com (JIRA)

unread,
Feb 5, 2020, 4:43:04 AM2/5/20
to jenkinsc...@googlegroups.com
Daniel Estermann assigned an issue to Daniel Estermann

yves.schumann@ti8m.ch (JIRA)

unread,
Apr 22, 2020, 8:59:03 AM4/22/20
to jenkinsc...@googlegroups.com
Yves Schumann commented on Bug JENKINS-60969
 
Re: Unwanted job interaction if using global pipeline libraries

Any news here? Spot the same issue again...

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

soundcracker@gmail.com (JIRA)

unread,
Apr 23, 2020, 11:02:05 AM4/23/20
to jenkinsc...@googlegroups.com

Tried to reproduce. Unsuccessfully so far...

soundcracker@gmail.com (JIRA)

unread,
Apr 23, 2020, 11:02:06 AM4/23/20
to jenkinsc...@googlegroups.com

soundcracker@gmail.com (JIRA)

unread,
Apr 24, 2020, 9:01:07 AM4/24/20
to jenkinsc...@googlegroups.com
 
Re: Unwanted job interaction if using global pipeline libraries

I managed to reproduce it, for that purpose I had to set workspaceDir in $JENKINS_HOME/config.xml to the recommended value ${JENKINS_HOME}/workspace/${ITEM_FULL_NAME} (as in WorkspaceLocatorImpl.java:337). Hence this is related to path sanitization.

soundcracker@gmail.com (JIRA)

unread,
Apr 24, 2020, 9:02:02 AM4/24/20
to jenkinsc...@googlegroups.com
Daniel Estermann started work on Bug JENKINS-60969
 
Change By: Daniel Estermann
Status: Open In Progress

jglick@cloudbees.com (JIRA)

unread,
Apr 24, 2020, 12:23:03 PM4/24/20
to jenkinsc...@googlegroups.com

try -Djenkins.branch.WorkspaceLocatorImpl.MODE=ENABLED

jglick@cloudbees.com (JIRA)

unread,
Apr 24, 2020, 12:23:03 PM4/24/20
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
 
Change By: Jesse Glick
Component/s: branch-api-plugin
Component/s: workflow-cps-global-lib-plugin
Component/s: core
Component/s: pipeline
Reply all
Reply to author
Forward
0 new messages