[JIRA] (JENKINS-58967) Credentials not available after upgrade to LTS 2.176.2

1,635 views
Skip to first unread message

bram.mertens@anubex.com (JIRA)

unread,
Aug 21, 2019, 8:41:01 AM8/21/19
to jenkinsc...@googlegroups.com
Bram Mertens updated an issue
 
Jenkins / Bug JENKINS-58967
Credentials not available after upgrade to LTS 2.176.2
Change By: Bram Mertens
Summary: Credentials not available in multibranch project after upgrade to LTS 2.176.2
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

bram.mertens@anubex.com (JIRA)

unread,
Aug 21, 2019, 8:47:01 AM8/21/19
to jenkinsc...@googlegroups.com
Bram Mertens commented on Bug JENKINS-58967
 
Re: Credentials not available after upgrade to LTS 2.176.2

As a workaround I tried to setup a regular jenkins pipeline job using the same jenkinsfile.

This fails with the same error:

[BFA] Done. 0s
ERROR: Could not find credentials entry with ID 'customer-testmatch-creds'
Finished: FAILURE

The jenkins pipeline looks like:

          withCredentials([usernamePassword(credentialsId: 'customer-testmatch-creds', passwordVariable: 'TM_PASSWORD', usernameVariable: 'TM_USER')]) {
          ...
          }

If I use the Snippet Generator at this level the dropdown is correctly populated with both the existing and the new credentials.

The snippet that is generated looks ok:

withCredentials([usernamePassword(credentialsId: 'customer-testmatch-creds', passwordVariable: 'TM_PASSWORD', usernameVariable: 'TM_USER')]) {
    // some block
}

I cannot find any errors in the jenkins log file.

How can I troubleshoot this further? This is blocking our regression tests and delivery to our customer.

bram.mertens@anubex.com (JIRA)

unread,
Aug 21, 2019, 8:53:02 AM8/21/19
to jenkinsc...@googlegroups.com

I can reproduce the problem with a minimal new pipeline:

node('serverX&&windows') {
    withCredentials([usernamePassword(credentialsId: 'customer-testmatch-creds2', passwordVariable: 'TM_PASSWORD', usernameVariable: 'TM_USER')]) {
    echo 'running with new credentials'
    }
}

The credentials with ID customer-testmatch-creds2 was created after the upgrade.

bram.mertens@anubex.com (JIRA)

unread,
Aug 21, 2019, 9:16:02 AM8/21/19
to jenkinsc...@googlegroups.com
Bram Mertens updated an issue
Change By: Bram Mertens
We recently performed a long overdue upgrade from 2.95 to LTS 2.176.2.

Jenkins master runs on RHEL 7.7 64 bit, the node where the pipeline is failing is a Windows Server 2012 R2 (64 bit).

Master is running:
{noformat}
java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode){noformat}
Node is running:
{noformat}
C:\>java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode){noformat}
Credentials binding plugin is at 1.20

Credentials plugin is at 2.2.1

Pipeline: Multibranch is at 2.21

Jenkins is running directly without reverse proxy, installed via rpm from "jenkins-stable" ([http://pkg.jenkins.io/redhat-stable)]

We're using chrome as a browser but the issue occurs on builds that are triggered via SCM polling so the browser is probably not relevant.

 

In a Multibranch pipeline we're using the withCredentials syntax in several places:

{noformat}
withCredentials([usernamePassword(credentialsId: 'name-of-credentials', passwordVariable: 'PASSWORD', usernameVariable: 'USER')]) {
// some block
}
{noformat}


 

This has worked fine for several weeks but since the upgrade we get:
{noformat}
[Pipeline] withCredentials
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] echo
*15:07:06*  failed
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
[BFA] Scanning build for known causes...
[BFA] No failure causes found
[BFA] Done. 0s
ERROR: Could not find credentials entry with ID 'name-of-credentials'
Finished: FAILURE
{noformat}


The pipeline has not changed, the credentials have not been updated.

I have found that the snippet generator displays an empty drop down list when I start it from within the "trunk" directory of the Multibranch pipeline.

At one point I added a new credential and limited it to the folder of the Multibranch pipeline.
When I start the snippet generator in that directory the new credential as well as the old credentials are both visible.
When I start the snippet generator at the top level only the old credential is visible (as expected).

The problem is that within the branches (currently only trunk) of the multibranch pipeline the credentials are not visible.

bram.mertens@anubex.com (JIRA)

unread,
Aug 21, 2019, 9:18:02 AM8/21/19
to jenkinsc...@googlegroups.com
Bram Mertens updated an issue
Change By: Bram Mertens
Environment:
We recently performed a long overdue upgrade from 2.95 to LTS 2.176.2.

Jenkins master runs on RHEL 7.7 64 bit, the node where the pipeline is failing is a Windows Server 2012 R2 (64 bit).

Master is running:

java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
Node is running:


C:\>java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)

Credentials binding plugin is at 1.20
Credentials plugin is at 2.2.1
Pipeline: Multibranch is at 2.21

Jenkins is running directly without reverse proxy, installed via rpm from "jenkins-stable" (http://pkg.jenkins.io/redhat-stable)

bram.mertens@anubex.com (JIRA)

unread,
Aug 21, 2019, 9:20:02 AM8/21/19
to jenkinsc...@googlegroups.com
Bram Mertens updated an issue
Change By: Bram Mertens
Attachment: jenkins-system-properties.txt

bram.mertens@anubex.com (JIRA)

unread,
Aug 21, 2019, 9:37:02 AM8/21/19
to jenkinsc...@googlegroups.com
 
Re: Credentials not available after upgrade to LTS 2.176.2

Using credentials works fine on a test jenkins setup:

RHEL 7.6 version 2.179

credentials plugin 2.2.0

credentials binding plugin 1.19

Pipeline plugin 2.6

Pipeline: Multibranch plugin 2.21

 

Also works fine on another older installation:

Jenkins 2.143 on RHEL 6.7

credentials plugin 2.1.18

crdentials binding plugin 1.16

pipeline: multibranch: 2.20 (not used in this setup)

 

bram.mertens@anubex.com (JIRA)

unread,
Aug 21, 2019, 9:52:03 AM8/21/19
to jenkinsc...@googlegroups.com
Bram Mertens updated an issue
Change By: Bram Mertens
Attachment: jenkins-credentials-failure-stacktrace.txt

bram.mertens@anubex.com (JIRA)

unread,
Aug 21, 2019, 9:53:01 AM8/21/19
to jenkinsc...@googlegroups.com
 
Re: Credentials not available after upgrade to LTS 2.176.2

The attached stacktrace was generated by running the test pipeline (not multibranch) after downgrading the credentials binding plugin to 1.13.

bram.mertens@anubex.com (JIRA)

unread,
Aug 22, 2019, 5:30:03 AM8/22/19
to jenkinsc...@googlegroups.com

The credentials test pipeline I created also works on another instance of the same LTS version.

RHEL 7.6 , jenkins-2.176.2-1.1.noarch, 

Several plugins are slightly older. The ones that catch my eye are:

credentials 2.2.0

credentials-binding 1.19 

 

Versus on the jenkins master that fails:

credentials 2.2.1

credentials-binding 1.20

 

bram.mertens@anubex.com (JIRA)

unread,
Aug 22, 2019, 8:34:02 AM8/22/19
to jenkinsc...@googlegroups.com

I tried to dowgrade the credentials and credentials binding plugins to credentials 2.2.0 and credentials binding 1.19 and even created a new credentials entry but the problem persists.

bram.mertens@anubex.com (JIRA)

unread,
Aug 26, 2019, 11:58:01 AM8/26/19
to jenkinsc...@googlegroups.com

A colleague found out that the problem is caused by the fact that as part of the upgrade the build authorization was configured to run as a specific user.

The user that is configured does not have the Jobs/Configure permission.

If we grant that permission the job is able to get the credentials as before.

We will investigate the different options described in the Credentials API user guide to avoid granting that user Jobs/Configure permission.

dbeck@cloudbees.com (JIRA)

unread,
Oct 7, 2019, 3:47:02 AM10/7/19
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
 
Change By: Daniel Beck
Component/s: authorize-project-plugin
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

Kalle.Niemitalo@procomp.fi (JIRA)

unread,
Nov 28, 2019, 8:05:03 AM11/28/19
to jenkinsc...@googlegroups.com
Kalle Niemitalo commented on Bug JENKINS-58967
 
Re: Credentials not available after upgrade to LTS 2.176.2

This seems to be working:

  • Add a "jenkins-build" user to the security realm.
  • Configure Authorize Project to run all builds as the "jenkins-build" user, without allowing per-job configuration.
  • Add -Dcom.cloudbees.plugins.credentials.UseItemPermission=true to the Java options of the Jenkins master.
  • Configure Role-based Authorization Strategy like this:
    • Define the global role "build" with only these global permissions:
      • Overall/Read (might not be necessary)
      • Credentials/UseItem (this requires the option that was set above)
      • Agent/Build
      • Job/Read
    • Assign the global role "build" to the user "jenkins-build".
  • Add a certificate credential to the global credential domain of a multibranch pipeline job.
  • Reference the credential using withCredentials in the Jenkinsfiles of branches of that job.

If I then log in as the "jenkins-build" user, I do not see the credentials, because of the missing Credentials/View permission. However, the builds can use the credentials just fine.

msicker@cloudbees.com (JIRA)

unread,
May 8, 2020, 2:37:02 PM5/8/20
to jenkinsc...@googlegroups.com
Matt Sicker resolved as Not A Defect
 

Sounds like this wasn't a regression then?

Change By: Matt Sicker
Status: Open Resolved
Resolution: Not A Defect
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages