[JIRA] (JENKINS-58170) Allow credential parameters to shadow credential ids in lookup

0 views
Skip to first unread message

boards@gmail.com (JIRA)

unread,
Jun 24, 2019, 2:43:02 PM6/24/19
to jenkinsc...@googlegroups.com
Matt Sicker created an issue
 
Jenkins / New Feature JENKINS-58170
Allow credential parameters to shadow credential ids in lookup
Issue Type: New Feature New Feature
Assignee: Matt Sicker
Components: credentials-plugin
Created: 2019-06-24 18:42
Priority: Minor Minor
Reporter: Matt Sicker

As a pipeline author, suppose I have a pipeline that performs a deployment action that requires credentials from Jenkins. A naive approach might be to add my credentials to the global Jenkins credential store (call it deployKey as its id); then I can reference it in my pipeline later like so:

 

node {
  // ...
  stage('Deploy') {
    withCredentials([usernameColonPassword(credentialsId: 'deployKey', variable: 'DEPLOY_CREDENTIALS')]) {
      sh 'curl -u "${DEPLOY_CREDENTIALS}" ...'
    }
  }
}

But then I want to make my pipeline more secure. One option I might have is to have a dedicated Jenkins instance for running deployments. This is a large maintenance burden (e.g., trusted.ci.j.io versus ci.j.io), so digging deeper into Jenkins' existing features, I find user-scoped credentials. I can simply use credentials attached to my user account instead of Jenkins itself, though in order for my pipeline to recognize and use these credentials, I find that I must make them parameterized to the build itself so that I can select my credentials upon starting a build (there's an alternative option using the Authorize Project plugin, but this plugin is not widely used). A neat side-effect of this functionality is that other users can also deploy using the same pipeline but with their own user-scoped credentials. This can also allow me the flexibility of using a default global credential and allowing users to override that specifically for a pipeline run.

Thus, I modify my pipeline to add a build parameter for the deploy credentials as well as using a modified syntax in withCredentials to indicate that the credentialId parameter being provided is in fact a build parameter name and not the actual credential id. Combined, we get the following updated pipeline:

properties([parameters([credentials(credentialType: 'com.cloudbees.plugins.credentials.common.StandardUsernamePasswordCredentials', name: 'deployKey', required: true)])])
node {
  // ...
  stage('Deploy') {
    withCredentials([usernameColonPassword(credentialsId: '${deployKey}', variable: 'DEPLOY_CREDENTIALS')]) {
      sh 'curl -u "${DEPLOY_CREDENTIALS}" ...'
    }
  }
}

Notice how we need to use a special syntax: credentialsId: '${deployKey}'. This syntax is misleading because it can be easily confused with the similar but not equivalent GString syntax: credentialsId: "${deployKey}". In fact, this latter syntax would never work for a user-scoped credential because user-scoped credentials are currently only looked up if the credential id is provided using that template syntax.

What I am proposing we change here is to make credential parameters referenced by name take precedent over a globally-scoped credential with an id of the same name as the parameter name itself. Thus, we could update the above pipeline using this feature to be a little simpler:

 

properties([parameters([credentials(credentialType: 'com.cloudbees.plugins.credentials.common.StandardUsernamePasswordCredentials', name: 'deployKey', required: true)])])
node {
  // ...
  stage('Deploy') {
  withCredentials([usernameColonPassword(credentialsId: 'deployKey', variable: 'DEPLOY_CREDENTIALS')]) {
    sh 'curl -u "${DEPLOY_CREDENTIALS}" ...'
  }
 }
}

The beauty of this approach is that it allows me to go from the first pipeline to the third pipeline simply by updating my build to add a credential parameter with the same name as the credential id I was using before that I want to migrate to a user account. I don't even have to touch the rest of the pipeline!

This issue relates to JENKINS-47699 in that it's a useful point to help support that feature.

 

 

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

boards@gmail.com (JIRA)

unread,
Jun 24, 2019, 2:46:03 PM6/24/19
to jenkinsc...@googlegroups.com
Matt Sicker started work on New Feature JENKINS-58170
 
Change By: Matt Sicker
Status: Open In Progress

boards@gmail.com (JIRA)

unread,
Jun 24, 2019, 2:46:03 PM6/24/19
to jenkinsc...@googlegroups.com

boards@gmail.com (JIRA)

unread,
Jun 24, 2019, 3:31:02 PM6/24/19
to jenkinsc...@googlegroups.com
Matt Sicker updated an issue
As a pipeline author, suppose I have a pipeline that performs a deployment action that requires credentials from Jenkins. A naive approach might be to add my credentials to the global Jenkins credential store (call it {{deployKey}} as its id); then I can reference it in my pipeline later like so:

 
{code:java}

node {
  // ...
  stage('Deploy') {
    withCredentials([usernameColonPassword(credentialsId: 'deployKey', variable: 'DEPLOY_CREDENTIALS')]) {
      sh 'curl -u "${DEPLOY_CREDENTIALS}" ...'
    }
  }
}
{code}
But then I want to make my pipeline more secure. One option I might have is to have a dedicated Jenkins instance for running deployments. This is a _large_ maintenance burden (e.g., trusted.ci.j.io versus ci.j.io), so digging deeper into Jenkins' existing features, I find user-scoped credentials. I can simply use credentials attached to my user account instead of Jenkins itself, though in order for my pipeline to recognize and use these credentials, I find that I must make them parameterized to the build itself so that I can select my credentials upon starting a build (there's an alternative option using the Authorize Project plugin, but this plugin is not widely used). A neat side-effect of this functionality is that other users can also deploy using the same pipeline but with their own user-scoped credentials. This can also allow me the flexibility of using a default global credential and allowing users to override that specifically for a pipeline run.


Thus, I modify my pipeline to add a build parameter for the deploy credentials as well as using a modified syntax in {{withCredentials}} to indicate that the {{credentialId}} parameter being provided is in fact a build parameter name and not the actual credential id. Combined, we get the following updated pipeline:
{code:java}

properties([parameters([credentials(credentialType: 'com.cloudbees.plugins.credentials.common.StandardUsernamePasswordCredentials', name: 'deployKey', required: true)])])
node {
  // ...
  stage('Deploy') {
    withCredentials([usernameColonPassword(credentialsId: '${deployKey}', variable: 'DEPLOY_CREDENTIALS')]) {
      sh 'curl -u "${DEPLOY_CREDENTIALS}" ...'
    }
  }
}
{code}
Notice how we need to use a special syntax: {{credentialsId: '${deployKey}'}}. This syntax is misleading because it can be easily confused with the similar but not equivalent GString syntax: {{credentialsId: "${deployKey}"}}. In fact, this latter syntax would never work for a user-scoped credential because user-scoped credentials are currently only looked up if the credential id is provided using
that the former template syntax.


What I am proposing we change here is to make credential parameters referenced by name take precedent over a globally-scoped credential with an id of the same name as the parameter name itself. Thus, we could update the above pipeline using this feature to be a little simpler:

 
{code:java}

properties([parameters([credentials(credentialType: 'com.cloudbees.plugins.credentials.common.StandardUsernamePasswordCredentials', name: 'deployKey', required: true)])])
node {
  // ...
  stage('Deploy') {
  withCredentials([usernameColonPassword(credentialsId: 'deployKey', variable: 'DEPLOY_CREDENTIALS')]) {
    sh 'curl -u "${DEPLOY_CREDENTIALS}" ...'
  }
}
}
{code}

The beauty of this approach is that it allows me to go from the first pipeline to the third pipeline simply by updating my build to add a credential parameter with the same name as the credential id I was using before that I want to migrate to a user account. I don't even have to touch the rest of the pipeline!

This issue relates to JENKINS-47699 in that it's a useful point to help support that feature.

 

 

boards@gmail.com (JIRA)

unread,
Aug 1, 2019, 3:03:02 PM8/1/19
to jenkinsc...@googlegroups.com
Matt Sicker commented on New Feature JENKINS-58170
 
Re: Allow credential parameters to shadow credential ids in lookup

Linking related issues that can be implemented through this feature.

wfollonier@cloudbees.com (JIRA)

unread,
Aug 23, 2019, 6:17:03 AM8/23/19
to jenkinsc...@googlegroups.com

Adding 4 other tickets that could be potentially closed after this one is released.

boards@gmail.com (JIRA)

unread,
Aug 26, 2019, 3:12:02 PM8/26/19
to jenkinsc...@googlegroups.com
 

Merged to master. Will be released in 2.3.0.

Change By: Matt Sicker
Status: In Review Fixed but Unreleased
Resolution: Fixed

boards@gmail.com (JIRA)

unread,
Aug 26, 2019, 3:20:02 PM8/26/19
to jenkinsc...@googlegroups.com
 

Credentials 2.3.0 is now released.

Change By: Matt Sicker
Status: Fixed but Unreleased Resolved
Released As: credentials-2.3.0

bernard@bondos.org (JIRA)

unread,
Aug 27, 2019, 9:01:06 AM8/27/19
to jenkinsc...@googlegroups.com
Bernard Bondos commented on New Feature JENKINS-58170
 
Re: Allow credential parameters to shadow credential ids in lookup

With the new version there is an issue with triggering build of other jobs with user-scoped credential passed as parameter. Consider two jobs:

source:

properties([parameters([credentials(credentialType: 'com.cloudbees.plugins.credentials.common.StandardUsernamePasswordCredentials', name: 'deployKey', required: true
)])])

node {  
  stage('call target') {
    build job: 'target', parameters: [credentials(description: '', name: 'deployKey', value: params.deployKey )]
  }
}

target: 

properties([parameters([credentials(credentialType: 'com.cloudbees.plugins.credentials.common.StandardUsernamePasswordCredentials', name: 'deployKey', required: true
)])])

node {
  stage('use creds') {
    withCredentials([usernamePassword(credentialsId: 'deployKey', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
        echo "My name is ${USERNAME}"
      }
  }
} 

When building source with user-scoped credential, target will fail with

ERROR: Could not find credentials entry with ID 'deployKey' 

but when you run "replay" on the failed build of target, it will actually work.

Is it a bug or desired behavior?

boards@gmail.com (JIRA)

unread,
Aug 27, 2019, 3:59:03 PM8/27/19
to jenkinsc...@googlegroups.com

That is somewhat expected. In the first build, the user providing the build parameters is associated there, but when it invokes another build, that's technically being done by the Jenkins system user. When you went to replay the second pipeline, that updated the build to use your user for the parameters again rather than the system. I'd imagine if you were using authorize-project, that would have likely passed the credential as such.

However, I think you have a valid use case here. If this feature were to be supported, it would have to be supported in pipeline-build-step somewhere: https://github.com/jenkinsci/pipeline-build-step-plugin/tree/master/src/main/java/org/jenkinsci/plugins/workflow/support/steps/build

Similarly, this feature is being supported in pipeline-input-step, so it's not that out there to do it in pipeline-build-step as well. Devin Nusbaum, what do you think?

dnusbaum@cloudbees.com (JIRA)

unread,
Aug 27, 2019, 4:11:02 PM8/27/19
to jenkinsc...@googlegroups.com

Matt Sicker Yeah, if someone chooses to pass user-scoped credentials to the build step, binding those credentials to the downstream build in such a way that they are available regardless of the actual authentication running the downstream build seems like it would match the behavior you are adding to the input step. As long as users see the "only pass user-scoped credentials to trusted builds" warning in the snippet generator for the build step, and the credentials aren't available if the downstream build is a Pipeline and it gets replayed, I think that feature would make sense.

boards@gmail.com (JIRA)

unread,
Aug 27, 2019, 4:25:03 PM8/27/19
to jenkinsc...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages