[JIRA] (JENKINS-58339) Support new types of credentials

31 views
Skip to first unread message

kike.fdez.polo@gmail.com (JIRA)

unread,
Jul 4, 2019, 4:45:02 AM7/4/19
to jenkinsc...@googlegroups.com
Enrique Fernández-Polo created an issue
 
Jenkins / Improvement JENKINS-58339
Support new types of credentials
Issue Type: Improvement Improvement
Assignee: Enrique Fernández-Polo
Components: aws-secrets-manager-credentials-provider-plugin
Created: 2019-07-04 08:44
Priority: Minor Minor
Reporter: Enrique Fernández-Polo

Hello!

I've created a [fork|https://github.com/Quaiks/aws-secrets-manager-credentials-provider-plugin] and I am adding new type of credentials.

The principle is simple: add a tag in AWS with the secret type and store a JSON as value. I am looking for some feedback of the implementation. Right now only happy paths are implemented but I would like to know if I am following the proper path.

 

Thanks in advance!
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

kike.fdez.polo@gmail.com (JIRA)

unread,
Jul 4, 2019, 4:46:01 AM7/4/19
to jenkinsc...@googlegroups.com
Enrique Fernández-Polo updated an issue
Hello!

I've created a [fork|[https://github.com/Quaiks/aws-secrets-manager-credentials-provider-plugin]]   and I am adding new type of credentials.

The principle is simple: add a tag in AWS with the secret type and store a JSON as value. I am looking for some feedback of the implementation. Right now only happy paths are implemented but I would like to know if I am following the proper path.

 

Thanks in advance!
Add Comment Add Comment
 

chris+jenkins@chriskilding.com (JIRA)

unread,
Jul 4, 2019, 9:18:02 AM7/4/19
to jenkinsc...@googlegroups.com
Chris Kilding commented on Improvement JENKINS-58339
 
Re: Support new types of credentials

Hi Enrique,

This is a feature I’ve been musing on for a little while. As I see it, there’s a few different solutions possible.

I initially developed the plugin with just the Secret Text credential Type for the following reasons:

  1. It was easier to build for an initial release (obviously).
  2. Travis CI, Circle CI, and some other CI products only have a notion of environment variables and encrypted environment variables (equivalent to Jenkins Secret Text credentials). They’ve been able to get along fine with these, and haven’t needed to add extra credential types, which suggests that maybe Jenkins is over-complicating credentials (when Secret Text alone would do the job). (Of course, if you really need an SSH key, it’s still possible in Travis/Circle - you can base64-encode the SSH key and store it in an encrypted env var.)
  3. It makes storage and retrieval of credentials a lot easier if you know that a credential can only be a simple arbitrary string (or byte array).
  4. It enables a secret to be saved once in AWS and then shared across any consuming application - not just Jenkins.

Point 4 is probably the strongest reason not to support multiple credential types in the AWS Secrets Manager provider. If we invent a Jenkins-proprietary schema for complex credential types eg SSH key (username, key, passphrase (optional)), only Jenkins will understand it, and we’ll have to either duplicate that credential for our other applications, or update them all to understand the Jenkins credential schema.

However... this is a tricky architectural problem. There’s probably not a single perfect answer to it. I’d be interested to hear other views on this

mathewpeterson@me.com (JIRA)

unread,
Jul 18, 2019, 12:50:02 PM7/18/19
to jenkinsc...@googlegroups.com

Hello,

I first wanted to say thank you for putting this plugin together.

Secondly, this implementation is different than how we have tried to solve this so, maybe my suggestion is incompatible.

We are using a very crude method to handle pulling secrets from AWS Secrets Manager and creating the Jenkins Credentials for them. Our solution is to create a file (`init.groovy.d/credential-x.groovy`) which executes aws cli command to retrieve the secret value for the supplied secret ARN. This file is generated from an Ansible template where we pass in the credential name and ARN, such that there exists one file per credential. In Ansible we have different templates for the different credential implementations, e.g. `AWSCredentialsImpl`, `StringCredentialsImpl`, `BasicSSHUserPrivateKey`, etc. The downside to this is that the secret is only updated once Jenkins reboots or the script gets ran manual via Script Console or similar. 

To restate, we have a specific ARN mapped to a specific Credential Implementation.

Again, I know this is somewhat at odds to this plugin implementation but I think it could be beneficial to allow the end user to create the mappings of ARN and then have a schema for each Implementation. 

mathewpeterson@me.com (JIRA)

unread,
Jul 18, 2019, 1:29:02 PM7/18/19
to jenkinsc...@googlegroups.com
Mathew Peterson edited a comment on Improvement JENKINS-58339

kike.fdez.polo@gmail.com (JIRA)

unread,
Jul 24, 2019, 10:13:04 AM7/24/19
to jenkinsc...@googlegroups.com
Hello!

[~chriskilding] We like to use the different types of credentials that Jenkins support because we think Jenkins provides some security measures when using their secrets natively. Avoid accessing files, avoid printing strings and so on. There are some plugins which are expecting a specific type of credential, user and password for bitbucket, Slack token and some others. We have a lot of processes sharing credentials with Jenkins, we are using [summon|[https://cyberark.github.io/summon/]] so everything is just an environment variable. I am going to continue working in the plugin supporting a lot of different credentials just for us, or creating a new plugin.

 

[~mathewpeterson] We we want to avoid to store the credentials in Jenkins. We prefer to retrieve the credential every time we need it. 

 

We are still playing around with the credentials. Our project is in a highly regulated market and we are still analyzing the best way to store our credentials and how to retrieve them.

kike.fdez.polo@gmail.com (JIRA)

unread,
Jul 24, 2019, 10:13:04 AM7/24/19
to jenkinsc...@googlegroups.com

Hello!

Chris Kilding We like to use the different types of credentials that Jenkins support because we think Jenkins provides some security measures when using their secrets natively. Avoid accessing files, avoid printing strings and so on. There are some plugins which are expecting a specific type of credential, user and password for bitbucket, Slack token and some others. We have a lot of processes sharing credentials with Jenkins, we are using [summon|https://cyberark.github.io/summon/] so everything is just an environment variable. I am going to continue working in the plugin supporting a lot of different credentials just for us, or creating a new plugin.

 

Mathew Peterson We we want to avoid to store the credentials in Jenkins. We prefer to retrieve the credential every time we need it. 

 

We still playing around with the credentials. Our project is in a highly regulated market and we are still analyzing the best way to store our credentials and how to retrieve them.

chris+jenkins@chriskilding.com (JIRA)

unread,
Jul 25, 2019, 5:34:04 AM7/25/19
to jenkinsc...@googlegroups.com

Hi Enrique,

Ah yes, I had overlooked that some plugins might only be able to handle certain credential types. The weight of legacy I suppose.

I’m thinking of a way of supporting multiple credential types that would at least keep the important part of the credential (the Secret) compatible with non-Jenkins apps like it is today.

Something like the following:

  • The secretString remains like it is now - a simple string (or byte array). Not JSON.
  • The extra fields (eg username for SSH key or X509 cert) are added as tags on the AWS Secret. This is fine, because most of those extra fields should be non-secret anyway.
  • The only secret extra field I’m aware of is a passphrase on a private key or cert. This, however, is only really relevant for local on-disk credentials stores - in a remote store like Secrets Manager the passphrase loses its protective power, as it would have to be stored inside... the key it’s protecting, so if an attacker got the key out of SM then they’d have got the passphrase with it. Therefore we’d just populate the passphrase field with blank or null.
  • When a Secret is loaded we determine the type by parsing the secretString and tags, maybe with a chain pattern or ADT: If the secretString contains the SSH ASCII armor construct an SSH key credential (get username from tags), if it contains the X509 header etc then construct a client cert credential (get username from tags), and so on.

With this approach, the core bit of the Secret is still readable by any application using Secrets Manager (not just Jenkins), but the extra non-secret bits can be stored there for Jenkins’ sake. And we don’t need an artificial ‘type’ field/string to be stored either - the chain decoder doesn’t need it.

Off the top of my head this should work for:

  • Secret text (there’s no extra fields to store)
  • SSH key (secretString format test + AWS tags)
  • Client cert (secretString format test + AWS tags)
  • Username / password (if doesn’t pass any secretString format tests, and there’s a username field in the AWS tags then it’s a username / password pair)

Thoughts? (This is only a first take on the idea, and there might be other credentials types that I’ve overlooked.)

chris+jenkins@chriskilding.com (JIRA)

unread,
Jul 25, 2019, 5:36:02 AM7/25/19
to jenkinsc...@googlegroups.com
Chris Kilding edited a comment on Improvement JENKINS-58339
Hi Enrique,

Ah yes, I had overlooked that some plugins might only be able to handle certain credential types. The weight of legacy I suppose.

I’m thinking of a way of supporting multiple credential types that would at least keep the important part of the credential (the Secret) compatible with non-Jenkins apps like it is today.

Something like the following:
* The secretString remains like it is now - a simple string (or byte array). Not JSON.
* The extra fields (eg username for SSH key or X509 cert) are added as tags on the AWS Secret. This is fine, because most of those extra fields should be non-secret anyway.
* The only secret extra field I’m aware of is a passphrase on a private key or cert. This, however, is only really relevant for local on-disk credentials stores - in a remote store like Secrets Manager the passphrase loses its protective power, as it would have to be stored inside... the key it’s protecting, so if an attacker got the key out of SM then they’d have got the passphrase with it. Therefore we’d just populate the passphrase field of the Credential constructor with blank or null.
* When a Secret is loaded we determine the type by parsing the secretString and tags, maybe with a chain pattern or ADT: If the secretString contains the SSH ASCII armor construct an SSH key credential (get username from tags), if it contains the X509 header etc then construct a client cert credential (get username from tags), and so on.


With this approach, the core bit of the Secret is still readable by any application using Secrets Manager (not just Jenkins), but the extra non-secret bits can be stored there for Jenkins’ sake. And we don’t need an artificial ‘type’ field/string to be stored either - the chain decoder doesn’t need it.

Off the top of my head this should work for:
* Secret text (there’s no extra fields to store)
* SSH key (secretString format test + AWS tags)
* Client cert (secretString format test + AWS tags)
* Username / password (if doesn’t pass any secretString format tests, and there’s a username field in the AWS tags then it’s a username / password pair)


Thoughts? (This is only a first take on the idea, and there might be other credentials types that I’ve overlooked.)

chris+jenkins@chriskilding.com (JIRA)

unread,
Jul 25, 2019, 7:09:01 AM7/25/19
to jenkinsc...@googlegroups.com
Chris Kilding edited a comment on Improvement JENKINS-58339
Hi Enrique,

Ah yes, I had overlooked that some plugins might only be able to handle certain credential types. The weight of legacy I suppose.

I’m thinking of a way of supporting multiple credential types that would at least keep the important part of the credential (the Secret) compatible with non-Jenkins apps like it is today.

Something like the following:
* The secretString remains like it is now - a simple string (or byte array). Not JSON.
* The extra fields (eg username for SSH key or X509 cert) are added as tags on the AWS Secret. This is fine, because most of those extra fields should be non-secret anyway.
* The only secret extra field I’m aware of is a passphrase on a private key or cert. This, however, is only really relevant for local on-disk credentials stores - in a remote store like Secrets Manager the passphrase loses its protective power, as it would have to be stored inside... the key it’s protecting, so if an attacker got the key out of SM then they’d have got the passphrase with it. Therefore we’d just populate the passphrase field of the Credential constructor with blank or null.
* When a Secret is loaded we determine the type by parsing the secretString and tags, maybe with a chain pattern or ADT: If the secretString contains the SSH ASCII armor construct an SSH key credential (get username from tags), if it contains the X509 header etc then construct a client cert credential (get username from tags), and so on.

With this approach, the core bit of the Secret is still readable by any application using Secrets Manager (not just Jenkins), but the extra non-secret bits can be stored there for Jenkins’ sake. And we don’t need an artificial ‘type’ field/string to be stored either - the chain decoder doesn’t need it.

Off the top of my head this should work for:
* Secret text (there’s no extra fields to store)
*
SSH key (secretString format test + AWS tags)

* Client cert (secretString format test + AWS tags)
* Username / password (if doesn’t pass any secretString format tests, and there’s a username field in the AWS tags then it )
* Secret text (if didn ’ s a username / password pair t match any of the above criteria )


Thoughts? (This is only a first take on the idea, and there might be other credentials types that I’ve overlooked.)

chris+jenkins@chriskilding.com (JIRA)

unread,
Aug 10, 2019, 11:08:02 AM8/10/19
to jenkinsc...@googlegroups.com
Chris Kilding started work on Improvement JENKINS-58339
 
Change By: Chris Kilding
Status: Open In Progress

chris+jenkins@chriskilding.com (JIRA)

unread,
Aug 10, 2019, 11:08:04 AM8/10/19
to jenkinsc...@googlegroups.com
Chris Kilding assigned an issue to Chris Kilding
 
Change By: Chris Kilding
Assignee: Enrique Fernández-Polo Chris Kilding

chris+jenkins@chriskilding.com (JIRA)

unread,
Aug 20, 2019, 1:10:02 PM8/20/19
to jenkinsc...@googlegroups.com
Chris Kilding commented on Improvement JENKINS-58339
 
Re: Support new types of credentials

I have started working on this auto-sensing implementation at https://github.com/jenkinsci/aws-secrets-manager-credentials-provider-plugin/pull/8

Thus far I have got a WIP implementation down that supports Secret Text, Username with Password, SSH Private Key credential types.

I’m looking to support the client certificate type but there is a problem - I can’t find any decent examples of:

  • How to use client credentials via withCredentials()
  • Plugins that operate on certificate credentials instances directly (to which we only pass the cert credential ID as a string).

Can someone point me to a simple of using Jenkins Certificate credentials to do something in a build?

chris+jenkins@chriskilding.com (JIRA)

unread,
Aug 20, 2019, 1:16:05 PM8/20/19
to jenkinsc...@googlegroups.com

(I have seen https://jenkins.io/doc/book/pipeline/jenkinsfile/#handling-credentials but the certificate example does not say what format the certificate’s KeyStore is written out in, eg JKS, PEM, P12, and it does not provide an example of an application actually consuming the bound credential.)

chris+jenkins@chriskilding.com (JIRA)

unread,
Aug 23, 2019, 11:02:03 AM8/23/19
to jenkinsc...@googlegroups.com

I’ve now completed a first draft implementation of multiple credential types support over at https://github.com/jenkinsci/aws-secrets-manager-credentials-provider-plugin/pull/8

Your questions and comments are welcome

talbot.nat@gmail.com (JIRA)

unread,
Aug 23, 2019, 11:21:02 AM8/23/19
to jenkinsc...@googlegroups.com

Hi Chris,

Thanks for doing this implementation! I started using the plugin with AWS Secrets Manager this week it's a feature that is definitely really really helpful. I'm going to try to manually install the plugin from the built .hpi and test the new features, I'll provide feedback as needed.

talbot.nat@gmail.com (JIRA)

unread,
Aug 23, 2019, 11:22:03 AM8/23/19
to jenkinsc...@googlegroups.com
Nat Talbot edited a comment on Improvement JENKINS-58339
Hi Chris,

Thanks for doing this implementation! I started using the plugin with AWS Secrets Manager this week it's a feature that is definitely really really helpful. I'm going to try to manually install the plugin from the built .hpi and test the new features, I'll can provide feedback as needed on the PR if there's anything worth mentioning .

chris+jenkins@chriskilding.com (JIRA)

unread,
Nov 7, 2019, 6:59:01 AM11/7/19
to jenkinsc...@googlegroups.com

Enrique Fernández-Polo I have found a way to add support for this feature in the linked GitHub PR above. Since you originally requested this feature I would be interested to hear your feedback and whether it works for your company.

We have some Jenkins deployments coming up at work soon, so if I don’t hear anything by the 22nd November I will merge that PR to master. And then publish a Release Candidate artifact to Jenkins Experimentals (this will contain the final functionality, and any changes after will only be bug fixes).
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

chris+jenkins@chriskilding.com (JIRA)

unread,
Nov 7, 2019, 7:02:03 AM11/7/19
to jenkinsc...@googlegroups.com
Chris Kilding edited a comment on Improvement JENKINS-58339
[~quaiks] I have found a way to add support for this feature in the linked this [ GitHub PR above | https://github . com/jenkinsci/aws-secrets-manager-credentials-provider-plugin/pull/8] . Since you originally requested this feature I would be interested to hear your feedback and whether it works for your company.


We have some Jenkins deployments coming up at work soon, so if I don’t hear anything by the 22nd November I will merge that PR to master. And then publish a Release Candidate artifact to Jenkins Experimentals (this will contain the final functionality, and any changes after will only be bug fixes).

jenkins-ci.org@cowsgomoo.org (JIRA)

unread,
Nov 7, 2019, 8:36:02 AM11/7/19
to jenkinsc...@googlegroups.com
C commented on Improvement JENKINS-58339

Chris Kilding - I'll open another issue on it, but curious what you think about using parameter store as a backend.  Do you think that having that functionality in this plugin (so the plugin can scan either secrets manager, or parameter store) is a good idea?

chris+jenkins@chriskilding.com (JIRA)

unread,
Nov 7, 2019, 8:55:04 AM11/7/19
to jenkinsc...@googlegroups.com

I don’t have an intuition for that off the top of my head, but it’s certainly worth a discussion as I know of a couple of projects in our place that put their secrets in parameter store.

Open a ticket and we’ll discuss

chris+jenkins@chriskilding.com (JIRA)

unread,
Nov 28, 2019, 5:54:02 AM11/28/19
to jenkinsc...@googlegroups.com
Chris Kilding resolved as Fixed
 
Change By: Chris Kilding
Status: In Progress Resolved
Resolution: Fixed
Released As: 0.1.0

chris+jenkins@chriskilding.com (JIRA)

unread,
Nov 28, 2019, 5:54:03 AM11/28/19
to jenkinsc...@googlegroups.com
Chris Kilding closed an issue as Fixed
Change By: Chris Kilding
Status: Resolved Closed
Reply all
Reply to author
Forward
0 new messages