[JIRA] (JENKINS-60132) Add support for AWS parameter store as a backend for storing credentials

[jenkins-ci.org@cowsgomoo.org (JIRA)]

C created an issue

Jenkins / JENKINS-60132 Add support for AWS parameter store as a backend for storing credentials Issue Type: New Feature Assignee: Chris Kilding Components: aws-secrets-manager-credentials-provider-plugin Created: 2019-11-11 14:11 Priority: Minor Reporter: C Particularly, storing credentials as secure strings parameters can have a lower cost relative to secrets manager. https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-securestring.html   Questions: Does this belong in its own project/plugin or is there opportunity for code reuse as part of aws-secrets-manager-credentials-provider-plugin? Is there opportunity to reuse the conventions, such as tagging, that were defined when adding multiple credential support (JENKINS-58339)? Any other comments? Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[chris+jenkins@chriskilding.com (JIRA)]

Chris Kilding updated an issue

Jenkins / JENKINS-60132 Add support for AWS parameter store as a backend for storing credentials

Change By: Chris Kilding Particularly, storing *Feature* Allow Jenkins to look up credentials in AWS Parameter Store. (They will be stored as secure strings Secure String parameters can have a lower cost relative to secrets manager. [https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-securestring.html] ).   *Benefits* - Storing credentials in Parameter Store can be cheaper than storing them  in Secrets Manager. *Pitfalls* - Secrets Manager may offer higher API rate limits or better QoS since it is a paid-for service. We would need to find out whether Parameter Store offers acceptable rate limits and performance for Jenkins to rely on. * Questions : * # - Does this belong in its own project/plugin or is there opportunity for code reuse as part of aws-secrets-manager-credentials-provider-plugin? # Is there opportunity to - Could we reuse the naming or storage conventions, such as tagging, that were defined when adding multiple credential support (JENKINS-58339)? # Any *Constraints* - Jenkins should be able to source credentials from both Secrets Manager and Parameter Store. (Using one should not rule out using the other comments? .) - If Jenkins encounters an error looking up secrets in one of the services, this should not impede lookups in the other. (An exception from a Secrets Manager API call should not break secret resolution in Parameter Store if PS is still functioning.) - Tag naming conventions should be shared in both PS and SM. (Eg a username tag should be called jenkins:credentials:username in PS, just like it is in SM today.)

Add Comment

[chris+jenkins@chriskilding.com (JIRA)]

Chris Kilding commented on JENKINS-60132

Re: Add support for AWS parameter store as a backend for storing credentials I’ve refactored the feature definition to be more structured, and I’ve provided a few constraints to focus the ticket. Identifying how to proceed on this is a bit confusing, in part because AWS themselves seem to be confused. They have effectively built 2 services (SM and PS) which, for the task of managing secrets, seem to do the same thing (with 1 or 2 feature variations). In 2018 it got more confusing when they allowed PS to act as a passthrough, so it can retrieve secrets from SM (presumably with the implied extra API call and associated cost).

Add Comment

[chris+jenkins@chriskilding.com (JIRA)]

Chris Kilding updated an issue

Jenkins / JENKINS-60132

Add support for AWS parameter store as a backend for storing credentials

Change By: Chris Kilding *Feature* Allow Jenkins to look up credentials in AWS Parameter Store. (They will be stored as Secure String parameters [https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-securestring.html]). * Benefits Rationale *

- Storing credentials in Parameter Store can be cheaper than storing them  in Secrets Manager.

*Pitfalls* - Secrets Manager may offer higher API rate limits or better QoS since it is a paid-for service. We would need to find out whether Parameter Store offers acceptable rate limits and performance for Jenkins to rely on. TODO anything else? *Questions*

- Does this belong in its own project/plugin or is there opportunity for code reuse as part of aws-secrets-manager-credentials-provider-plugin?

- Does Parameter Store offer acceptable API rate limits and QoS for Jenkins to rely on? - Could we reuse naming or storage conventions, such as tagging, that were defined when adding multiple store all major credential support types ( JENKINS-58339 especially binary ones like the PKCS#12 certificate ) in Parameter Store ? *Constraints* - Jenkins should be able to source credentials from both Secrets Manager and Parameter Store. (Using one should not rule out using the other.)

- If Jenkins encounters an error looking up secrets in one of the services, this should not impede lookups in the other. (An exception from a Secrets Manager API call should not break secret resolution in Parameter Store if PS is still functioning.) - Tag naming conventions should be shared in both PS and SM. (Eg a username tag should be called

{{ jenkins:credentials:username }} in PS, just like it is in SM today.)

Add Comment

[chris+jenkins@chriskilding.com (JIRA)]

Chris Kilding updated an issue

Jenkins / JENKINS-60132 Add support for AWS parameter store as a backend for storing credentials Change By: Chris Kilding

*Feature* Allow Jenkins to look up credentials in AWS Parameter Store. (They will be stored as Secure String parameters [https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-securestring.html]).

*Rationale*

- Storing credentials in Parameter Store can be cheaper than storing them  in Secrets Manager.

- TODO anything else? *Questions*

- Does this belong in its own project/plugin or is there opportunity for code reuse as part of aws-secrets-manager-credentials-provider-plugin? - Does

*Comparison of services* $ = chargeable ||Feature||Secrets Manager||Standard Parameter Store offer acceptable API rate limits and QoS for Jenkins to rely on? ||Advanced Parameter|| |Size|10.24kb|4kb|8kb| |Storage cost / month|$|Free|$| |IAM per - Could we store all major credential types secret policy|Yes| No|Yes| |Max API calls / sec ( especially binary ones like the PKCS#12 certificate retrieval ) in Parameter Store? |1,500 ($)|40 (free) 1,000 ($)|40 (free) 1,000 ($)| |Max num secrets|40,000|10,000|100,000| |String secrets|Yes|Yes|Yes| |Binary secrets|Yes|No|No|

*Constraints* - Jenkins should be able to source credentials from both Secrets Manager and Parameter Store. (Using one should not rule out using the other.) - If Jenkins encounters an error looking up secrets in one of the services, this should not impede lookups in the other. (An exception from a Secrets Manager API call should not break secret resolution in Parameter Store if PS is still functioning.) - Tag naming conventions should be shared in both PS and SM. (Eg a username tag should be called {{jenkins:credentials:username}} in PS, just like it is in SM today.)

Add Comment

[chris+jenkins@chriskilding.com (JIRA)]

Chris Kilding updated an issue

Jenkins / JENKINS-60132 Add support for AWS parameter store as a backend for storing credentials Change By: Chris Kilding

*Feature* Allow Jenkins to look up credentials in AWS Parameter Store. (They will be stored as Secure String parameters [https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-securestring.html]). *Rationale* - Storing credentials in Parameter Store can be cheaper than storing them in Secrets Manager. - TODO anything else? *Questions* - Does this belong in its own project/plugin or is there opportunity for code reuse as part of aws-secrets-manager-credentials-provider-plugin?

*Comparison of services* $ = chargeable

||Feature||Secrets Manager||Standard Parameter||Advanced Parameter|| |Size|10.24kb|4kb|8kb| |Storage cost / month|$ 0.40 |Free|$ 0.05 | |IAM per-secret policy|Yes| No|Yes| |Max API calls / sec (retrieval)|1,500 ($)|40 (free) 1,000 ($)|40 ( free $ )

1,000 ($)| |Max num secrets|40,000|10,000|100,000| |String secrets|Yes|Yes|Yes| |Binary secrets|Yes|No|No| *Constraints* - Jenkins should be able to source credentials from both Secrets Manager and Parameter Store. (Using one should not rule out using the other.) - If Jenkins encounters an error looking up secrets in one of the services, this should not impede lookups in the other. (An exception from a Secrets Manager API call should not break secret resolution in Parameter Store if PS is still functioning.) - Tag naming conventions should be shared in both PS and SM. (Eg a username tag should be called {{jenkins:credentials:username}} in PS, just like it is in SM today.)

Add Comment

[chris+jenkins@chriskilding.com (JIRA)]

Chris Kilding updated an issue

Jenkins / JENKINS-60132 Add support for AWS parameter store as a backend for storing credentials Change By: Chris Kilding

*Feature* Allow Jenkins to look up credentials in AWS Parameter Store. (They will be stored as Secure String parameters [https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-securestring.html]). *Rationale* - Storing credentials in Parameter Store can be cheaper than storing them in Secrets Manager. - TODO anything else? *Questions* - Does this belong in its own project/plugin or is there opportunity for code reuse as part of aws-secrets-manager-credentials-provider-plugin?   *Comparison of services* $ = chargeable ||Feature||Secrets Manager||Standard Parameter||Advanced Parameter|| |Size|10.24kb|4kb|8kb|

| Storage Monthly cost / month per secret |$0.40|Free|$0.05|

|IAM per-secret policy|Yes| No|Yes|

|Max API calls / per sec (retrieval)|1,500 ($)|40 (free) 1,000 ($)|40 ($)

1,000 ($)| |Max num secrets|40,000|10,000|100,000| |String secrets|Yes|Yes|Yes| |Binary secrets|Yes|No|No| *Constraints* - Jenkins should be able to source credentials from both Secrets Manager and Parameter Store. (Using one should not rule out using the other.) - If Jenkins encounters an error looking up secrets in one of the services, this should not impede lookups in the other. (An exception from a Secrets Manager API call should not break secret resolution in Parameter Store if PS is still functioning.) - Tag naming conventions should be shared in both PS and SM. (Eg a username tag should be called {{jenkins:credentials:username}} in PS, just like it is in SM today.)

Add Comment