Jira (PUP-8913) Agents should re-download CRLs when they are newer than the local copy

1 view
Skip to first unread message

Maggie Dreyer (JIRA)

unread,
Jun 7, 2018, 5:23:02 PM6/7/18
to puppe...@googlegroups.com
Maggie Dreyer moved an issue
 
Puppet / Story PUP-8913
Agents should re-download CRLs when they are newer than the local copy
Change By: Maggie Dreyer
Key: SERVER PUP - 2174 8913
Project: Puppet Server
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Amy Sahli (JIRA)

unread,
Jul 11, 2018, 2:05:03 PM7/11/18
to puppe...@googlegroups.com

Maggie Dreyer (JIRA)

unread,
Jul 17, 2018, 6:41:02 PM7/17/18
to puppe...@googlegroups.com

Maggie Dreyer (JIRA)

unread,
Jul 17, 2018, 7:09:02 PM7/17/18
to puppe...@googlegroups.com

Maggie Dreyer (JIRA)

unread,
Jul 17, 2018, 7:18:03 PM7/17/18
to puppe...@googlegroups.com

Maggie Dreyer (JIRA)

unread,
Aug 1, 2018, 6:09:03 PM8/1/18
to puppe...@googlegroups.com
Maggie Dreyer commented on Story PUP-8913
 
Re: Agents should re-download CRLs when they are newer than the local copy

This had to be reverted due to an issue where during failover, the CA cannot be contacted (though the compile master can), but the reverse-proxy service in pe-puppetserver forward the CRL redownload request anyway, causing the agent run to error. This error brought to light the fact that we have built up quite a few assumptions around not needing to contact the CA post-SSL bootstrapping. This feature will need further evaluation.

Maggie Dreyer (JIRA)

unread,
Aug 1, 2018, 6:09:06 PM8/1/18
to puppe...@googlegroups.com

Maggie Dreyer (JIRA)

unread,
Aug 1, 2018, 6:10:04 PM8/1/18
to puppe...@googlegroups.com

Justin Stoller (JIRA)

unread,
Sep 17, 2018, 1:47:04 PM9/17/18
to puppe...@googlegroups.com
Justin Stoller commented on Story PUP-8913
 
Re: Agents should re-download CRLs when they are newer than the local copy

Note: the server side work to enable this will go out in Puppet Server 6.0.0.

Justin Stoller (JIRA)

unread,
Sep 17, 2018, 1:47:05 PM9/17/18
to puppe...@googlegroups.com

Justin Stoller (JIRA)

unread,
Sep 17, 2018, 3:20:05 PM9/17/18
to puppe...@googlegroups.com
Justin Stoller updated an issue
This is the original ticket, but is being used for the Server side work that will ship in Puppet Server 6.0.0. For the remaining, agent-side work, see the cloned ticket PUP-9149.

 

-----

 

While my agents are running, I expect them to check the CRL to see if it's been updated since the last download, and if so to get the latest one from the CA, so that they can correctly validate that the masters they are talking to are still valid.

This is mostly relevant for agents talking to compile masters, which also get certificates from the CA. We don't want agents to try to talk to compile masters whose certificates have been revoked.

See PUP-2310.

Q: Do CRLs have TTLs that we could respect?
A: Yes, but it's complicated and we probably should just check last modified.

*Details:*

So the plan going forward will be this (see [https://github.com/puppetlabs/puppet/blob/76043d833f70d70fd2b5e3c43f13c5226223d8f9/lib/puppet/ssl/host.rb#L301):]

1.  From from `ssl_store` => `build_ssl_store`=> We will alter the `build_ssl_store` method to ALWAYS call the altered/renamed `download_and_save_crl_bundle` method. 

2.  That will continue to call the `Puppet::Rest::Routes.get_crls` method, but the `get_crls` method will send a *header* (ex *{{If-Modified-Since)}}*containing the last_modified timestamp of the current crl.  The
  *   /certificate_revocation_list/name* endpoint on puppetserver will then be altered to either accept a header with the timestamp and,
* If the *{{If-Modified-Since}}* header  is present AND the timestamp is OLDER than the master's crl, will return the NEW crl to the agent in the response body and 200 response. 
* If no *{{If-Modified-Since}}* header is included, the endpoint will function the same way as it currently functions.  
* If the *{{If-Modified-Since}}* header is present AND the timestamp is EQUAL to or NEWER than the master's crl, will return an empty body and something like a 204 response.

-We need to decide if a header or query param is more appropriate for our case.- [~josh] -thoughts on this?- See comment below.

*Acceptance Criteria:*
* The *certificate_revocation_list/name* endpoint on puppetserver continues to function as it always did if no *{{If-Modified-Since}}* header are received.
* If the *{{If-Modified-Since}}* header  is passed to the above endpoint, it will return a crl if the timestamp provided is OLDER than the master crl file's timestamp.
* If the *{{If-Modified-Since}}* header  is passed to the above endpoint, it will return an empty body (no crl) if the timestamp provided is EQUAL to or NEWER than the master crl file's timestamp.
* New endpoint behavior with headers are documented.

Kenn Hussey (JIRA)

unread,
Sep 19, 2018, 4:07:05 PM9/19/18
to puppe...@googlegroups.com
Kenn Hussey updated an issue
Change By: Kenn Hussey
Fix Version/s: SERVER 6.0.0
Fix Version/s: PUP 6.0.0

Justin Stoller (JIRA)

unread,
Sep 19, 2018, 4:15:04 PM9/19/18
to puppe...@googlegroups.com
Justin Stoller updated an issue
Change By: Justin Stoller
Fix Version/s: PUP 6.0.0
Fix Version/s: SERVER 6.0.0

Josh Cooper (JIRA)

unread,
Sep 25, 2018, 1:49:05 PM9/25/18
to puppe...@googlegroups.com
Josh Cooper commented on Story PUP-8913
 
Re: Agents should re-download CRLs when they are newer than the local copy

Justin Stoller could reword the summary and release notes so it's clear the agent behavior is unchanged for 6.0?

Justin Stoller (JIRA)

unread,
Sep 25, 2018, 2:02:05 PM9/25/18
to puppe...@googlegroups.com
Justin Stoller updated an issue
Change By: Justin Stoller
In Puppet Server 6.0.0 the {{puppet-ca/v1/certificate_revocation_list/ca}} endpoint will honor **If-Modified-Since** headers, serving the CRL content if modified since the time given and returning 204 and an empty body if not.

The Puppet Agent had no change of behavior in 6 and currently, and previously, does not send **If-Modified-Since** headers, or ever try to download a CRL if one exists on disk.

This ticket was originally for both the Server and the Agent side work, both landed, however the Agent work was reverted as it caused issues with PE's HA functionality that would require large-ish changes to how HA works.


This
is the original ticket, but is being used for the Server side work that will ship in Puppet Server 6.0.0. For the remaining, agent-side work, see the cloned ticket PUP-9149.


 

-----

 

While my agents are running, I expect them to check the CRL to see if it's been updated since the last download, and if so to get the latest one from the CA, so that they can correctly validate that the masters they are talking to are still valid.

This is mostly relevant for agents talking to compile masters, which also get certificates from the CA. We don't want agents to try to talk to compile masters whose certificates have been revoked.

See PUP-2310.

Q: Do CRLs have TTLs that we could respect?
A: Yes, but it's complicated and we probably should just check last modified.

*Details:*

So the plan going forward will be this (see [https://github.com/puppetlabs/puppet/blob/76043d833f70d70fd2b5e3c43f13c5226223d8f9/lib/puppet/ssl/host.rb#L301):]

1.  From from `ssl_store` => `build_ssl_store`=> We will alter the `build_ssl_store` method to ALWAYS call the altered/renamed `download_and_save_crl_bundle` method. 

2.  That will continue to call the `Puppet::Rest::Routes.get_crls` method, but the `get_crls` method will send a *header* (ex *{{If-Modified-Since)}}*containing the last_modified timestamp of the current crl.  The* /certificate_revocation_list/name* endpoint on puppetserver will then be altered to either accept a header with the timestamp and,

* If the *{{If-Modified-Since}}* header  is present AND the timestamp is OLDER than the master's crl, will return the NEW crl to the agent in the response body and 200 response. 
* If no *{{If-Modified-Since}}* header is included, the endpoint will function the same way as it currently functions.  
* If the *{{If-Modified-Since}}* header is present AND the timestamp is EQUAL to or NEWER than the master's crl, will return an empty body and something like a 204 response.

-We need to decide if a header or query param is more appropriate for our case.- [~josh] -thoughts on this?- See comment below.

*Acceptance Criteria:*
* The *certificate_revocation_list/name* endpoint on puppetserver continues to function as it always did if no *{{If-Modified-Since}}* header are received.
* If the *{{If-Modified-Since}}* header  is passed to the above endpoint, it will return a crl if the timestamp provided is OLDER than the master crl file's timestamp.
* If the *{{If-Modified-Since}}* header  is passed to the above endpoint, it will return an empty body (no crl) if the timestamp provided is EQUAL to or NEWER than the master crl file's timestamp.
* New endpoint behavior with headers are documented.

Justin Stoller (JIRA)

unread,
Sep 25, 2018, 2:03:07 PM9/25/18
to puppe...@googlegroups.com
Justin Stoller updated an issue
In Puppet Server 6.0.0 the {{puppet-ca/v1/certificate_revocation_list/ca}} endpoint will honor **If-Modified-Since** headers, serving the CRL content if modified since the time given and returning 204 and an empty body if not.

The Puppet Agent had no change of behavior in 6 and currently, and previously, does not send **If-Modified-Since** headers, or ever try to download a CRL if one exists on disk.

This ticket was originally for both the Server and the Agent side work, both landed, however the Agent work was reverted as it caused issues with PE's HA functionality that would require large-ish changes to how HA works.


This is
PUP-9149 tracks updating the original ticket, but is being used for the Server side work that will ship Agent to take advantage of this ability in Puppet Server 6 once PE HA issues have been resolved . 0.0. For the remaining, agent-side work, see the cloned ticket PUP-9149.

 

-----

 

While my agents are running, I expect them to check the CRL to see if it's been updated since the last download, and if so to get the latest one from the CA, so that they can correctly validate that the masters they are talking to are still valid.

This is mostly relevant for agents talking to compile masters, which also get certificates from the CA. We don't want agents to try to talk to compile masters whose certificates have been revoked.

See PUP-2310.

Q: Do CRLs have TTLs that we could respect?
A: Yes, but it's complicated and we probably should just check last modified.

*Details:*

So the plan going forward will be this (see [https://github.com/puppetlabs/puppet/blob/76043d833f70d70fd2b5e3c43f13c5226223d8f9/lib/puppet/ssl/host.rb#L301):]

1.  From from `ssl_store` => `build_ssl_store`=> We will alter the `build_ssl_store` method to ALWAYS call the altered/renamed `download_and_save_crl_bundle` method. 

2.  That will continue to call the `Puppet::Rest::Routes.get_crls` method, but the `get_crls` method will send a *header* (ex *{{If-Modified-Since)}}*containing the last_modified timestamp of the current crl.  The* /certificate_revocation_list/name* endpoint on puppetserver will then be altered to either accept a header with the timestamp and,
* If the *{{If-Modified-Since}}* header  is present AND the timestamp is OLDER than the master's crl, will return the NEW crl to the agent in the response body and 200 response. 
* If no *{{If-Modified-Since}}* header is included, the endpoint will function the same way as it currently functions.  
* If the *{{If-Modified-Since}}* header is present AND the timestamp is EQUAL to or NEWER than the master's crl, will return an empty body and something like a 204 response.

-We need to decide if a header or query param is more appropriate for our case.- [~josh] -thoughts on this?- See comment below.

*Acceptance Criteria:*
* The *certificate_revocation_list/name* endpoint on puppetserver continues to function as it always did if no *{{If-Modified-Since}}* header are received.
* If the *{{If-Modified-Since}}* header  is passed to the above endpoint, it will return a crl if the timestamp provided is OLDER than the master crl file's timestamp.
* If the *{{If-Modified-Since}}* header  is passed to the above endpoint, it will return an empty body (no crl) if the timestamp provided is EQUAL to or NEWER than the master crl file's timestamp.
* New endpoint behavior with headers are documented.

Justin Stoller (JIRA)

unread,
Sep 25, 2018, 2:03:08 PM9/25/18
to puppe...@googlegroups.com
Justin Stoller updated an issue
In Puppet Server 6.0.0 the {{puppet-ca/v1/certificate_revocation_list/ca}} endpoint will honor * * If-Modified-Since* * headers, serving the CRL content if modified since the time given and returning 204 and an empty body if not.

The Puppet Agent had no change of behavior in 6 and currently, and previously, does not send *
* If-Modified-Since* * headers, or ever try to download a CRL if one exists on disk.


This ticket was originally for both the Server and the Agent side work, both landed, however the Agent work was reverted as it caused issues with PE's HA functionality that would require large-ish changes to how HA works.

PUP-9149 tracks updating the Agent to take advantage of this ability in Server once PE HA issues have been resolved.

 

-----

 

While my agents are running, I expect them to check the CRL to see if it's been updated since the last download, and if so to get the latest one from the CA, so that they can correctly validate that the masters they are talking to are still valid.

This is mostly relevant for agents talking to compile masters, which also get certificates from the CA. We don't want agents to try to talk to compile masters whose certificates have been revoked.

See PUP-2310.

Q: Do CRLs have TTLs that we could respect?
A: Yes, but it's complicated and we probably should just check last modified.

*Details:*

So the plan going forward will be this (see [https://github.com/puppetlabs/puppet/blob/76043d833f70d70fd2b5e3c43f13c5226223d8f9/lib/puppet/ssl/host.rb#L301):]

1.  From from `ssl_store` => `build_ssl_store`=> We will alter the `build_ssl_store` method to ALWAYS call the altered/renamed `download_and_save_crl_bundle` method. 

2.  That will continue to call the `Puppet::Rest::Routes.get_crls` method, but the `get_crls` method will send a *header* (ex *{{If-Modified-Since)}}*containing the last_modified timestamp of the current crl.  The* /certificate_revocation_list/name* endpoint on puppetserver will then be altered to either accept a header with the timestamp and,
* If the *{{If-Modified-Since}}* header  is present AND the timestamp is OLDER than the master's crl, will return the NEW crl to the agent in the response body and 200 response. 
* If no *{{If-Modified-Since}}* header is included, the endpoint will function the same way as it currently functions.  
* If the *{{If-Modified-Since}}* header is present AND the timestamp is EQUAL to or NEWER than the master's crl, will return an empty body and something like a 204 response.

-We need to decide if a header or query param is more appropriate for our case.- [~josh] -thoughts on this?- See comment below.

*Acceptance Criteria:*
* The *certificate_revocation_list/name* endpoint on puppetserver continues to function as it always did if no *{{If-Modified-Since}}* header are received.
* If the *{{If-Modified-Since}}* header  is passed to the above endpoint, it will return a crl if the timestamp provided is OLDER than the master crl file's timestamp.
* If the *{{If-Modified-Since}}* header  is passed to the above endpoint, it will return an empty body (no crl) if the timestamp provided is EQUAL to or NEWER than the master crl file's timestamp.
* New endpoint behavior with headers are documented.

Justin Stoller (JIRA)

unread,
Sep 25, 2018, 2:04:16 PM9/25/18
to puppe...@googlegroups.com
Justin Stoller updated an issue
Change By: Justin Stoller
Release Notes Summary: In Puppet Server 6.0.0 the {{puppet-ca/v1/certificate_revocation_list/ca}} endpoint will honor *If-Modified-Since* headers, serving the CRL content if modified since the time given and returning 204 and an empty body if not.

The agent however has not changed to take advantage of this ability.
Release Notes: New Feature

Justin Stoller (JIRA)

unread,
Sep 25, 2018, 2:21:06 PM9/25/18
to puppe...@googlegroups.com
Justin Stoller commented on Story PUP-8913
 
Re: Agents should re-download CRLs when they are newer than the local copy

Updated the description and docs tab, went to do the docs update myself and realized that we don't have a great story around puppet-ca endpoint docs now that we've removed the Ruby CA. Opened SERVER-2333 to work that out and linked the issues so we can document this when we update our CA docs.

Josh Cooper (JIRA)

unread,
Feb 3, 2020, 12:53:05 AM2/3/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
 
Change By: Josh Cooper
Fix Version/s: SERVER 6.0.0
Fix Version/s: PUP 6.0.0
Reply all
Reply to author
Forward
0 new messages