Jira (PUP-5069) Puppet HTTP connection code does not allow callers to trust system cacerts

[Josh Cooper (JIRA)]

Josh Cooper created an issue

Puppet / PUP-5069 Puppet HTTP connection code does not allow callers to trust system cacerts Issue Type: Bug Assignee: Unassigned Created: 2015/08/24 11:15 AM Priority: Normal Reporter: Josh Cooper Puppet's HTTP connection code does not allow callers to specify that they want to trust the default set of cacerts. To do so, you need to get access to the Net::HTTP#store and call OpenSSL::X509::Store#set_default_paths, but this is not currently possible. The PMT works around this by rolling its own Net::HTTP object, leading to duplicate proxy handling logic. It would be nice to DRY this up, but to do so, our HTTP code needs to provide a way for callers to trust the system's cacerts. This would also be useful for providers that need to make network connections, and not rely on open-uri to do so. Add Comment

This message was sent by Atlassian JIRA (v6.4.5#64020-sha1:78acd6c)

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-5069

Puppet's HTTP API does not allow callers to trust system cacerts Change By: Josh Cooper Summary: Puppet 's  HTTP  connection code  API  does not allow callers to trust system cacerts

Add Comment

[William Hopper (JIRA)]

William Hopper updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: William Hopper Story Points: 2 3

Add Comment

This message was sent by Atlassian JIRA (v6.4.13#64028-sha1:b7939e9)

[Adrien Thebo (JIRA)]

Adrien Thebo updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Adrien Thebo Labels: ssl

Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Branan Riley (JIRA)]

Branan Riley updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Branan Riley Story Points: 3

Add Comment

[Branan Riley (JIRA)]

Branan Riley updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Branan Riley Labels: ssl  triaged

Add Comment

[Branan Riley (JIRA)]

Branan Riley updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Branan Riley Team: Agent

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Josh Cooper Sub-team: Coremunity

Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts Change By: Josh Cooper

underlined textPuppet Puppet 's HTTP connection code does not allow callers to specify that they want to trust the default set of cacerts. To do so, you need to get access to the {{Net::HTTP#store}} and call {{OpenSSL::X509::Store#set_default_paths}}, but this is not currently possible.

The PMT works around this by rolling its own {{Net::HTTP}} object, leading to duplicate proxy handling logic. It would be nice to DRY this up, but to do so, our HTTP code needs to provide a way for callers to trust the system's cacerts. This would also be useful for providers that need to make network connections, and not rely on open-uri to do so.

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts Change By: Josh Cooper

Puppet underlined textPuppet 's HTTP connection code does not allow callers to specify that they want to trust the default set of cacerts. To do so, you need to get access to the {{Net::HTTP#store}} and call {{OpenSSL::X509::Store#set_default_paths}}, but this is not currently possible.

The PMT works around this by rolling its own {{Net::HTTP}} object, leading to duplicate proxy handling logic. It would be nice to DRY this up, but to do so, our HTTP code needs to provide a way for callers to trust the system's cacerts. This would also be useful for providers that need to make network connections, and not rely on open-uri to do so.

Add Comment

[Rob Braden (JIRA)]

Rob Braden updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Rob Braden Sprint: Coremunity Hopper

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Josh Cooper Puppet's HTTP connection code does not allow callers to specify that they want to trust the default set of cacerts. To do so, you need to get access to the {{Net::HTTP#store}} and call {{OpenSSL::X509::Store#set_default_paths}}, but this is not currently possible.

The PMT works around this by rolling its own {{Net::HTTP}} object, leading to duplicate proxy handling logic. It would be nice to DRY this up, but to do so, our HTTP code needs to provide a way for callers to trust the system's cacerts. This would also be useful for providers that need to make network connections, and not rely on open-uri to do so.

Next Steps: 1. Update puppet, report and fileserver http services to accept an optional {{trust_system_store}} argument which defaults to false. 2. Remove the {{ssl_context}} argument from the same set of services. Note the ca server needs to remain as-is, because the agent needs to pass in an {{ssl_context}} during ssl bootstrapping. 3. Update the HTTP client get, post, etc methods to accept either {{trust_system_store}} or {{ssl_context}} but not both. 4. Add a {{Puppet::HTTP::Client#system_ssl_context}} method that calls {{Puppet::SSL::SSLProvider.create_system_context}} and memoizes it 5. If {{trust_system_store}} is true, then {{Puppet::HTTP::Client#get}} (etc) should use the {{system_ssl_context}} for that connection. Acceptance Criteria: 1. The client should be able to connect to well-known servers like github.com, whose SSL cert is not issued by the puppet PKI. 2. The client should not perform revocation checking, since the CA cert won't be in puppet's CRL bundle. 3. A connection created with the system_ssl_context should not be reused when making a connection to puppet infrastructure (using the puppet_ssl_context) and vice-versa. This can be verified looking at the http_debug log and seeing when new connections are started vs reused.

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Josh Cooper Fix Version/s: PUP 6.14.0

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper assigned an issue to Josh Cooper

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Josh Cooper Assignee: Josh Cooper

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Josh Cooper Sprint: Coremunity Hopper Platform Core KANBAN

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-5069 Puppet's HTTP API does not allow callers to trust system cacerts Change By: Josh Cooper

Puppet's HTTP connection code does not allow callers to specify that they want to trust the default set of cacerts. To do so, you need to get access to the {{Net::HTTP#store}} and call {{OpenSSL::X509::Store#set_default_paths}}, but this is not currently possible. The PMT works around this by rolling its own {{Net::HTTP}} object, leading to duplicate proxy handling logic. It would be nice to DRY this up, but to do so, our HTTP code needs to provide a way for callers to trust the system's cacerts. This would also be useful for providers that need to make network connections, and not rely on open-uri to do so. Next Steps:

1. Update puppet, report and fileserver http services to accept an optional {{ trust_system_store include_system_store }} argument which defaults to false.

2. Remove the {{ssl_context}} argument from the same set of services. Note the ca server needs to remain as-is, because the agent needs to pass in an {{ssl_context}} during ssl bootstrapping.

3. Update the HTTP client get, post, etc methods to accept either {{ trust_system_store include_system_store }} or {{ssl_context}} but not both.

4. Add a {{Puppet::HTTP::Client#system_ssl_context}} method that calls {{Puppet::SSL::SSLProvider.create_system_context}} and memoizes it

5. If {{ trust_system_store include_system_store }} is true, then {{Puppet::HTTP::Client#get}} (etc) should use the {{system_ssl_context}} for that connection.

Acceptance Criteria: 1. The client should be able to connect to well-known servers like github.com, whose SSL cert is not issued by the puppet PKI. 2. The client should not perform revocation checking, since the CA cert won't be in puppet's CRL bundle. 3. A connection created with the system_ssl_context should not be reused when making a connection to puppet infrastructure (using the puppet_ssl_context) and vice-versa. This can be verified looking at the http_debug log and seeing when new connections are started vs reused.

Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-5069

Re: Puppet's HTTP API does not allow callers to trust system cacerts Changed trust_system_store to include_system_store

Add Comment

[Melissa Stone (Jira)]

Melissa Stone commented on PUP-5069

Re: Puppet's HTTP API does not allow callers to trust system cacerts

Merged into master with https://github.com/puppetlabs/puppet/commit/001a141fc18db46e3274256784614ed8a8e2b79a

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-5069

Puppet's HTTP API does not allow callers to trust system cacerts

Change By: Josh Cooper Release Notes: Enhancement Release Notes Summary: The HTTP client now accepts a parameter to trust the system certificate store when making HTTPS connections. By default, it only trusts the puppet PKI. The API is private and may change in a future release.

Add Comment