Jira (PUP-10226) URL encode paths in http client

Issue Type:	Task
Assignee:	Unassigned
Created:	2020/01/08 1:33 PM
Priority:	Normal
Reporter:	Josh Cooper

The various Puppet::HTTP::Service::* methods need to URL encode the HTTP request path. It's not currently an issue for the CA or Report services in practice, because the request path is something like /puppet/v3/certificate/<certname> where <certname> is generally safe US-ASCII. However, in theory it could contain restricted characters based on this regex, and we'll soon support arbitrary file paths in the FileServer service, e.g. /puppet/v3/file_metadata/module/mymodule/a path with spaces"

REST based requests using the indirector current encode the URL path using:

URI.parse(Puppet::Util.uri_encode(key))

So we should do something similar to maintain compatibility.

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

Josh Cooper (JIRA)

unread,

Jan 8, 2020, 4:53:04 PM1/8/20

to puppe...@googlegroups.com

Josh Cooper updated an issue

Puppet /

Change By:	Josh Cooper
Sprint:	Coremunity Grooming

Josh Cooper (JIRA)

unread,

Jan 13, 2020, 7:33:03 PM1/13/20

to puppe...@googlegroups.com

Josh Cooper commented on

Grooming Notes:

1. Target master
2. Add calls to Puppet::Util.uri_encode(str) for any user input that ends up in the request path, such as node name, file metadata/content path, etc.
3. The encoded string, e.g. /foo%20bar, should be passed to URI::HTTPS.build(...path: "/foo%20bar"), which will convert the encoded string into a URL.

Josh Cooper (JIRA)

unread,

Jan 21, 2020, 12:46:04 PM1/21/20

to puppe...@googlegroups.com

Josh Cooper updated an issue

Puppet /

Change By:	Josh Cooper
Sprint:	Coremunity Grooming Hopper

Melissa Stone (JIRA)

unread,

Jan 24, 2020, 6:54:04 PM1/24/20

to puppe...@googlegroups.com

Melissa Stone updated an issue

Puppet /

Change By:	Melissa Stone
Sprint:	Coremunity Hopper Platform Core KANBAN

Melissa Stone (JIRA)

unread,

Jan 24, 2020, 6:55:04 PM1/24/20

to puppe...@googlegroups.com

Melissa Stone assigned an issue to Melissa Stone

Puppet /

Change By:	Melissa Stone
Assignee:	Melissa Stone

Melissa Stone (JIRA)

unread,

Jan 29, 2020, 7:02:04 PM1/29/20

to puppe...@googlegroups.com

Melissa Stone commented on

Josh Cooper I have a question on note 3 above. We're already calling `URI::HTTPS.build` when we call `build_url` in the service inits. Are you imagining we'll call `URI::HTTPS.build` twice, or change around the logic to call it only once, right before we send the request?

Josh Cooper (JIRA)

unread,

Jan 30, 2020, 1:19:04 PM1/30/20

to puppe...@googlegroups.com

Josh Cooper commented on

I was hoping we could pass the encoded strings as:

 with_base_url("/certificate/#{Puppet::Util.uri_encode(name)}"),

which will pass the URL encoded string to the URI::HTTPS.build method.

Melissa Stone (JIRA)

unread,

Jan 30, 2020, 1:33:04 PM1/30/20

to puppe...@googlegroups.com

Melissa Stone commented on

From my understanding, `with_base_url` only tacks on the extra string to the url that's already been built with `URI::HTTPS.build`?

Josh Cooper (JIRA)

unread,

Jan 30, 2020, 5:33:04 PM1/30/20

to puppe...@googlegroups.com

Josh Cooper updated an issue

Puppet /

Change By:	Josh Cooper
Fix Version/s:	PUP 6.13.0

Josh Cooper (JIRA)

unread,

Jan 30, 2020, 5:33:04 PM1/30/20

to puppe...@googlegroups.com

Josh Cooper commented on

Merged to master in https://github.com/puppetlabs/puppet/commit/9bdbdc1f605adad3f1363edacd1dfa91551cc6a9

Melissa Stone (JIRA)

unread,

Feb 3, 2020, 12:45:04 PM2/3/20

to puppe...@googlegroups.com

Melissa Stone updated an issue

Puppet /

Change By:	Melissa Stone
Release Notes Summary:	HTTP client encodes url paths
Release Notes:	Not Needed

Melissa Stone (JIRA)

unread,

Feb 4, 2020, 2:20:03 PM2/4/20

to puppe...@googlegroups.com

Melissa Stone commented on

This passed ci with puppet-agent 6.12.0.177.g77ef0268