Jira (PUP-7090) Better error message for source => puppet://foo/bar

[Moses Mendoza (JIRA)]

Moses Mendoza created an issue

Puppet / PUP-7090 Better error message for source => puppet://foo/bar Issue Type: Task Assignee: Unassigned Created: 2017/01/11 9:29 AM Labels: maintenance Priority: Normal Reporter: Moses Mendoza In PUP-6448 we were troubleshooting what we believed was a UTF-8 related problem, when in fact the cause was that the source string took the form puppet://foo/bar instead of {{puppet:///foo/bar} (note the extra slash). The error message we got was: Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not get metadata for puppet://modules/utf_8/?? /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/rest.rb:212:in `is_http_200?' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/rest.rb:110:in `find' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:194:in `find' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:377:in `block in retrieve_new_catalog' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:386:in `block in thinmark' /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:385:in `thinmark' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:376:in `retrieve_new_catalog' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:78:in `retrieve_catalog' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:147:in `prepare_and_retrieve_catalog' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:281:in `run_internal' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:186:in `block in run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:240:in `override' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:185:in `run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:45:in `block (4 levels) in run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent/locker.rb:21:in `lock' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:45:in `block (3 levels) in run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:98:in `with_client' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:42:in `block (2 levels) in run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:65:in `run_in_fork' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:41:in `block in run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:179:in `call' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:179:in `controlled_run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:39:in `run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:353:in `onetime' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:331:in `run_command' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:344:in `block in run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:540:in `exit_on_fail' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:344:in `run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:128:in `run' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute' /opt/puppetlabs/puppet/bin/puppet:5:in `<main>' Warning: Not using cache on failed catalog Error: Could not retrieve catalog; skipping run I can imagine it's not that uncommon to forget the third slash (which means you're omitting the file server and using the default which is your puppet master) so maybe we could supply better validation and warn/error when we detect the slash is missing (or something else is malformed)? Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Ethan Brown (JIRA)]

Ethan Brown updated an issue

Puppet / PUP-7090 Better error message for source => puppet://foo/bar

Change By: Ethan Brown

In PUP-6448 we were troubleshooting what we believed was a UTF-8 related problem, when in fact the cause was that the source string took the form {{puppet://foo/bar}} instead of {{puppet:///foo/bar} (note the extra slash) }

.

The error message we got was:

{code}

{code}

I can imagine it's not that uncommon to forget the third slash (which means you're omitting the file server and using the default which is your puppet master) so maybe we could supply better validation and warn/error when we detect the slash is missing (or something else is malformed)?

Add Comment

[Ethan Brown (JIRA)]

[Ethan Brown (JIRA)]

Ethan Brown commented on PUP-7090

Re: Better error message for source => puppet://foo/bar Totally agree. The equivalent server logs (from puppetserver.log) present like: 2017-01-10 15:34:48,547 ERROR [qtp1582563747-192] [puppetserver] Puppet Server Error: Could not get metadata for puppet://modules/utf_8/静的 /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:233:in `inline_metadata' org/jruby/RubyArray.java:1613:in `each' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:159:in `inline_metadata' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:279:in `compile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:278:in `compile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `benchmark' /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/benchmark.rb:295:in `realtime' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:277:in `compile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:264:in `compile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `benchmark' /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/benchmark.rb:295:in `realtime' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:262:in `compile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:53:in `find' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:194:in `find' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:121:in `do_find' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:48:in `call' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:241:in `override' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:47:in `call' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in `process' org/jruby/RubyArray.java:1613:in `each' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in `process' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:87:in `process' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:87:in `process' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:60:in `process' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:58:in `process' file:/opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/puppetserver-lib/puppet/server/master.rb:42:in `handleRequest' Puppet$$Server$$Master_465464500.gen:13:in `handleRequest' request_handler_core.clj:273:in `invoke' jruby_request.clj:46:in `invoke' jruby_request.clj:31:in `invoke' request_handler_service.clj:34:in `handle_request' request_handler.clj:3:in `invoke' request_handler.clj:3:in `invoke' core.clj:2493:in `invoke' ring_middleware.clj:284:in `invoke' core.clj:168:in `invoke' core.clj:211:in `invoke' core.clj:45:in `invoke' core.clj:343:in `invoke' core.clj:51:in `invoke' ringutils.clj:86:in `invoke' master_core.clj:428:in `invoke' ring.clj:21:in `invoke' ring.clj:12:in `invoke' comidi.clj:249:in `invoke' http.clj:152:in `invoke' http.clj:152:in `invoke' http.clj:148:in `invoke' comidi.clj:336:in `invoke' jetty9_core.clj:424:in `invoke' normalized_uri_helpers.clj:80:in `invoke'

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper commented on PUP-7090

Re: Better error message for source => puppet://foo/bar

This is tricky. It is completely valid to specify a host in the URI. So puppet://foo/bar says to connect to host foo on port 8140 (the default masterport) and use the bar mount point. Whereas puppet:///foo/bar says to connect to the default server or server_list on port 8140, use the foo mount point, and the relative file path bar. In this case, we could validate that the URI path has at least two components (foo and bar corresponding to the mount point and relative path). But that wouldn't help things like puppet://foo/bar/baz. The other thing that complicates this is the user can create custom mount points.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-7090

Re: Better error message for source => puppet://foo/bar

Sounds like a whitelist approach could be used. It would need to start (by default) with only allowing the three slashes variant.

Add Comment

[Ethan Brown (JIRA)]

Ethan Brown commented on PUP-7090

Re: Better error message for source => puppet://foo/bar

I was thinking that when 2 slashes are present, and the metadata could not be retrieved that we simply amend the error message to include something like: puppet:// URI specified a host of FOO. If FOO was not the intended host, use a URI like puppet:/// with 3 slashes to omit a host name I'm mostly looking for a pointer to fix a costly, but easy to make mistake.

Add Comment

[Russell Mull (JIRA)]

Russell Mull updated an issue

Puppet / PUP-7090

Better error message for source => puppet://foo/bar

Change By: Russell Mull Labels: maintenance  triaged

Add Comment

[Russell Mull (JIRA)]

Russell Mull updated an issue

Puppet / PUP-7090 Better error message for source => puppet://foo/bar

Change By: Russell Mull Team: Agent

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-7090

Re: Better error message for source => puppet://foo/bar After the http client rewrite, the error message has a bit more information: $ bx puppet apply -e "file { '/tmp/foo': ensure => file, source => 'puppet://modules/foo' }" Notice: Compiled catalog for localhost in environment production in 0.01 seconds Error: /Stage[main]/Main/File[/tmp/foo]: Could not evaluate: Could not retrieve file metadata for puppet://modules/foo: Request to https://modules:8140/puppet/v3 failed after 15.27 seconds: Failed to open TCP connection to modules:8140 (getaddrinfo: Temporary failure in name resolution)

Add Comment

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)