Jira (PUP-6494) exec resources leak the command string when execution fails

[Adrien Thebo (JIRA)]

Brian Conner created an issue

Puppet / PUP-6494 exec resources leak the command string when execution fails Issue Type: Bug Assignee: Unassigned Created: 2016/07/12 9:19 AM Labels: secrets Priority: Normal Reporter: Brian Conner If an exec is run as an inline sh script with an eyaml'd password variable, the password will get logged in plaintext on the console and agent if it fails. loglevel and logoutput don't do anything in this situation, as it's the command that's being displayed, not the output of the command. It was suggested to make the command into a script and run it that way, but that presents putting the password plaintext in the script, not a viable long-term solution. Having the exec's inline sh script executed in this manner presents another issue. The same data is present in the cached catalog on agents in /opt/puppetlabs/puppet/cache/client_data/catalog/*.json. I imagine this issue is caught somewhere in https://tickets.puppetlabs.com/browse/PUP-1974. Just an idea to branch off of the "sensitive" resource type mentioned in PUP-1974: Most passwords and sensitve data will be coming from an eyaml'd variable(at least, in our scenario). If there were a setting in puppet.conf that would mark all eyaml data as "sensitive", hashing or masking it in logs and cached catalogs, that might take care of the lion's share of sensitive information leaks. This would be in addition to being able "to give manifest/module authors the ability to specify resource properties (such as attributes or titles) which are sensitive". Add Comment

This message was sent by Atlassian JIRA (v6.4.13#64028-sha1:b7939e9)

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-6494

Re: exec resources leak the command string when execution fails The eyaml is currently a hiera (project) backend which makes it difficult to have settings for it in puppet. It would be best if it is was available as a data provider instead as it is then possible to pass arguments to the provider via the configuration file (a different format than for the 3.x hiera project). In general, data providers should be able to return rich data types such as a Sensitive[T] which requires a new set of providers than the current set of json/yaml ones. A port of eyaml could be the first.

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper commented on PUP-6494

Re: exec resources leak the command string when execution fails

For this specific issue, can we just not echo the executed command if the command fails? Is it acceptable to only echo the command if debug is enabled (which also prints out the commands being executed)?

Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Thomas Mueller (JIRA)]

Thomas Mueller commented on PUP-6494

Re: exec resources leak the command string when execution fails

I see value in having the command shown on error by default. I'd normally want to know what exact command failed. I'd rather like some show_echo (or whatever name) param like the file types show_diff. So a user can define a resouce default or set it on specific resources.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-6494

Re: exec resources leak the command string when execution fails

If the exec resource provider honoured the Sensitive bit, the command string could be given as a Sensitive value and it could then say "redacted" in the output. (Just an idea).

Add Comment

[Brian Conner (JIRA)]

Brian Conner commented on PUP-6494

Re: exec resources leak the command string when execution fails

Josh, that is a possibility, however, won't the password still be on the agent catalog, which isn't desired? Thomas, the command that prompted this ticket was adcli join, with the password. Any failing exec where the password has to be passed inline as a variable(eyaml or not) will do this. Henrik, I like this idea. Perhaps that could be further implemented on the agent side as well so that the password is a harmless hash on the stored catalog? I believe conjur does something along these lines, I did see it discussed elsewhere in the Puppet tickets.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-6494

Re: exec resources leak the command string when execution fails

Brian Conner Using Sensitive is a half step - while the value is in clear text in the catalog, the fact that it is a sensitive value is kept and logic using it can do its best not to spill the secret. What we are also working on with a slightly longer fuse is the ability to have encrypted data, or handle to remote service like conjur. There is still the issue with a clear text value being needed at some point in the computation and the risk of spilling that secret. The pattern for that will follow that of Sensitive - that is, that you ask for the clear text value and use it in a very controlled manner. To get to that place we need to get a number of more general features in first - one that is now an experimental features "rich data in catalog" that can be played with, but is not production ready. The reason for this feature is that our current catalog can only send a particular set of data values to the agent. Best we can do at the moment is using the Sensitive data type.

Add Comment

[Thomas Mueller (JIRA)]

Thomas Mueller commented on PUP-6494

Re: exec resources leak the command string when execution fails

Brian Conner my comment was based on Josh Cooper's comment ", can we just not echo the executed command if the command fails?" and me thinking he meant that the command should by default not printed anymore when it fails. Most of the time it will be essential to know the exact command and only for (IMHO) very few cases where for example secrets are passed it's not desired. Another issue with passing a password in CLI is that it is available to everybody logged in with rights to view run "ps ef" (but as you running an AD join there wont be much users around). But the whole password/secrets story with Puppet is still bit tricky as things get probably stored at different places (PuppetDB, Agent cached catalog, log output, cached catalog on the server maybe, ...). I'd look forward to use this Sensitive type (and types in general). Unfortunatly still preparing the Puppet 4 upgrade. : /

Add Comment

[Thomas Mueller (JIRA)]

Thomas Mueller commented on PUP-6494

Re: exec resources leak the command string when execution fails

Brian Conner idea for a workaround: create a custom fact node_is_adjoined (boolean) and create the script which is called by exec only if it is false and remove the script if it is true. (still its a param). Or pre-create the ad computer object with a preshared-secret (maybe the mac address of the node, something derivable on the node itself but not known to the whole world) and use the computer account itself to join.

Add Comment

[Karen Van der Veer (JIRA)]

Karen Van der Veer assigned an issue to Peter Huene

Puppet / PUP-6494

exec resources leak the command string when execution fails

Change By: Karen Van der Veer Assignee: Peter Huene

Add Comment

[Karen Van der Veer (JIRA)]

Karen Van der Veer updated an issue

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: Karen Van der Veer

Team: Systems Engineering Sprint: SE 2017-01-25

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson commented on PUP-6494

Re: exec resources leak the command string when execution fails Is there a path to use a type and provider for this, which would enable a resource attribute specifically containing the password? That would make it much easier to use Sensitive[T] since it's a distinct value in the resource rather than having to overload the exec type.

Add Comment

[Peter Huene (JIRA)]

Peter Huene updated an issue

Puppet / PUP-6494

exec resources leak the command string when execution fails

Change By: Peter Huene Story Points: 1

Add Comment

[Peter Huene (JIRA)]

Peter Huene assigned an issue to Unassigned

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: Peter Huene Assignee: Peter Huene

Add Comment

[Karen Van der Veer (JIRA)]

Karen Van der Veer updated an issue

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: Karen Van der Veer Sprint: SE 2017-01-25 , SE 2017-02-08

Add Comment

[Karen Van der Veer (JIRA)]

Karen Van der Veer updated an issue

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: Karen Van der Veer

Sprint: SE 2017-01-25, SE 2017-02-08 , SE 2017-02-22

Add Comment

[Karen Van der Veer (JIRA)]

Karen Van der Veer updated an issue

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: Karen Van der Veer

Sprint: SE 2017-01-25, SE 2017-02-08, SE 2017-02-22 , SE 2017-03-08

Add Comment

[Peter Huene (JIRA)]

Peter Huene assigned an issue to qa

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: Peter Huene Status: Ready for  CI  Test Assignee: qa

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols assigned an issue to John Duarte

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: Geoff Nichols Assignee: qa John Duarte

Add Comment

[John Duarte (JIRA)]

John Duarte commented on PUP-6494

Re: exec resources leak the command string when execution fails This passes on Ubuntu 16.04 amd64 using puppet-agent at sha 4a671ea6829e987be83ab0934389940ce68724a1 containing puppet at sha ad6df55a248a9fdf19afdce42a00eea7adeeae45. An exec resource that has been defined as Sensitive will be redacted from the catalog execution output if it fails. Output: root@p7l7p03h0rd5gwr:~# cat sensitive_exec.pp exec { 'sensitive_false': command => Sensitive.new("/bin/false TOPSECRET"), } exec { 'sensitive_true': command => Sensitive.new("/bin/true TOPSECRET"), } root@p7l7p03h0rd5gwr:~# puppet apply sensitive_exec.pp Notice: Compiled catalog for p7l7p03h0rd5gwr.delivery.puppetlabs.net in environment production in 0.13 seconds Error: [command redacted] returned 1 instead of one of [0] Error: /Stage[main]/Main/Exec[sensitive_false]/returns: change from notrun to 0 failed: [command redacted] returned 1 instead of one of [0] Notice: /Stage[main]/Main/Exec[sensitive_true]/returns: executed successfully Notice: Applied catalog in 0.04 seconds

Add Comment

[John Duarte (JIRA)]

John Duarte updated an issue

Puppet / PUP-6494

exec resources leak the command string when execution fails

Change By: John Duarte QA Contact: John Duarte

Add Comment

[John Duarte (JIRA)]

John Duarte updated an issue

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: John Duarte QA Risk Assessment: Manual

Add Comment

[John Duarte (JIRA)]

John Duarte updated an issue

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: John Duarte Release Notes: New Feature

Add Comment

[John Duarte (JIRA)]

John Duarte updated an issue

Puppet / PUP-6494 exec resources leak the command string when execution fails

Change By: John Duarte Release Notes Summary: Added support for sensitive commands in the Exec resource type: command parameters that are specified as Sensitive.new(...) are now properly redacted when the command fails. This supports using data from lookup and hiera.

Add Comment

[Brian Conner (JIRA)]

Brian Conner commented on PUP-6494

Re: exec resources leak the command string when execution fails Great work John Duarte! Will it be redacted like that in the agent's /opt/puppetlabs/puppet/cache/client_data/catalog/*.json also? Which Puppet release will this be planned for? Appreciate it!

Add Comment

[John Duarte (JIRA)]

John Duarte commented on PUP-6494

Re: exec resources leak the command string when execution fails

Brian Conner all credit goes to Peter Huene for this. The redaction is not part of the agent's catalog in /opt/puppetlabs/puppet/cache/client_data/catalog/*.json therefore the sensitive data CAN be seen in the catalog. Sample catalog contents: { "catalog_format": 1, "catalog_uuid": "ab92ecb0-29e1-458f-9ca3-bce57c5c34b8", "classes": [ "settings", "default", "sensitive_exec" ], "code_id": null, ... "environment": "production", "name": "t3z16k3wvmq7442.delivery.puppetlabs.net", "resources": [ ... { "exported": false, "file": "/etc/puppetlabs/code/environments/production/modules/sensitive_exec/manifests/init.pp", "line": 3, "parameters": { "command": "/bin/false TOPSECRET" }, "sensitive_parameters": [ "command" ], "tags": [ "exec", "sensitive_false", "class", "sensitive_exec", "node", "default" ], "title": "sensitive_false", "type": "Exec" }, { "exported": false, "file": "/etc/puppetlabs/code/environments/production/modules/sensitive_exec/manifests/init.pp", "line": 6, "parameters": { "command": "/bin/true TOPSECRET" }, "sensitive_parameters": [ "command" ], "tags": [ "exec", "sensitive_true", "class", "sensitive_exec", "node", "default" ], "title": "sensitive_true", "type": "Exec" } ... Please see Peter Huene's comment in the PR that the provider will need to be changed to address this. https://github.com/puppetlabs/puppet/pull/5502

Add Comment

[John Duarte (JIRA)]

John Duarte updated an issue

Puppet / PUP-6494

exec resources leak the command string when execution fails

Change By: John Duarte Fix Version/s: PUP 5.0.0

Add Comment

[John Duarte (JIRA)]

John Duarte commented on PUP-6494

Re: exec resources leak the command string when execution fails Brian Conner, my apologies. I did not answer your release question earlier. This change is targeted at the Puppet 5.0.0 release.

Add Comment

[Brian Conner (JIRA)]

Brian Conner commented on PUP-6494

Re: exec resources leak the command string when execution fails

John Duarte my mistake, Peter Huene, thanks for the work! I just saw you update the Fix Version, so it seems that answers my other question about which release this will go into. Haha we're typing at the same time, no apologies needed!

Add Comment

[Ethan Brown (JIRA)]

Ethan Brown updated an issue

Puppet / PUP-6494

exec resources leak the command string when execution fails

Change By: Ethan Brown Fix Version/s: PUP 5.0.0 Fix Version/s: PUP 4.10.0

Add Comment

[Sean McDonald (JIRA)]

Sean McDonald commented on PUP-6494

Re: exec resources leak the command string when execution fails John Duarte I'm assuming this ticket should have the 5.0.0 fix version?

Add Comment

[John Duarte (JIRA)]

John Duarte updated an issue

Puppet / PUP-6494

exec resources leak the command string when execution fails

Change By: John Duarte Fix Version/s: PUP 5.0.0

Add Comment

[John Duarte (JIRA)]

John Duarte commented on PUP-6494

Re: exec resources leak the command string when execution fails Sean McDonald, yes 5.0.0 fix version. I have added it. This was also backported to 4.10.0

Add Comment

[Bengu Karaalioglu Yener (JIRA)]

Bengu Karaalioglu Yener commented on PUP-6494

Re: exec resources leak the command string when execution fails

Hi all, I am trying to write a puppet program for the ldap configuration. To be able to hide the server password, I have created an eyaml file and I am getting the password from the file and using the password in exec command as below:        exec {"RHEL_ldap_configuration_check":            provider => shell,            command => Sensitive.new("ipa-client-install --domain=$domain --realm=$realm --server=$server1 --hostname=$hostname -w $password -U --force"),        } However altough I run the command with Sensitive.new, the decrypted password seems in /opt/puppetlabs/puppet/cache/client_data/catalog/...json file from the puppet client side. How can I hide the password completely, I don't want puppet clients to see the password. If anyone can help, I would really appreciate. Thank you so much.

Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Kledson Basso (JIRA)]

Kledson Basso commented on PUP-6494

Re: exec resources leak the command string when execution fails

Can we reopen this? Even enclosing the whole command with Sensitive, exec resources leak the command string as 'Notice' when execution fails. Simmilarly to Brian Conner, most of our passwords and sensitve data will be coming from an eyaml'd variable. Could we just not echo/notice in clear-text values redacted with Sensitive? agent run #[mynode.openstack.plat.lab]/> puppet agent --version 5.5.2 #[mynode.openstack.plat.lab]/> puppet agent --test Info: Using configured environment 'production' Info: Retrieving pluginfacts Info: Retrieving plugin Info: Retrieving locales Info: Loading facts Info: Caching catalog for gitlab.openstack.plat.lab Info: Applying configuration version '1534166358' Notice: /Stage[main]/My_module::Test_sensitive/Exec[sensitive command with error]/returns: /usr/bin/ls: cannot access /tmp/my_password: No such file or directory Error: /Stage[main]/My_module::Test_sensitive/Exec[sensitive command with error]/returns: change from 'notrun' to ['0'] failed: [command redacted] returned 2 instead of one of [0] Info: Class[My_module::Test_sensitive]: Unscheduling all events on Class[My_module::Test_sensitive] Notice: Applied catalog in 0.10 seconds #[mynode.openstack.plat.lab]/> code content #[mynode.openstack.plat.lab]/> cat manifests/test_sensitive.pp # class: my_module::test_sensitive class my_module::test_sensitive ( Sensitive[String] $my_param = Sensitive('my_password'), ) { exec { 'sensitive command with error' : command => Sensitive("/usr/bin/ls /tmp/${my_param.unwrap}"), } } #[mynode.openstack.plat.lab]/>

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-6494

Re: exec resources leak the command string when execution fails

Well, the error you see is in clear text because it is a string that is generated by the ls command. Think a new ticket should be opened for this case - a feature request to add redaction of sensitive parts of error messages from the system for the exec resource. (Don't think it is a good idea to redact the entire error message).

Add Comment

[Adam Winberg (JIRA)]

Adam Winberg commented on PUP-6494

Re: exec resources leak the command string when execution fails

So this is supposed to be fixed? I get Error: Failed to apply catalog: Validation of Exec[populate_luksfile] failed: 'echo "supersecretpassword"' is not qualified and no path was specified. Please qualify the command or specify a path.   with the following code: exec { "echo_passphrase": command => Sensitive("echo \"${secretpw.unwrap}\""), } with puppet-agent-6.10.0-1.el8.x86_64

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper commented on PUP-6494

Re: exec resources leak the command string when execution fails

Adam Winberg The validation takes place before the command is executed, which is variation on this ticket (where the command is executed but the result is non-zero). Filed as PUP-10100

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper commented on PUP-6494

Re: exec resources leak the command string when execution fails

Bengu Karaalioglu Yener please take a look at the Deferred data type https://puppet.com/docs/puppet/latest/integrating_secrets_and_retrieving_agent-side_data.html. Puppet can retrieve the password from a secret store like Vault, so that the password is not contained in the cached catalog.

Add Comment