Is "Sensitive" broken?

[Dirk Heinrichs]

Hi,

tried to use the "Sensitive" data type for the first time (with Puppet 6.18.0), but it doesn't work as expected (found and followed several tutorials on the net, see links below), also using the "lookup_options" method to ensure the Hiera-provided value is indeed converted to sensitive.

Here's my sample code (Hiera files omitted):

class test (Sensitive $foo) {

  file {'/tmp/foo':

    content => "${foo}\n",

  }

}

When I run this, I get:

Notice: /Stage[main]/Test/File[/tmp/foo]/content: 

--- /tmp/foo    2020-09-09 07:53:40.166807782 +0200

+++ /tmp/puppet-file20200909-18841-zq93gr       2020-09-09 14:55:05.569695841 +0200

@@ -1 +1 @@

-bar

+Sensitive [value redacted]

Notice: /Stage[main]/Test/File[/tmp/foo]/content: content changed '{md5}fc552...' to '{md5}48a07...'

and then the file indeed looks like this:

# cat /tmp/foo 

Sensitive [value redacted]

instead of containing the real value provided in Hiera.

Any ideas?

Thanks...

Dirk

https://blog.example42.com/2019/04/04/puppet_sensitive_data/

https://www.puppetcookbook.com/posts/hide-sensitive-values.html

https://puppet.com/blog/my-journey-securing-sensitive-data-puppet-code/

-- 

Dirk Heinrichs

Senior Systems Engineer, Delivery Pipeline

OpenText ™ Discovery | Recommind

Phone: +49 2226 15966 18

Email: dhei...@opentext.com

Website: www.recommind.de

Recommind GmbH, Von-Liebig-Straße 1, 53359 Rheinbach

Vertretungsberechtigte Geschäftsführer Gordon Davies, Madhu Ranganathan, Christian Waida, Registergericht Amtsgericht Bonn, Registernummer HRB 10646

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail sind nicht gestattet.

[Marty Ewings]

Hi There

https://puppet.com/docs/puppet/6.17/lang_data_sensitive.html

You need to "unwrap" the sensitive data in order to consume the original data in a function

[Dirk Heinrichs]

Am Mittwoch, den 09.09.2020, 06:34 -0700 schrieb Marty Ewings:

https://puppet.com/docs/puppet/6.17/lang_data_sensitive.html

You need to "unwrap" the sensitive data in order to consume the original data in a function

Yes, did that, too. And that works. However, the data is then shown in clear again (in the diff, or in the command of an exec) , so I gained exactly nothing.

Bye...

Dirk

[Mattias Giese]

On 09/09/20 13:53:36, 'Dirk Heinrichs' via Puppet Users wrote:
> Am Mittwoch, den 09.09.2020, 06:34 -0700 schrieb Marty Ewings:
>
> > https://puppet.com/docs/puppet/6.17/lang_data_sensitive.html
> > You need to "unwrap" the sensitive data in order to consume the
> > original data in a function
>
> Yes, did that, too. And that works. However, the data is then shown in
> clear again (in the diff, or in the command of an exec) , so I gained
> exactly nothing.

The following snippet redacts the content from log output but stores the cleartext in the resulting
file, thus doing what you are looking for

```
file { "${home}/.meraki_env":
ensure => file,
owner => $user,
group => $group,
content => Sensitive("export MERAKI_DASHBOARD_API_KEY=${dashboard_api_key}"),
mode => '0600',
}
```

Cheers,

Mattias

--
Mattias Giese
Linux Consultant und Trainer
Mail: gi...@b1-systems.de

B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt, HRB 3537

[Josh Cooper]

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/b8b44d0a3859790edae6d420ab256d629df227a1.camel%40opentext.com.

The issue you're running into is due to the sensitive value being interpolated:

content => "${foo}\n"

If you reference the Sensitive variable directly, then it will work as expected:

content => $foo

This issue and some possible solutions have been discussed in https://tickets.puppetlabs.com/browse/PUP-10092. For example, Henrik suggested a `rewrap` function https://tickets.puppetlabs.com/browse/PUP-10093.

Josh

--

Josh Cooper | Software Engineer

jo...@puppet.com

[Dirk Heinrichs]

Am Mittwoch, den 09.09.2020, 09:14 -0500 schrieb Mattias Giese:

The following snippet redacts the content from log output but stores the cleartext in the resulting
file, thus doing what you are looking for

```
  file { "${home}/.meraki_env":
    ensure  => file,
    owner   => $user,
    group   => $group,
    content => Sensitive("export MERAKI_DASHBOARD_API_KEY=${dashboard_api_key}"),
    mode    => '0600',
  }
```

Thanks a lot. That's at least a workaround.

[Dirk Heinrichs]

Am Mittwoch, den 09.09.2020, 08:59 -0700 schrieb Josh Cooper:

The issue you're running into is due to the sensitive value being interpolated:

content => "${foo}\n"

If you reference the Sensitive variable directly, then it will work as expected:

content => $foo

OK, thanks a lot. But then I'd loose the trailing "\n". And it's also quite counterintuitive, isn't it? Think "exec", where it's sometimes needed to provide a password as part of the command. One would have to define $password as String instead of Sensitive and then wrap the whole command in a Sensitive() call (as Mattias suggested).

This issue and some possible solutions have been discussed in https://tickets.puppetlabs.com/browse/PUP-10092. For example, Henrik suggested a `rewrap` function https://tickets.puppetlabs.com/browse/PUP-10093.

Reg. the solution(s) discussed in there: Wouldn't the addition of a string concatenation operator (+) solve the problem right away (to make it content => $foo + "\n")?

Bye...

Dirk

[Josh Cooper]

On Wed, Sep 9, 2020 at 11:16 PM 'Dirk Heinrichs' via Puppet Users <puppet...@googlegroups.com> wrote:

Am Mittwoch, den 09.09.2020, 08:59 -0700 schrieb Josh Cooper:

The issue you're running into is due to the sensitive value being interpolated:

content => "${foo}\n"

If you reference the Sensitive variable directly, then it will work as expected:

content => $foo

OK, thanks a lot. But then I'd loose the trailing "\n".

To interpolate sensitive values, you currently have to unwrap the sensitive value, interpolate the value, and then rewrap as sensitive:

$var = Sensitive('a')
$var2 = Sensitive("${var.unwrap}\n")

file { '/tmp/sensitive.txt':
  ensure => file,
  content => $var2

}

 

And it's also quite counterintuitive, isn't it?

Yes, I agree the current behavior is surprising and not easy to work with, which is why I filed PUP-10092 :)

Think "exec", where it's sometimes needed to provide a password as part of the command. One would have to define $password as String instead of Sensitive and then wrap the whole command in a Sensitive() call (as Mattias suggested).

Note the `exec` resource specifically treats the entire command/unless/onlyif parameters as sensitive, so those values don't accidentally end up in logs and reports. Also note this capability is available for any provider by passing `sensitive: true` to Puppet::Util::Execution.execute.

This issue and some possible solutions have been discussed in https://tickets.puppetlabs.com/browse/PUP-10092. For example, Henrik suggested a `rewrap` function https://tickets.puppetlabs.com/browse/PUP-10093.

Reg. the solution(s) discussed in there: Wouldn't the addition of a string concatenation operator (+) solve the problem right away (to make it content => $foo + "\n")?

That might work, but is a bit magic and may not work if the order is reversed (as each type would need to account for concatenation with a sensitive):

content => "prefix" + $foo

I'm not sure if it's feasible for puppet to cast the "prefix" string to a sensitive so it can be concatenated with foo. Which I think is why Henrik was suggesting the "rewrap" function. That way it's explicit and it works regardless of argument order. Also it would handle hashes/arrays.

Josh

[Henrik Lindberg]

On 2020-09-11 20:52, Josh Cooper wrote:
>
>
> On Wed, Sep 9, 2020 at 11:16 PM 'Dirk Heinrichs' via Puppet Users

> <puppet...@googlegroups.com <mailto:puppet...@googlegroups.com>>

> <https://github.com/puppetlabs/puppet/blob/6.18.0/lib/puppet/util/execution.rb#L159>.

>
>
>> This issue and some possible solutions have been discussed in
>> https://tickets.puppetlabs.com/browse/PUP-10092

>> <https://urldefense.com/v3/__https://tickets.puppetlabs.com/browse/PUP-10092__;!!Obbck6kTJA!O4r1x_A74uV9gBlwYRiL9nM4gus0ekJBAjar-QkllcvgQnLkdaGjPKxAsiQQM7O9$>.

>> For example, Henrik suggested a `rewrap` function
>> https://tickets.puppetlabs.com/browse/PUP-10093

>> <https://urldefense.com/v3/__https://tickets.puppetlabs.com/browse/PUP-10093__;!!Obbck6kTJA!O4r1x_A74uV9gBlwYRiL9nM4gus0ekJBAjar-QkllcvgQnLkdaGjPKxAsjvV9iIs$>.

>
> Reg. the solution(s) discussed in there: Wouldn't the addition of a
> string concatenation operator (+) solve the problem right away (to
> make it content => $foo + "\n")?
>
>
> That might work, but is a bit magic and may not work if the order is
> reversed (as each type would need to account for concatenation with a
> sensitive):
>
> content => "prefix" + $foo
>
> I'm not sure if it's feasible for puppet to cast the "prefix" string to
> a sensitive so it can be concatenated with foo. Which I think is why

Puppet does not have string concatenation with `+` since it used to mean
numerical addition if the operands can be converted to numeric. Having +
mean string concatenation if both operands are strings could be
introduced in the language since puppet since some time back no longer
automatically convert numerical strings to numbers.

The implementation would be in the evaluator, and it could treat any
operation on a Sensitive as an operation on the unwrapped value and with
a wrap of the result. If that is done it would need to be done to a
large number of operators

Sensitive(1) + Sensitive(2) # would yield Sensitive(3)
Sensitive("foo") + Sensitive("bar") # would yield Sensitive("foobar")

etc... but where should it stop? conditionals and case/selector matches?
Regexp matches etc?

What happens if there is an error when executing an operation on an
unwrapped value? The code raising the error does not know it was
sensitive to begin with and could reveal the sensitive value "that was
wrong" in the eyes of the operation.

For many such reasons, the use of the Sensitive type forces the
implementor to deal with these things - you unwrapped it, you better not
spill the secret! And yes, it is a bit cumbersome to do operation on
sensitive values.

- henrik

> Henrik was suggesting the "rewrap" function. That way it's explicit and
> it works regardless of argument order. Also it would handle hashes/arrays.
>
> Josh
> --
> Josh Cooper | Software Engineer

> jo...@puppet.com <mailto:jo...@puppet.com>

>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com

> <mailto:puppet-users...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/CA%2Bu97umre_yBB_NAxX2%3DeaUJeJE4yXSnZCfrKfH99GGx%2BkTexA%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-users/CA%2Bu97umre_yBB_NAxX2%3DeaUJeJE4yXSnZCfrKfH99GGx%2BkTexA%40mail.gmail.com?utm_medium=email&utm_source=footer>.


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/