Jira (PUP-10928) Puppet falls back to non-rich data if there is binary data in the catalog

[Josh Cooper (Jira)]

Josh Cooper created an issue

Puppet / PUP-10928 Puppet falls back to non-rich data if there is binary data in the catalog Issue Type: Bug Assignee: Unassigned Created: 2021/02/18 9:51 PM Priority: Normal Reporter: Josh Cooper If a catalog contains binary data, such as kerberos keytab files, then puppetserver will fallback to PSON. However, if the catalog also contains Sensitive/Binary/Deferred etc data types, then they will not work properly since rich data can't be serialized via PSON currently. We should either fail the catalog, provide more information that the conversion is lossy, or look at adding a rich_data_pson serialization format. Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[Henrik Lindberg (Jira)]

Henrik Lindberg commented on PUP-10928

Re: Puppet falls back to non-rich data if there is binary data in the catalog IIRC, when rich data serialization encounters ascii-8-bit strings then it turns those into proper Binary instances I may be wrong though as that may not have made it in for some reason). It is only if the receiver does not accept rich-data there will be a real problem - and it should then fail when there is rich data.

Add Comment

[Mihai Buzgau (Jira)]

Mihai Buzgau updated an issue

Puppet / PUP-10928

Puppet falls back to non-rich data if there is binary data in the catalog

Change By: Mihai Buzgau Team: Night's Watch

Add Comment

[Nick Hall (Jira)]

Nick Hall commented on PUP-10928

Re: Puppet falls back to non-rich data if there is binary data in the catalog This has been open a year - we've recently hit this. Is that at least a workaround so that if somebody adds binary data to a catalogue mistakenly that we can stop puppet overwriting files with the Deferred() (e.g) strings rather than the results of the call? e.g. can we stop it falling back to JSON? If not, this has the potential to have a lot of impact.

Add Comment

This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-10928

Re: Puppet falls back to non-rich data if there is binary data in the catalog

It would be fairly easy to add a puppet setting to disable falling back to *PSON*. The code where the fallback happens is:https://github.com/puppetlabs/puppet/blob/4772afa194402a3876069785a178611797a8eb7d/lib/puppet/indirector/catalog/json.rb#L21-L23. So to implement this, I think we'd want: A new setting in defaults.rb, located near "preferred_serialization_format" something like "allow_pson_serialization=true|false". The new setting should allow the fallback by default. If the fallback is triggered, but the fallback is not allowed, then log an error and exception at "err" level instead of "debug". If the fallback is triggered and the fallback is allowed, then log a warning (instead of info) saying that PSON is deprecated and will be removed in a future release. Another option would be to add a "rich_data_pson" serialization format, but I dislike that because 1) it doesn't solve the "accidentally putting binary data in the catalog" problem and 2) we want to get away from PSON as much as possible, see PUP-3852.

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-10928

Puppet falls back to non-rich data if there is binary data in the catalog

Change By: Josh Cooper Team: Night's Watch Phoenix

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-10928

Re: Puppet falls back to non-rich data if there is binary data in the catalog In Puppet 8, we are going to drop PSON, but make it possible to install a puppet pson gem, like we do for msgpack. This will prevent accidental downgrading to PSON. However, we may still want to add a setting to puppet 7.x to prevent downgrading.

Add Comment

This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)

[Henrik Lindberg (Jira)]

Henrik Lindberg commented on PUP-10928

Re: Puppet falls back to non-rich data if there is binary data in the catalog

Isn't the "accidental binary in the catalog" solved by automatically transforming binary string to a proper Binary intsance? (Thus requiring rich data transfer).

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-10928

Puppet falls back to non-rich data if there is binary data in the catalog

Change By: Josh Cooper

If a catalog contains binary data, such as kerberos keytab files, then puppetserver will fallback to PSON. However, if the catalog also contains Sensitive/Binary/Deferred etc data types, then they will not work properly since rich data can't be serialized via PSON currently.

We should either fail the catalog PUP-11787 will add a warning when fallback occurs. In this ticket , provide more information that the conversion is lossy, or look at adding we'll add a {{rich_data_pson}} serialization format setting to prevent fallback .

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-10928

Re: Puppet falls back to non-rich data if there is binary data in the catalog

Isn't the "accidental binary in the catalog" solved by automatically transforming binary string to a proper Binary intsance

Yes, agreed. PSON has been a cause of great many problems. In PUP-11878, we'll add a warning to 7.x that we are falling back. In this ticket we'll add a setting to prevent fallback. And in Puppet 8 we're removing PSON.

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-10928

Puppet falls back to non-rich data if there is binary data in the catalog

Change By: Josh Cooper Epic Link: PUP-11659

Add Comment