Jira (FACT-3191) Allow Facter 4 to handle UTF-16

[Aaron Hilton (Jira)]

Aaron Hilton created an issue

Facter / FACT-3191 Allow Facter 4 to handle UTF-16 Issue Type: Improvement Affects Versions: FACT 4.2.11 Assignee: Unassigned Components: Facter 4 Created: 2023/03/15 5:44 AM Priority: Low Reporter: Aaron Hilton Customer is requesting that Facter 4 be more open to using UTF-16 and UTF-16LE after upgrading to newer version of PE with agent 7.x and Facter 4 caused agent failures due to custom facts not resolving. Customer states "UTF-8 is rarely the default encoding for file creation from Windows utilities" and Windows registry keys are frequently not UTF-8 Add Comment

This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)

[Charmaine Pritchett (Jira)]

Charmaine Pritchett updated an issue

Facter / FACT-3191 Allow Facter 4 to handle UTF-16

Change By: Charmaine Pritchett Zendesk Ticket Count: 1 Zendesk Ticket IDs: 51235

Add Comment

[Charmaine Pritchett (Jira)]

Charmaine Pritchett updated an issue

Facter / FACT-3191 Allow Facter 4 to handle UTF-16

Change By: Charmaine Pritchett Labels: jira_escalated

Add Comment

[Marty Ewings (Jira)]

Marty Ewings assigned an issue to Marty Ewings

Facter / FACT-3191 Allow Facter 4 to handle UTF-16

Change By: Marty Ewings Assignee: Marty Ewings

Add Comment

[Marty Ewings (Jira)]

Marty Ewings assigned an issue to Unassigned

[Tony Vu (Jira)]

Tony Vu updated an issue

Facter / FACT-3191 Allow Facter 4 to handle UTF-16

Change By: Tony Vu Labels: jira_escalated needs-validation

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on FACT-3191

Re: Allow Facter 4 to handle UTF-16 I think this is a regression from Facter 3 for YAML-based external facts. Facter 3 used yaml-cpp to parse external facts https://github.com/puppetlabs/facter/blob/6d7ffc6efdfbc3b1fc79311cdeb4581ac2098d9c/lib/src/facts/external/yaml_resolver.cc#L29 The yaml-cpp library examines the first several bytes of the stream to determine the encoding, see https://github.com/jbeder/yaml-cpp/blob/74f63c1181533923d5312596ba06b4fc8d10a387/src/stream.cpp#L198-L238 Facter 4 uses libyaml built into Ruby, which has similar logic for determining the encoding, though not exactly the same: https://github.com/yaml/libyaml/blob/f8f760f7387d2cc56a2fc7b1be313a3bf3f7f58c/src/reader.c#L51-L87. libyaml it assumes UTF-8 if the file does not have a BOM, whereas yaml-cpp always tries to detect the encoding. Note how these cases are different: https://github.com/jbeder/yaml-cpp/blob/74f63c1181533923d5312596ba06b4fc8d10a387/src/stream.cpp#L24-L26 That said we should still verify the regression and also check behavior with JSON, INI and external executable facts.

Add Comment

[Michael Hashizume (Jira)]

Michael Hashizume updated an issue

Facter / FACT-3191

Allow Facter 4 to handle UTF-16

Change By: Michael Hashizume Issue Type: Improvement Bug

Add Comment

[Michael Hashizume (Jira)]

Michael Hashizume updated an issue

Facter / FACT-3191 Allow Facter 4 to handle UTF-16

Change By: Michael Hashizume Epic Link: PUP-11659

Add Comment