Jira (FACT-3068) Facter timezone utf8 problem

[Dominique Arpin (Jira)]

Dominique Arpin created an issue

Facter / FACT-3068 Facter timezone utf8 problem Issue Type: Bug Affects Versions: FACT 3.9.z Assignee: Unassigned Components: Facter 3 Created: 2021/08/27 4:37 AM Priority: Normal Reporter: Dominique Arpin Hi, I got a utf8 problem after upgrading from puppet 6.24.0 (opensource) to 7.9 or 7.10. Puppet 6.24.0 (facter -j) "timezone": "Est (heure dÆÚtÚ)",   Puppet 7.10.0 (facter -j) "timezone": "Est (heure d?été)", I got a direct impact when you use Ansible (2.9): { "exception": "C:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/2.7.0/json/common.rb:286:in `to_json': source sequence is illegal/malformed utf-8 (JSON::GeneratorError)\r\nAu caractère Ligne:243 : 9\r\n+ &facter -j | Tee-Object -Variable facter_output > $null\r\n+ ~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (C:/Program File...GeneratorError):String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n\r\nScriptStackTrace:\r\nà <ScriptBlock>, <Aucun fichier> : ligne 243\r\n", "msg": "Unhandled exception while executing module: C:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/2.7.0/json/common.rb:286:in `to_json': source sequence is illegal/malformed utf-8 (JSON::GeneratorError)", "_ansible_verbose_override": true, "_ansible_no_log": false, "changed": false } Regards, Dominique Add Comment

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)

[Josh Cooper (Jira)]

Josh Cooper commented on FACT-3068

Re: Facter timezone utf8 problem Dominique Arpin what is the output of: C:\>"C:\Program Files\Puppet Labs\Puppet\puppet\bin\ruby" -e "puts Time.now.localtime.strftime('%Z').encoding" I suspect that ruby is returning a UTF-16LE string?

Add Comment

[Dominique Arpin (Jira)]

Dominique Arpin commented on FACT-3068

Re: Facter timezone utf8 problem

C:\windows\system32>"C:\Program Files\Puppet Labs\Puppet\puppet\bin\ruby" -e "pu ts Time.now.localtime.strftime('%Z').encoding" {noformat} CP850 I will try run this command on more computer, to see if I got the same value.

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Ciprian Badescu Sprint: NW - 2021-10-06

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Ciprian Badescu Team: Night's Watch

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Ciprian Badescu Story Points: 2

Add Comment

[Luchian Nemes (Jira)]

Luchian Nemes assigned an issue to Luchian Nemes

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Luchian Nemes Assignee: Luchian Nemes

Add Comment

[Luchian Nemes (Jira)]

Luchian Nemes updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Luchian Nemes Attachment: image-2021-09-22-14-18-43-384.png

Add Comment

[Luchian Nemes (Jira)]

Luchian Nemes updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Luchian Nemes Attachment: image-2021-09-22-14-21-56-439.png

Add Comment

[Luchian Nemes (Jira)]

Luchian Nemes commented on FACT-3068

Re: Facter timezone utf8 problem

Dominique Arpin, we were unable to reproduce this. From my understanding, you’re seeing the expected behaviour with Puppet 6.24.0 and the issue with Puppet 7+ (please correct me if I’m wrong) but from my investigation I got the exact opposite results. With the following versions installed: C:\> puppet --version 6.24.0   C:\> facter --version 3.14.19 (commit f3c68a6f636c5e5eb94fd4add78ee5fc8ed33182)   C:\> facter-ng --version 4.2.2 Got this output: (Pasted output as image to not let JIRA create an extra layer of confusion)   And with the following versions installed: C:\> puppet --version 7.11.0 C:\> facter --version 4.2.4 Got this output: (Pasted output as image to not let JIRA create an extra layer of confusion)    Please let us know if you're still seeing this issue, if changing the code page to 1252 helps or if you have any information that could help us further this investigation.

Add Comment

[Luchian Nemes (Jira)]

Luchian Nemes updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Luchian Nemes Attachment: image-2021-10-01-16-49-51-786.png

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu commented on FACT-3068

Re: Facter timezone utf8 problem

https://docs.microsoft.com/en-us/windows/win32/api/winnls/nf-winnls-getacp + we might need fallback if value is to known by ruby

Add Comment

[Dominique Arpin (Jira)]

Dominique Arpin commented on FACT-3068

Re: Facter timezone utf8 problem

I can make some test if you want to check if the value is known by ruby.

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Ciprian Badescu Sprint: NW - 2021-10-06 , NW - 2021-10-20

Add Comment

[Gabriel Nagy (Jira)]

Gabriel Nagy commented on FACT-3068

Re: Facter timezone utf8 problem

Merged to main in https://github.com/puppetlabs/facter/commit/142ad0203f2dcabe1b8f703741b2813f004ca69f Dominique Arpin the commit above should fix the issue you're encountering

Add Comment

[Gabriel Nagy (Jira)]

Gabriel Nagy updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Gabriel Nagy Component/s: Facter 3 Component/s: Facter 4

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Ciprian Badescu Fix Version/s: FACT 4.2.6

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on FACT-3068

Re: Facter timezone utf8 problem

This broke Facter acceptance on Japanese Windows, and I left a comment in https://github.com/puppetlabs/facter/pull/2446#issuecomment-939040529. We used to retrieve the time zone as: Time.now.localtime.strftime('%Z') And now we're calling: timezone = Time.now.zone @fact_list[:timezone] = timezone.force_encoding("CP#{codepage}") When running this script on Japanese Windows: zone1 = Time.now.zone puts zone1 puts zone1.encoding.name puts zone1.valid_encoding? zone1.force_encoding("CP932") puts zone1.encoding.name puts zone1.valid_encoding?   puts zone2 = Time.now.localtime.strftime('%Z') puts zone2.encoding puts zone2.valid_encoding?   puts zone1.encoding == zone2.encoding puts zone1 == zone2 I get: C:\ProgramData\PuppetLabs>ruby zone.rb 協定世界時 Windows-31J true Windows-31J true 協定世界時 UTF-8 true false false So it seems like somehow we used to rely on the timezone encoding being incorrect. It may be that we can call Time.now.zone, but we should transcode the value to UTF8. Something like: zone = Time.now.zone zone.force_encoding("CP932") zone.encode!('UTF-8')

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Ciprian Badescu Sprint: NW - 2021-10-06, NW - 2021-10-20 , NW - 2021-11-03

Add Comment

[Luchian Nemes (Jira)]

Luchian Nemes updated an issue

Facter / FACT-3068 Facter timezone utf8 problem

Change By: Luchian Nemes Release Notes: Bug Fix Release Notes Summary: This release adds a new `timezone` resolver specific to Windows which checks the system codepage and uses it for encoding the timezone fact to avoid unwanted characters on non-English OS.

Add Comment