Jira (PUP-11196) puppet warns about BOM and US-ASCII encoding

39 views
Skip to first unread message

Josh Cooper (Jira)

unread,
Aug 3, 2021, 1:58:04 PM8/3/21
to puppe...@googlegroups.com
Josh Cooper created an issue
 
Puppet / Bug PUP-11196
puppet warns about BOM and US-ASCII encoding
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2021/08/03 10:57 AM
Priority: Normal Normal
Reporter: Josh Cooper

The fix for PUP-8243 causes ruby to generate a warning if the default external encoding is US-ASCII:

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_system/file_impl.rb:83: warning: BOM with non-UTF encoding US-ASCII is nonsense

Since ASCII characters are single bytes, there is no need for a BOM to detect byte ordering (LSB/MSB), hence the message. I had assumed ruby would just ignore the `bom` prefix.

We'll need to check that the current external encoding starts with `utf-` (case-insensitively I think), and only add the `bom` prefix if it does.

See https://github.com/puppetlabs/puppet/pull/8639#issuecomment-891939114
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Josh Cooper (Jira)

unread,
Aug 3, 2021, 2:03:02 PM8/3/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
The fix for PUP-8243 causes ruby to generate a warning if the default external encoding is US-ASCII:

{noformat}

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_system/file_impl.rb:83: warning: BOM with non-UTF encoding US-ASCII is nonsense
{noformat}


Since ASCII characters are single bytes, there is no need for a BOM to detect byte ordering (LSB/MSB), hence the message. I had assumed ruby would just ignore the `bom` prefix.

We'll need to check that the current external encoding starts with `utf-` (case-insensitively I think), and only add the `bom` prefix if it does.

See https://github.com/puppetlabs/puppet/pull/8639#issuecomment-891939114

To reproduce:

{noformat}
$ bx ruby -e 'Encoding.default_external = "US-ASCII"; require "puppet"; Puppet::FileSystem.read_preserve_line_endings("Gemfile")'
/home/josh/work/puppet/lib/puppet/file_system/file_impl.rb:83: warning: BOM with non-UTF encoding US-ASCII is nonsense
{noformat}

Gheorghe Popescu (Jira)

unread,
Aug 16, 2021, 10:48:02 AM8/16/21
to puppe...@googlegroups.com
Gheorghe Popescu updated an issue
Change By: Gheorghe Popescu
Sprint: NW-2021-09-08

John Ericson (Jira)

unread,
Aug 17, 2021, 4:52:02 AM8/17/21
to puppe...@googlegroups.com
John Ericson commented on Bug PUP-11196
 
Re: puppet warns about BOM and US-ASCII encoding

I can confirm this. I started getting the warning:

C:/Program Files/Puppet Labs/DevelopmentKit/private/puppet/ruby/2.5.0/gems/puppet-6.24.0-x64-mingw32/lib/puppet/file_system/windows.rb:113: warning: BOM with non-UTF encoding IBM437 is nonsense

after I upgraded PDK from version 2.1.1 to 2.2.0.

The warning is created when:

Puppet::FileSystem.read_preserve_line_endings 

is reading our templates.

Ciprian Badescu (Jira)

unread,
Aug 25, 2021, 6:45:02 AM8/25/21
to puppe...@googlegroups.com
Ciprian Badescu updated an issue
 
Change By: Ciprian Badescu
Team: Night's Watch

Ciprian Badescu (Jira)

unread,
Aug 25, 2021, 6:45:03 AM8/25/21
to puppe...@googlegroups.com
Ciprian Badescu updated an issue
The fix for PUP-8243 causes ruby to generate a warning if the default external encoding is US-ASCII:
{noformat}
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_system/file_impl.rb:83: warning: BOM with non-UTF encoding US-ASCII is nonsense
{noformat}
Since ASCII characters are single bytes, there is no need for a BOM to detect byte ordering (LSB/MSB), hence the message. I had assumed ruby would just ignore the `bom` prefix.

We'll need to check that the current external encoding starts with `utf-` (case-insensitively I think), and only add the `bom` prefix if it does.

See [ https://github.com/puppetlabs/puppet/pull/8639#issuecomment-891939114 ]

To reproduce:
{noformat}
$ bx ruby -e 'Encoding.default_external = "US-ASCII"; require "puppet"; Puppet::FileSystem.read_preserve_line_endings("Gemfile")'
/home/josh/work/puppet/lib/puppet/file_system/file_impl.rb:83: warning: BOM with non-UTF encoding US-ASCII is nonsense
{noformat}

Ciprian Badescu (Jira)

unread,
Aug 25, 2021, 8:08:03 AM8/25/21
to puppe...@googlegroups.com
Ciprian Badescu updated an issue
Change By: Ciprian Badescu
Story Points: 1

Luchian Nemes (Jira)

unread,
Aug 25, 2021, 8:39:02 AM8/25/21
to puppe...@googlegroups.com
Luchian Nemes assigned an issue to Luchian Nemes
Change By: Luchian Nemes
Assignee: Luchian Nemes

Josh Cooper (Jira)

unread,
Aug 30, 2021, 11:37:02 AM8/30/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 7.11.0
Fix Version/s: PUP 6.25.0

Josh Cooper (Jira)

unread,
Aug 30, 2021, 11:37:03 AM8/30/21
to puppe...@googlegroups.com

Luchian Nemes (Jira)

unread,
Sep 6, 2021, 4:30:02 AM9/6/21
to puppe...@googlegroups.com
Luchian Nemes updated an issue
 
Change By: Luchian Nemes
Release Notes: Bug Fix
Release Notes Summary: This fix removes BOM for non-UTF encoding and its warnings. Since ASCII characters are single bytes, there is no need for a BOM to detect byte ordering (LSB/MSB).

Claire Cadman (Jira)

unread,
Sep 8, 2021, 5:53:03 AM9/8/21
to puppe...@googlegroups.com
Claire Cadman updated an issue
Change By: Claire Cadman
Labels: doc-reviewed
Reply all
Reply to author
Forward
0 new messages