Jira (PUP-11049) Ruby 3 changes default external encoding on Windows

6 views
Skip to first unread message

Josh Cooper (Jira)

unread,
Apr 29, 2021, 7:22:04 PM4/29/21
to puppe...@googlegroups.com
Josh Cooper created an issue
 
Puppet / Bug PUP-11049
Ruby 3 changes default external encoding on Windows
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2021/04/29 4:21 PM
Priority: Normal Normal
Reporter: Josh Cooper

Some encoding related tests fail on Windows on Ruby 3 in github actions. It appears ruby changed the default external encoding on Windows to be UTF-8, instead of the one that matches the native code page, eg. Encoding::CP437:

Run # https://github.com/ruby/ruby/pull/2791/files#diff-ff5ff976e81bebd977f0834e60416abbR97-R100
Active code page: 437
 
LCID             Name             DisplayName
----             ----             -----------
1033             en-US            English (United States)
...
openssl (default: 2.2.0)
OpenSSL Version - OpenSSL 1.1.1k  25 Mar 2021
OpenSSL Library Version - OpenSSL 1.1.1k  25 Mar 2021
PATH
  remote: .
  specs:
    puppet (7.7.0)
      CFPropertyList (~> 2.2)
      concurrent-ruby (~> 1.0)
      deep_merge (~> 1.0)
      facter (>= 2.4.0, < 5)
      fast_gettext (~> 1.1)
      ffi (> 1.9.24, < 2)
      hiera (>= 3.2.1, < 4)
      locale (~> 2.1)
      minitar (~> 0.9)
      multi_json (~> 1.13)
      scanf (~> 1.0)
      semantic_puppet (~> 1.0)
      sorted_set (~> 1.0)
 
GEM
  remote: https://rubygems.org/
  specs:
    CFPropertyList (2.3.6)
    addressable (2.7.0)
      public_suffix (>= 2.0.2, < 5.0)
    artifactory (2.8.2)
    concurrent-ruby (1.1.8)
    crack (0.4.5)
      rexml
    csv (3.1.5)
    deep_merge (1.2.1)
    diff-lcs (1.4.4)
    docopt (0.6.1)
    facter (2.5.7-x64-mingw32)
      ffi (~> 1.9)
    fast_gettext (1.8.0)
    ffi (1.15.0-x64-mingw32)
    hashdiff (1.0.1)
    hiera (3.7.0)
    hiera-eyaml (3.2.1)
      highline
      optimist
    highline (2.0.3)
    hocon (1.3.1)
    json-schema (2.8.1)
      addressable (>= 2.4)
    locale (2.1.3)
    minitar (0.9)
    msgpack (1.4.2)
    multi_json (1.15.0)
    optimist (3.0.1)
    packaging (0.99.76)
      artifactory (~> 2)
      csv (= 3.1.5)
      rake (>= 12.3)
      release-metrics
    public_suffix (4.0.6)
    puppet-resource_api (1.8.13)
      hocon (>= 1.0)
    puppetserver-ca (2.1.0)
      facter (>= 2.0.1, < 5)
    rake (13.0.3)
    rbtree (0.4.4)
    release-metrics (1.1.0)
      csv
      docopt
    rexml (3.2.5)
    rspec (3.10.0)
      rspec-core (~> 3.10.0)
      rspec-expectations (~> 3.10.0)
      rspec-mocks (~> 3.10.0)
    rspec-core (3.10.1)
      rspec-support (~> 3.10.0)
    rspec-expectations (3.10.1)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (~> 3.10.0)
    rspec-its (1.3.0)
      rspec-core (>= 3.0.0)
      rspec-expectations (>= 3.0.0)
    rspec-mocks (3.10.2)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (~> 3.10.0)
    rspec-support (3.10.2)
    ruby-prof (1.4.3-x64-mingw32)
    scanf (1.0.0)
    semantic_puppet (1.0.3)
    set (1.0.1)
    sorted_set (1.0.3)
      rbtree
      set (~> 1.0)
    vcr (5.1.0)
    webmock (3.12.2)
      addressable (>= 2.3.6)
      crack (>= 0.3.2)
      hashdiff (>= 0.4.0, < 2.0.0)
    webrick (1.7.0)
    yard (0.9.26)
 
PLATFORMS
  x64-mingw32
 
DEPENDENCIES
  diff-lcs (~> 1.3)
  ffi
  gettext-setup (~> 0.28)
  hiera-eyaml
  hocon (~> 1.0)
  json-schema (~> 2.0)
  memory_profiler
  minitar (~> 0.9)
  msgpack (~> 1.2)
  packaging (~> 0.99)
  pry
  puppet!
  puppet-resource_api (~> 1.5)
  puppetserver-ca (~> 2.0)
  racc (= 1.4.9)
  rake (~> 13.0)
  rdoc (~> 6.0)
  ronn (~> 0.7.3)
  rspec (~> 3.1)
  rspec-expectations (~> 3.9, != 3.9.3)
  rspec-its (~> 1.1)
  rubocop (~> 0.49)
  rubocop-i18n (~> 1.2.0)
  ruby-prof (>= 0.16.0)
  semantic_puppet (~> 1.0)
  vcr (~> 5.0)
  webmock (~> 3.0)
  webrick (~> 1.7)
  yard
 
BUNDLED WITH
   2.2.16
ruby 3.0.1p64 (2021-04-05 revision 0fb782ee38) [x64-mingw32]
3.2.15
Bundler version 2.2.16
...
Puppet::Util#withenv on Windows works around Ruby bug 8822 (which fails to preserve UTF-8 properly when accessing ENV) (Ruby >= 2.3.x) 
     Failure/Error: raise 'This test requires a non-UTF8 codepage' if Encoding.default_external == Encoding::UTF_8
     RuntimeError:
       This test requires a non-UTF8 codepage
     # ./spec/unit/util_spec.rb:147:in `block (3 levels) in <top (required)>'
     # ./spec/spec_helper.rb:189:in `block (2 levels) in <top (required)>'
     # util/rspec_runner:44:in `run'
     # util/rspec_runner:59:in `<main>'
Finished in 55.78 seconds (files took 3.11 seconds to load)
 
  1) Puppet::Pops::Parser::Lexer2 when lexing files from disk should always read files as UTF-8
     Failure/Error: raise 'This test must be run in a codepage other than 65001 to validate behavior'
     RuntimeError:
       This test must be run in a codepage other than 65001 to validate behavior
     # ./spec/unit/pops/parser/lexer2_spec.rb:922:in `block (3 levels) in <top (required)>'
     # ./spec/spec_helper.rb:189:in `block (2 levels) in <top (required)>'
     # util/rspec_runner:44:in `run'
     # util/rspec_runner:59:in `<main>'

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Josh Cooper (Jira)

unread,
May 3, 2021, 1:25:02 PM5/3/21
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
May 3, 2021, 1:25:02 PM5/3/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Platform Core KANBAN

Josh Cooper (Jira)

unread,
May 25, 2021, 11:55:03 AM5/25/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Epic Link: PUP- 10957 11071

Josh Cooper (Jira)

unread,
Jun 8, 2021, 10:01:08 AM6/8/21
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Jun 8, 2021, 11:32:03 AM6/8/21
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Jun 8, 2021, 11:36:03 AM6/8/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 7.8.0

Josh Cooper (Jira)

unread,
Jun 9, 2021, 4:26:01 PM6/9/21
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Jun 10, 2021, 12:58:02 PM6/10/21
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Jun 10, 2021, 12:58:02 PM6/10/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 7.8.0

Josh Cooper (Jira)

unread,
Jun 10, 2021, 12:59:02 PM6/10/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 7.9.0

Gabriel Nagy (Jira)

unread,
Jun 11, 2021, 10:45:02 AM6/11/21
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Jun 14, 2021, 1:02:02 PM6/14/21
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages