Jira (PUP-11775) Use valid encoding for metadata file reading

[Tony Vu (Jira)]

Tony Vu created an issue

Puppet / PUP-11775 Use valid encoding for metadata file reading Issue Type: Bug Assignee: Unassigned Created: 2023/03/03 4:06 PM Priority: Normal Reporter: Tony Vu The File.read method allows for a encoding to be specified, such as UTF-8. However, when an invalid encoding is specified, then no error is return but an empty string is seen. An invalid encoding string of utf-8 is used here when reading the metadata for a module. This should be fixed to used UTF-8 instead.  Add Comment

This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-11775

Re: Use valid encoding for metadata file reading I'm wondering if you're running into PUP-11754 as I'm able to pass the lower case 'utf-8' string: irb(main):004:0> puts File.read('Gemfile', encoding: 'utf-8') # frozen_string_literal: true   source ENV['GEM_SOURCE'] || 'https://rubygems.org'   gemspec name: 'facter'   group(:release, optional: true) do gem 'octokit', '~> 4.18.0' end   gem 'packaging', require: false   local_gemfile = File.expand_path('Gemfile.local', __dir__) eval_gemfile(local_gemfile) if File.exist?(local_gemfile)   group(:documentation) do gem 'ronn', '~> 0.7.3', require: false, platforms: [:ruby] end But if File.read returns an empty string, then you'll hit the multijson issue: irb(main):002:0> Puppet::Util::Json.load('') Traceback (most recent call last): ... 3: from /home/josh/work/puppet/lib/puppet/util/json.rb:64:in `load' 2: from /home/josh/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/multi_json-1.15.0/lib/multi_json.rb:122:in `load' 1: from /home/josh/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/multi_json-1.15.0/lib/multi_json/adapter.rb:20:in `load' Puppet::Util::Json::ParseError (JSON::ParserError)

Add Comment

[Tony Vu (Jira)]

Tony Vu commented on PUP-11775

Re: Use valid encoding for metadata file reading

Hmm, that's really weird, I of course can reproduce the behavior Josh Cooper  has described above, but this is what I see in the repl when I pause the test—it looks like there is some weird behavior with relative vs absolute strings????: [36] pry(#<Puppet::Module>)> md_file => "/Users/toneloc/repos/puppet/spec/fixtures/unit/pops/loaders/loaders/dependent_modules_with_metadata/modules/user/metadata.json" [37] pry(#<Puppet::Module>)> File.read("/Users/toneloc/repos/puppet/spec/fixtures/unit/pops/loaders/loaders/dependent_modules_with_metadata/modules/user/metadata.json", encoding: 'utf-8') => "" [38] pry(#<Puppet::Module>)> File.read("spec/fixtures/unit/pops/loaders/loaders/dependent_modules_with_metadata/modules/user/metadata.json", encoding: 'utf-8') => "{\n  \"name\": \"test-user\",\n  \"author\": \"test\",\n  \"description\": \"\",\n  \"license\": \"\",\n  \"source\": \"\",\n  \"version\": \"1.0.0\",\n  \"dependencies\": [\n  ]\n}\n" [39] pry(#<Puppet::Module>)> File.read(md_file, encoding:  'UTF-8') => "{\n  \"name\": \"test-user\",\n  \"author\": \"test\",\n  \"description\": \"\",\n  \"license\": \"\",\n  \"source\": \"\",\n  \"version\": \"1.0.0\",\n  \"dependencies\": [\n  ]\n}\n" [40] pry(#<Puppet::Module>)> File.read(md_file, encoding:  'utf-8') => "" [41] pry(#<Puppet::Module>)> whereamiFrom: /Users/toneloc/repos/puppet/lib/puppet/module.rb:218 Puppet::Module#read_metadata:    215: def read_metadata     216:   md_file = metadata_file     217:   require 'pry-byebug';binding.pry  => 218:   md_file.nil? ? {} : Puppet::Util::Json.load(File.read(md_file, :encoding => 'utf-8'))     219: rescue Errno::ENOENT     220:   {}     221: rescue Puppet::Util::Json::ParseError => e     222:   #TRANSLATORS 'metadata.json' is a specific file name and should not be translated.     223:   msg = _("%{name} has an invalid and unparsable metadata.json file. The parse error: %{error}") % { name: name, error: e.message }     224:   case Puppet[:strict]     225:   when :off     226:     Puppet.debug(msg)     227:   when :warning     228:     Puppet.warning(msg)     229:   when :error     230:     raise FaultyMetadata, msg     231:   end     232:   {}     233: end

Add Comment