Puppet 3.0 - Windows - Char Encoding Path Problem

[Edward Sumerfield]

The problem starts with requiring rspec-puppet but ends up in the puppet 3.0 run_mode trying to determine Windows APPDATA pathing. The Dir::COMMON_APPDATA constant from win32-dir appears to be unicode which blows on the File.join call in run_mode line 67

: irb

irb(main):001:0> require "rspec-puppet"

Failed to load feature test for root: uninitialized constant Windows::Synchronize

ArgumentError: string contains null byte

        from C:/ruby193/lib/ruby/gems/1.9.1/gems/puppet-3.0.0/lib/puppet/util/run_mode.rb:67:in `join'

        from C:/ruby193/lib/ruby/gems/1.9.1/gems/puppet-3.0.0/lib/puppet/util/run_mode.rb:67:in `conf_dir'

        from C:/ruby193/lib/ruby/gems/1.9.1/gems/puppet-3.0.0/lib/puppet/settings.rb:495:in `user_config_file'

        from C:/ruby193/lib/ruby/gems/1.9.1/gems/puppet-3.0.0/lib/puppet/settings.rb:1234:in `which_configuration_file'

        from C:/ruby193/lib/ruby/gems/1.9.1/gems/puppet-3.0.0/lib/puppet/settings.rb:475:in `parse_config_files'

        from C:/ruby193/lib/ruby/gems/1.9.1/gems/puppet-3.0.0/lib/puppet/settings.rb:147:in `initialize_global_settings'

        from C:/ruby193/lib/ruby/gems/1.9.1/gems/puppet-3.0.0/lib/puppet.rb:135:in `do_initialize_settings_for_run_mode'

        from C:/ruby193/lib/ruby/gems/1.9.1/gems/puppet-3.0.0/lib/puppet.rb:123:in `initialize_settings'

        from C:/ruby193/lib/ruby/gems/1.9.1/gems/rspec-puppet-0.1.5/lib/rspec-puppet.rb:10:in `<top (required)>'

        from C:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:60:in `require'

        from C:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:60:in `rescue in require'

        from C:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:35:in `require'

        from (irb):1

        from C:/ruby193/bin/irb:12:in `<main>'

Reproducing the specific line in question in irb like this:

: irb

irb(main):002:0> require 'win32/dir'

=> true

irb(main):003:0> Dir::COMMON_APPDATA

=> "C:\\ProgramData"

irb(main):006:0> File.join [Dir::COMMON_APPDATA, "PuppetLabs", "puppet", "etc"]

ArgumentError: string contains null byte

        from (irb):6:in `join'

        from (irb):6

        from C:/ruby193/bin/irb:12:in `<main>'

Installed versions:

: ruby -v

ruby 1.9.3p194 (2012-04-20) [i386-mingw32]

: gem list

*** LOCAL GEMS ***

...

facter (1.6.13, 1.6.12)

puppet (3.0.0, 2.7.19)

puppetlabs_spec_helper (0.3.0)

rspec (2.11.0, 2.10.0)

rspec-puppet (0.1.5, 0.1.4)

sys-admin (1.5.6 x86-mingw32)

win32-api (1.4.8 x86-mingw32)

win32-dir (0.4.1)

win32-process (0.7.0)

win32-security (0.1.4, 0.1.3)

win32-service (0.7.2 x86-mingw32)

win32-taskscheduler (0.2.2)

windows-api (0.4.2)

windows-pr (1.2.2)

Thanks

Ed Sumerfield

[Andy Parker]

Hi Edward,

what is the output in irb of:

require 'win32/dir'
Dir::COMMON_APPDATA.encoding
Dir::COMMON_APPDATA.bytes.to_a

I'm interested where this null byte is in that innocuous looking string.

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To post to this group, send email to puppe...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-dev+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-dev?hl=en.

[Edward Sumerfield]

: irb

irb(main):001:0> require 'win32/dir'

=> true

irb(main):002:0> Dir::COMMON_APPDATA

=> "C:\\ProgramData"

irb(main):003:0> Dir::COMMON_APPDATA.encoding

=> #<Encoding:UTF-16LE>

irb(main):004:0> Dir::COMMON_APPDATA.bytes.to_a

=> [67, 0, 58, 0, 92, 0, 80, 0, 114, 0, 111, 0, 103, 0, 114, 0, 97, 0, 109, 0, 68, 0, 97, 0, 116, 0, 97, 0]

[Andy Parker]

Looks like this has hit a bug in ruby itself. I've reported it as
https://bugs.ruby-lang.org/issues/7168

There might be a workaround that we can do where we force all of the
encodings before we hand over to File.join, but this will probably end
up being a problem in other places as well.

On Mon, Oct 15, 2012 at 11:44 AM, Edward Sumerfield

[Edward Sumerfield]

Thanks Andy, so to follow up. nobu closed the ticket indicating it was fixed in r37207. However, this is not committed to the 1_9_3 branch and is only in trunk at the moment. 

tags: svn log http://svn.ruby-lang.org/repos/ruby/tags/v1_9_3_286 --stop-on-copy

------------------------------------------------------------------------

r37165 | usa | 2012-10-12 05:22:37 -0400 (Fri, 12 Oct 2012) | 1 line

Here is a brute force monkey path that might solve the problem until this fix is released.

class File

  class << self

    alias_method :original_join, :join

  end

  def self.join(*args)

    new_args = args.collect { |questionableEncoding|

      questionableEncoding.encode("UTF-8")

    }

    self.send(:original_join, new_args)

  end

end

[Edward Sumerfield]

An improvement on the patch to handle the different types that can be passed into the join.

class File

  class << self

    alias_method :original_join, :join

  end

  def self.join(*args)

    new_args = args.collect { |questionableEncoding|

      join_encoding_fix(questionableEncoding)

    }

    self.send(:original_join, new_args)

  end

  def self.join_encoding_fix(value)

    if (value.instance_of?(String))

      value = value.encode("UTF-8")

    elsif (value.instance_of?(Array))

      value = value.collect { |subValue|

        join_encoding_fix(subValue)

      }

    end

    value

  end

end

[Andy Parker]

The "fix" that they committed only seems to make the exception
explicit instead of an accident. So waiting for it won't really fix
this problem

I'm loath to monkey patch join like this, but it may be the only
option. Can you open a bug against puppet and submit this as a pull
request? It should go in lib/puppet/util/monkey_patches.rb

On Tue, Oct 16, 2012 at 9:16 AM, Edward Sumerfield

[Edward Sumerfield]

https://projects.puppetlabs.com/issues/17010