#2157 - help on windows paths

[Ken Barber]

Hi everyone,

I need some advice on a ticket I'm working on - #2157:

https://projects.puppetlabs.com/issues/2157

This introduces scripting support for facter. This new change
introduces two new path requirements:

* A directory for scripted facts (for linux now its /etc/facter/facts.d/)
* A file for the cache (for linux now its /tmp/facts_cache.yml - which
on a separate note probably needs to change)

Now I'm about to fix these paths for Windows and since some of you
have already investigated this topic before I would like some advice.

I've looked at how we do this for Puppet now which is something like:

require 'win32/dir'
base = File.join(Dir::COMMON_APPDATA, "Puppetlabs", "facter")

This would make an adequate base directory where I can put sub-dirs
for facts.d and the cache. Does anyone have any comment or feelings
about this approach? Feel free to comment in the ticket or here :-).

I believe introducing the need for win32-dir may affect Jenkins/CI ...
while we already use win32/resolv in facter (so dependencies should
already be met) we may not already have win32-dir. I'm not sure how
new gem requirements work in this case and I wouldn't mind someone
familiar with our CI environment to comment.

Thanks heaps.

ken.

--
"Join us for PuppetConf, September 22nd and 23rd in Portland, OR:
http://bit.ly/puppetconfsig"

[Jacob Helwig]

Generally speaking: Facter should probably still work (though in a
degraded mode) without the win32-dir gem installed.

As far as CI goes, we would likely set things up similarly to how the
Puppet jobs are configured. There are "allFeatures", and "noFeatures"
gemsets that make sure the Puppet tests still run & pass when all of the
optional extras are installed, and when none of them are.

Hope that this is the info you were looking for.

--
Jacob Helwig
,----

| Join us for PuppetConf, September 22nd and 23rd in Portland, OR
| http://bit.ly/puppetconfsig

`----

[Ken Barber]

> Generally speaking: Facter should probably still work (though in a
> degraded mode) without the win32-dir gem installed.

Hmm. ENV["ProgramData"] is actually the same as Dir::COMMON_APPDATA on
2008R2. I can avoid the gem completely this way, but this is not how
Puppet does it today ... as it seems Puppet depends on
Dir::COMMON_APPDATA being around on windows.

I could avoid allowing script & cache handling altogether without the
gem, but a workaround like using the ENV var is so simple it seems
overkill.

I can always look for the gem, and fall back to the environment var.
Or just go with the environment var only ... avoiding the lot - not
sure what problems this may bring.

> As far as CI goes, we would likely set things up similarly to how the
> Puppet jobs are configured.  There are "allFeatures", and "noFeatures"
> gemsets that make sure the Puppet tests still run & pass when all of the
> optional extras are installed, and when none of them are.

Sure that makes sense. Thanks.

ken.

--

[Josh Cooper]

On Tue, Aug 23, 2011 at 11:31 AM, Ken Barber <k...@puppetlabs.com> wrote:

> Generally speaking: Facter should probably still work (though in a
> degraded mode) without the win32-dir gem installed.

Hmm. ENV["ProgramData"] is actually the same as Dir::COMMON_APPDATA on
2008R2. I can avoid the gem completely this way, but this is not how
Puppet does it today ... as it seems Puppet depends on
Dir::COMMON_APPDATA being around on windows.

I could avoid allowing script & cache handling altogether without the
gem, but a workaround like using the ENV var is so simple it seems
overkill.

I can always look for the gem, and fall back to the environment var.
Or just go with the environment var only ... avoiding the lot - not
sure what problems this may bring.

The ProgramData environment variable is not supported on 2003. The equivalent directory on 2003 (and other legacy systems) is %ALLUSERSPROFILE%\Application Data.[1]

Since we were already using the win32/dir gem in puppet, it made sense to use it to resolve the programdata directory. Also one good thing about the gem is that it queries the system for "known folders" and dynamically generates constants that reflect these locations, so it is not dependent on environment variables.

Josh

[1] http://msdn.microsoft.com/en-us/library/dd378457(v=vs.85).aspx

[Ken Barber]

Thanks Josh.

> The ProgramData environment variable is not supported on 2003. The
> equivalent directory on 2003 (and other legacy systems)
> is %ALLUSERSPROFILE%\Application Data.[1]

Gah! (/me goes off to download Windows 2003 ...).

So in 2008R2 ALLUSERSPROFILE points at C:\ProgramData ... so if you
use the suffix Application Data there its going to be in 2008R2:

C:\ProgramData\Application Data

Which is wrong. So I'll need to be careful about this depending on the
different versions of windows ...

> Since we were already using the win32/dir gem in puppet, it made sense to
> use it to resolve the programdata directory. Also one good thing about the
> gem is that it queries the system for "known folders" and dynamically
> generates constants that reflect these locations, so it is not dependent on
> environment variables.

Yeah lovely.

Hmm ... the reality check is that the fallback behavior for lack of
win32/dir and Dir::COMMON_APPDATA is really for systems that are
running Facter but not Puppet where the admin hasn't installed
win32/dir yet themselves ... as they would have _had_ to install it if
they wished to use the latest Puppet support.

So I guess the question here is - do I break script support when
win32-dir is missing or try my hardest to figure out the path using
some other mechanism such as environment variables ... the problem is
that the path cannot change if later someone does install win32-dir so
the fallback mechanism would have to be reliable.

ken.

[Jacob Helwig]

The degraded mode I was referring to was that scripted facts wouldn't
work at all, so my vote would be for that behavior.

--
Jacob Helwig
,----

| Join us for PuppetConf, September 22nd and 23rd in Portland, OR
| http://bit.ly/puppetconfsig

`----

[Josh Cooper]

On Tue, Aug 23, 2011 at 12:46 PM, Ken Barber <k...@puppetlabs.com> wrote:

Thanks Josh.

> The ProgramData environment variable is not supported on 2003. The
> equivalent directory on 2003 (and other legacy systems)
> is %ALLUSERSPROFILE%\Application Data.[1]

Gah! (/me goes off to download Windows 2003 ...).

So in 2008R2 ALLUSERSPROFILE points at C:\ProgramData ... so if you
use the suffix Application Data there its going to be in 2008R2:

C:\ProgramData\Application Data

Which is wrong. So I'll need to be careful about this depending on the
different versions of windows ...

If PROGRAMDATA is defined, I would use that value.

Otherwise, if ALLUSERSPROFILE is defined (and it should be), I would use %ALLUSERSPROFILE%\Application Data

The other alternative is to call the SHGetFolderPath Win32 API directly, similar to what win32-dir does[1]

Something else to keep in mind is that puppet only uses Dir::COMMON_APPDATA when running as "root" on Windows. When running as a non-privileged user it stores configuration data in the user's ~/.puppet directory.

Josh

[1] https://github.com/djberg96/win32-dir/blob/master/lib/win32/dir.rb

[Ken Barber]

> If PROGRAMDATA is defined, I would use that value.
> Otherwise, if ALLUSERSPROFILE is defined (and it should be), I would use
> %ALLUSERSPROFILE%\Application Data

I actually like this methodology over relying on win32-dir myself
because if it works its a win-win and I don't need to degrade. I guess
I can "suck it and see" and fall back to win32-dir if it appears to be
the wrong approach. I like options.

> The other alternative is to call the SHGetFolderPath Win32 API directly,
> similar to what win32-dir does[1]

Hmm. Okay. I'll look at that as well.

> Something else to keep in mind is that puppet only uses Dir::COMMON_APPDATA
> when running as "root" on Windows. When running as a non-privileged user it
> stores configuration data in the user's ~/.puppet directory.

Fair point ... I wonder if this case matches Facter as well.

Thanks - now I have more to think about. You guys have been a great
help ... cheers ...

[Ken Barber]

>> If PROGRAMDATA is defined, I would use that value.
>> Otherwise, if ALLUSERSPROFILE is defined (and it should be), I would use
>> %ALLUSERSPROFILE%\Application Data
>
> I actually like this methodology over relying on win32-dir myself
> because if it works its a win-win and I don't need to degrade. I guess
> I can "suck it and see" and fall back to win32-dir if it appears to be
> the wrong approach. I like options.

By this I mean I'll test on all the Windows variations we aim to
support. I need to test the external scripts code on those platforms
anyway.

Do we have a target list of facter/puppet support for windows?

[Josh Cooper]

On Tue, Aug 23, 2011 at 4:27 PM, Ken Barber <k...@puppetlabs.com> wrote:

>> If PROGRAMDATA is defined, I would use that value.
>> Otherwise, if ALLUSERSPROFILE is defined (and it should be), I would use
>> %ALLUSERSPROFILE%\Application Data
>
> I actually like this methodology over relying on win32-dir myself
> because if it works its a win-win and I don't need to degrade. I guess
> I can "suck it and see" and fall back to win32-dir if it appears to be
> the wrong approach. I like options.

By this I mean I'll test on all the Windows variations we aim to
support. I need to test the external scripts code on those platforms
anyway.

Do we have a target list of facter/puppet support for windows?

The windows release will support 2003 R2 and 2008 R2 running puppet as an agent only. See http://projects.puppetlabs.com/issues/8268 for more details.

Josh