Installing Java software on Windows using Puppet

[Jeff Sussna]

I want to use Puppet to manage Groovy and Gradle on Windows machines. The install process for each looks like:

1. Download zip file from a URL
2. Unzip
3. Create a Windows environment variable to point to the location where you unzipped
4. Add that environment variable/bin to the global PATH environment variable.

As far as I can Puppet for Windows supports none of these actions. I'm guessing I have to write a .bat script to call from Puppet? Any plans on adding support for these seemingly common actions?

[Ryan Coleman]

On Thu, Aug 16, 2012 at 12:44 PM, Jeff Sussna <j...@ingineering.it> wrote:
> I want to use Puppet to manage Groovy and Gradle on Windows machines. The
> install process for each looks like:
>
> 1. Download zip file from a URL
> 2. Unzip

The nanliu/staging module on the Puppet Forge will at least get you
the first two points. http://forge.puppetlabs.com/nanliu/staging

With a couple of Puppet resources, it will retrieve the zip file from
a url, store it somewhere and unzip it wherever you want. He lists
some examples on the Forge page. They're for tar.gz files but the
resources automatically handle zipped files should they have the
proper .zip extension.

> 3. Create a Windows environment variable to point to the location where you
> unzipped
> 4. Add that environment variable/bin to the global PATH environment
> variable.

I'm sorry I can't help you with these two. Hopefully someone else on
the list can. :-)

[Jeff Sussna]

Thanks for the pointer to nanliu/staging. Though from the forge page it sounds like it requires a master and hiera. I'm running standalone.

Re env vars, I managed to figure that out. The following resource sets the global system var FOO_BAR to the current value of PATH, with ";baz" appended to the end of it:

exec { "cmd.exe /c reg add \"HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Environment\" /v FOO_BAR /t REG_SZ /d \"$::path;baz\" /f":
     path => $::path
}

Note the escaped quotes.

On Windows 7 you could use the builtin 'setx' command instead, but I'm being wacky and using Puppet on XP, which doesn't have setx.

[Josh Cooper]

Hi Jeff,

I've recently added support for MSI and executable packages on
Windows[1], which will be out in Puppet 3. That way you can just
download the java exe installer (step 1) and manage it via a `package`
resource, thereby avoiding the zip issue. Obviously, it doesn't help
you out now.

In 2.7.x, I'd recommend using one of the later puppet versions as the
MSI support is much better[2]

On Thu, Aug 16, 2012 at 1:45 PM, Jeff Sussna <j...@ingineering.it> wrote:
> Thanks for the pointer to nanliu/staging. Though from the forge page it
> sounds like it requires a master and hiera. I'm running standalone.

You can install and run modules locally, e.g.

puppet apply <manifest> --modulepath <path>

In 2.7.x, make sure to use forward slashes for the module path[3]

>
> Re env vars, I managed to figure that out. The following resource sets the
> global system var FOO_BAR to the current value of PATH, with ";baz" appended
> to the end of it:
>
> exec { "cmd.exe /c reg add \"HKLM\SYSTEM\CurrentControlSet\Control\Session
> Manager\Environment\" /v FOO_BAR /t REG_SZ /d \"$::path;baz\" /f":
> path => $::path
> }
>

You'll need to add an `unless` otherwise puppet will append baz each
time it runs.

> Note the escaped quotes.
>
> On Windows 7 you could use the builtin 'setx' command instead, but I'm being
> wacky and using Puppet on XP, which doesn't have setx.

Didn't know about setx, cool!

>
> On Thursday, August 16, 2012 3:20:11 PM UTC-5, Ryan Coleman wrote:
>>
>> On Thu, Aug 16, 2012 at 12:44 PM, Jeff Sussna <j...@ingineering.it> wrote:
>> > I want to use Puppet to manage Groovy and Gradle on Windows machines.
>> > The
>> > install process for each looks like:
>> >
>> > 1. Download zip file from a URL

If you can host the package on a puppet fileserver, then you can just
use a 'puppet' URI scheme.

If not, then you'll have to use nan's module (which requires curl) to download.

>> > 2. Unzip
>>
>> The nanliu/staging module on the Puppet Forge will at least get you
>> the first two points. http://forge.puppetlabs.com/nanliu/staging
>>
>> With a couple of Puppet resources, it will retrieve the zip file from
>> a url, store it somewhere and unzip it wherever you want. He lists
>> some examples on the Forge page. They're for tar.gz files but the
>> resources automatically handle zipped files should they have the
>> proper .zip extension.
>>

Nan's module requires unzip, which isn't present by default. You'd
probably want to install 7zip. They have an MSI installer, so you can
manage as a package resource.

>> > 3. Create a Windows environment variable to point to the location where
>> > you
>> > unzipped

You can exec reg.exe or use the registry module[4][5]:

So to create an environment variable JAVA_HOME=C:\java7, you'd do:

registry_value { 'HKLM\System\CurrentControlSet\Control\Session
Manager\Environment\JAVA_HOME':
ensure => present,
type => string,
value => 'C:\java7'
}

The `registry_value` type supports all common registry types, such as
REG_EXPAND_SZ, so you can do `value => '%SYSTEMDRIVE%\java7'`

>> > 4. Add that environment variable/bin to the global PATH environment
>> > variable.
>>
>> I'm sorry I can't help you with these two. Hopefully someone else on
>> the list can. :-)
>

This would best be handled by an environment type & provider (to
account for the inclusive/exclusive behavior of the multi-valued PATH
environment variable). In the meantime, probably best to use the exec
resource.

Josh

[1] http://projects.puppetlabs.com/issues/11870
[2] http://projects.puppetlabs.com/issues/11868
[3] http://projects.puppetlabs.com/issues/13055
[4] http://forge.puppetlabs.com/puppetlabs/registry
[5] http://puppetlabs.com/blog/module-of-the-week-puppetlabs-registry-windows/

--
Josh Cooper
Developer, Puppet Labs

[Jeff Sussna]

Thanks! Very helpful. Though I don't follow your discussion of MSI's, since Groovy/Gradle don't have MSI installers. My understanding is MSI development is non-trivial.

[ad]

Jeff,

Just to give you more ideas, and as someone who tends to reinvint wheels, what I do for this sort of thing on Windows is make custom types and use Ruby to handle download, unzip, md5, etc. Here's how a couple custom types I use look in a Puppet manifest:

    media_player_msi { 'ensure_msi_version':

        ensure          => present,

        version         => $version,

        download_dir    => $download_dir,

        schedule        => 'msi_upgrades',

        require         => Media_Player_Stage_Msi['ensure_msi_download'],

        notify          => Exec['reboot'],

    }

I have a standardlib type module with generic functions for things like download, md5, etc. that are used in other custom types. Lacking a robust shell, tools, and package management on Windows, I find this approach cleaner than trying to ship binaries and use cmd.exe.

Also, check out Ruby's win32 stuff, or even just wmic. You may find working with MSI's easier this way than using the 2.7x type in Puppet. Working with the registry is pretty easy too. Of course you can also use the registry module in the Forge. To give you another example, here's a winfacts module I made that uses wmi (note: I am still new to Ruby, this isn't the prettiest, but it works :)

https://github.com/adenning/winfacts

Regards,

Adam

[Josh Cooper]

Hi Adam

On Thu, Aug 16, 2012 at 3:19 PM, ad <adam.d...@gmail.com> wrote:
> Jeff,
>
> Just to give you more ideas, and as someone who tends to reinvint wheels,
> what I do for this sort of thing on Windows is make custom types and use
> Ruby to handle download, unzip, md5, etc. Here's how a couple custom types I
> use look in a Puppet manifest:
>
> media_player_msi { 'ensure_msi_version':
> ensure => present,
> version => $version,
> download_dir => $download_dir,
> schedule => 'msi_upgrades',
> require => Media_Player_Stage_Msi['ensure_msi_download'],
> notify => Exec['reboot'],
> }
>
> I have a standardlib type module with generic functions for things like
> download, md5, etc. that are used in other custom types. Lacking a robust
> shell, tools, and package management on Windows, I find this approach
> cleaner than trying to ship binaries and use cmd.exe.
>

It would be great to get this on the forge, would you be willing to publish it?

> Also, check out Ruby's win32 stuff, or even just wmic. You may find working
> with MSI's easier this way than using the 2.7x type in Puppet.

Be careful with wmic and msi's. Simply enumerating the list of
installed products will cause wmi to perform a consistency check
across all installed products[1]. Probably not what you wanted!

I've made some improvements to the MSI package provider on Windows in
2.7.19[2] (currently in rc and will be out "soon"). It would be great
if you could give that a try.

Also, we'll have support for executable packages in 3.x[3].

And, Rich has a package provider using chocolately (a la apt-get for
windows)[4][5].

Josh

[1] http://gregramsey.net/2012/02/20/win32_product-is-evil/
[2] http://projects.puppetlabs.com/issues/11868
[3] http://projects.puppetlabs.com/issues/11870
[4] https://github.com/rismoney/puppet-chocolatey
[5] http://projects.puppetlabs.com/issues/15541

[ad]

Hey Josh,

On Friday, August 17, 2012 11:38:08 AM UTC-5, Josh Cooper wrote:

>> I have a standardlib type module with generic functions for things like 

>> download, md5, etc. that are used in other custom types. Lacking a robust 

>> shell, tools, and package management on Windows, I find this approach 

>> cleaner than trying to ship binaries and use cmd.exe. 

> It would be great to get this on the forge, would you be willing to publish it? 

It needs some refactoring to get company specific stuff out right now, but sure I'll try to make time to do that.

>> Also, check out Ruby's win32 stuff, or even just wmic. You may find working 

>> with MSI's easier this way than using the 2.7x type in Puppet. 

> Be careful with wmic and msi's. Simply enumerating the list of 

> installed products will cause wmi to perform a consistency check 

> across all installed products[1]. Probably not what you wanted! 

Yeah I know, it's crazy! This is why I get product versions from HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall instead.

> I've made some improvements to the MSI package provider on Windows in 

> 2.7.19[2] (currently in rc and will be out "soon"). It would be great 

> if you could give that a try. 

> Also, we'll have support for executable packages in 3.x[3]. 

Awesome, I'll check it out soon, thanks.

> And, Rich has a package provider using chocolately (a la apt-get for 

> windows)[4][5]. 

I hadn't even heard of chocolately before, thanks, good stuff.

Regards,

Adam

[Josh Cooper]

Hi Adam,

On Fri, Aug 17, 2012 at 11:27 AM, ad <adam.d...@gmail.com> wrote:
>> Hey Josh,
>>
>> On Friday, August 17, 2012 11:38:08 AM UTC-5, Josh Cooper wrote:
>>
>> >> I have a standardlib type module with generic functions for things like
>> >> download, md5, etc. that are used in other custom types. Lacking a
>> >> robust
>> >> shell, tools, and package management on Windows, I find this approach
>> >> cleaner than trying to ship binaries and use cmd.exe.
>>
>> > It would be great to get this on the forge, would you be willing to
>> > publish it?
>>
>> It needs some refactoring to get company specific stuff out right now, but
>> sure I'll try to make time to do that.

Awesome, thanks!

>> >> Also, check out Ruby's win32 stuff, or even just wmic. You may find
>> >> working
>> >> with MSI's easier this way than using the 2.7x type in Puppet.
>>
>> > Be careful with wmic and msi's. Simply enumerating the list of
>> > installed products will cause wmi to perform a consistency check
>> > across all installed products[1]. Probably not what you wanted!
>>
>> Yeah I know, it's crazy! This is why I get product versions from
>> HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall instead.

This is the same approach I'm using in 3.x, including handling
registry redirection for 32 vs 64 bit apps:

https://github.com/puppetlabs/puppet/blob/3.x/lib/puppet/provider/package/windows/package.rb#L32

Josh

[Jeff Sussna]

Hmm..it appears that $::path is returning multiple copies of the Puppet paths, along with "/user/bin" (which as you can imagine causes problems for Windows), in addition to the explicitly defined PATH string. Any idea why?

On Thursday, August 16, 2012 2:44:16 PM UTC-5, Jeff Sussna wrote:

[Josh Cooper]

On Fri, Aug 17, 2012 at 2:38 PM, Jeff Sussna <j...@ingineering.it> wrote:
> Hmm..it appears that $::path is returning multiple copies of the Puppet
> paths, along with "/user/bin" (which as you can imagine causes problems for
> Windows), in addition to the explicitly defined PATH string. Any idea why?
>

So I get:

C:\work\puppet> envpuppet puppet apply -e "notice($::path)"
notice: Scope(Class[main]):
c:\work\puppet\ext\..\bin;c:\work\puppet\ext\..\..\facter\bin;c:\puppetwinbuilder\sys\ruby\bin;c:\ruby187\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;c:\bin;c:\puppetwinbuilder\sys\git\bin;c:\puppetwinbuilder\sys\git\bin;C:\Chocolatey\bin;c:\work\puppet\ext;/usr/sbin;/sbin

The `:path` setting is populated from the environment, but we also add
in /usr/sbin and /sbin (in defaults.rb), obviously not helpful. Filed
as https://projects.puppetlabs.com/issues/16019. With that said it
shouldn't prevent the exec from working.

You can do something like the following to see how puppet resolves
which cmd.exe to run:

C:\work\puppet>envpuppet irb
irb(main):001:0> require 'puppet'
=> true
irb(main):002:0> Puppet::Util.which('cmd.exe')
=> "C:/Windows/system32/cmd.exe"

Or is it not finding reg.exe?

[Jeff Sussna]

I discovered a gotcha with exec'ing "reg add": the change to PATH doesn't show up in the current cmd environment (the one where puppet is being run). That means that, if you add something to PATH multiple times, only the last one will stick.

I tried using puppet-registry, but am getting "unknown function validate_re" on line 44 of value.pp. I'm running in standalone mode, and put the entire contents of the puppet-windows zip from Github in my modules directory. Do I need any special config to make it work (or does it only work with a master)?

On Thursday, August 16, 2012 2:44:16 PM UTC-5, Jeff Sussna wrote:

[Jeff Sussna]

OK, I got registry::value working by commenting out the calls to validate_re. I will file a ticket for that. Unfortunately, though, the Windows gotchas continue. If you manually edit PATH in the System control panel, the new value shows up the next time you open a new cmd window. If, however, you edit it in the registry, the change doesn't show up until you reboot the machine. I've verified this behavior in XP and Server 2003.

I also discovered an "inconvenience" with the puppet registry module. registry::value calls registry_value, which appends the value to the key to create a namevar. That means you can't edit PATH within a module that has a dependency on another module that edits path; if you do, you get a duplicate declaration error.

[Jeff Sussna]

Belay my comment about validate_re. It's in the stdlib module. For some reason the last time I googled it I didn't find anything.

If anyone has any bright ideas re getting around the reboot problem I'm all ears.

[Josh Cooper]

Hi Jeff,

On Mon, Aug 20, 2012 at 11:43 AM, Jeff Sussna <j...@ingineering.it> wrote:
> OK, I got registry::value working by commenting out the calls to
> validate_re. I will file a ticket for that.

The registry module has a dependency on the stdlib module. When
installing the module on a unix puppet master, the dependency will be
resolved automatically (stdlib module will be downloaded if
necessary). And since the running the module tool on Windows is not
currently supported (in large part because of the tar.gz format, see
[1]), you'll have to manually download the stdlib module to your
Windows box.

> Unfortunately, though, the
> Windows gotchas continue. If you manually edit PATH in the System control
> panel, the new value shows up the next time you open a new cmd window. If,
> however, you edit it in the registry, the change doesn't show up until you
> reboot the machine. I've verified this behavior in XP and Server 2003.

I found this post:
http://mnaoumov.wordpress.com/2012/07/24/powershell-add-directory-to-environment-path-variable/

Broadcast to all windows that the environment has changed.

> I also discovered an "inconvenience" with the puppet registry module.
> registry::value calls registry_value, which appends the value to the key to
> create a namevar. That means you can't edit PATH within a module that has a
> dependency on another module that edits path; if you do, you get a duplicate
> declaration error.

Really we should have a type & provider for managing the system path.
I've filed this as https://projects.puppetlabs.com/issues/16048

Josh

[1] http://projects.puppetlabs.com/issues/11276

[Jeff Sussna]

A path type/provider would be great, since the need is so common. For example, 7Zip has an MSI, but to use the command line tools you have to edit PATH.

Also, since facter prepends the Puppet dirs to PATH, every time I update PATH from $::path it adds another copy of the Puppet dirs, and quickly fills up the entire variable with Puppet stuff. I don't actually understand why you do that anyway. If you can point me to the source where it happens I can hack it out myself for now.

[Jeff Sussna]

After trying out http://mnaoumov.wordpress.com/2012/07/24/powershell-add-directory-to-environment-path-variable/ I realized it's smart enough to:

1. Read the old PATH value from the registry (thus getting around the facter path problem)
2. Only add the new dir if it's not already in PATH
3. Broadcast PATH change to new cmd prompts without reboot

So now everything is working just as desired. I've written a a Puppet defined type that takes a zip file and the desired home directory and home env var name, does the unzip, sets the home env var,and adds home\bin to PATH. Using it to automate install of groovy, gradle, and so on.

Really appreciate your help.

Jeff