[Puppet - Bug #11868] (Accepted) msi package provider can only manage packages it installed

[tic...@puppetlabs.com]

Issue #11868 has been reported by Josh Cooper.

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Accepted
• Priority: Normal
• Assignee: Josh Cooper
• Category: windows
• Target version:
• Affected Puppet version:
• Keywords:
• Branch:

The msi package provider keeps track of packages it installed in a yaml file. As a result, it doesn’t know about packages it didn’t install.

The msi package provider should query the registry http://msdn.microsoft.com/en-us/library/aa372105(VS.85).aspx, or access that information through WMI’s Win32_Product http://msdn.microsoft.com/en-us/library/windows/desktop/aa394378(v=vs.85).aspx This way it will accurately report on the state of the system.

---

You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

• Affected Puppet version set to 2.7.6

I think we’d be better off querying the Windows Installer Automation interface: http://msdn.microsoft.com/en-us/library/windows/desktop/aa369432(v=VS.85).aspx, especially since ruby provides decent ole/com support. For example, to query for installed products:

require 'win32ole'

installer = WIN32OLE.new("WindowsInstaller.Installer")
installer.Products.each do |guid|
  # http://msdn.microsoft.com/en-us/library/windows/desktop/aa370130(v=VS.85).aspx
  puts "#{installer.ProductInfo(guid, 'ProductName')} #{guid}"
  %w[Language PackageCode Transforms AssignmentType PackageName InstalledProductName VersionString RegCompany RegOwner ProductID ProductIcon InstallLocation InstallSource InstallDate Publisher LocalPackage HelpLink HelpTelephone URLInfoAbout URLUpdateInfo InstanceType].sort.each do |prop|
    puts "  #{prop}=#{installer.ProductInfo(guid, prop)}"
  end
  puts
end

Generates:

Splunk {936c99c8-74c3-4ca3-9688-e39fe2237d4b}
  AssignmentType=1
  HelpLink=http://www.splunk.com/page/submit_issue
  HelpTelephone=
  InstallDate=20120110
  InstallLocation=c:\splunk\
  InstallSource=c:\packages\
  InstalledProductName=Splunk
  InstanceType=0
  Language=1033
  LocalPackage=c:\WINDOWS\Installer\1399f6e.msi
  PackageCode={AF681FAA-772A-4AA8-8861-352609FFEF83}
  PackageName=splunk-4.2.4-110225-x64-release.msi
  ProductID=none
  ProductIcon=c:\WINDOWS\Installer\{936c99c8-74c3-4ca3-9688-e39fe2237d4b}\ARPPRODUCTICON.exe
  Publisher=Splunk, Inc.
  RegCompany=
  RegOwner=josh
  Transforms=
  URLInfoAbout=http://www.splunk.com
  URLUpdateInfo=http://www.splunk.com/download
  VersionString=108.2.13425

VMware Tools {FE2F6A2C-196E-4210-9C04-2B1BC21F07EF}
  AssignmentType=1
  HelpLink=
  HelpTelephone=
  InstallDate=20110817
  InstallLocation=C:\Program Files\VMware\VMware Tools\
  InstallSource=C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\000036e3\
  InstalledProductName=VMware Tools
  InstanceType=0
  Language=0
  LocalPackage=C:\WINDOWS\Installer\1afa8.msi
  PackageCode={F4B7F59E-4135-4735-97A6-3A98E2AA3A55}
  PackageName=VMware Tools64.msi
  ProductID=none
  ProductIcon=C:\WINDOWS\Installer\{FE2F6A2C-196E-4210-9C04-2B1BC21F07EF}\ARPPRODUCTICON.exe
  Publisher=VMware, Inc.
  RegCompany=
  RegOwner=josh
  Transforms=@1033.mst
  URLInfoAbout=http://www.vmware.com
  URLUpdateInfo=
  VersionString=8.4.7.12773

And to uninstall a product given its productcode:

# msiUILevelNone = 2
installer.UILevel = 2

# INSTALLSTATE_ABSENT = 2
installer.ConfigureProduct("{936c99c8-74c3-4ca3-9688-e39fe2237d4b}", 0, 2)

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Accepted
• Priority: Normal
• Assignee: Josh Cooper
• Category: windows
• Target version:

• Affected Puppet version: 2.7.6
• Keywords:
• Branch:

[tic...@puppetlabs.com]

Issue #11868 has been updated by Nan Liu.

The registry keys are unreliable. Puppet installed on a 64 bit system is actually located at:

HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall

instead of

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Uninstall

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Accepted
• Priority: Normal
• Assignee: Josh Cooper
• Category: windows
• Target version:

• Affected Puppet version: 2.7.6
• Keywords:
• Branch:

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

Lance provided a wmi-based msi package provider that uses wmi to query installed package status. I’m including the implementation here, however, there are two issues that prevent this from being a real solution. 1) It can only discover products installed via MSI, 2) When query WMI for Win32_Product, windows will perform a consistency check on each product, possibly repairing currently installed products. See http://gregramsey.wordpress.com/2012/02/20/win32_product-is-evil/ It’s also very slow.

# TODO: Figure out how to automatically install pre-requisites
# Note that putting the package statement in the init.pp of the module does NOT work
# Ideally, we want to require this gem automatically prior to this provider being included
# Install the Ruby WMI gem required for msiwmi provider
#package { 'ruby-wmi':
#   provider    => gem,
#   ensure      => installed
#}

require 'ruby-wmi'

Puppet::Type.type(:package).provide :msiwmi, :parent => :msi, :source => :msi do

  desc "Package management by installing and removing MSIs via Windows Management Instrumentation (WMI)."

  confine    :operatingsystem => :windows

  has_feature :install_options

  MSIEXEC='msiexec.exe'

  # TODO: Cache WMI results to increase performance

  def self.instances
    packages = []

    # Query all installed products
    self.debug "Running WMI query for installed products..."
    
    if products = WMI::Win32_Product.find(:all)
      self.debug "Found #{products.count} installed products"
      products.each { |product|
        package = new(:name => product.Name, :provider => self.name, :ensure => :installed)
        packages << package
      }
    else
      self.warning "No installed products found - WMI query returned no results"
    end

    packages
  end
  
  # TODO: Cache WMI results to increase performance

  def query
    self.debug "Querying for #{resource[:name]}"
    if products = WMI::Win32_Product.find(:all)
      if product = products.find { |product| product.Name.casecmp(resource[:name]) == 0 }
        return { :name => resource[:name], :ensure => :installed }
      end
    else
      self.warning "No installed products found - WMI query returned no results"
    end
  end
  
  def install
    # TODO: Move this into a common function since it's nearly the same for un/install
    quoted_source = shell_quote resource[:source]
    quoted_options = quote_options resource[:install_options]
    command = [MSIEXEC, '/i', quoted_source, quoted_options, '/qn', '/norestart'].flatten.compact.join(' ')
    
    self.debug "Executing '#{command}'"
    execute command
  end

  def uninstall
    # TODO: Move this into a common function since it's nearly the same for un/install
    quoted_source = shell_quote resource[:source]
    quoted_options = quote_options resource[:install_options]
    command = [MSIEXEC, '/x', quoted_source, quoted_options, '/qn', '/norestart'].flatten.compact.join(' ')
    
    self.debug "Executing '#{command}'"
    execute command
  end

  def quote_options(options)
    quoted = options ? resource[:install_options].collect do |k,v|
      property = shell_quote k
      value    = shell_quote v
      "#{property}=#{value}"
    end : nil
    quoted
  end
  
  # Package name is expected to be a name with an optional version appended
  # Example: Company WWCOM_MAIN_US_PROD 11.8.0.1
  # Name: Company WWCOM_MAIN_US_PROD
  # Version: 11.8.0.1
  # Note the '?' after the '+' - this makes it lazy rather than greedy
  # See: http://www.regular-expressions.info/repeat.html
  NAME_REGEX = /^(.+?)(?:\s+)?([0-9]\S+)?$/

  # TODO: Remove this - it's not used
  def self.parse_product
    hash = nil

    if (m = NAME_REGEX.match(product.Name))
      hash = {
        :provider   => self.name,
        
    :name       => m[1],
        :version    => product.Version,
        :ensure     => :installed,
      }
    else
      self.warning "ERROR: Unable to parse product name (name=#{product.Name})"
    end

    hash

  end

end

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Accepted
• Priority: Normal
• Assignee: Josh Cooper
• Category: windows
• Target version:

• Affected Puppet version: 2.7.6
• Keywords:
• Branch:

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

When I asked Lance “Do you know if WMI’s Win32_Product class provides information that is not available in the Uninstall registry key, http://msdn.microsoft.com/en-us/library/aa372105(VS.85).aspx, for each installed application?” he replied:

“The answer here gets a little messy, so bear with me. My assumption, perhaps false, was that the WMI product provider would be a suitable abstraction over the underlying registry. I really don’t want to go spelunking around the registry if I can avoid it, especially given the multiple locations available that can contain installed product information. For example, the key you cite is only one of a handful that can contain this information – don’t forget the user-specific (HKEY_USER) variant and 32-bit applications installed on 64-bit machines – these go under the Wow6432Node "special” location.

Oh, and be aware that that “special” WOW64 location gets mapped differently based on the version of Windows you’re on. Details here: http://msdn.microsoft.com/en-us/library/aa384253(v=vs.85).aspx

What I just realized today is that this affects what products are visible via WMI. On 64-bit Windows, it won’t list 32-bit products. Lovely. Case in point, try installing WANdisco Subversion on 64-bit Windows 2008 (or Windows 7, I believe). It’ll show up in Programs and Features, but not WMI. Btw, a quick way to see the WMI list is to issue the following command from the command line – “wmic product get name”. You can, supposedly, force WMI to use 32-bit, but, of course, I haven’t tried. That’s just going off the deep end, IMHO. WMI 32-bit on 64-bit platform details here: http://msdn.microsoft.com/en-us/library/aa393067(v=vs.85).aspx

Another link: http://stackoverflow.com/questions/673233/wmi-installed-query-different-from-add-remove-programs-list

What I really want is the ability to hook into whatever logic populates the Programs and Features list in Windows and its respective uninstall functionality. If you find it, please let me know. I hate reinventing wheels.

So, yes, the original attraction behind WMI was getting a representative list and being able to call Uninstall, if need be, without crawling through registry entries or knowing the underlying details. If all of the above sounds ridiculous, good, it should. I knew nothing of such details a couple of months back, having successfully avoided Windows servers (and operations) until a good friend needed help :) Of course, that also means you should take my comments with a grain of salt."

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Accepted
• Priority: Normal
• Assignee: Josh Cooper
• Category: windows
• Target version:

• Affected Puppet version: 2.7.6
• Keywords:
• Branch:

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

Lance also had issues uninstalling when using a multi-instance MSI:

Lance: The Windows (MSI) package manager should allow for specification of uninstall options. I raise this to support a specific use case we have regarding multi-instance MSIs. A multi-instance MSI can be installed multiple times on a single machine, each time specifying a different transform via the command line. For example, we’ll have a web application packaged as a single MSI that targets multiple countries. The transform specifies what country is being installed. The same transform, as we’ve implemented the MSIs, must be specified during uninstall to ensure proper operation. For our use case, reusing the install_options in the custom provider was sufficient.

Josh: If we queried the registry for the UninstallString and passed that to msiexec, then would this still be an issue? Or are there cases, where you would want to specify additional options beyond what the Windows Installer writes to the UninstallString? Said another way, if you specify an MST during install, then the Windows Installer writes the ‘/t package.mst’ option into the UninstallString, right?

Lance: Yeah, I would think the same thing. Here’s what I can tell you – executing the UninstallString does not uninstall the product, at least not in my case. Neither does calling the Uninstall method on the respective WMI object. However, selecting Uninstall via Programs and Features works like a charm and uninstalls only the specific instance, as desired. Confused? Yeah, me too. I really don’t understand the unnecessary complexity or what’s going on behind the scenes, but there it is. Admittedly, the use cases I’m citing is on the more complex side given the multi-instance nature of the installs we’re doing. I doubt the bulk of your customers will encounter them, but they should be easy to handle via configuration, or a trivial extension, of an existing provider, IMHO. One last refinement – I mentioned that reusing the install_options would work. I take that back. The MSINEWINSTANCE property must be present during install but absent during uninstall.

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Accepted
• Priority: Normal
• Assignee: Josh Cooper
• Category: windows
• Target version:

• Affected Puppet version: 2.7.6
• Keywords:
• Branch:

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

I trying using the Windows Installer Automation interface: http://msdn.microsoft.com/en-us/library/windows/desktop/aa369432(v=VS.85).aspx, which seemed promising. For example, to query for installed products:

Interestingly, it captures the transform(s) used to install the VMware Tools package, so I’m optimistic that Uninstall would work as expected.

And you can programmatically uninstall products:

I've also verified you can programmatically uninstall products. For example, this will uninstall splunk silently:

# msiUILevelNone = 2
installer.UILevel = 2

# INSTALLSTATE_ABSENT = 2
installer.ConfigureProduct("{936c99c8-74c3-4ca3-9688-e39fe2237d4b}", 0, 2)

And Lance confirmed that “uninstall does indeed work properly with our multi-instance MSIs.” And he added, “Down the road, the ability to enumerate patches should be a bonus for rounding out your Windows support.”

But we later determined that the automation interface only describes packages installed via MSI, not self-extracting executables. So it’s still not clear how the Add/Remove Program dialog is populated (seems to be from multiple sources)

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Accepted
• Priority: Normal
• Assignee: Josh Cooper
• Category: windows
• Target version:

• Affected Puppet version: 2.7.6
• Keywords:
• Branch:

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

• Parent task deleted (#8268)

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Accepted
• Priority: Normal
• Assignee: Josh Cooper
• Category: windows
• Target version:

• Affected Puppet version: 2.7.6
• Keywords:
• Branch:

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

• Target version set to 2.7.x
• Keywords set to windows msi package

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Accepted
• Priority: Normal
• Assignee: Josh Cooper
• Category: windows

• Target version: 2.7.x
• Affected Puppet version: 2.7.6
• Keywords: windows msi package
• Branch:

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

• Status changed from Accepted to In Topic Branch Pending Review
• Branch set to https://github.com/puppetlabs/puppet/pull/851

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: In Topic Branch Pending Review

• Priority: Normal
• Assignee: Josh Cooper
• Category: windows

• Target version: 2.7.x
• Affected Puppet version: 2.7.6
• Keywords: windows msi package

• Branch: https://github.com/puppetlabs/puppet/pull/851

[tic...@puppetlabs.com]

Issue #11868 has been updated by Patrick Carlisle.

• Status changed from In Topic Branch Pending Review to Merged - Pending Release
• Target version changed from 2.7.x to 2.7.18

Merged in https://github.com/puppetlabs/puppet/commit/ecc52ed6dd105e156a9ac4be58cb5ebbb24b4b2c

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Merged - Pending Release

• Priority: Normal
• Assignee: Josh Cooper
• Category: windows

• Target version: 2.7.18

• Affected Puppet version: 2.7.6
• Keywords: windows msi package
• Branch: https://github.com/puppetlabs/puppet/pull/851

The msi package provider keeps track of packages it installed in a yaml file. As a result, it doesn’t know about packages it didn’t install.

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

Previously, Puppet recorded MSI packages it had installed in a YAML
file. However, if the file was deleted or the system modified, e.g.
Add/Remove Programs, then Puppet did not know the package state had
changed.

Also, if the name of the package did not change across versions, e.g.
VMware Tools, then puppet would report the package as insync even though
the installed version could be different than the one pointed to by the
source parameter.

Also, msiexec.exe returns non-zero exit codes when either the package
requests a reboot (194), the system requires a reboot (3010), e.g. due
to a locked file, or the system initiates a reboot (1641). This would
cause puppet to think the install failed, and it would try to reinstall
the packge the next time it ran (since the YAML file didn't get
updated).

This commit changes the msi package provider to use the Installer
Automation (COM) interface to query the state of the system[1]. It will
now accurately report on installed packages, even those it did not
install, including Puppet itself (#13444). If a package is removed via
Add/Remove Programs, Puppet will re-install it the next time it runs.

The MSI package provider will now warn in the various reboot scenarios,
but report the overall install/uninstall as successful (#14055).

When using the msi package resource, the resource title should match the
'ProductName' property in the MSI Property table, which is also the
value displayed in Add/Remove Programs, e.g.

package { 'Microsoft Visual C++ 2008 Redistributable - x86 9.0.30729.4148':
  ensure => installed,
  ...
}
In cases where the ProductName does not change across versions, e.g.
VMware Tools, you MUST use the PackageCode as the name of the resource
in order for puppet to accurately determine the state of the system:

package { '{0E3AA38E-EAD3-4348-B5C5-051B6852CED6}':
  ensure => installed,
  ...
}
You can obtain the PackageCode in ruby using:

require 'win32ole'
installer = WIN32OLE.new('WindowsInstaller.Installer')
db = installer.OpenDatabase(path, 0)
puts db.SummaryInformation.Property(9)
where is the path to the MSI.

The msi provider does not automatically compare PackageCodes when
determining if the resource is insync, because the source MSI could be
on a network share, and we do not want to copy the potentially large
file just to see if changes need to be made.

The msi provider does not use the Installer interface to perform
install and uninstall, because I have not found a way to obtain useful
error codes when reboots are requested. Instead the methods
InstallProduct and ConfigureProduct raise exceptions with the
general 0x80020009 error, which means 'Exception occurred'. So for now
we continue to use msiexec.exe for install and uninstall, though the msi
provider may not uninstall multi-instance transforms correctly, since
the transform (MST) used to install the package needs to be respecified
during uninstall. This could be resolved by allowing uninstall_options
to be specified, or figuring out how to obtain useful error codes when
using the Installer interface.

[1] http://msdn.microsoft.com/en-us/library/windows/desktop/aa369432(v=vs.85).aspx

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Merged - Pending Release

• Priority: Normal
• Assignee: Josh Cooper
• Category: windows

• Target version: 2.7.18
• Affected Puppet version: 2.7.6
• Keywords: windows msi package
• Branch: https://github.com/puppetlabs/puppet/pull/851

The msi package provider keeps track of packages it installed in a yaml file. As a result, it doesn’t know about packages it didn’t install.

[tic...@puppetlabs.com]

Issue #11868 has been updated by Josh Cooper.

• Target version changed from 2.7.18 to 2.7.19

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Merged - Pending Release

• Priority: Normal
• Assignee: Josh Cooper
• Category: windows

• Target version: 2.7.19

• Affected Puppet version: 2.7.6
• Keywords: windows msi package
• Branch: https://github.com/puppetlabs/puppet/pull/851

The msi package provider keeps track of packages it installed in a yaml file. As a result, it doesn’t know about packages it didn’t install.

[tic...@puppetlabs.com]

Issue #11868 has been updated by Moses Mendoza.

• Status changed from Merged - Pending Release to Closed

Released in 2.7.19rc1.

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Closed

• Priority: Normal
• Assignee: Josh Cooper
• Category: windows

• Target version: 2.7.19
• Affected Puppet version: 2.7.6
• Keywords: windows msi package
• Branch: https://github.com/puppetlabs/puppet/pull/851

The msi package provider keeps track of packages it installed in a yaml file. As a result, it doesn’t know about packages it didn’t install.

[tic...@puppetlabs.com]

Issue #11868 has been updated by Matthaus Litteken.

Released in Puppet 3.0.0-rc4

---

Bug #11868: msi package provider can only manage packages it installed

• Author: Josh Cooper
• Status: Closed

• Priority: Normal
• Assignee: Josh Cooper
• Category: windows

• Target version: 2.7.19
• Affected Puppet version: 2.7.6
• Keywords: windows msi package
• Branch: https://github.com/puppetlabs/puppet/pull/851

The msi package provider keeps track of packages it installed in a yaml file. As a result, it doesn’t know about packages it didn’t install.