Jira (PUP-9974) allow_virtual => true matches too much for dpkg provider

5 views
Skip to first unread message

Matthias Hörmann

unread,
Aug 23, 2019, 11:19:04 AM8/23/19
to puppe...@googlegroups.com
Matthias Hörmann created an issue
 
Puppet / Bug PUP-9974
allow_virtual => true matches too much for dpkg provider
Issue Type: Bug Bug
Affects Versions: PUP 6.8.0
Assignee: Unassigned
Created: 2019/08/23 8:18 AM
Priority: Major Major
Reporter: Matthias Hörmann

Puppet Version: 6.8.0
Puppet Server Version: 6.5.0
OS Name/Version: Debian buster (10.x)

Since the change of allow_virtual to true in Puppet 6.8.0 some of our package resources behaved strangely and required to change the parameter back to the old default. At first we thought this only affected packages where the name and a provides value are identical (e.g. a resource to uninstall vim and install vim-nox now tried to unistall vim-nox since it also provides vim).

After installing a new server with the new Puppet version we noticed that the package bc had not been installed. It is defined with the simplest of package resources:

package { 'bc':
    ensure => latest
}

Looking at the provider source code we can see the relevant part here

dpkg.rb

      if @resource.allow_virtual?
        output = dpkgquery(
          "-W",
          "--showformat",
          self.class::DPKG_QUERY_PROVIDES_FORMAT_STRING
        ).lines.find {|package| package.match(/\[.*#{@resource[:name]}.*\]/)}
        if output          
          hash = self.class.parse_line(output,self.class::FIELDS_REGEX_WITH_PROVIDES)
          Puppet.info("Package #{@resource[:name]} is virtual, defaulting to #{hash[:name]}")
          @resource[:name] = hash[:name]
        end
      end

with the constant used being defined further up in the file as

dpkg.rb

  self::DPKG_QUERY_PROVIDES_FORMAT_STRING = %Q{'${Status} ${Package} ${Version} [${Provides}]\\n'}

Running the command

dpkg-query -W --showformat '${Status} ${Package} ${Version} [${Provides}]\n"'

on the affected system we can see the output the provider processes in the above code, a line might look like one of these

"install ok installed php-apcu-bc 1.0.4-4 [php7.3-apcu-bc]
"install ok installed php7.3-bcmath 7.3.4-2 [php-bcmath]
"install ok installed php7.3-odbc 7.3.4-2 [php-odbc, php-pdo-odbc, php7.3-pdo-odbc]

It is easy to see that all of these would also match the package resource named bc since the code just matches .name. within the comma-separated list of dpkg provides values, with no concern for matching a full name.

Desired Behavior:

Virtual packages (provides values) should only match exactly. The default for allow_virtual should be false for at least the apt and dpkg provider since there are many packages in the apt repositories that have identical names and provides values which leads to broken systems when puppet starts uninstalling packages on ensure => absent.

Actual Behavior:

Package resources are currently satisfied that they have found a matching package when they find the resource name as a substring in the list of provides values anywhere.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Thomas Kishel (JIRA)

unread,
Aug 23, 2019, 4:06:03 PM8/23/19
to puppe...@googlegroups.com
Thomas Kishel commented on Bug PUP-9974
 
Re: allow_virtual => true matches too much for dpkg provider

Maybe:

      if @resource.allow_virtual?
        query_output = dpkgquery(
          "-W",
          "--showformat",
          self.class::DPKG_QUERY_PROVIDES_FORMAT_STRING
        )
        virtual_package_line = query_output.lines.find { |line|
          if matched = line.match(/\[(?<package_provides>.*)\]/)
            if matched[:package_provides].split(',').include?(@resource[:name])
              matched
            end
          end
        }
        if virtual_package_line
          hash = self.class.parse_line(virtual_package_line, self.class::FIELDS_REGEX_WITH_PROVIDES)
          Puppet.info("Package #{@resource[:name]} is virtual, defaulting to #{hash[:name]}")
          @resource[:name] = hash[:name]
        end
      end

Tested with:

test.rb

#!/opt/puppetlabs/puppet/bin/ruby
 
def test(query_output)
  package_name = 'bc'
 
        virtual_package_line = query_output.lines.find { |line|
          if matched = line.match(/\[(?<package_provides>.*)\]/)
            if matched[:package_provides].split(',').include?(package_name)
              matched
            end
          end
        }
end
 
puts 'YES'
puts test("install ok installed bc 1.2.3 [bc]\n")
puts
puts 'NOO'
puts test("install ok installed phpodbc 4.5.6 [php-odbc, php-pdo-odbc, php7.3-pdo-odbc]\n")

root@pe-20192nightly-agent-ubuntu:~# ./test.rb 
YES
install ok installed bc 1.2.3 [bc]
 
NOO

Josh Cooper (JIRA)

unread,
Aug 23, 2019, 4:48:03 PM8/23/19
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-9974

This is caused by PUP-9824. In addition to the regex not being specific enough, there is a bigger problem that virtual packages are not enabled by default on apt/dpkg. See PUP-2182 when the same problem occurred with yum. The solution at that time was to define the allow_virtual parameter which defaulted to true. In Puppet 4, the default changed to true, but it only affected those providers currently supporting virtual packages.

To resolve this problem we need to update the regex and the default value of allow_virtual needs to be false for apt/dpkg.

Reply all
Reply to author
Forward
0 new messages