Jira (PUP-4039) Add ubnt/ubic deploy keys to CentOS 7 Base

[Robin Bowes (JIRA)]

Robin Bowes created an issue

Puppet / PUP-4039 Add ubnt/ubic deploy keys to CentOS 7 Base Issue Type: Bug Affects Versions: PUP 3.7.4 Assignee: Kylo Ginsberg Components: Types and Providers Created: 2015/02/24 4:06 PM Environment: CentOS 7, puppet 3.7.4 Priority: Normal Reporter: Robin Bowes I created a new meta RPM and added it to my yum repo. I then added a package resource to install the new RPM. The manifest fails as follows: Error: /Stage[main]/Profile_sensu_client::Nagios_plugins/Package[ubnt-meta-nagios-plugins]/ensure: change from absent to present failed: Execution of '/bin/yum -d 0 -e 0 -y list ubnt-meta-nagios-plugins' returned 1: Error: No matching Packages to list This is because "yum list" does not update the yum cache: # yum list ubnt-meta-nagios-plugins Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.solarvps.com * epel: mirror.cogentco.com * extras: mirror.us.leaseweb.net * updates: mirror.steadfast.net Error: No matching Packages to list However, if I just try "yum install" directly, the cache is updated, and the package is found and installed: # yum install ubnt-meta-nagios-plugins Loaded plugins: fastestmirror base | 3.6 kB 00:00:00 epel | 4.4 kB 00:00:00 extras | 3.4 kB 00:00:00 foreman | 2.9 kB 00:00:00 foreman-plugins | 2.9 kB 00:00:00 puppetlabs-deps | 2.5 kB 00:00:00 puppetlabs-products | 2.5 kB 00:00:00 updates | 3.4 kB 00:00:00 yo61 | 2.9 kB 00:00:00 (1/4): foreman-plugins/x86_64/primary_db | 49 kB 00:00:00 (2/4): epel/x86_64/primary_db | 4.0 MB 00:00:00 (3/4): yo61/primary_db | 242 kB 00:00:00 (4/4): updates/7/x86_64/primary_db | 6.6 MB 00:00:07 (1/2): epel/x86_64/updateinfo | 288 kB 00:00:00 (2/2): epel/x86_64/pkgtags | 1.4 MB 00:00:00 Loading mirror speeds from cached hostfile * base: distro.ibiblio.org * epel: mirror.cogentco.com * extras: mirror.us.leaseweb.net * updates: mirror.steadfast.net Resolving Dependencies --> Running transaction check ---> Package ubnt-meta-nagios-plugins.noarch 0:1.0.0-1 will be installed --> Finished Dependency Resolution   Dependencies Resolved   ============================================================================================================================================= Package Arch Version Repository Size ============================================================================================================================================= Installing: ubnt-meta-nagios-plugins noarch 1.0.0-1 yo61 3.5 k   Transaction Summary ============================================================================================================================================= Install 1 Package   Total download size: 3.5 k Installed size: 0 Is this ok [y/d/N]: It seems there is no way to install the package on this node without manually refreshing the cache in this way. Add Comment

This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a)

[Robin Bowes (JIRA)]

Robin Bowes updated an issue

Puppet / PUP-4039

package resource doesn't find new packages in yum repo Change By: Robin Bowes Summary: Add ubnt/ubic deploy keys to CentOS 7 Base package resource doesn't find new packages in yum repo

Add Comment

[Kylo Ginsberg (JIRA)]

Kylo Ginsberg updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Kylo Ginsberg Scrum Team: Release Engineering

Add Comment

[Kylo Ginsberg (JIRA)]

Kylo Ginsberg assigned an issue to Unassigned

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Kylo Ginsberg Assignee: Kylo Ginsberg

Add Comment

[Michael Stahnke (JIRA)]

Michael Stahnke commented on PUP-4039

Re: package resource doesn't find new packages in yum repo I think the best course of action here might be to just have an Exec that does a metadata refresh for yum. If puppet did that for each package resource in the graph, that would add lots of time to the puppet run. I do find it odd that yum behaves differently on list vs install though.

Add Comment

This message was sent by Atlassian JIRA (v6.4.5#64020-sha1:78acd6c)

[Michael Stahnke (JIRA)]

Michael Stahnke commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

I'm also curious if this behaves the same way if ensure => latest is set vs installed.

Add Comment

[Kenn Hussey (JIRA)]

Kenn Hussey updated an issue

Puppet / PUP-4039

package resource doesn't find new packages in yum repo

Change By: Kenn Hussey Scrum Team: Release Engineering

Add Comment

This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4)

[Jonathan Lynch (JIRA)]

Jonathan Lynch commented on PUP-4039

Re: package resource doesn't find new packages in yum repo Adding an exec to refresh the metadata is a valid workaround, but I really think puppet should handle that on it's own. Otherwise the documentation for the yumrepo type should be updated to say "if you want this to work properly you'll need to add an exec to your manifest".

Add Comment

This message was sent by Atlassian JIRA (v6.4.13#64028-sha1:b7939e9)

[Michael Stahnke (JIRA)]

Michael Stahnke commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

The issue is a yum issue though. It's not a puppet issue. Puppet is calling out to yum. If yum can't find the package without cleaning metadata, then I guess that a state to enforce. This doesn't happen in all cases (we use the yum_repo type all the time). I'm wondering what your cache settings look like, or your expiry on metadata. Is there a proxy in between your client and the yum repo?

Add Comment

[Jonathan Lynch (JIRA)]

Jonathan Lynch commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Thank you for responding, because you helped me realize my error. The actual problem was that the repository wasn't getting installed before the package. Once I ensured the repository was installed first the problem went away.

Add Comment

[Michael Stahnke (JIRA)]

Michael Stahnke commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Excellent.

Add Comment

[Robin Bowes (JIRA)]

Robin Bowes commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Michael Stahnke I am the original reporter of this. Did you actually try the steps detailed in the original post? The problem occurs because the yum command used by the yum provider to test for package availability does not update the cache. yum list new_package does not find new_package if it is not in the local cache yum install new_package does find new_package even if it is not in the cache because it updates the cache. I don't consider this to be resolved. R.

Add Comment

[Michael Stahnke (JIRA)]

Michael Stahnke assigned an issue to Robin Bowes

Puppet / PUP-4039

package resource doesn't find new packages in yum repo

Change By: Michael Stahnke Assignee: Robin Bowes

Add Comment

[Michael Stahnke (JIRA)]

Michael Stahnke commented on PUP-4039

Re: package resource doesn't find new packages in yum repo Sorry Robin Bowes, I thought the last commenter was the reporter. Are you suggesting that we don't use yum list? What should be used instead? To me this seems like a yum issue still. If list uses a cache and install doesn't, what should puppet do? If we update the cache on every package resource, the performance hit will be huge. People already don't like how long packages take to manage. That performance hit would also only be needed in the case where the repo was updated, but the cache hasn't been updated yet. If this occurs often, you could clear the yum cache at the start of the puppet run or something. I don't see us making a change to optimize for this case when it hurts the general case so much. If there is another way to make this work, please suggest something.

Add Comment

[Robin Bowes (JIRA)]

Robin Bowes commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

I'm not sure why it is using yum list in the install method of the yum provider. There's some comment about virtual packages: unless @resource.allow_virtual? yum *['-d', '0', '-e', '0', '-y', install_options, :list, wanted].compact end To my mind, if we were calling "install" then we should just run "yum install", not run "yum list" before running "yum install" R.

Add Comment

[Robin Bowes (JIRA)]

Robin Bowes commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Has there been any further progress with this?

Add Comment

[Thomas Mueller (JIRA)]

Thomas Mueller commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

EL7 has metadata_expire and metadata_expire_filter yum.conf options. IMHO for yum list to upgrade metadata the metadata_expire_filter needs to be adjusted. and metadata_expire to be set to 0 to ensure metadata is checked on every yum call. or just run yum clean expire-cache before any package seems the yum providers considers a package which is not found with "yum list" as virtual.

Add Comment

[Robin Bowes (JIRA)]

Robin Bowes commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Following some input from Richard Clamp, it seems this is the cause of the regression: https://github.com/puppetlabs/puppet/commit/21322de612356b231830ec8cce57371b84306520 The relevant ticket is PUP-2182

Add Comment

[Jonathan Lynch (JIRA)]

Jonathan Lynch commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Interesting, so it seems like if we find a better way to detect a virtual package then this issue will be resolved.

Add Comment

[Kylo Ginsberg (JIRA)]

Kylo Ginsberg updated an issue

Puppet / PUP-4039

package resource doesn't find new packages in yum repo

Change By: Kylo Ginsberg Sprint: Client Triage

Add Comment

[Kylo Ginsberg (JIRA)]

Kylo Ginsberg updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Kylo Ginsberg Scrum Team: Client Platform

Add Comment

[Robin Bowes (JIRA)]

Robin Bowes commented on PUP-4039

Re: package resource doesn't find new packages in yum repo I would also add that $current_client is on EL6 R.

Add Comment

[Kylo Ginsberg (JIRA)]

Kylo Ginsberg updated an issue

Puppet / PUP-4039

package resource doesn't find new packages in yum repo

Change By: Kylo Ginsberg Sprint: Client Triage CI.next 2016-05-04

Add Comment

[Steve Barlow (JIRA)]

Steve Barlow updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Steve Barlow Sprint: CI.next Client  2016-05-04

Add Comment

[Steve Barlow (JIRA)]

Steve Barlow updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Steve Barlow Story Points: 3

Add Comment

[Eric Thompson (JIRA)]

Eric Thompson updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Eric Thompson QA Risk Assessment Reason: work around available QA Highest Test Level: Acceptance QA Risk Assessment: Medium

Add Comment

[John Duarte (JIRA)]

John Duarte updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: John Duarte QA Status: Reviewed

Add Comment

[Branan Riley (JIRA)]

Branan Riley commented on PUP-4039

Re: package resource doesn't find new packages in yum repo For now, it looks like there's a workaround - refresh the yum cache in between adding a new repo and trying to install packages from it. I know that's not awesome, but it's also a thing that modules which handle Debian already have to deal with, so I'm not too beat up about it.

Add Comment

This message was sent by Atlassian JIRA (v6.4.13#64028-sha1:b7939e9)

[Branan Riley (JIRA)]

Branan Riley commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Unless there's a better way than yum list for us to find out whether a package is virtual, I don't think we can do much to fix this without regressing on handling of virtual packages. None of us working on Puppet right now are yum experts, and when I try to google how to determine if a package is virtual what I end up getting is old Puppet tickets, which doesn't bode well.

Add Comment

This message was sent by Atlassian JIRA (v6.4.13#64028-sha1:b7939e9)

[Steve Barlow (JIRA)]

Steve Barlow updated an issue

Puppet / PUP-4039

package resource doesn't find new packages in yum repo

Change By: Steve Barlow Fix Version/s: PUP 5.0.0

Add Comment

[Steve Barlow (JIRA)]

Steve Barlow updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Steve Barlow Sprint: Client 2016-05-04 , Client 2016-05-18 (Freeze)

Add Comment

[Branan Riley (JIRA)]

Branan Riley updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Branan Riley

Sprint: Client 2016-05-04 , Client 2016-05-18 (Freeze)

Add Comment

[Branan Riley (JIRA)]

Branan Riley commented on PUP-4039

Re: package resource doesn't find new packages in yum repo My thoughts haven't really changed - fixing this in the package type would either regress on PUP-2182 , or cause a pretty major slowdown. Making the yumrepo type do the sync would also be super slow for anyone who adds a bunch of repositories. At this point I'm inclined to sit on this until we're ready to work PUP-1061 - that will require doing enough surgery to how packages are installed that we can more easily do the metadata refresh in a way that's amortized across many packages.

Add Comment

[Robin Bowes (JIRA)]

Robin Bowes commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

I've done a bit of research into this and wonder if "yum info" is a better command to use? This also exits 1 if the specified package is not found, but updates the yum cache if necessary.

Add Comment

[Matthew Cover (JIRA)]

Matthew Cover commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

This hit us as well. I got a chance to test a bit and found that yum info does not update the metadata (at least not on centos 7). I started with a default centos/7 vagrant box. I then installed and started httpd. We use mirrorlist, so my test includes that. /var/www/html/mirrorlist: http://localhost/repo/ /etc/yum.repos.d/test.repo: [test] name=test mirrorlist=http://localhost/mirrorlist enabled=1 gpgcheck=0 metadata_expire=30 I built some test packages using a very simple spec. Name: test-package-a Version: 0.0.1 Release: 1%{?dist} License: NA BuildArch: noarch BuildRoot: %{_tmppath}/%{name}-%{version}-build Requires: bash, coreutils Group: System Environment/Base Summary: test package a. URL: http://example.com   %description test package a.   %prep   %build   %install %{__install} -d %{buildroot}%{_sharedstatedir}/test_package_a   %clean %{__rm} -rf %{buildroot}   %files %defattr(0644,root,root) %dir %{_sharedstatedir}/test_package_a   %changelog * Tue Mar 07 2017 Matt Cover <matt....@highwinds.com> - Built initial spec for test_package_a I then used the following script to run through the tests. doit: #!/bin/bash   run() { echo "==> Running: $@" "$@" 2>&1 | awk '{ print "\t", $0; fflush(); }' }   metadata_age() { echo "`date '+%s'`-`stat -c '%Y' /var/cache/yum/x86_64/7/test/cachecookie`" | bc -l }   deploy_package() { cp /home/vagrant/test-package-${1}-0.0.1-1.el7.centos.noarch.rpm /var/www/html/repo/ && createrepo --update /var/www/html/repo/ }   remove_package() { mv /var/www/html/repo/test-package-${1}-0.0.1-1.el7.centos.noarch.rpm /var/tmp/$i.`date +%s` 2>/dev/null createrepo --update /var/www/html/repo/ }   # clean up run remove_package a   # make sure all repos have reasonably new metadata run yum clean all run yum makecache fast   # make sure test has really new metadata so metadata_age is small run yum clean all --disablerepo="*" --enablerepo=test run yum makecache fast --disablerepo="*" --enablerepo=test run metadata_age   # deploy test package and wait for a little longer than metadata_expire run deploy_package a run sleep 35 run metadata_age   # test yum list run yum list test-package-a run metadata_age   # test yum info run yum info test-package-a run metadata_age   # test yum install run bash -c 'echo "n" | yum install test-package-a' run metadata_age Running this outputs the following. [root@localhost ~]# ./doit ==> Running: remove_package a Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete ==> Running: yum clean all Loaded plugins: fastestmirror Cleaning repos: base extras test updates Cleaning up everything Cleaning up list of fastest mirrors ==> Running: yum makecache fast Loaded plugins: fastestmirror Determining fastest mirrors * base: mirror.hmc.edu * extras: denver.gaminghost.co * updates: centos.mirror.ndchost.com Metadata Cache Created ==> Running: yum clean all --disablerepo=* --enablerepo=test Loaded plugins: fastestmirror Cleaning repos: test Cleaning up everything Cleaning up list of fastest mirrors ==> Running: yum makecache fast --disablerepo=* --enablerepo=test Loaded plugins: fastestmirror Determining fastest mirrors Metadata Cache Created ==> Running: metadata_age 0 ==> Running: deploy_package a Spawning worker 0 with 1 pkgs Spawning worker 1 with 0 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete ==> Running: sleep 35 ==> Running: metadata_age 36 ==> Running: yum list test-package-a Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.hmc.edu * extras: mirrors.usc.edu * updates: centos.mirror.ndchost.com Error: No matching Packages to list ==> Running: metadata_age 38 ==> Running: yum info test-package-a Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.hmc.edu * extras: mirrors.usc.edu * updates: centos.mirror.lstn.net Error: No matching Packages to list ==> Running: metadata_age 39 ==> Running: bash -c echo "n" | yum install test-package-a Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.hmc.edu * extras: mirrors.usc.edu * updates: centos.mirror.ndchost.com Resolving Dependencies --> Running transaction check ---> Package test-package-a.noarch 0:0.0.1-1.el7.centos will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: test-package-a noarch 0.0.1-1.el7.centos test 2.1 k Transaction Summary ================================================================================ Install 1 Package Total download size: 2.1 k Installed size: 0 Is this ok [y/d/N]: Exiting on user command Your transaction was saved, rerun it with: yum load-transaction /tmp/yum_save_tx.2017-03-08.19-03.GlSNww.yumtx ==> Running: metadata_age 1 I agree that an exec of yum makecache fast or similar works around this, but I'd still like to see the yum provider handle this. The solution requirements I see are as follows. 1. Allow the package resource to properly install new package names when matadata_expire has been reached (this was the behavior before PUP-2182 ). 2. Keep the virtual package fix provided by PUP-2182 . 3. Not cause a major slow down when using the yum provider. One possible solution would be something like this. # pseudo code yum list or { yum makecache fast and yum list } This would meet the requirements as I understand it. It would only cause a slow down when a package would otherwise fail. If this is acceptable to the group, I can look into putting a PR together.

Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Matthew Cover (JIRA)]

Matthew Cover commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Submitted a PR https://github.com/puppetlabs/puppet/pull/5699.

Add Comment

[Matthew Cover (JIRA)]

Matthew Cover commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

I also tested this in vagrant, with the same set up as above. This time using puppet.

#!/bin/bash   run() { echo "==> Running: $@" "$@" 2>&1 | awk '{ print "\t", $0; fflush(); }' }   metadata_age() { echo "`date '+%s'`-`stat -c '%Y' /var/cache/yum/x86_64/7/test/cachecookie`" | bc -l }   deploy_package() { cp /home/vagrant/test-package-${1}-0.0.1-1.el7.centos.noarch.rpm /var/www/html/repo/ && createrepo --update /var/www/html/repo/ }   remove_package() { mv /var/www/html/repo/test-package-${1}-0.0.1-1.el7.centos.noarch.rpm /var/tmp/$i.`date +%s` 2>/dev/null createrepo --update /var/www/html/repo/ }   # clean up

run yum erase test-package-a

run remove_package a   # make sure all repos have reasonably new metadata run yum clean all run yum makecache fast   # make sure test has really new metadata so metadata_age is small run yum clean all --disablerepo="*" --enablerepo=test run yum makecache fast --disablerepo="*" --enablerepo=test run metadata_age   # deploy test package and wait for a little longer than metadata_expire run deploy_package a run sleep 35 run metadata_age

# test puppet bundle exec puppet apply -e 'package { "test-package-a": ensure => "installed", allow_virtual => false }' Without the patch, the last step fails. Notice: Compiled catalog for localhost.localdomain in environment production in 0.57 seconds Error: Execution of '/bin/yum -d 0 -e 0 -y list test-package-a' returned 1: Error: No matching Packages to list Error: /Stage[main]/Main/Package[test-package-a]/ensure: change from purged to present failed: Execution of '/bin/yum -d 0 -e 0 -y list test-package-a' returned 1: Error: No matching Packages to list Notice: Applied catalog in 0.73 seconds With the patch, the last step succeeds. Notice: Compiled catalog for localhost.localdomain in environment production in 0.71 seconds Notice: /Stage[main]/Main/Package[test-package-a]/ensure: created Notice: Applied catalog in 8.92 seconds This was tested in rvm on ruby 2.3.1.

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-4039

package resource doesn't find new packages in yum repo

Change By: Geoff Nichols Fix Version/s: PUP 5.0.0 Fix Version/s: PUP 6.0.0

Add Comment

[Robin Bowes (JIRA)]

Robin Bowes commented on PUP-4039

Re: package resource doesn't find new packages in yum repo Two things: 1. This ticket is marked as "Needs Information". What information does it need? 2. Why has fix version been moved to 6.0.0? Why not fix it in the next release? R.

Add Comment

[Branan Riley (JIRA)]

Branan Riley commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Robin Bowes This was marked Needs Info almost a year ago. It looks like Michael Stahnke marked it as that when he last commented, and I think it should go back to accepted at this point (I'm doing that as I post this). As for the version bump: We're likely not going to tackle this ourselves any time soon, for the same reasons as last year (there are workarounds, it's a Yum issue, etc.) If we end up accepting Matthew Cover's PR (which I'm glancing at now), the actual fix version could end up being 4.10 or 5.0.

Add Comment

[Branan Riley (JIRA)]

Branan Riley updated an issue

Puppet / PUP-4039

package resource doesn't find new packages in yum repo

Change By: Branan Riley Story Points: 3

Add Comment

[Branan Riley (JIRA)]

Branan Riley updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Branan Riley Team: Agent

Add Comment

[Matthew Cover (JIRA)]

Matthew Cover commented on PUP-4039

Re: package resource doesn't find new packages in yum repo I think I've figured out a better way to solve this than (#5699|https://github.com/puppetlabs/puppet/pull/5699). I'd still like to see (#5699|https://github.com/puppetlabs/puppet/pull/5699) get merged as it fixes this issue. Then we can work the proposal below as a separate effort. `yum install` has 3 alternate versions which can be seen in the (rhel7 documentation|https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/sec-Working_with_Packages.html). yum install-n name yum install-na name.architecture yum install-nevra name-epoch:version-release.architecture Unlike `yum install`, `yum install-n` will not install virtual packages ('virtual capabilities' in RPM parlance), but will install with the real package name. [root@localhost ~]# yum install-n inetd

Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile

* base: mirror.sigmanet.com * extras: mirrors.tummy.com * updates: centos.mirror.ndchost.com No package inetd available. Error: Nothing to do [root@localhost ~]# yum install-n xinetd

Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile

* base: mirror.sigmanet.com * extras: mirrors.tummy.com

* updates: centos.mirror.ndchost.com Resolving Dependencies --> Running transaction check

---> Package xinetd.x86_64 2:2.3.15-13.el7 will be installed

--> Finished Dependency Resolution   Dependencies Resolved

======================================================================================================================================================================================================================================================

Package Arch Version Repository Size

====================================================================================================================================================================================================================================================== Installing: xinetd x86_64 2:2.3.15-13.el7 base 128 k   Transaction Summary ====================================================================================================================================================================================================================================================== Install 1 Package   Total download size: 128 k Installed size: 261 k Is this ok [y/d/N]: n

Exiting on user command Your transaction was saved, rerun it with:

yum load-transaction /tmp/yum_save_tx.2017-03-13.22-41.5vklSW.yumtx In addition to this, `yum install-n` does update the metadata cache in the same manner as `yum install`. We should be able to refactor the code and do something like this. if @resource.allow_virtual? operation = :install else operation = :install-n end Doing so will remove the need for `yum list` which in turn removes the need for `yum makecache fast`. Now, it's actually slightly more complex in that we also have to support arch and version. package { 'glibc.i686': ensure => 'present', } package { 'glibc': ensure => '2.17-157.el7_3.1', } Based on my testing so far, it looks like both of these use cases can just keep using regular `yum install`. The virtual package/capability does not work with arch in the multilib packages I've tested. [root@localhost ~]# repoquery --provides glibc | grep ldconfig ldconfig ldconfig [root@localhost ~]# yum install glibc.i686 Loaded plugins: fastestmirror test | 2.9 kB 00:00:00

Loading mirror speeds from cached hostfile

* base: mirror.sigmanet.com * extras: mirrors.tummy.com

* updates: centos.mirror.ndchost.com Resolving Dependencies --> Running transaction check

---> Package glibc.i686 0:2.17-157.el7_3.1 will be installed --> Processing Dependency: libfreebl3.so(NSSRAWHASH_3.12.3) for package: glibc-2.17-157.el7_3.1.i686 --> Processing Dependency: libfreebl3.so for package: glibc-2.17-157.el7_3.1.i686 --> Running transaction check ---> Package nss-softokn-freebl.i686 0:3.16.2.3-14.4.el7 will be installed

--> Finished Dependency Resolution   Dependencies Resolved

======================================================================================================================================================================================================================================================

Package Arch Version Repository Size

====================================================================================================================================================================================================================================================== Installing: glibc i686 2.17-157.el7_3.1 updates 4.2 M Installing for dependencies: nss-softokn-freebl i686 3.16.2.3-14.4.el7 base 188 k   Transaction Summary ====================================================================================================================================================================================================================================================== Install 1 Package (+1 Dependent package)   Total download size: 4.4 M Installed size: 15 M Is this ok [y/d/N]: n

Exiting on user command Your transaction was saved, rerun it with:

yum load-transaction /tmp/yum_save_tx.2017-03-13.22-47.r7co40.yumtx [root@localhost ~]# yum install ldconfig.i686

Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile

* base: mirror.sigmanet.com * extras: mirrors.tummy.com * updates: centos.mirror.ndchost.com No package ldconfig.i686 available. Error: Nothing to do The virtual package/capability also does not work with version in the multilib packages I've tested. [root@localhost ~]# yum install glibc-2.17-157.el7_3.1.x86_64

Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile

* base: mirror.sigmanet.com * extras: mirrors.tummy.com * updates: centos.mirror.ndchost.com Package glibc-2.17-157.el7_3.1.x86_64 already installed and latest version Nothing to do [root@localhost ~]# yum install ldconfig-2.17-157.el7_3.1.x86_64

Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile

* base: mirror.sigmanet.com * extras: mirrors.tummy.com * updates: centos.mirror.ndchost.com No package ldconfig-2.17-157.el7_3.1.x86_64 available. Error: Nothing to do There is already some `operation` based logic in yum.rb (to see if we need install, downgrade, install with version). It shouldn't be too difficult to add the new `allow_virtual` logic into the existing logic. We'll also have to look into when `install-n` was added to yum to make sure it will be available on most systems which use the yum package provider.

Add Comment

[Robin Bowes (JIRA)]

Robin Bowes commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

It seems to be in this patch: http://yum.baseurl.org/gitweb?p=yum.git;a=commit;h=d56dcaa8ba266873b32832e713fc8199c9a9f45f A quick check on CentOS 6 vs. CentOS 7 reveals it's not in EL6 but is in EL7.

Add Comment

[Matthew Cover (JIRA)]

Matthew Cover commented on PUP-4039

Re: package resource doesn't find new packages in yum repo

Thanks for checking on that. Yeah, I don't see it on CentOS 6.8... so that's not going to work in every case. #5699 is a better general fix. It may be worthwhile to check for the presence of install-n (one way would be via yum help install) and create a fast path when it's available. I'm not entirely sure the gain is worth the effort though.

Add Comment

[Scott Garman (JIRA)]

Scott Garman updated an issue

Puppet / PUP-4039

package resource doesn't find new packages in yum repo

Change By: Scott Garman Labels: triaged

Add Comment

[Moses Mendoza (JIRA)]

Moses Mendoza updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Moses Mendoza Labels: triaged

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Josh Cooper Fix Version/s: PUP 6.0.0 Fix Version/s: PUP 5.y

Add Comment

[Scott Garman (JIRA)]

Scott Garman assigned an issue to Geoff Nichols

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Scott Garman Assignee: Robin Bowes Geoff Nichols

Add Comment

[Scott Garman (JIRA)]

Scott Garman updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Scott Garman Sprint: Client 2016-05-04 , Agent Grooming

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Geoff Nichols Sprint: Client 2016-05-04,  Agent  Platform OS  Grooming

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Geoff Nichols Team: Agent Platform OS

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols assigned an issue to Unassigned

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Geoff Nichols Assignee: Geoff Nichols

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Geoff Nichols Sprint: Client 2016-05-04, Platform OS  Grooming  Community

Add Comment

[Melissa Stone (JIRA)]

Melissa Stone updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Melissa Stone Labels: community

Add Comment

This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db)

[Branan Riley (JIRA)]

Branan Riley updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Branan Riley Labels: community linux package redhat triaged type_and_provider yum

Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Branan Riley (JIRA)]

Branan Riley updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Branan Riley Sprint: Client 2016-05-04, Platform OS Community PR Kanban

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-4039 package resource doesn't find new packages in yum repo

Change By: Geoff Nichols Sprint: Client 2016-05-04 , Platform OS Kanban

Add Comment

[jagadish honnappa (JIRA)]

jagadish honnappa commented on PUP-4039

Re: package resource doesn't find new packages in yum repo Hello, As we know this problem is due to "yum list" will look only local cache. Any method we apply, it should refresh the metadata then only problem will get resolved .. right?!! i have applied below code and it works fine.  package { 'new-pkg-name': ensure => 'present', require => Exec['yum-clean-expire-cache'], } exec { 'yum-clean-expire-cache': path => '/usr/bin:/usr/sbin:/bin', command => '/usr/bin/yum clean expire-cache', unless => [ 'yum -d 0 -e 0 -y list new-pkg-name' ], }   Regards Jagadish

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-4039

package resource doesn't find new packages in yum repo

Change By: Geoff Nichols

Labels: community linux package redhat triaged type_and_provider yum

Add Comment

[Nate McCurdy (Jira)]

Nate McCurdy commented on PUP-4039

Re: package resource doesn't find new packages in yum repo I've found another problem caused by the patch made for PUP-2182, and it has to do with installing RPM's using the package resource and yum provider in cases where the RPM is specified in the source parameter. For example: # This causes 'yum list pdk' to be run before 'yum install pdk'. # And in the case where we don't have the 'puppet6' repo installed, 'yum list' fails\ # as it can't find the 'pdk' package. package { 'pdk': ensure => 'installed', source => 'http://yum.puppet.com/puppet6/el/8/x86_64/pdk-1.18.1.0-1.el8.x86_64.rpm', provider => 'yum', allow_virtual => false, } # This works because 'yum list' is not called prior to 'yum install'. package { 'pdk': ensure => 'installed', source => 'http://yum.puppet.com/puppet6/el/8/x86_64/pdk-1.18.1.0-1.el8.x86_64.rpm', provider => 'yum', allow_virtual => true, } The problem is this check for allow_virtual: https://github.com/puppetlabs/puppet/blob/7.1.0/lib/puppet/provider/package/yum.rb#L237-L240 In my example above the pdk package is not a virtual package, so one would think that the allow_virtual attribute should not come into play when trying to install the package with yum. But due to the patch in PUP-2182, it does come into play. I know this is a bit of an edge case, but installing raw RPM's with yum install does work and is fully supported by Yum. Of course it's better to setup a repo and pull from the repo, but sometimes that's not possible and you'll be left needing to run yum install https://some.artifact.store/rpms/foo.rpm. There's no reason for Puppet to prevent this valid use case from working if it doesn't have to. allow_virtual does default to true, but it's been common practice for a long time for folks to set a resource default for all packages in site.pp flipping it to false. Package { allow_virtual => false, } Whether or not that resource default is still valid nowadays is not the point though, it's that when installing a non-virtual package with yum install http://some.remote/package.rpm, I would expect the yum provider to work without having to worry about the un-related allow_virtual attribute.

Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)