Re: [Puppet-dev] Apt package provider and --force-yes

[Andy Parker]

On Tue, Oct 16, 2012 at 1:24 PM, John Downey
<john....@getbraintree.com> wrote:
> Puppeteers,
>
> I've been using the Debian apt provider for package to help ensure a
> specific version of a package is installed such as:
>
> package { "rsyslog": ensure => "5.8.11-1.1" }
>
> However this appears[1] to add --force-yes to the apt command line in
> addition to specifying the exact version requirement. The --force-yes option
> is considered a dangerous option per the debian manual:
>

The --force-yes was added as part of being able to downgrade packages
(https://projects.puppetlabs.com/issues/1999) when the version number
specified forces that.

This may be overkill for what it was trying to achieve. Is there a
safer way of doing this?

> --force-yes
> Force yes; This is a dangerous option that will cause apt to
> continue without prompting if it is doing something potentially harmful. It
> should not be used except in very special situations. Using force-yes can
> potentially destroy your system! Configuration Item: APT::Get::force-yes.
>
> Is there a reason puppet shouldn't use another option to explicitly add
> --force-yes? Since this option essentially tells apt to ignore everything it
> knows, I've run into cases where this can cause package dependencies to
> become broken without it being obvious. The force flag is not necessary for
> explicitly setting a version number, so it seems like a dangerous option is
> not needed for this use case. Ideally there would be a force option to
> Package that could set this flag to disregard dependencies.
>
> [1]
> https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/apt.rb#L57-65
>
> -John
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-dev/-/mN-SAGSjpnwJ.
> To post to this group, send email to puppe...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-dev+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-dev?hl=en.

[John Downey]

I think that may be a different problem. Downgrading packages is not supported in apt[1]: "Downgrading is not officially supported by the Debian by design." The recommended path is to remove the old package and install the downgraded package in its place. What the force option does instead is tell apt to ignore all dependency checks and proceed with the install without prompting. Since this option is so dangerous it seems like it should not be used by any form of automation system.

As it exists, if the --force-yes option is removed it would warn you that the package that is installed is not what you specified. I feel this is the behavior I would expect from the ensure => "version string" option.

[1] http://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_emergency_downgrading

[Andy Parker]

On Fri, Oct 19, 2012 at 11:43 AM, John Downey
<john....@getbraintree.com> wrote:
> I think that may be a different problem. Downgrading packages is not
> supported in apt[1]: "Downgrading is not officially supported by the Debian
> by design." The recommended path is to remove the old package and install
> the downgraded package in its place. What the force option does instead is
> tell apt to ignore all dependency checks and proceed with the install
> without prompting. Since this option is so dangerous it seems like it should
> not be used by any form of automation system.
>
> As it exists, if the --force-yes option is removed it would warn you that
> the package that is installed is not what you specified. I feel this is the
> behavior I would expect from the ensure => "version string" option.

So the apt provider should uninstall and then reinstall when the
ensured version is less than the installed version? That sounds
reasonable on the surface, but seems like it will have a lot of
unintended consequences. For instance, what should be done with
dependencies?

Has puppet using --force-yes caused specific problems that could be addressed?

> https://groups.google.com/d/msg/puppet-dev/-/OGu7n3QdhgAJ.

[Jesse Hathaway]

Andy Parker <andy <at> puppetlabs.com> writes:
> So the apt provider should uninstall and then reinstall when the
> ensured version is less than the installed version? That sounds
> reasonable on the surface, but seems like it will have a lot of
> unintended consequences. For instance, what should be done with
> dependencies?

Downgrading should result in an error, and not be automatically completed as it
is not supported. Uninstalling and reinstalling a package will not always work
if the configuration format has changed or the data format has changed.
However, the lack of support for downgrading does not negate the value of
specifying the package version. Specifying a package version ensures that your
currently supported version is consistently installed across all nodes.

> Has puppet using --force-yes caused specific problems that could be addressed?

Here is an example of where --force-yes causes problems. This demonstrates two
ways using force-yes can break your system. The first being installing a
package and breaking dependencies. The second being downgrading a package where
the packages configuration or data format is not backward compatible.

I think it is suprising to a user of puppet that merely specifying the version
of a package results in this flag being set, given the dire warning in the
manpage, "Using force-yes can potentially destroy your system!". This flag
should be a separate puppet option on the package type.

1. Install bacula 5.2.6 on Debian Squeeze

$ sudo apt-get install -t squeeze-backports bacula-server

2. Try to downgrade to 5.0.2, apt-get correctly refuses

$ sudo apt-get -y install bacula-common=5.0.2-2.2+b1
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
python-minimal bacula-common-sqlite3 sqlite mt-st dbconfig-common
libsqlite0 python bacula-common mtx sqlite3
Use 'apt-get autoremove' to remove them.
Suggested packages:
bacula-doc
The following packages will be REMOVED:
bacula bacula-client bacula-console bacula-director-common
bacula-director-sqlite3 bacula-fd bacula-sd bacula-sd-sqlite3 bacula-server
bacula-traymonitor
The following packages will be DOWNGRADED:
bacula-common
0 upgraded, 0 newly installed, 1 downgraded, 10 to remove and 2 not upgraded.
Need to get 635 kB of archives.
After this operation, 3,817 kB disk space will be freed.
E: There are problems and -y was used without --force-yes

3. Ignore the warning and downgrade anyway

$ sudo apt-get -y --force-yes install bacula-common=5.0.2-2.2+b1
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
python-minimal bacula-common-sqlite3 sqlite mt-st dbconfig-common
libsqlite0 python bacula-common mtx sqlite3
Use 'apt-get autoremove' to remove them.
Suggested packages:
bacula-doc
The following packages will be REMOVED:
bacula bacula-client bacula-console bacula-director-common
bacula-director-sqlite3 bacula-fd bacula-sd bacula-sd-sqlite3 bacula-server
bacula-traymonitor
The following packages will be DOWNGRADED:
bacula-common
0 upgraded, 0 newly installed, 1 downgraded, 10 to remove and 2 not upgraded.
Need to get 635 kB of archives.
After this operation, 3,817 kB disk space will be freed.
Get:1 http://debian.mirrors.tds.net/debian/ squeeze/main bacula-common amd64
5.0.2-2.2+b1 [635 kB]
Fetched 635 kB in 0s (2,010 kB/s)
(Reading database ... 22487 files and directories currently installed.)
Removing bacula ...
Removing bacula-client ...
Removing bacula-console ...
Removing bacula-server ...
Removing bacula-director-sqlite3 ...
Stopping Bacula Director...:.
Removing bacula-director-common ...
Removing bacula-fd ...
Stopping Bacula File daemon...:.
Removing bacula-sd-sqlite3 ...
Removing bacula-sd ...
Stopping Bacula Storage daemon... :.
Removing bacula-traymonitor ...
Processing triggers for man-db ...
dpkg: warning: downgrading bacula-common from 5.2.6+dfsg-1~bpo60+1 to
5.0.2-2.2+b1.
(Reading database ... 22387 files and directories currently installed.)
Preparing to replace bacula-common 5.2.6+dfsg-1~bpo60+1 (using
.../bacula-common_5.0.2-2.2+b1_amd64.deb) ...
Unpacking replacement bacula-common ...
dpkg: warning: unable to delete old directory '/etc/tmpfiles.d': Directory
not empty
Processing triggers for man-db ...
Setting up bacula-common (5.0.2-2.2+b1) ...
Installing new version of config file /etc/bacula/scripts/btraceback.gdb ...

- Package list

$ dpkg -l|grep bacula
ii bacula-common 5.0.2-2.2+b1
network backup, recovery and verification - common support files
ii bacula-common-sqlite3 5.2.6+dfsg-1~bpo60+1
network backup, recovery and verification - SQLite v3 common files
rc bacula-console 5.2.6+dfsg-1~bpo60+1
network backup, recovery and verification - text console
rc bacula-director-common 5.2.6+dfsg-1~bpo60+1
network backup, recovery and verification - Director common files
rc bacula-director-sqlite3 5.2.6+dfsg-1~bpo60+1
network backup, recovery and verification - SQLite 3 storage for Director
rc bacula-fd 5.2.6+dfsg-1~bpo60+1
network backup, recovery and verification - file daemon
rc bacula-sd 5.2.6+dfsg-1~bpo60+1
network backup, recovery and verification - storage daemon
rc bacula-traymonitor 5.2.6+dfsg-1~bpo60+1
network backup, recovery and verification - tray monitor

4. Result package is broken. If you then install missing dependencies, the
package will still be broken as the bacula database format has changed
between 5.0.2 & 5.2.6

Thanks, Jesse

[Andy Parker]

On Fri, Oct 26, 2012 at 1:17 PM, Jesse Hathaway
<jesse.h...@getbraintree.com> wrote:
> Andy Parker <andy <at> puppetlabs.com> writes:
>> So the apt provider should uninstall and then reinstall when the
>> ensured version is less than the installed version? That sounds
>> reasonable on the surface, but seems like it will have a lot of
>> unintended consequences. For instance, what should be done with
>> dependencies?
>
> Downgrading should result in an error, and not be automatically completed as it
> is not supported. Uninstalling and reinstalling a package will not always work
> if the configuration format has changed or the data format has changed.
> However, the lack of support for downgrading does not negate the value of
> specifying the package version. Specifying a package version ensures that your
> currently supported version is consistently installed across all nodes.
>

This would be a complete regression of issue #1999, which would be a
majorly breaking change to the apt provider and so we can't do.

We can maybe do the original suggestion that there is a flag with
disables this, but it would have to be off by default.

> --
> You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

[Jesse Hathaway]

Andy Parker <andy <at> puppetlabs.com> writes:

> This would be a complete regression of issue #1999, which would be a
> majorly breaking change to the apt provider and so we can't do.
>
> We can maybe do the original suggestion that there is a flag with
> disables this, but it would have to be off by default.

The problem with keeping this option as the default is two fold.

1. Debian does not officially support downgrading. Sometimes it works fine,
other times the downgrade leaves the service completely unusable and has the
potential to corrupt data.

2. The force option also breaks package dependencies, which can leave the system
unusable.

I understand the need for backward compatibility, however having a force flag
set by default should never have been added. Also, if downgrading is a critical
path in your puppet configuration then you have larger problems.

Thanks,

Jesse Hathaway