Proposal new meta attribute: meta.permitAutoUpgrade

[Freddy Rietdijk]

Hi,

For the Python package set I would like to introduce a new meta attribute to indicate to update scripts whether the package may be upgraded to a newer version. The motivation is that I want to exclude some packages from my update script. While I am only interest in Python packages, I think it is good to consider the whole of Nixpkgs before introducing attributes like these.

I propose an attribute `meta.permitAutoUpgade` which takes either a boolean, or one of "major", "minor", "patch". When set to true, the package may be upgraded to any version, and when set to false tools should not update the expression. When "major" is set major version upgrades may be performed; this is effectively the same as passing true. With "minor" only minor upgrades and "patch" only patch-level updates. 

What do you think of such new attribute?

Kind regards,

Freddy

[Volth]

What is expected number of Python packages which do not allow auto upgrade?

(For Perl there is only about 10 such packages so I ended up with
keeping their list inside the update script)

> --
> You received this message because you are subscribed to the Google Groups
> "nix-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nix-devel+...@googlegroups.com.
> To post to this group, send email to nix-...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/nix-devel/CAOQtOH2mpypwJDtGOCzFOhTujhsJ8yfrZ65MKGz8skJ96g8uLA%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.
>

[Volth]

What is expected number of Python packages which do not allow auto upgrade?

(For Perl there is only about 10 such packages so I ended up with
keeping their list inside the update script)

On 6/22/18, Freddy Rietdijk <freddyr...@fridh.nl> wrote:

[Freddy Rietdijk]

I think about 10 to 20. 

Keeping it in the script is a pragmatic solution but at the risk of having the data spread out and getting out of sync. What I also would like to have is a way that package maintainers can say "Please don't do any major upgrades of this package".

> email to nix-devel+unsubscribe@googlegroups.com.

[Orivej Desh]

Do you have a use for "minor" and "patch" attributes?

Is there a need to distinguish between "upgrade = false" and
"upgrade.major = false", or is it enough to keep only one kind of
restriction (the second)?

In the latter case I'd prefer the setting to be a boolean
"keepMajorVersion".

* Freddy Rietdijk <freddyr...@fridh.nl> [2018-06-22]

[Freddy Rietdijk]

Typically we should always permit "patch" level updates, simply from a security point of view. "major" upgrades typically break things whereas "minor" do so only occasionally.

There are some packages for which major updates are not allowed but minor are. One example is `ipython`, for which the Python 2 version will remain at 5.x.y.

[Matthew Pickering]

I think this information is better suited to live in the tooling.

For the Haskell package set there is a .yaml file which lives in
nixpkgs which lists some meta information. There is also some meta
information baked into `cabal2nix`.

As we have already discovered in this thread, there are many
meta-attributes which tools can require. Adding one special one won't
go very far in the end.

Matt

> --
> You received this message because you are subscribed to the Google Groups
> "nix-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to nix-devel+...@googlegroups.com.

> To post to this group, send email to nix-...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/nix-devel/CAOQtOH3OeAW92P7AEr7j9BvA2Ei7fvtgVACGZLq-qPMiBnLTQw%40mail.gmail.com.

[Leo Gaspard]

On 06/22/2018 01:14 PM, Freddy Rietdijk wrote:
> Typically we should always permit "patch" level updates, simply from a
> security point of view. "major" upgrades typically break things whereas
> "minor" do so only occasionally.

Except for OpenSMTPD, for which I thought a patch-level update was OK,
but it wasn't (a `sed` performed by the derivation was no longer
changing anything, and the failure was thus silent during build and even
execution, but a configuration parameter was ill-set)

Or for Dovecot, for which the 2.3.0.1 -> 2.3.1 upgrade included a
rewrite of the message parser, that ended up rejecting all incoming mail
from OpenSMTPD (which broke my installation until I noticed that mail
was no longer being delivered)

In an ideal world, all patch-level upgrades would be fully automatable,
but I fear it wouldn't actually work out well. And if such a meta
attribute was added, it would have to have a meaning for *all* packages.

Also, with the dovecot (2.3.0.1) or the OpenSMTPD (6.0.3p1) numbering
schemes, I'm not sure “major”, “minor” and “patch” could fit them well.
And if this was added to nixpkgs (and not to the language-specific
tooling), it would have to handle all possible version numbers IMO.
Maybe have it be a function like this?

meta.permitAutoUpgrade = versionFrom: versionTo: boolIsOk

This would have the bonus of being fully-configurable, and helper
functions could be provided in lib, so that it could be set at
meta.permitAutoUpgrade = lib.permitMajorUpgrades if the package follows
the x.y.z versioning scheme.

Anyway, I must say I believe that at some point there must be human
intervention to check the changelog looks good, etc. for nixpkgs. And
I'm more and more thinking that all these fully-automated bumps should
be moved to language-specific overlays, that could even consist in a
full dump of the language package manager, where reasonably possible.

But on the other hand, I guess that even including only the strict
minimum of packages required for the applications in nixpkgs, there are
still too many packages for reasonable human-done maintenance? If so…
meh. We're doomed anyway, and things will break due to automatic
upgrades, as upstreams aren't always following their own guidelines, so…