[gentoo-user] python mess - random winge!

[William Kenworthy]

I synced portage a couple of days now and now my systems are rebuilding
python modules for 3.10 without any input from me (prior to this 3.10
was on the system but wasn't picked up by applications.)  This is
breaking non portage apps like homeassistant which are still not fully
3.10 safe - ok that's sort of expected and in this case will be fixed,
but I cant find anything definitive on the task of "I want to control
which python is used" and when to update.

I eventually found that changing the order in python-exec.conf helped on
the homeassistant system.  There is a LOT of out of date documentation
out there, particularly with eselect being used but is actually not used
with python anymore (why? - from a user point of view having consistent
access to configuration is a no brainer!) - so how can one get python to
behave reliably and override its automatic get things wrong installation
system?  Is manually editing python-exec.conf the way (which seems to
get overwritten - shouldn't that be a protected config file then?)

BillK

[Nikos Chantziaras]

On 05/07/2022 08:04, William Kenworthy wrote:
> I synced portage a couple of days now and now my systems are rebuilding

> python modules for 3.10 without any input from me [...]
Yes, that's normal and there was a news item about it. Do:

eselect news list

to get the and then use the NUMBER of the "Python 3.10 to become the
default on 2022-07-01" news item:

eselect read NUMBER

This will give you details on how to do the upgrade properly.

[cal]

On 7/4/22 22:04, William Kenworthy wrote:
> I synced portage a couple of days now and now my systems are rebuilding
> python modules for 3.10 without any input from me (prior to this 3.10
> was on the system but wasn't picked up by applications.)  This is
> breaking non portage apps like homeassistant which are still not fully
> 3.10 safe - ok that's sort of expected and in this case will be fixed,
> but I cant find anything definitive on the task of "I want to control
> which python is used" and when to update.

As Nikos mentioned in another reply, you should pay attention to the
eselect news items as python upgrades are usually announced weeks in
advance.

>
> I eventually found that changing the order in python-exec.conf helped on
> the homeassistant system.

> There is a LOT of out of date documentation
> out there, particularly with eselect being used but is actually not used
> with python anymore (why? - from a user point of view having consistent
> access to configuration is a no brainer!)

I don't see any deprecation notice on
https://packages.gentoo.org/packages/app-eselect/eselect-python, but
someone could correct me.

> - so how can one get python to
> behave reliably and override its automatic get things wrong installation
> system?  Is manually editing python-exec.conf the way (which seems to
> get overwritten - shouldn't that be a protected config file then?)

The python package supports installing multiple versions to different
slots. The easiest way I've found to run python software that depends
on a specific version, without mucking around with global python
settings, is to install the relevant version (e.g. emerge python:3.8)
and then use venv to run that software in a virtual environment
(python3.8 -m venv .env; .env/bin/python something.py).

>
> BillK

[wk...@op.pl]

Dnia 2022-07-05, o godz. 13:04:07
William Kenworthy <bi...@iinet.net.au> napisał(a):

Hello!

In "eselect news" info about python update there is a paragraph about
blocking the upgrade. It just means adding:

*/* PYTHON_TARGETS: -* python3_9
*/* PYTHON_SINGLE_TARGET: -* python3_9

to /etc/portage/make.conf or /etc/portage/package.use or
/etc/portage/package.use/zz-somename - whichever suites you best.

You can also change these settings just for some packages, by adding:

cat/pkg PYTHON_TARGETS: -* python3_9 PYTHON_SINGLE_TARGET: -* python3_9

to one of aforementioned files.

Hope that helps!

--
xWK

[William Kenworthy]

I did read the news item and set the systems as above with multiple python targets - there is no mention of python-exec and its role in which python version is in use for packages that just call "python".  Perhaps I should have been clearer - what I see is with multiple python targets present the python ebuild automatically selects the latest version that is stable via python-exec - ok, some would want that.  But what it should do is respect the users choice of running version and not automaticly overide it without asking.  It looks like python-exec is the controlling factor so I'll try CONFIG_PROTECT on that file and manually manage it via ansible.

BillK

[Neil Bothwick]

On Tue, 5 Jul 2022 15:31:38 +0800, William Kenworthy wrote:

> I did read the news item and set the systems as above with multiple
> python targets - there is no mention of python-exec and its role in
> which python version is in use for packages that just call "python". 
> Perhaps I should have been clearer - what I see is with multiple python
> targets present the python ebuild automatically selects the latest
> version that is stable via python-exec - ok, some would want that.  But
> what it should do is respect the users choice of running version and not
> automaticly overide it without asking.  It looks like python-exec is the

> controlling factor so I'll try CONFIG_PROTECTon that file and manually
> manage it via ansible.

It is already CONFIG_PROTECTed, I had to approve the changes in the usual
way before they went ahead. What I find slightly odd is that this file is
also managed by eselect-python, but that is not installed by default. I
would have expected it to be part of @system.


--
Neil Bothwick

Quick!! Act as if nothing has happened!

[Petr Vaněk]

This change is described in
https://www.gentoo.org/support/news-items/2021-01-30-python-preference-to-follow-python-targets.html

Petr

[Neil Bothwick]

On Tue, 5 Jul 2022 11:17:27 +0200, Petr Vaněk wrote:

> > It is already CONFIG_PROTECTed, I had to approve the changes in the
> > usual way before they went ahead. What I find slightly odd is that
> > this file is also managed by eselect-python, but that is not
> > installed by default. I would have expected it to be part of @system.
> >
>
> This change is described in
> https://www.gentoo.org/support/news-items/2021-01-30-python-preference-to-follow-python-targets.html

Worth reading, I missed that one, thanks.


--
Neil Bothwick

"Do you reply to our surveys.?"
[X]Never [ ]Always [ ]Sometimes

[Grant Edwards]

On 2022-07-05, William Kenworthy <bi...@iinet.net.au> wrote:

> I synced portage a couple of days now and now my systems are rebuilding

> python modules for 3.10 without any input from me [...]

Every time there's a Python upgrade like this, it turns into a bit of
an ordeal because I always have a small handful of packages that don't
support the newer version.

The news item offers no advice on what to do in this situation other
than completely postponing the upgrade of everything (which doesn't
seem like the best choice.)

It would be nice if the news item explained how to let the upgrade
procede while holding back a few packages.

Can you set 3_9 and 3_10 globally, and then disable 3_10 for a few
individual packages that can't be built with 3_10?

--
Grant

[Jack]

As far as I can tell, you just need to add python_targets_python3_9 for
the package in the appropriate package.use file.

[Grant Edwards]

I've tried that, and it takes forever. Everything time you do it, the
next emerge attempt will fail because one of that package's
dependencies doesn't have 3_9 set. So you set that one, do an emerge,
and it fails because there's another depenency that doesn't have 3_9
set. Repeat this for an hour or two...

If it would tell you about all of them at once, it wouldn't be so bad.
But, if you're trying to hold back a large application with dozens and
dozens of dependancies it makes you want to scream.

Or doesn't it torture you like that any longer?

--
Grant

[Rich Freeman]

That and its dependencies. Obviously you need to be caught up before
things get removed from the repo, but the offending package itself
will get removed when that happens anyway.

You can always just globally keep the older version around longer if
you don't want to deal with a bunch of cruft in
/etc/portage/package.use. The news item explains how to do this.

--
Rich

[Jack]

Oh, it still does. I just think I anticipate it more often, so I am
frequently able to deal with more than one "layer" in each round, so
the total process doesn't take so long. I've run into problems both
for dependencies of the package I changed first, and packages for which
It is a dependency.

On major updates, such as KDE, emerge @world often shows me a long list
of packages it will upgrade, followed by a long list of slot
conflicts. Over the years I've not necessarily gotten better at
directly interpreting portage's output, but at least figuring out I've
got one package somehow related to the ones to be upgraded (usually
somewhere in the dependency tree) which itself doesn't have an upgrade,
but does need to be rebuilt at the same time. I'll often end up doing
"eix-installed -a | grep kde" (or python) and look for a package that
isn't being upgraded, but is in the dependency tree of one or more
which is. It is also common that the problem is a package I've added
under /etc/portage, and I just need to figure out if it still needs to
be there, or else how that non-standard entry affects other packages.

[Wols Lists]

On 05/07/2022 17:44, Rich Freeman wrote:
> hat and its dependencies. Obviously you need to be caught up before
> things get removed from the repo, but the offending package itself
> will get removed when that happens anyway.
>
> You can always just globally keep the older version around longer if
> you don't want to deal with a bunch of cruft in
> /etc/portage/package.use. The news item explains how to do this.

I'd be inclined to lock the older version of the package you want so it
won't upgrade automatically (don't know how to do this, probably put
that particular version in @world), and also give that a python 9
dependency in package.use.

That way, when they update it to python 10, you can delete the version
lock on your package, and it will upgrade and get rid of all the python
9 stuff without you having to worry about it.

I always try and make sure anything I don't care about but that needs
specific stuff is always specified by version. That way, as the system
moves on, all that cruft just falls off the wagon ...

Cheers,
Wol

[Rich Freeman]

On Tue, Jul 5, 2022 at 1:10 PM Wols Lists <antl...@youngman.org.uk> wrote:
>
> On 05/07/2022 17:44, Rich Freeman wrote:
> > hat and its dependencies. Obviously you need to be caught up before
> > things get removed from the repo, but the offending package itself
> > will get removed when that happens anyway.
> >
> > You can always just globally keep the older version around longer if
> > you don't want to deal with a bunch of cruft in
> > /etc/portage/package.use. The news item explains how to do this.
>
> I'd be inclined to lock the older version of the package you want so it
> won't upgrade automatically (don't know how to do this, probably put
> that particular version in @world), and also give that a python 9
> dependency in package.use.

The news item explains how to get the upgrade earlier, delay the
upgrade entirely, or have support for both versions. I'd suggest
doing it the way the news item suggests, because when you start
fighting the devs who maintain the python update logic, you're
probably going to find yourself swearing a lot.

--
Rich