[PATCH 1/1] wic: disable generation of python bytecode cache

[Felix Moessbauer]

Wic is executed as root, so the python bytecode cache is also
created as root. This is problematic as the cache is created inside the
source tree in a folder__pycache__ next to the python script itself. By
that, we end up with files belonging to root inside the repo source
tree, which makes it impossible to delete the source tree as a regular
user.

This problem became visible with the kas purge plugin that removes the
fetched layers as a regular user (these layers are fetched and managed
by kas). Also the read-only mounting of repos in kas does not help here,
as the fetched repos are not mounted ro for obvious reasons.

Anyways, we should not create files inside the source tree that do not
belong to the calling user. To fix this, we just disable the python
cache for the wic task. This is the only task that executes a python
script from the source tree as root.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
meta/classes/imagetypes_wic.bbclass | 1 +
1 file changed, 1 insertion(+)

diff --git a/meta/classes/imagetypes_wic.bbclass b/meta/classes/imagetypes_wic.bbclass
index 7a050e73..fb0b81a9 100644
--- a/meta/classes/imagetypes_wic.bbclass
+++ b/meta/classes/imagetypes_wic.bbclass
@@ -157,6 +157,7 @@ generate_wic_image() {
export FAKEROOTCMD=${FAKEROOTCMD}
export BUILDDIR=${TOPDIR}
export MTOOLS_SKIP_CHECK=1
+ export PYTHONDONTWRITEBYTECODE=1
mkdir -p ${IMAGE_ROOTFS}/../pseudo
touch ${IMAGE_ROOTFS}/../pseudo/files.db

--
2.49.0

[Heinisch, Alexander]

On Thu, 2025-06-05 at 16:58 +0200, 'Felix Moessbauer' via isar-users
wrote:

> Wic is executed as root, so the python bytecode cache is also
> created as root. This is problematic as the cache is created inside
> the
> source tree in a folder__pycache__ next to the python script itself.
> By
> that, we end up with files belonging to root inside the repo source
> tree, which makes it impossible to delete the source tree as a
> regular
> user.
>
> This problem became visible with the kas purge plugin that removes
> the
> fetched layers as a regular user (these layers are fetched and
> managed
> by kas). Also the read-only mounting of repos in kas does not help
> here,
> as the fetched repos are not mounted ro for obvious reasons.
>
> Anyways, we should not create files inside the source tree that do
> not
> belong to the calling user. To fix this, we just disable the python
> cache for the wic task. This is the only task that executes a python
> script from the source tree as root.

Hi Felix,

What about moving the cache to the build dir?

I just did a quick test using `PYTHONPYCACHEPREFIX` (see [1]) which was
added in Python 3.8 (so works for hosts >= bullseye)

```
diff --git a/meta/classes/imagetypes_wic.bbclass
b/meta/classes/imagetypes_wic.bbclass
index 38b5f0e1..7b8dc38c 100644

--- a/meta/classes/imagetypes_wic.bbclass
+++ b/meta/classes/imagetypes_wic.bbclass
@@ -157,6 +157,7 @@ generate_wic_image() {
export FAKEROOTCMD=${FAKEROOTCMD}
export BUILDDIR=${TOPDIR}
export MTOOLS_SKIP_CHECK=1

+ export PYTHONPYCACHEPREFIX="${TOPDIR}/__pycache__"

mkdir -p ${IMAGE_ROOTFS}/../pseudo

touch ${IMAGE_ROOTFS}/../pseudo/files.db```

and it seems to do the job. In depth testing still required!

[1]:
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPYCACHEPREFIX

BR Alexander

[Jan Kiszka]

What's yocto doing about this BTW? Are we just missing some of their
bits, or are they affected by a similar issue (except for creating the
cache as root)?

Jan

--
Siemens AG, Foundational Technologies
Linux Expert Center

[MOESSBAUER, Felix]

This script is not called frequently, so there is no real benefit in
caching it. If we want to cache, we should probably cache in the
WORKDIR.

> >      mkdir -p ${IMAGE_ROOTFS}/../pseudo
> >      touch ${IMAGE_ROOTFS}/../pseudo/files.db```
> >
> > and it seems to do the job. In depth testing still required!
> >
> > [1]:
> > https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPYCACHEPREFIX
> >
>
> What's yocto doing about this BTW? Are we just missing some of their
> bits, or are they affected by a similar issue (except for creating
> the
> cache as root)?

Yocto also disables the python cache for their fakeroot environments:

https://github.com/openembedded/openembedded-core/blob/bd8fc4c59a137a37bd7a54f398949617982d447e/meta/conf/bitbake.conf#L753

But as isar differs here, we either need to add this statement where I
added it (take the patch as-is), or implement this logic in
`wic_fakeroot`.

Felix

>
> Jan

--
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

[Jan Kiszka]

We have more python code than wic in isar. You should then make sure to
disable caching in general, not just for wic.

[Heinisch, Alexander]

Probably that's already the root cause :-)

The chosen directory was just for a quick PoC.

> >
> > > > mkdir -p ${IMAGE_ROOTFS}/../pseudo
> > > > touch ${IMAGE_ROOTFS}/../pseudo/files.db```
> > > >
> > > > and it seems to do the job. In depth testing still required!
> > > >
> > > > [1]:
> > > > https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPYCACHEPREFIX
> > > >
> > >
> > > What's yocto doing about this BTW? Are we just missing some of
> > > their
> > > bits, or are they affected by a similar issue (except for
> > > creating
> > > the
> > > cache as root)?
> >
> > Yocto also disables the python cache for their fakeroot
> > environments:
> >
> > https://github.com/openembedded/openembedded-core/blob/bd8fc4c59a137a37bd7a54f398949617982d447e/meta/conf/bitbake.conf#L753
> >
> > But as isar differs here, we either need to add this statement
> > where I
> > added it (take the patch as-is), or implement this logic in
> > `wic_fakeroot`.
>
> We have more python code than wic in isar. You should then make sure
> to
> disable caching in general, not just for wic.

Maybe we should investigate in not adding/modifying files in the source
tree at all? While most buildsystems provide some kind of PREFIX, we
could put all generated artifacts someplace below TOPDIR.
(Same applies to yocto)

>
> Jan
>
BR Alexander

[MOESSBAUER, Felix]

On Fri, 2025-06-06 at 07:42 +0000, Heinisch, Alexander (FT RPD CED SES-

We have, but no more python code that is called as root.
We know that, as we exactly know which files in the source tree cannot
be deleted after a build.

>
> Maybe we should investigate in not adding/modifying files in the
> source
> tree at all? While most buildsystems provide some kind of PREFIX, we
> could put all generated artifacts someplace below TOPDIR.
> (Same applies to yocto)

Yes, but this should be aligned with Yocto as they also only fix the
issue for root environments (or fakeroot in their case). That's a far
bigger topic than what this patch fixes.

Felix

>
> >
> > Jan
> >
> BR Alexander

[Heinisch, Alexander]

On Fri, 2025-06-06 at 07:54 +0000, Moessbauer, Felix (FT RPD CED OES-

Yes and yes!

Most probably nothing to fix in a single patch, rather a long term
strategy. (Iff we decide to do so (open for discussion) and if we could
align with yocto)

>
> Felix
>
> >
> > >
> > > Jan
> > >
> > BR Alexander
>

[Baurzhan Ismagulov]

On 2025-06-05 16:58, 'Felix Moessbauer' via isar-users wrote:
> Wic is executed as root, so the python bytecode cache is also
> created as root. This is problematic as the cache is created inside the
> source tree in a folder__pycache__ next to the python script itself. By
> that, we end up with files belonging to root inside the repo source
> tree, which makes it impossible to delete the source tree as a regular
> user.
>
> This problem became visible with the kas purge plugin that removes the
> fetched layers as a regular user (these layers are fetched and managed
> by kas). Also the read-only mounting of repos in kas does not help here,
> as the fetched repos are not mounted ro for obvious reasons.
>
> Anyways, we should not create files inside the source tree that do not
> belong to the calling user. To fix this, we just disable the python
> cache for the wic task. This is the only task that executes a python
> script from the source tree as root.

If there is consensus on this, I'd like to merge this in the next days because
it provides a specific improvement.

Any broader / more optimal changes can be merged on top.

With kind regards,
Baurzhan

[Baurzhan Ismagulov]

On 2025-06-05 16:58, 'Felix Moessbauer' via isar-users wrote:

> Wic is executed as root, so the python bytecode cache is also
> created as root. This is problematic as the cache is created inside the
> source tree in a folder__pycache__ next to the python script itself. By
> that, we end up with files belonging to root inside the repo source
> tree, which makes it impossible to delete the source tree as a regular
> user.
>
> This problem became visible with the kas purge plugin that removes the
> fetched layers as a regular user (these layers are fetched and managed
> by kas). Also the read-only mounting of repos in kas does not help here,
> as the fetched repos are not mounted ro for obvious reasons.
>
> Anyways, we should not create files inside the source tree that do not
> belong to the calling user. To fix this, we just disable the python
> cache for the wic task. This is the only task that executes a python
> script from the source tree as root.

Applied to next, thanks.

With kind regards,
Baurzhan