image-postproc-extension.bbclass modifying /etc/os-release
53 views
Skip to first unread message
Silvano Cirujano Cuesta
Jan 15, 2021, 9:26:18 AM1/15/21
to isar-...@googlegroups.com
Hi,
I've noticed that '/etc/os-release' is being changed on the image in meta/classes/image-postproc-extension.bbclass [1]. What BTW ends up changing '/usr/lib/os-release', since it's only a symlink. But both '/etc/os-release' and '/usr/lib/os-release' are owned by 'base-files'...
An upgrade of 'base-files' would be replacing (silently, since is not marked as a configuration file) '/usr/lib/os-release' with the version of the upstream 'base-files' package and possibly breaking any tools in the system relying on certain values in that file.
Is there a reason that I'm missing for doing so instead of the Debian-way (file diversion with dpkg-divert)? Or any hack that I've overseen that avoids the mentioned breakage?
Hasta la vista,
Silvano
[1] https://github.com/ilbers/isar/blob/a0d6de4ac7dfb926b38cfcd44f2bc8852f2e43e3/meta/classes/image-postproc-extension.bbclass#L20
I've noticed that '/etc/os-release' is being changed on the image in meta/classes/image-postproc-extension.bbclass [1]. What BTW ends up changing '/usr/lib/os-release', since it's only a symlink. But both '/etc/os-release' and '/usr/lib/os-release' are owned by 'base-files'...
An upgrade of 'base-files' would be replacing (silently, since is not marked as a configuration file) '/usr/lib/os-release' with the version of the upstream 'base-files' package and possibly breaking any tools in the system relying on certain values in that file.
Is there a reason that I'm missing for doing so instead of the Debian-way (file diversion with dpkg-divert)? Or any hack that I've overseen that avoids the mentioned breakage?
Hasta la vista,
Silvano
[1] https://github.com/ilbers/isar/blob/a0d6de4ac7dfb926b38cfcd44f2bc8852f2e43e3/meta/classes/image-postproc-extension.bbclass#L20
Silvano Cirujano Cuesta
Jan 18, 2021, 6:35:33 AM1/18/21
to isar-...@googlegroups.com, Claudius Heine
I might try to provide a fix, if we agree that the current implementation has an issue.
@Claudius: you wrote the original code [1]. Do you remember why you implemented it this way? Do you remember if you were aware of the issue I mentioned and you provided a mitigation for the issue that I see (assuming my analysis is right)?
[1] https://github.com/ilbers/isar/commit/13ce96e5bc84b60f2fa7ccfe93dde045461884e6
Silvano
@Claudius: you wrote the original code [1]. Do you remember why you implemented it this way? Do you remember if you were aware of the issue I mentioned and you provided a mitigation for the issue that I see (assuming my analysis is right)?
[1] https://github.com/ilbers/isar/commit/13ce96e5bc84b60f2fa7ccfe93dde045461884e6
Silvano
Claudius Heine
Jan 18, 2021, 7:35:56 AM1/18/21
to Silvano Cirujano Cuesta, isar-...@googlegroups.com
Hi Silvano,
On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
> I might try to provide a fix, if we agree that the current implementation has an issue.
>
> @Claudius: you wrote the original code [1]. Do you remember why you implemented it this way? Do you remember if you were aware of the issue I mentioned and you provided a mitigation for the issue that I see (assuming my analysis is right)?
>
> [1] https://github.com/ilbers/isar/commit/13ce96e5bc84b60f2fa7ccfe93dde045461884e6
>
> Silvano
>
> On 15/01/2021 15:26, [ext] Silvano Cirujano Cuesta wrote:
>> Hi,
>>
>> I've noticed that '/etc/os-release' is being changed on the image in meta/classes/image-postproc-extension.bbclass [1]. What BTW ends up changing '/usr/lib/os-release', since it's only a symlink. But both '/etc/os-release' and '/usr/lib/os-release' are owned by 'base-files'...
>>
>> An upgrade of 'base-files' would be replacing (silently, since is not marked as a configuration file) '/usr/lib/os-release' with the version of the upstream 'base-files' package and possibly breaking any tools in the system relying on certain values in that file.
>>
>> Is there a reason that I'm missing for doing so instead of the Debian-way (file diversion with dpkg-divert)? Or any hack that I've overseen that avoids the mentioned breakage?
Interesting, I didn't remember that `/etc/os-release` is a symlink,
could that be something that has changed in more recent debian versions?
If so then, of course that needs to be fixed.
regards,
Claudius
On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
> I might try to provide a fix, if we agree that the current implementation has an issue.
>
> @Claudius: you wrote the original code [1]. Do you remember why you implemented it this way? Do you remember if you were aware of the issue I mentioned and you provided a mitigation for the issue that I see (assuming my analysis is right)?
>
> [1] https://github.com/ilbers/isar/commit/13ce96e5bc84b60f2fa7ccfe93dde045461884e6
>
> Silvano
>
> On 15/01/2021 15:26, [ext] Silvano Cirujano Cuesta wrote:
>> Hi,
>>
>> I've noticed that '/etc/os-release' is being changed on the image in meta/classes/image-postproc-extension.bbclass [1]. What BTW ends up changing '/usr/lib/os-release', since it's only a symlink. But both '/etc/os-release' and '/usr/lib/os-release' are owned by 'base-files'...
>>
>> An upgrade of 'base-files' would be replacing (silently, since is not marked as a configuration file) '/usr/lib/os-release' with the version of the upstream 'base-files' package and possibly breaking any tools in the system relying on certain values in that file.
>>
>> Is there a reason that I'm missing for doing so instead of the Debian-way (file diversion with dpkg-divert)? Or any hack that I've overseen that avoids the mentioned breakage?
could that be something that has changed in more recent debian versions?
If so then, of course that needs to be fixed.
regards,
Claudius
Silvano Cirujano Cuesta
Jan 18, 2021, 9:52:11 AM1/18/21
to Claudius Heine, isar-...@googlegroups.com
I've tested it right now with the Qemu AMD64 image:
1. Modified it so that /etc/os-release has "my information".
2. Confirmed that /etc/os-release in the VM has "my information".
3. Made the current 'base-files' upstream package [1] available to the VM (additional disk, editing the generated image,...).
4. Installed the current 'base-files' upstream package.
5. Confirmed that "my" /etc/os-release has been silently replaced by a symlink to /usr/lib/os-release.
Anybody not doing package updates on 'base-files' won't notice anything, but the bug will still be there. Only covered by a convention, uncover it and it'll bite you...
Please convince me :-) that it's not a bug or that there are no realistic scenarios where it can be triggered... if not I'll contribute a fix.
Silvano
[1] http://ftp.de.debian.org/debian/pool/main/b/base-files/base-files_10.3+deb10u7_amd64.deb
On 18/01/2021 13:35, Claudius Heine wrote:
> Hi Silvano,
>
> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
>> I might try to provide a fix, if we agree that the current implementation has an issue.
>>
>> @Claudius: you wrote the original code [1]. Do you remember why you implemented it this way? Do you remember if you were aware of the issue I mentioned and you provided a mitigation for the issue that I see (assuming my analysis is right)?
>>
>> [1] https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Filbers%2Fisar%2Fcommit%2F13ce96e5bc84b60f2fa7ccfe93dde045461884e6&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C0bf1a83dda684b9e03e408d8bbad9a58%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637465701582684886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rRq%2F%2BKNrHdUn1SV%2FxVTIOSH6BofosIhUrFZWyy5PmSc%3D&reserved=0
1. Modified it so that /etc/os-release has "my information".
2. Confirmed that /etc/os-release in the VM has "my information".
3. Made the current 'base-files' upstream package [1] available to the VM (additional disk, editing the generated image,...).
4. Installed the current 'base-files' upstream package.
5. Confirmed that "my" /etc/os-release has been silently replaced by a symlink to /usr/lib/os-release.
Anybody not doing package updates on 'base-files' won't notice anything, but the bug will still be there. Only covered by a convention, uncover it and it'll bite you...
Please convince me :-) that it's not a bug or that there are no realistic scenarios where it can be triggered... if not I'll contribute a fix.
Silvano
[1] http://ftp.de.debian.org/debian/pool/main/b/base-files/base-files_10.3+deb10u7_amd64.deb
On 18/01/2021 13:35, Claudius Heine wrote:
> Hi Silvano,
>
> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
>> I might try to provide a fix, if we agree that the current implementation has an issue.
>>
>> @Claudius: you wrote the original code [1]. Do you remember why you implemented it this way? Do you remember if you were aware of the issue I mentioned and you provided a mitigation for the issue that I see (assuming my analysis is right)?
>>
Henning Schild
Jan 19, 2021, 3:25:35 AM1/19/21
to Claudius Heine, Silvano Cirujano Cuesta, isar-...@googlegroups.com
Am Mon, 18 Jan 2021 13:35:53 +0100
schrieb Claudius Heine <c...@denx.de>:
the problem seems to be that it is a symlink, otherwise one would
assume that changes in /etc/ are allowed and covered by the config file
exception and will be subject to merging if an updated package comes
around.
My guess would be that we need to
- make it a copy instead of a symlink
- modify it
In this case an update of the base-files package should leave it alone
or ask for a merge. And i think that would be OK behaviour.
Henning
> regards,
> Claudius
>
schrieb Claudius Heine <c...@denx.de>:
assume that changes in /etc/ are allowed and covered by the config file
exception and will be subject to merging if an updated package comes
around.
My guess would be that we need to
- make it a copy instead of a symlink
- modify it
In this case an update of the base-files package should leave it alone
or ask for a merge. And i think that would be OK behaviour.
Henning
> regards,
> Claudius
>
Henning Schild
Jan 19, 2021, 3:33:29 AM1/19/21
to Claudius Heine, Silvano Cirujano Cuesta, isar-...@googlegroups.com
Am Tue, 19 Jan 2021 09:25:31 +0100
schrieb "[ext] Henning Schild" <henning...@siemens.com>:
Alternative would be to create /etc/os-release-isar as that modified
copy, and replace the symlink target /etc/os-release ->
/etc/os-release-isar
We can not have the modified file be part of any package, at least not
with its final content. The version control command has to happen on
every rootfs build and has to come "late", and a package would not have
that property. Plus a package would become subject to potential false
sharing in the multiconfig case ... The fields we change are mostly
image-properties, and not generic
Henning
schrieb "[ext] Henning Schild" <henning...@siemens.com>:
copy, and replace the symlink target /etc/os-release ->
/etc/os-release-isar
We can not have the modified file be part of any package, at least not
with its final content. The version control command has to happen on
every rootfs build and has to come "late", and a package would not have
that property. Plus a package would become subject to potential false
sharing in the multiconfig case ... The fields we change are mostly
image-properties, and not generic
Henning
Silvano Cirujano Cuesta
Jan 19, 2021, 3:43:23 AM1/19/21
to Henning Schild, Claudius Heine, isar-...@googlegroups.com
On 19/01/2021 09:25, Henning Schild wrote:
> Am Mon, 18 Jan 2021 13:35:53 +0100
> schrieb Claudius Heine <c...@denx.de>:
>
>> Hi Silvano,
>>
>> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
>>> I might try to provide a fix, if we agree that the current
>>> implementation has an issue.
>>>
>>> @Claudius: you wrote the original code [1]. Do you remember why you
>>> implemented it this way? Do you remember if you were aware of the
>>> issue I mentioned and you provided a mitigation for the issue that
>>> I see (assuming my analysis is right)?
>>>
>>> [1]
>>>
>>> Silvano
>>>
>>> On 15/01/2021 15:26, [ext] Silvano Cirujano Cuesta wrote:
>>>> Hi,
>>>>
>>>> I've noticed that '/etc/os-release' is being changed on the image
>>>> in meta/classes/image-postproc-extension.bbclass [1]. What BTW
>>>> ends up changing '/usr/lib/os-release', since it's only a symlink.
>>>> But both '/etc/os-release' and '/usr/lib/os-release' are owned by
>>>> 'base-files'...
>>>>
>>>> An upgrade of 'base-files' would be replacing (silently, since is
>>>> not marked as a configuration file) '/usr/lib/os-release' with the
>>>> version of the upstream 'base-files' package and possibly breaking
>>>> any tools in the system relying on certain values in that file.
>>>>
>>>> Is there a reason that I'm missing for doing so instead of the
>>>> Debian-way (file diversion with dpkg-divert)? Or any hack that
>>>> I've overseen that avoids the mentioned breakage?
>> Interesting, I didn't remember that `/etc/os-release` is a symlink,
>> could that be something that has changed in more recent debian
>> versions?
>>
>> If so then, of course that needs to be fixed.
> the problem seems to be that it is a symlink, otherwise one would
> assume that changes in /etc/ are allowed and covered by the config file
> exception and will be subject to merging if an updated package comes
> around.
No, that's wrong. The issue is not that it's a symlink, the issue is that it's a package-managed file that hasn't been declared as a configuration file.
>>> Silvano
>>>
>>> On 15/01/2021 15:26, [ext] Silvano Cirujano Cuesta wrote:
>>>> Hi,
>>>>
>>>> I've noticed that '/etc/os-release' is being changed on the image
>>>> in meta/classes/image-postproc-extension.bbclass [1]. What BTW
>>>> ends up changing '/usr/lib/os-release', since it's only a symlink.
>>>> But both '/etc/os-release' and '/usr/lib/os-release' are owned by
>>>> 'base-files'...
>>>>
>>>> An upgrade of 'base-files' would be replacing (silently, since is
>>>> not marked as a configuration file) '/usr/lib/os-release' with the
>>>> version of the upstream 'base-files' package and possibly breaking
>>>> any tools in the system relying on certain values in that file.
>>>>
>>>> Is there a reason that I'm missing for doing so instead of the
>>>> Debian-way (file diversion with dpkg-divert)? Or any hack that
>>>> I've overseen that avoids the mentioned breakage?
>> Interesting, I didn't remember that `/etc/os-release` is a symlink,
>> could that be something that has changed in more recent debian
>> versions?
>>
>> If so then, of course that needs to be fixed.
> the problem seems to be that it is a symlink, otherwise one would
> assume that changes in /etc/ are allowed and covered by the config file
> exception and will be subject to merging if an updated package comes
> around.
I agree with you that files under /etc should be changeable, but... Debian found sort of a compromise between using the standard path /etc/os-release and not expecting it to be changed (symlink to /usr/lib and no configuration). You can like it or not, but that's what we have to live with...
> My guess would be that we need to
> - make it a copy instead of a symlink
> - modify it
>
> In this case an update of the base-files package should leave it alone
> or ask for a merge. And i think that would be OK behaviour.
But that's not the case if updating upstream Debian "base-files", it silently replaces whatever we place there (file or symlink) with a symlink to /usr/lib/os-release.
> In this case an update of the base-files package should leave it alone
> or ask for a merge. And i think that would be OK behaviour.
Silvano
>
> Henning
>
>> regards,
>> Claudius
>>
--
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux
Henning Schild
Jan 19, 2021, 3:43:58 AM1/19/21
to [ext] Silvano Cirujano Cuesta, isar-...@googlegroups.com
Hi,
i wonder if there is any command we can run for QA. One that makes sure
that all files in the rootfs are owned by a package and are not
modified. That should take /etc /var /tmp and other such locations into
account, but should clearly detect the problematic change in /usr/lib
that slipped in via that unexpected symlink.
Does anyone know such a command?
Henning
Am Fri, 15 Jan 2021 15:26:14 +0100
schrieb "[ext] Silvano Cirujano Cuesta"
<silvano.cir...@siemens.com>:
i wonder if there is any command we can run for QA. One that makes sure
that all files in the rootfs are owned by a package and are not
modified. That should take /etc /var /tmp and other such locations into
account, but should clearly detect the problematic change in /usr/lib
that slipped in via that unexpected symlink.
Does anyone know such a command?
Henning
Am Fri, 15 Jan 2021 15:26:14 +0100
schrieb "[ext] Silvano Cirujano Cuesta"
<silvano.cir...@siemens.com>:
Silvano Cirujano Cuesta
Jan 19, 2021, 3:50:31 AM1/19/21
to Henning Schild, Claudius Heine, isar-...@googlegroups.com
On 19/01/2021 09:33, Henning Schild wrote:
> Am Tue, 19 Jan 2021 09:25:31 +0100
> schrieb "[ext] Henning Schild" <henning...@siemens.com>:
>
>> Am Mon, 18 Jan 2021 13:35:53 +0100
>> schrieb Claudius Heine <c...@denx.de>:
>>
>>> Hi Silvano,
>>>
>>> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
>>>> I might try to provide a fix, if we agree that the current
>>>> implementation has an issue.
>>>>
>>>> @Claudius: you wrote the original code [1]. Do you remember why
>>>> you implemented it this way? Do you remember if you were aware of
>>>> the issue I mentioned and you provided a mitigation for the issue
>>>> that I see (assuming my analysis is right)?
>>>>
>>>> [1]
>
> We can not have the modified file be part of any package, at least not
> with its final content. The version control command has to happen on
> every rootfs build and has to come "late", and a package would not have
> that property. Plus a package would become subject to potential false
> sharing in the multiconfig case ... The fields we change are mostly
> image-properties, and not generic
Image post-processing is deemed to fail.
> We can not have the modified file be part of any package, at least not
> with its final content. The version control command has to happen on
> every rootfs build and has to come "late", and a package would not have
> that property. Plus a package would become subject to potential false
> sharing in the multiconfig case ... The fields we change are mostly
> image-properties, and not generic
Sorry, I don't understand what you mean with "a package would become subject to potential false sharing in the multiconfig case". A package created on build-time can contain those image properties, right? I might be missing something...
Silvano
>
> Henning
>
>> In this case an update of the base-files package should leave it alone
>> or ask for a merge. And i think that would be OK behaviour.
>>
>> Henning
>>
>>> regards,
>>> Claudius
>>>
Silvano Cirujano Cuesta
Jan 19, 2021, 4:03:59 AM1/19/21
to Henning Schild, isar-...@googlegroups.com
On 19/01/2021 09:43, Henning Schild wrote:
> Hi,
>
> i wonder if there is any command we can run for QA. One that makes sure
> that all files in the rootfs are owned by a package and are not
> modified. That should take /etc /var /tmp and other such locations into
> account, but should clearly detect the problematic change in /usr/lib
> that slipped in via that unexpected symlink.
>
> Does anyone know such a command?
Silvano
>
> Henning
>
> Am Fri, 15 Jan 2021 15:26:14 +0100
> schrieb "[ext] Silvano Cirujano Cuesta"
> <silvano.cir...@siemens.com>:
>
>> Hi,
>>
>> I've noticed that '/etc/os-release' is being changed on the image in
>> meta/classes/image-postproc-extension.bbclass [1]. What BTW ends up
>> changing '/usr/lib/os-release', since it's only a symlink. But both
>> '/etc/os-release' and '/usr/lib/os-release' are owned by
>> 'base-files'...
>>
>> An upgrade of 'base-files' would be replacing (silently, since is not
>> marked as a configuration file) '/usr/lib/os-release' with the
>> version of the upstream 'base-files' package and possibly breaking
>> any tools in the system relying on certain values in that file.
>>
>> Is there a reason that I'm missing for doing so instead of the
>> Debian-way (file diversion with dpkg-divert)? Or any hack that I've
>> overseen that avoids the mentioned breakage?
>>
>> Hasta la vista,
>>
>> Silvano
>>
>> [1]
Henning Schild
Jan 19, 2021, 4:08:30 AM1/19/21
to Silvano Cirujano Cuesta, Claudius Heine, isar-...@googlegroups.com
Am Tue, 19 Jan 2021 09:43:17 +0100
schrieb Silvano Cirujano Cuesta <silvano.cir...@siemens.com>:
> On 19/01/2021 09:25, Henning Schild wrote:
> > Am Mon, 18 Jan 2021 13:35:53 +0100
> > schrieb Claudius Heine <c...@denx.de>:
> >
> >> Hi Silvano,
> >>
> >> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
> >>> I might try to provide a fix, if we agree that the current
> >>> implementation has an issue.
> >>>
> >>> @Claudius: you wrote the original code [1]. Do you remember why
> >>> you implemented it this way? Do you remember if you were aware of
> >>> the issue I mentioned and you provided a mitigation for the issue
> >>> that I see (assuming my analysis is right)?
> >>>
> >>> [1]
> >>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Filbers%2Fisar%2Fcommit%2F13ce96e5bc84b60f2fa7ccfe93dde045461884e6&data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7Cf108d28909264c483e3d08d8bc5646d1%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637466426022621736%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nXdIXZTWYDOaDvvJkYCD1q1rYY2dsKuKPj0nUexZz5U%3D&reserved=0
We should not follow that symlink, but i would still hope that we are
allowed to recreate that file without the risk of that package
overwriting the file in /etc/ if it became a file and is modified.
Problem with the copy is that a major update would not be reflected.
Say your isar image starts off with debian9, a dist-upgrade might leave
you with 9 in there.
> > My guess would be that we need to
> > - make it a copy instead of a symlink
> > - modify it
> As long as the file remains managed by "base-files", none of this
> should be done. Either we replace "base-files" or we create file
> diversions for those files. I wanted to focus this thread on
> confirming the issue, I've opened another thread to align on how to
> fix this issue. My proposal there is to contact the Debian
> Derivatives mailing list.
Dont know what a file diversion is, but also sounds like a hack that
would not cover the dist-upgrade correctly.
> >
> > In this case an update of the base-files package should leave it
> > alone or ask for a merge. And i think that would be OK behaviour.
>
> But that's not the case if updating upstream Debian "base-files", it
> silently replaces whatever we place there (file or symlink) with a
> symlink to /usr/lib/os-release.
Are you sure about that? That would have to be an "ln -sf" in a
postinst or something, maybe you can share a link to the code doing
that.
If there is code that overwrites a file in /etc without config file
protection, that would clearly be a reason to start talking to
upstream. Because it would be a violation of how config files work in
general.
Henning
> Silvano
>
> >
> > Henning
> >
> >> regards,
> >> Claudius
> >>
schrieb Silvano Cirujano Cuesta <silvano.cir...@siemens.com>:
> On 19/01/2021 09:25, Henning Schild wrote:
> > Am Mon, 18 Jan 2021 13:35:53 +0100
> > schrieb Claudius Heine <c...@denx.de>:
> >
> >> Hi Silvano,
> >>
> >> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
> >>> I might try to provide a fix, if we agree that the current
> >>> implementation has an issue.
> >>>
> >>> @Claudius: you wrote the original code [1]. Do you remember why
> >>> you implemented it this way? Do you remember if you were aware of
> >>> the issue I mentioned and you provided a mitigation for the issue
> >>> that I see (assuming my analysis is right)?
> >>>
> >>> [1]
allowed to recreate that file without the risk of that package
overwriting the file in /etc/ if it became a file and is modified.
Problem with the copy is that a major update would not be reflected.
Say your isar image starts off with debian9, a dist-upgrade might leave
you with 9 in there.
> > My guess would be that we need to
> > - make it a copy instead of a symlink
> > - modify it
> As long as the file remains managed by "base-files", none of this
> should be done. Either we replace "base-files" or we create file
> diversions for those files. I wanted to focus this thread on
> confirming the issue, I've opened another thread to align on how to
> fix this issue. My proposal there is to contact the Debian
> Derivatives mailing list.
would not cover the dist-upgrade correctly.
> >
> > In this case an update of the base-files package should leave it
> > alone or ask for a merge. And i think that would be OK behaviour.
>
> But that's not the case if updating upstream Debian "base-files", it
> silently replaces whatever we place there (file or symlink) with a
> symlink to /usr/lib/os-release.
postinst or something, maybe you can share a link to the code doing
that.
If there is code that overwrites a file in /etc without config file
protection, that would clearly be a reason to start talking to
upstream. Because it would be a violation of how config files work in
general.
Henning
> Silvano
>
> >
> > Henning
> >
> >> regards,
> >> Claudius
> >>
Claudius Heine
Jan 19, 2021, 4:11:30 AM1/19/21
to Silvano Cirujano Cuesta, isar-...@googlegroups.com
Hi Silvano,
On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
> I might try to provide a fix, if we agree that the current implementation has an issue.
>
> @Claudius: you wrote the original code [1]. Do you remember why you implemented it this way? Do you remember if you were aware of the issue I mentioned and you provided a mitigation for the issue that I see (assuming my analysis is right)?
>
> [1] https://github.com/ilbers/isar/commit/13ce96e5bc84b60f2fa7ccfe93dde045461884e6
I now looked at the code, and in that commit I just copied the
>
> @Claudius: you wrote the original code [1]. Do you remember why you implemented it this way? Do you remember if you were aware of the issue I mentioned and you provided a mitigation for the issue that I see (assuming my analysis is right)?
>
> [1] https://github.com/ilbers/isar/commit/13ce96e5bc84b60f2fa7ccfe93dde045461884e6
`update_etc_os_release` function to fit into the post processing
framework. The original commit is actually [1], so Henning might have
more insight into this than me.
regards,
Claudius
[1]
https://github.com/ilbers/isar/commit/2c99c8d2b821c493dfc4eab65ea0d205c992cd4d
Henning Schild
Jan 19, 2021, 4:14:50 AM1/19/21
to Silvano Cirujano Cuesta, Claudius Heine, isar-...@googlegroups.com
Am Tue, 19 Jan 2021 10:08:25 +0100
schrieb Henning Schild <henning...@siemens.com>:
Indeed, it does overwrite a modified copy ignoring config file
protection.
rm /etc/os-release
cp /usr/lib/os-release /etc/os-release
vim /etc/os-release
apt-get install base-files --reinstall
ls -al /etc/os-release
It is also replaced when it is a symlink to another location.
Henning
schrieb Henning Schild <henning...@siemens.com>:
protection.
rm /etc/os-release
cp /usr/lib/os-release /etc/os-release
vim /etc/os-release
apt-get install base-files --reinstall
ls -al /etc/os-release
It is also replaced when it is a symlink to another location.
Henning
Henning Schild
Jan 19, 2021, 4:22:51 AM1/19/21
to Silvano Cirujano Cuesta, Claudius Heine, isar-...@googlegroups.com
Am Tue, 19 Jan 2021 09:50:27 +0100
No a package can not do the job, unfortunately. That is why we violate
the rule that every modification should be part of a package. It is not
because that file is potentially already owned by another package, that
can always be solved with overwriting in a package hook.
The package would need to have a unique name and version for every new
output of "git describe" and for every image description. In
multiconfig we build multiple images in one tree, sharing isar-apt.
Henning
> On 19/01/2021 09:33, Henning Schild wrote:
> > Am Tue, 19 Jan 2021 09:25:31 +0100
> > schrieb "[ext] Henning Schild" <henning...@siemens.com>:
> >
> >> Am Mon, 18 Jan 2021 13:35:53 +0100
> >> schrieb Claudius Heine <c...@denx.de>:
> >>
> >>> Hi Silvano,
> >>>
> >>> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
> >>>> I might try to provide a fix, if we agree that the current
> >>>> implementation has an issue.
> >>>>
> >>>> @Claudius: you wrote the original code [1]. Do you remember why
> >>>> you implemented it this way? Do you remember if you were aware of
> >>>> the issue I mentioned and you provided a mitigation for the issue
> >>>> that I see (assuming my analysis is right)?
> >>>>
> >>>> [1]
> >>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Filbers%2Fisar%2Fcommit%2F13ce96e5bc84b60f2fa7ccfe93dde045461884e6&data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7Cfb6a5b02cdb74a01fe7708d8bc5745c5%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637466430297681379%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MLD8tTQoJprCpQsG%2BRL6oSg8jiR%2BUBUHG4J89VAEWDg%3D&reserved=0
> > Am Tue, 19 Jan 2021 09:25:31 +0100
> > schrieb "[ext] Henning Schild" <henning...@siemens.com>:
> >
> >> Am Mon, 18 Jan 2021 13:35:53 +0100
> >> schrieb Claudius Heine <c...@denx.de>:
> >>
> >>> Hi Silvano,
> >>>
> >>> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
> >>>> I might try to provide a fix, if we agree that the current
> >>>> implementation has an issue.
> >>>>
> >>>> @Claudius: you wrote the original code [1]. Do you remember why
> >>>> you implemented it this way? Do you remember if you were aware of
> >>>> the issue I mentioned and you provided a mitigation for the issue
> >>>> that I see (assuming my analysis is right)?
> >>>>
> >>>> [1]
the rule that every modification should be part of a package. It is not
because that file is potentially already owned by another package, that
can always be solved with overwriting in a package hook.
The package would need to have a unique name and version for every new
output of "git describe" and for every image description. In
multiconfig we build multiple images in one tree, sharing isar-apt.
Henning
Silvano Cirujano Cuesta
Jan 19, 2021, 4:30:35 AM1/19/21
to Henning Schild, Claudius Heine, isar-...@googlegroups.com
On 19/01/2021 10:08, Henning Schild wrote:
> Am Tue, 19 Jan 2021 09:43:17 +0100
> schrieb Silvano Cirujano Cuesta <silvano.cir...@siemens.com>:
>
>> On 19/01/2021 09:25, Henning Schild wrote:
>>> Am Mon, 18 Jan 2021 13:35:53 +0100
>>> schrieb Claudius Heine <c...@denx.de>:
>>>
>>>> Hi Silvano,
>>>>
>>>> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
>>>>> I might try to provide a fix, if we agree that the current
>>>>> implementation has an issue.
>>>>>
>>>>> @Claudius: you wrote the original code [1]. Do you remember why
>>>>> you implemented it this way? Do you remember if you were aware of
>>>>> the issue I mentioned and you provided a mitigation for the issue
>>>>> that I see (assuming my analysis is right)?
>>>>>
>>>>> [1]
What happens is that an update of "base-files" silently restores the symlink to /usr/lib/os-release, removing whatever is there.
>
> Problem with the copy is that a major update would not be reflected.
> Say your isar image starts off with debian9, a dist-upgrade might leave
> you with 9 in there.
>
>>> My guess would be that we need to
>>> - make it a copy instead of a symlink
>>> - modify it
>> As long as the file remains managed by "base-files", none of this
>> should be done. Either we replace "base-files" or we create file
>> diversions for those files. I wanted to focus this thread on
>> confirming the issue, I've opened another thread to align on how to
>> fix this issue. My proposal there is to contact the Debian
>> Derivatives mailing list.
> Dont know what a file diversion is, but also sounds like a hack that
> would not cover the dist-upgrade correctly.
That's exactly the opposite, it's the Debian way to support that a package can manage a file that was being managed previously by another one. And making all these modifications manageable by dpkg, even restoring the original files is supported.
>>> My guess would be that we need to
>>> - make it a copy instead of a symlink
>>> - modify it
>> As long as the file remains managed by "base-files", none of this
>> should be done. Either we replace "base-files" or we create file
>> diversions for those files. I wanted to focus this thread on
>> confirming the issue, I've opened another thread to align on how to
>> fix this issue. My proposal there is to contact the Debian
>> Derivatives mailing list.
> Dont know what a file diversion is, but also sounds like a hack that
> would not cover the dist-upgrade correctly.
"dpkg-divert --list" gives you all the file diversions you have on a Debian system. Packages doing diversions are "dash", "firefox-esr", "vim-runtime",...
See here: https://wiki.debian.org/DpkgDiversions
>
>>> In this case an update of the base-files package should leave it
>>> alone or ask for a merge. And i think that would be OK behaviour.
>> But that's not the case if updating upstream Debian "base-files", it
>> silently replaces whatever we place there (file or symlink) with a
>> symlink to /usr/lib/os-release.
> Are you sure about that? That would have to be an "ln -sf" in a
> postinst or something, maybe you can share a link to the code doing
> that.
>
> If there is code that overwrites a file in /etc without config file
> protection, that would clearly be a reason to start talking to
> upstream. Because it would be a violation of how config files work in
> general.
You might be right. I agree with Debian on considering that OS information is not configuration. But they probably didn't want to divert from the Linux common path /etc/os-release.
> If there is code that overwrites a file in /etc without config file
> protection, that would clearly be a reason to start talking to
> upstream. Because it would be a violation of how config files work in
> general.
Considering the content of /usr/lib/os-release something that shouldn't be changeable, they had to break one of following too Linux consents:
- files in /etc are configurations and changeable
- OS information is available under /etc/os-release
Silvano
>
> Henning
>
>> Silvano
>>
>>> Henning
>>>
>>>> regards,
>>>> Claudius
>>>>
Henning Schild
Jan 19, 2021, 4:38:58 AM1/19/21
to [ext] Silvano Cirujano Cuesta, isar-...@googlegroups.com
Am Fri, 15 Jan 2021 15:26:14 +0100
schrieb "[ext] Silvano Cirujano Cuesta"
<silvano.cir...@siemens.com>:
schrieb "[ext] Silvano Cirujano Cuesta"
<silvano.cir...@siemens.com>:
> Hi,
>
> I've noticed that '/etc/os-release' is being changed on the image in
> meta/classes/image-postproc-extension.bbclass [1]. What BTW ends up
> changing '/usr/lib/os-release', since it's only a symlink. But both
> '/etc/os-release' and '/usr/lib/os-release' are owned by
> 'base-files'...
>
> An upgrade of 'base-files' would be replacing (silently, since is not
> marked as a configuration file) '/usr/lib/os-release' with the
> version of the upstream 'base-files' package and possibly breaking
> any tools in the system relying on certain values in that file.
We found that we can not even place a file in /etc/ it will always be
>
> I've noticed that '/etc/os-release' is being changed on the image in
> meta/classes/image-postproc-extension.bbclass [1]. What BTW ends up
> changing '/usr/lib/os-release', since it's only a symlink. But both
> '/etc/os-release' and '/usr/lib/os-release' are owned by
> 'base-files'...
>
> An upgrade of 'base-files' would be replacing (silently, since is not
> marked as a configuration file) '/usr/lib/os-release' with the
> version of the upstream 'base-files' package and possibly breaking
> any tools in the system relying on certain values in that file.
replaced with that symlink, no config file protection on this one.
While we could talk with upstream about that violation, we would also
need a way of dealing with it, because it is unlikely to change in
stable distros.
> Is there a reason that I'm missing for doing so instead of the
> Debian-way (file diversion with dpkg-divert)? Or any hack that I've
> overseen that avoids the mentioned breakage?
- write a script that can create /etc/os-release from a given default
and a second file /etc/os-release-isar
- if variable in both, prefer isar
- if variable only in isar, append
- if variable only in original, use
- put that script into a package we install in every isar image
- place a dpkg or apt hook into that package
- test -f /etc/os-release-isar && test -h /etc/os-release
- create/change /etc/os-release-isar in image postproc and run hook
Henning
Silvano Cirujano Cuesta
Jan 19, 2021, 5:37:36 AM1/19/21
to Henning Schild, Claudius Heine, isar-...@googlegroups.com
On 19/01/2021 10:22, Henning Schild wrote:
> Am Tue, 19 Jan 2021 09:50:27 +0100
> schrieb Silvano Cirujano Cuesta <silvano.cir...@siemens.com>:
>
>> On 19/01/2021 09:33, Henning Schild wrote:
>>> Am Tue, 19 Jan 2021 09:25:31 +0100
>>> schrieb "[ext] Henning Schild" <henning...@siemens.com>:
>>>
>>>> Am Mon, 18 Jan 2021 13:35:53 +0100
>>>> schrieb Claudius Heine <c...@denx.de>:
>>>>
>>>>> Hi Silvano,
>>>>>
>>>>> On 2021-01-18 12:35, Silvano Cirujano Cuesta wrote:
>>>>>> I might try to provide a fix, if we agree that the current
>>>>>> implementation has an issue.
>>>>>>
>>>>>> @Claudius: you wrote the original code [1]. Do you remember why
>>>>>> you implemented it this way? Do you remember if you were aware of
>>>>>> the issue I mentioned and you provided a mitigation for the issue
>>>>>> that I see (assuming my analysis is right)?
>>>>>>
>>>>>> [1]
>
> The package would need to have a unique name and version for every new
> output of "git describe" and for every image description. In
> multiconfig we build multiple images in one tree, sharing isar-apt.
- /usr/lib/os-release is not managed by any package and that way post-processing it is not an issue and dpkg won't break anything
- Debian os-release manpage still applies
If going for the other approach (create our own "base-files" package) we can do pretty much the same:
- the package doesn't provide /usr/lib/os-release, leaving it package-unmanaged
- image post-processing to create the file becomes a requirement
Silvano
>
> Henning
>
>> Silvano
>>
>>> Henning
>>>
>>>> In this case an update of the base-files package should leave it
>>>> alone or ask for a merge. And i think that would be OK behaviour.
>>>>
>>>> Henning
>>>>
>>>>> regards,
>>>>> Claudius
>>>>>
Claudius Heine
Jan 22, 2021, 3:52:20 AM1/22/21
to Silvano Cirujano Cuesta, Henning Schild, isar-...@googlegroups.com
Hi Silvano,
On 2021-01-19 11:37, Silvano Cirujano Cuesta wrote:
> We can create a removal file diversion for /usr/lib/os-release (leaving /etc/os-release untouched, that way the Debian manpage for os-release still applies). This approach has following attributes:
>
> - /usr/lib/os-release is not managed by any package and that way post-processing it is not an issue and dpkg won't break anything
>
> - Debian os-release manpage still applies
>
> If going for the other approach (create our own "base-files" package) we can do pretty much the same:
>
> - the package doesn't provide /usr/lib/os-release, leaving it package-unmanaged
>
> - image post-processing to create the file becomes a requirement
I looked at the manpage of os-release and it does not state that
`/etc/os-release` can or should not be a file:
```
The file /etc/os-release takes precedence over
/usr/lib/os-release. Applications should check for the former, and
exclusively use its data if it exists, and only fall back to
/usr/lib/os-release if
it is missing. Applications should not read data from both files
at the same time. /usr/lib/os-release is the recommended place to store
OS release information as part of vendor trees.
/etc/os-release should be a relative symlink to
/usr/lib/os-release, to provide compatibility with applications only
looking at /etc/. A relative symlink instead of an absolute symlink is
necessary
to avoid breaking the link in a chroot or initrd environment
such as dracut.
os-release contains data that is defined by the operating system
vendor and should generally not be changed by the administrator.
As this file only encodes names and identifiers it should not be
localized.
The /etc/os-release and /usr/lib/os-release files might be
symlinks to other files, but it is important that the file is available
from earliest boot on, and hence must be located on the root file
system.
```
So why not make sure `/etc/os-release` is a file and do this:
dpkg-divert --add --local --no-rename /etc/os-release
in the post processing step? Shouldn't this prevent any future
base-files update from preventing to overwrite the `/etc/os-release`?
The only issue I would see if someone performs a distribution upgrade
and now the `/etc/os-release` no longer contains the correct version,
but other than an apt.conf hook that updates `/etc/os-release` based on
new information from `/usr/lib/os-release` after an upgrade, I don't see
a way to do this automatically.
I would rather divert '/etc/os-release' in a postprocessing script than
`/usr/lib/os-release` because, I see the post-processing more of a local
configuration.
For multiple reasons we cannot used a package to write the version
information, so that is out of the picture anyway.
The only other way would be to create a new file just for the image or
build version, but that would break compatibility now.
regards,
Claudius
On 2021-01-19 11:37, Silvano Cirujano Cuesta wrote:
> We can create a removal file diversion for /usr/lib/os-release (leaving /etc/os-release untouched, that way the Debian manpage for os-release still applies). This approach has following attributes:
>
> - /usr/lib/os-release is not managed by any package and that way post-processing it is not an issue and dpkg won't break anything
>
> - Debian os-release manpage still applies
>
> If going for the other approach (create our own "base-files" package) we can do pretty much the same:
>
> - the package doesn't provide /usr/lib/os-release, leaving it package-unmanaged
>
> - image post-processing to create the file becomes a requirement
`/etc/os-release` can or should not be a file:
```
The file /etc/os-release takes precedence over
/usr/lib/os-release. Applications should check for the former, and
exclusively use its data if it exists, and only fall back to
/usr/lib/os-release if
it is missing. Applications should not read data from both files
at the same time. /usr/lib/os-release is the recommended place to store
OS release information as part of vendor trees.
/etc/os-release should be a relative symlink to
/usr/lib/os-release, to provide compatibility with applications only
looking at /etc/. A relative symlink instead of an absolute symlink is
necessary
to avoid breaking the link in a chroot or initrd environment
such as dracut.
os-release contains data that is defined by the operating system
vendor and should generally not be changed by the administrator.
As this file only encodes names and identifiers it should not be
localized.
The /etc/os-release and /usr/lib/os-release files might be
symlinks to other files, but it is important that the file is available
from earliest boot on, and hence must be located on the root file
system.
```
So why not make sure `/etc/os-release` is a file and do this:
dpkg-divert --add --local --no-rename /etc/os-release
in the post processing step? Shouldn't this prevent any future
base-files update from preventing to overwrite the `/etc/os-release`?
The only issue I would see if someone performs a distribution upgrade
and now the `/etc/os-release` no longer contains the correct version,
but other than an apt.conf hook that updates `/etc/os-release` based on
new information from `/usr/lib/os-release` after an upgrade, I don't see
a way to do this automatically.
I would rather divert '/etc/os-release' in a postprocessing script than
`/usr/lib/os-release` because, I see the post-processing more of a local
configuration.
For multiple reasons we cannot used a package to write the version
information, so that is out of the picture anyway.
The only other way would be to create a new file just for the image or
build version, but that would break compatibility now.
regards,
Claudius
Silvano Cirujano Cuesta
Jan 22, 2021, 4:47:22 AM1/22/21
to Claudius Heine, Henning Schild, isar-...@googlegroups.com
Hi Claudius,
TL;DR: you're proposing here what appears to be a feasible solution to this issue without using a package
Nevertheless, more information inline.
On 22/01/2021 09:52, Claudius Heine wrote:
> Hi Silvano,
>
> On 2021-01-19 11:37, Silvano Cirujano Cuesta wrote:
>> We can create a removal file diversion for /usr/lib/os-release (leaving /etc/os-release untouched, that way the Debian manpage for os-release still applies). This approach has following attributes:
>>
>> - /usr/lib/os-release is not managed by any package and that way post-processing it is not an issue and dpkg won't break anything
>>
>> - Debian os-release manpage still applies
>>
>> If going for the other approach (create our own "base-files" package) we can do pretty much the same:
>>
>> - the package doesn't provide /usr/lib/os-release, leaving it package-unmanaged
>>
>> - image post-processing to create the file becomes a requirement
> I looked at the manpage of os-release and it does not state that `/etc/os-release` can or should not be a file:
That's probably a misunderstanding. I don't mean that we have to create "a file", but provide the os-release information according the specification. There are multiple different valid combinations:
- /etc/os-release symlink to /usr/lib/os-release
- /usr/lib/os-relase only (not /etc/os-release at all)
- /etc/os-release only, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)
- /etc/os-release is a symlink to another file like /usr/lib/isar-release, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)
...
>
> ```
> The file /etc/os-release takes precedence over /usr/lib/os-release. Applications should check for the former, and exclusively use its data if it exists, and only fall back to /usr/lib/os-release if
> it is missing. Applications should not read data from both files at the same time. /usr/lib/os-release is the recommended place to store OS release information as part of vendor trees.
> /etc/os-release should be a relative symlink to /usr/lib/os-release, to provide compatibility with applications only looking at /etc/. A relative symlink instead of an absolute symlink is necessary
> to avoid breaking the link in a chroot or initrd environment such as dracut.
>
> os-release contains data that is defined by the operating system vendor and should generally not be changed by the administrator.
>
> As this file only encodes names and identifiers it should not be localized.
>
> The /etc/os-release and /usr/lib/os-release files might be symlinks to other files, but it is important that the file is available from earliest boot on, and hence must be located on the root file
> system.
> ```
>
> So why not make sure `/etc/os-release` is a file and do this:
>
> dpkg-divert --add --local --no-rename /etc/os-release
>
> in the post processing step? Shouldn't this prevent any future base-files update from preventing to overwrite the `/etc/os-release`?
This command would so to say command dpkg not to touch /etc/os-release from the point it's executed on. That way you can have a local version of /etc/os-release and be sure that no package updates is modifying/replacing it. Such a command would go into the postinst hook and another postrm hook would be required to remove the file diversion for /etc/os-release and restoring the original one.
This would technically resolve the issue, but looks a bit weird to me. Debian and derivatives get that file managed by a package. But since I haven't understood what the issue mentioned by Henning regarding multiconfig and packages, this might be the only feasible solution...
>
> The only issue I would see if someone performs a distribution upgrade and now the `/etc/os-release` no longer contains the correct version, but other than an apt.conf hook that updates `/etc/os-release` based on new information from `/usr/lib/os-release` after an upgrade, I don't see a way to do this automatically.
Why should a distribution upgrade change that? I'm not aware that dist-upgrade changes file diversions... To my knowledge, once the file diversion has been created, dpkg should completely ignore that file.
File diversions don't apply to packages, but to files, therefore I'd expect it to survive a dist-upgrade. Therefore dpkg doesn't keep track of the packages that are "blocked" from touching a file. It keeps track of the file path and of the package that requested the file diversion (unless "--local" specified, as you propose).
>
> I would rather divert '/etc/os-release' in a postprocessing script than `/usr/lib/os-release` because, I see the post-processing more of a local configuration.
Diverting /etc/os-release or /usr/lib/os-release depends on what we want to achieve (it would be even better letting the ISAR user decide it):
1. Keep originating Debian for reference? Then change only /etc/os-release.
2. Wipe away all information about originating Debian? Then change /usr/lib/os-release.
/etc/os-release might appear like a local configuration for the ISAR builder, but IMO it's intrinsic information about the distribution being built with ISAR. As the documentation that you are quoting says "defined by the operating system vendor and should generally not be changed by the administrator." Therefore IMO the path /etc/os-release was kept for historical reasons (see footnote 1 of the announcement [1]), but it's an exception to the convention that files under "/etc" are configurations and can be changed.
[1] http://0pointer.de/blog/projects/os-release
> For multiple reasons we cannot used a package to write the version information, so that is out of the picture anyway.
As said above, if for the reasons you and Henning mention using a package is out of the picture, this is probably the only feasible solution... Except for those people who wrote their own base-files package to resolve this issue, since their package won't work anymore :-) They would have to manage this file using hooks instead of directly declaring the file as managed by their package.
>
> regards,
> Claudius
Silvano
--
Siemens AG, T RDA IOT SES-DE
TL;DR: you're proposing here what appears to be a feasible solution to this issue without using a package
Nevertheless, more information inline.
On 22/01/2021 09:52, Claudius Heine wrote:
> Hi Silvano,
>
> On 2021-01-19 11:37, Silvano Cirujano Cuesta wrote:
>> We can create a removal file diversion for /usr/lib/os-release (leaving /etc/os-release untouched, that way the Debian manpage for os-release still applies). This approach has following attributes:
>>
>> - /usr/lib/os-release is not managed by any package and that way post-processing it is not an issue and dpkg won't break anything
>>
>> - Debian os-release manpage still applies
>>
>> If going for the other approach (create our own "base-files" package) we can do pretty much the same:
>>
>> - the package doesn't provide /usr/lib/os-release, leaving it package-unmanaged
>>
>> - image post-processing to create the file becomes a requirement
> I looked at the manpage of os-release and it does not state that `/etc/os-release` can or should not be a file:
- /etc/os-release symlink to /usr/lib/os-release
- /usr/lib/os-relase only (not /etc/os-release at all)
- /etc/os-release only, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)
- /etc/os-release is a symlink to another file like /usr/lib/isar-release, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)
...
>
> ```
> The file /etc/os-release takes precedence over /usr/lib/os-release. Applications should check for the former, and exclusively use its data if it exists, and only fall back to /usr/lib/os-release if
> it is missing. Applications should not read data from both files at the same time. /usr/lib/os-release is the recommended place to store OS release information as part of vendor trees.
> /etc/os-release should be a relative symlink to /usr/lib/os-release, to provide compatibility with applications only looking at /etc/. A relative symlink instead of an absolute symlink is necessary
> to avoid breaking the link in a chroot or initrd environment such as dracut.
>
> os-release contains data that is defined by the operating system vendor and should generally not be changed by the administrator.
>
> As this file only encodes names and identifiers it should not be localized.
>
> The /etc/os-release and /usr/lib/os-release files might be symlinks to other files, but it is important that the file is available from earliest boot on, and hence must be located on the root file
> system.
> ```
>
> So why not make sure `/etc/os-release` is a file and do this:
>
> dpkg-divert --add --local --no-rename /etc/os-release
>
> in the post processing step? Shouldn't this prevent any future base-files update from preventing to overwrite the `/etc/os-release`?
This would technically resolve the issue, but looks a bit weird to me. Debian and derivatives get that file managed by a package. But since I haven't understood what the issue mentioned by Henning regarding multiconfig and packages, this might be the only feasible solution...
>
> The only issue I would see if someone performs a distribution upgrade and now the `/etc/os-release` no longer contains the correct version, but other than an apt.conf hook that updates `/etc/os-release` based on new information from `/usr/lib/os-release` after an upgrade, I don't see a way to do this automatically.
File diversions don't apply to packages, but to files, therefore I'd expect it to survive a dist-upgrade. Therefore dpkg doesn't keep track of the packages that are "blocked" from touching a file. It keeps track of the file path and of the package that requested the file diversion (unless "--local" specified, as you propose).
>
> I would rather divert '/etc/os-release' in a postprocessing script than `/usr/lib/os-release` because, I see the post-processing more of a local configuration.
1. Keep originating Debian for reference? Then change only /etc/os-release.
2. Wipe away all information about originating Debian? Then change /usr/lib/os-release.
/etc/os-release might appear like a local configuration for the ISAR builder, but IMO it's intrinsic information about the distribution being built with ISAR. As the documentation that you are quoting says "defined by the operating system vendor and should generally not be changed by the administrator." Therefore IMO the path /etc/os-release was kept for historical reasons (see footnote 1 of the announcement [1]), but it's an exception to the convention that files under "/etc" are configurations and can be changed.
[1] http://0pointer.de/blog/projects/os-release
> For multiple reasons we cannot used a package to write the version information, so that is out of the picture anyway.
>
> The only other way would be to create a new file just for the image or build version, but that would break compatibility now.
Do you mean using a completely different file? Something like /etc/isar-release or /usr/lib/isar-release? If that's what you mean, I wouldn't do it. Not only because of the compatibility issue that you mention, but also because ISAR is de-facto creating a Debian Derivative that information belongs logically into /etc/os-release. Someone finding vanilla Debian inputs in /etc/os-release would expect a vanilla Debian installation, not an ISAR-built distro.
> The only other way would be to create a new file just for the image or build version, but that would break compatibility now.
>
> regards,
> Claudius
Silvano
--
Siemens AG, T RDA IOT SES-DE
Claudius Heine
Jan 22, 2021, 5:33:29 AM1/22/21
to Silvano Cirujano Cuesta, Henning Schild, isar-...@googlegroups.com
Hi Silvano
On 2021-01-22 10:47, Silvano Cirujano Cuesta wrote:
> Hi Claudius,
>
> TL;DR: you're proposing here what appears to be a feasible solution to this issue without using a package
>
> Nevertheless, more information inline.
>
> On 22/01/2021 09:52, Claudius Heine wrote:
>> Hi Silvano,
>>
>> On 2021-01-19 11:37, Silvano Cirujano Cuesta wrote:
>>> We can create a removal file diversion for /usr/lib/os-release (leaving /etc/os-release untouched, that way the Debian manpage for os-release still applies). This approach has following attributes:
>>>
>>> - /usr/lib/os-release is not managed by any package and that way post-processing it is not an issue and dpkg won't break anything
>>>
>>> - Debian os-release manpage still applies
>>>
>>> If going for the other approach (create our own "base-files" package) we can do pretty much the same:
>>>
>>> - the package doesn't provide /usr/lib/os-release, leaving it package-unmanaged
>>>
>>> - image post-processing to create the file becomes a requirement
>> I looked at the manpage of os-release and it does not state that `/etc/os-release` can or should not be a file:
>
> That's probably a misunderstanding. I don't mean that we have to create "a file", but provide the os-release information according the specification. There are multiple different valid combinations:
>
> - /etc/os-release symlink to /usr/lib/os-release
>
> - /usr/lib/os-relase only (not /etc/os-release at all)
>
> - /etc/os-release only, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)
>
> - /etc/os-release is a symlink to another file like /usr/lib/isar-release, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)
Well I had this in mind:
- /etc/os-release is a file containing the isar version,
/usr/lib/os-release is a file containing the original debian version as
deployed by base-files.
This could also be done similar to your fourth solution:
- /etc/os-release is a symlink to another file like
/usr/lib/isar-release, original /usr/lib/os-release from base-files
The manual page states that /etc/os-release takes precedence over
/usr/lib/os-release, so if /etc/os-release exists, it does not matter if
/usr/lib/os-release does as well or what its content is.
>
> ...
>
>>
>> ```
>> The file /etc/os-release takes precedence over /usr/lib/os-release. Applications should check for the former, and exclusively use its data if it exists, and only fall back to /usr/lib/os-release if
>> it is missing. Applications should not read data from both files at the same time. /usr/lib/os-release is the recommended place to store OS release information as part of vendor trees.
>> /etc/os-release should be a relative symlink to /usr/lib/os-release, to provide compatibility with applications only looking at /etc/. A relative symlink instead of an absolute symlink is necessary
>> to avoid breaking the link in a chroot or initrd environment such as dracut.
>>
>> os-release contains data that is defined by the operating system vendor and should generally not be changed by the administrator.
>>
>> As this file only encodes names and identifiers it should not be localized.
>>
>> The /etc/os-release and /usr/lib/os-release files might be symlinks to other files, but it is important that the file is available from earliest boot on, and hence must be located on the root file
>> system.
>> ```
>>
>> So why not make sure `/etc/os-release` is a file and do this:
>>
>> dpkg-divert --add --local --no-rename /etc/os-release
>>
>> in the post processing step? Shouldn't this prevent any future base-files update from preventing to overwrite the `/etc/os-release`?
>
> This command would so to say command dpkg not to touch /etc/os-release from the point it's executed on. That way you can have a local version of /etc/os-release and be sure that no package updates is modifying/replacing it. Such a command would go into the postinst hook and another postrm hook would be required to remove the file diversion for /etc/os-release and restoring the original one.
Why would such a command go into a postinst hook? It contains '--local'
therefore I would expect the administrator doing it him or herself. Or
in Isars case a bitbake post installation process.
>
> This would technically resolve the issue, but looks a bit weird to me. Debian and derivatives get that file managed by a package. But since I haven't understood what the issue mentioned by Henning regarding multiconfig and packages, this might be the only feasible solution...
Simple example: Our /etc/os-release contains the build date and git
commit id. If we put this information into a package, then it we could
not cache it, because this information will change constantly. That also
means all dependent tasks and recipes (image creation) cannot be cached,
breaking the caching of a lot of stuff.
Calling 'bitbake isar-image' two times in short succession without any
change, would cause 2 full builds of the whole image.
>> The only issue I would see if someone performs a distribution upgrade and now the `/etc/os-release` no longer contains the correct version, but other than an apt.conf hook that updates `/etc/os-release` based on new information from `/usr/lib/os-release` after an upgrade, I don't see a way to do this automatically.
>
> Why should a distribution upgrade change that? I'm not aware that dist-upgrade changes file diversions... To my knowledge, once the file diversion has been created, dpkg should completely ignore that file.
>
> File diversions don't apply to packages, but to files, therefore I'd expect it to survive a dist-upgrade. Therefore dpkg doesn't keep track of the packages that are "blocked" from touching a file. It keeps track of the file path and of the package that requested the file diversion (unless "--local" specified, as you propose).
You misunderstood me.
In my example a base-files package would still update
/usr/lib/os-release to contain a new debian suite name, if you do a
distribution update. If you want our diverted /etc/os-release to contain
this new info automatically, you could do this via a apt.conf.d hook.
>
>>
>> I would rather divert '/etc/os-release' in a postprocessing script than `/usr/lib/os-release` because, I see the post-processing more of a local configuration.
>
> Diverting /etc/os-release or /usr/lib/os-release depends on what we want to achieve (it would be even better letting the ISAR user decide it):
I don't want to divert /usr/lib/os-release. Let it contain the upstream
version for reference. This file does not matter, because
/etc/os-release takes precedence over it.
>
> 1. Keep originating Debian for reference? Then change only /etc/os-release.
Yes.
>
> 2. Wipe away all information about originating Debian? Then change /usr/lib/os-release.
No.
>
> /etc/os-release might appear like a local configuration for the ISAR builder, but IMO it's intrinsic information about the distribution being built with ISAR. As the documentation that you are quoting says "defined by the operating system vendor and should generally not be changed by the administrator." Therefore IMO the path /etc/os-release was kept for historical reasons (see footnote 1 of the announcement [1]), but it's an exception to the convention that files under "/etc" are configurations and can be changed.
Well its a separate discussion if Isar is in the role of a distribution
vendor or of an administrator.
I would say its something in between.
>
> [1] http://0pointer.de/blog/projects/os-release
>
>> For multiple reasons we cannot used a package to write the version information, so that is out of the picture anyway.
> As said above, if for the reasons you and Henning mention using a package is out of the picture, this is probably the only feasible solution... Except for those people who wrote their own base-files package to resolve this issue, since their package won't work anymore :-) They would have to manage this file using hooks instead of directly declaring the file as managed by their package.
This was why I proposed a apt.conf.d hook. Maybe we could generally do
it like this.
Some package changes /usr/lib/os-release and a apt.conf.d hook would
merge this content with ours into /etc/os-release.
>>
>> The only other way would be to create a new file just for the image or build version, but that would break compatibility now.
> Do you mean using a completely different file? Something like /etc/isar-release or /usr/lib/isar-release? If that's what you mean, I wouldn't do it. Not only because of the compatibility issue that you mention, but also because ISAR is de-facto creating a Debian Derivative that information belongs logically into /etc/os-release. Someone finding vanilla Debian inputs in /etc/os-release would expect a vanilla Debian installation, not an ISAR-built distro.
While I agree that Isar in some configurations creates a Debian
Derivative, currently it puts image version and build date into
/etc/os-release. Which is strictly speaking not the right place for that.
It would be better to have something like /etc/base-image-version etc.
for that.
regards,
Claudius
On 2021-01-22 10:47, Silvano Cirujano Cuesta wrote:
> Hi Claudius,
>
> TL;DR: you're proposing here what appears to be a feasible solution to this issue without using a package
>
> Nevertheless, more information inline.
>
> On 22/01/2021 09:52, Claudius Heine wrote:
>> Hi Silvano,
>>
>> On 2021-01-19 11:37, Silvano Cirujano Cuesta wrote:
>>> We can create a removal file diversion for /usr/lib/os-release (leaving /etc/os-release untouched, that way the Debian manpage for os-release still applies). This approach has following attributes:
>>>
>>> - /usr/lib/os-release is not managed by any package and that way post-processing it is not an issue and dpkg won't break anything
>>>
>>> - Debian os-release manpage still applies
>>>
>>> If going for the other approach (create our own "base-files" package) we can do pretty much the same:
>>>
>>> - the package doesn't provide /usr/lib/os-release, leaving it package-unmanaged
>>>
>>> - image post-processing to create the file becomes a requirement
>> I looked at the manpage of os-release and it does not state that `/etc/os-release` can or should not be a file:
>
> That's probably a misunderstanding. I don't mean that we have to create "a file", but provide the os-release information according the specification. There are multiple different valid combinations:
>
> - /etc/os-release symlink to /usr/lib/os-release
>
> - /usr/lib/os-relase only (not /etc/os-release at all)
>
> - /etc/os-release only, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)
>
> - /etc/os-release is a symlink to another file like /usr/lib/isar-release, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)
- /etc/os-release is a file containing the isar version,
/usr/lib/os-release is a file containing the original debian version as
deployed by base-files.
This could also be done similar to your fourth solution:
- /etc/os-release is a symlink to another file like
The manual page states that /etc/os-release takes precedence over
/usr/lib/os-release, so if /etc/os-release exists, it does not matter if
/usr/lib/os-release does as well or what its content is.
>
> ...
>
>>
>> ```
>> The file /etc/os-release takes precedence over /usr/lib/os-release. Applications should check for the former, and exclusively use its data if it exists, and only fall back to /usr/lib/os-release if
>> it is missing. Applications should not read data from both files at the same time. /usr/lib/os-release is the recommended place to store OS release information as part of vendor trees.
>> /etc/os-release should be a relative symlink to /usr/lib/os-release, to provide compatibility with applications only looking at /etc/. A relative symlink instead of an absolute symlink is necessary
>> to avoid breaking the link in a chroot or initrd environment such as dracut.
>>
>> os-release contains data that is defined by the operating system vendor and should generally not be changed by the administrator.
>>
>> As this file only encodes names and identifiers it should not be localized.
>>
>> The /etc/os-release and /usr/lib/os-release files might be symlinks to other files, but it is important that the file is available from earliest boot on, and hence must be located on the root file
>> system.
>> ```
>>
>> So why not make sure `/etc/os-release` is a file and do this:
>>
>> dpkg-divert --add --local --no-rename /etc/os-release
>>
>> in the post processing step? Shouldn't this prevent any future base-files update from preventing to overwrite the `/etc/os-release`?
>
> This command would so to say command dpkg not to touch /etc/os-release from the point it's executed on. That way you can have a local version of /etc/os-release and be sure that no package updates is modifying/replacing it. Such a command would go into the postinst hook and another postrm hook would be required to remove the file diversion for /etc/os-release and restoring the original one.
therefore I would expect the administrator doing it him or herself. Or
in Isars case a bitbake post installation process.
>
> This would technically resolve the issue, but looks a bit weird to me. Debian and derivatives get that file managed by a package. But since I haven't understood what the issue mentioned by Henning regarding multiconfig and packages, this might be the only feasible solution...
commit id. If we put this information into a package, then it we could
not cache it, because this information will change constantly. That also
means all dependent tasks and recipes (image creation) cannot be cached,
breaking the caching of a lot of stuff.
Calling 'bitbake isar-image' two times in short succession without any
change, would cause 2 full builds of the whole image.
>> The only issue I would see if someone performs a distribution upgrade and now the `/etc/os-release` no longer contains the correct version, but other than an apt.conf hook that updates `/etc/os-release` based on new information from `/usr/lib/os-release` after an upgrade, I don't see a way to do this automatically.
>
> Why should a distribution upgrade change that? I'm not aware that dist-upgrade changes file diversions... To my knowledge, once the file diversion has been created, dpkg should completely ignore that file.
>
> File diversions don't apply to packages, but to files, therefore I'd expect it to survive a dist-upgrade. Therefore dpkg doesn't keep track of the packages that are "blocked" from touching a file. It keeps track of the file path and of the package that requested the file diversion (unless "--local" specified, as you propose).
In my example a base-files package would still update
/usr/lib/os-release to contain a new debian suite name, if you do a
distribution update. If you want our diverted /etc/os-release to contain
this new info automatically, you could do this via a apt.conf.d hook.
>
>>
>> I would rather divert '/etc/os-release' in a postprocessing script than `/usr/lib/os-release` because, I see the post-processing more of a local configuration.
>
> Diverting /etc/os-release or /usr/lib/os-release depends on what we want to achieve (it would be even better letting the ISAR user decide it):
version for reference. This file does not matter, because
/etc/os-release takes precedence over it.
>
> 1. Keep originating Debian for reference? Then change only /etc/os-release.
>
> 2. Wipe away all information about originating Debian? Then change /usr/lib/os-release.
>
> /etc/os-release might appear like a local configuration for the ISAR builder, but IMO it's intrinsic information about the distribution being built with ISAR. As the documentation that you are quoting says "defined by the operating system vendor and should generally not be changed by the administrator." Therefore IMO the path /etc/os-release was kept for historical reasons (see footnote 1 of the announcement [1]), but it's an exception to the convention that files under "/etc" are configurations and can be changed.
vendor or of an administrator.
I would say its something in between.
>
> [1] http://0pointer.de/blog/projects/os-release
>
>> For multiple reasons we cannot used a package to write the version information, so that is out of the picture anyway.
> As said above, if for the reasons you and Henning mention using a package is out of the picture, this is probably the only feasible solution... Except for those people who wrote their own base-files package to resolve this issue, since their package won't work anymore :-) They would have to manage this file using hooks instead of directly declaring the file as managed by their package.
it like this.
Some package changes /usr/lib/os-release and a apt.conf.d hook would
merge this content with ours into /etc/os-release.
>>
>> The only other way would be to create a new file just for the image or build version, but that would break compatibility now.
> Do you mean using a completely different file? Something like /etc/isar-release or /usr/lib/isar-release? If that's what you mean, I wouldn't do it. Not only because of the compatibility issue that you mention, but also because ISAR is de-facto creating a Debian Derivative that information belongs logically into /etc/os-release. Someone finding vanilla Debian inputs in /etc/os-release would expect a vanilla Debian installation, not an ISAR-built distro.
Derivative, currently it puts image version and build date into
/etc/os-release. Which is strictly speaking not the right place for that.
It would be better to have something like /etc/base-image-version etc.
for that.
regards,
Claudius
Silvano Cirujano Cuesta
Jan 22, 2021, 6:36:34 AM1/22/21
to Claudius Heine, Henning Schild, isar-...@googlegroups.com
Hi Claudius,
I also had foreseen that option as one of my favorites (either that or the first one), therefore that would be fine for me.
I might have a different understanding for "admin", but if not distribution builder, I'd say installation builder, but not administrator.
IMO ISAR is a distribution builder and as such a tool for distribution vendors. As a Debian user I find at home with many ISAR images, but not at all with some others.
A distribution according this definition "A Linux distribution may also be described as a particular assortment of application and utility software [...], packaged together with the Linux kernel in such a way that its capabilities meet the needs of many users.", taken from https://en.wikipedia.org/wiki/Linux_distribution
>
> I would say its something in between.
>
>>
>> [1] https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2F0pointer.de%2Fblog%2Fprojects%2Fos-release&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C4348b0c7cf544189355608d8bec12a38%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637469084129831158%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gobc49NK4Zh02%2BrVOBBU%2BQMg7sAu5XVstVzu39CquOA%3D&reserved=0
>
>>
>> ...
>>
>>>
>>> ```
>>> The file /etc/os-release takes precedence over /usr/lib/os-release. Applications should check for the former, and exclusively use its data if it exists, and only fall back to /usr/lib/os-release if
>>> it is missing. Applications should not read data from both files at the same time. /usr/lib/os-release is the recommended place to store OS release information as part of vendor trees.
>>> /etc/os-release should be a relative symlink to /usr/lib/os-release, to provide compatibility with applications only looking at /etc/. A relative symlink instead of an absolute symlink is necessary
>>> to avoid breaking the link in a chroot or initrd environment such as dracut.
>>>
>>> os-release contains data that is defined by the operating system vendor and should generally not be changed by the administrator.
>>>
>>> As this file only encodes names and identifiers it should not be localized.
>>>
>>> The /etc/os-release and /usr/lib/os-release files might be symlinks to other files, but it is important that the file is available from earliest boot on, and hence must be located on the root file
>>> system.
>>> ```
>>>
>>> So why not make sure `/etc/os-release` is a file and do this:
>>>
>>> dpkg-divert --add --local --no-rename /etc/os-release
>>>
>>> in the post processing step? Shouldn't this prevent any future base-files update from preventing to overwrite the `/etc/os-release`?
>>
>> This command would so to say command dpkg not to touch /etc/os-release from the point it's executed on. That way you can have a local version of /etc/os-release and be sure that no package updates is modifying/replacing it. Such a command would go into the postinst hook and another postrm hook would be required to remove the file diversion for /etc/os-release and restoring the original one.
>
> Why would such a command go into a postinst hook? It contains '--local' therefore I would expect the administrator doing it him or herself. Or in Isars case a bitbake post installation process.
Sorry, forgot that we cannot use a package. A bitbake post installation process is what I had in mind. I don't care in this mailing list about admins "doing stuff" that might break their systems ;-)
>>
>> ...
>>
>>>
>>> ```
>>> The file /etc/os-release takes precedence over /usr/lib/os-release. Applications should check for the former, and exclusively use its data if it exists, and only fall back to /usr/lib/os-release if
>>> it is missing. Applications should not read data from both files at the same time. /usr/lib/os-release is the recommended place to store OS release information as part of vendor trees.
>>> /etc/os-release should be a relative symlink to /usr/lib/os-release, to provide compatibility with applications only looking at /etc/. A relative symlink instead of an absolute symlink is necessary
>>> to avoid breaking the link in a chroot or initrd environment such as dracut.
>>>
>>> os-release contains data that is defined by the operating system vendor and should generally not be changed by the administrator.
>>>
>>> As this file only encodes names and identifiers it should not be localized.
>>>
>>> The /etc/os-release and /usr/lib/os-release files might be symlinks to other files, but it is important that the file is available from earliest boot on, and hence must be located on the root file
>>> system.
>>> ```
>>>
>>> So why not make sure `/etc/os-release` is a file and do this:
>>>
>>> dpkg-divert --add --local --no-rename /etc/os-release
>>>
>>> in the post processing step? Shouldn't this prevent any future base-files update from preventing to overwrite the `/etc/os-release`?
>>
>> This command would so to say command dpkg not to touch /etc/os-release from the point it's executed on. That way you can have a local version of /etc/os-release and be sure that no package updates is modifying/replacing it. Such a command would go into the postinst hook and another postrm hook would be required to remove the file diversion for /etc/os-release and restoring the original one.
>
> Why would such a command go into a postinst hook? It contains '--local' therefore I would expect the administrator doing it him or herself. Or in Isars case a bitbake post installation process.
>
>>
>> This would technically resolve the issue, but looks a bit weird to me. Debian and derivatives get that file managed by a package. But since I haven't understood what the issue mentioned by Henning regarding multiconfig and packages, this might be the only feasible solution...
>
> Simple example: Our /etc/os-release contains the build date and git commit id. If we put this information into a package, then it we could not cache it, because this information will change constantly. That also means all dependent tasks and recipes (image creation) cannot be cached, breaking the caching of a lot of stuff.
>
> Calling 'bitbake isar-image' two times in short succession without any change, would cause 2 full builds of the whole image.
The current image post-processing is happening pretty much in the end and not breaking caching, right? Wouldn't it be possible to modify/extend that recipe to create a Debian package and install it chrooting to the root filesystem?
>>
>> This would technically resolve the issue, but looks a bit weird to me. Debian and derivatives get that file managed by a package. But since I haven't understood what the issue mentioned by Henning regarding multiconfig and packages, this might be the only feasible solution...
>
> Simple example: Our /etc/os-release contains the build date and git commit id. If we put this information into a package, then it we could not cache it, because this information will change constantly. That also means all dependent tasks and recipes (image creation) cannot be cached, breaking the caching of a lot of stuff.
>
> Calling 'bitbake isar-image' two times in short succession without any change, would cause 2 full builds of the whole image.
>
>>> The only issue I would see if someone performs a distribution upgrade and now the `/etc/os-release` no longer contains the correct version, but other than an apt.conf hook that updates `/etc/os-release` based on new information from `/usr/lib/os-release` after an upgrade, I don't see a way to do this automatically.
>>
>> Why should a distribution upgrade change that? I'm not aware that dist-upgrade changes file diversions... To my knowledge, once the file diversion has been created, dpkg should completely ignore that file.
>>
>> File diversions don't apply to packages, but to files, therefore I'd expect it to survive a dist-upgrade. Therefore dpkg doesn't keep track of the packages that are "blocked" from touching a file. It keeps track of the file path and of the package that requested the file diversion (unless "--local" specified, as you propose).
>
> You misunderstood me.
>
> In my example a base-files package would still update /usr/lib/os-release to contain a new debian suite name, if you do a distribution update. If you want our diverted /etc/os-release to contain this new info automatically, you could do this via a apt.conf.d hook.
Diverting only /etc/os-release you get /usr/lib/os-release managed (and therefore updated) by base-files and ISAR takes care of /etc/os-release. I don't understand the need for an apt.conf.d hook... ISAR creates it and adds it to the root filesystem relying on neither the user nor dpkg modifying it.
>>> The only issue I would see if someone performs a distribution upgrade and now the `/etc/os-release` no longer contains the correct version, but other than an apt.conf hook that updates `/etc/os-release` based on new information from `/usr/lib/os-release` after an upgrade, I don't see a way to do this automatically.
>>
>> Why should a distribution upgrade change that? I'm not aware that dist-upgrade changes file diversions... To my knowledge, once the file diversion has been created, dpkg should completely ignore that file.
>>
>> File diversions don't apply to packages, but to files, therefore I'd expect it to survive a dist-upgrade. Therefore dpkg doesn't keep track of the packages that are "blocked" from touching a file. It keeps track of the file path and of the package that requested the file diversion (unless "--local" specified, as you propose).
>
> You misunderstood me.
>
> In my example a base-files package would still update /usr/lib/os-release to contain a new debian suite name, if you do a distribution update. If you want our diverted /etc/os-release to contain this new info automatically, you could do this via a apt.conf.d hook.
>
>>
>>>
>>> I would rather divert '/etc/os-release' in a postprocessing script than `/usr/lib/os-release` because, I see the post-processing more of a local configuration.
>>
>> Diverting /etc/os-release or /usr/lib/os-release depends on what we want to achieve (it would be even better letting the ISAR user decide it):
>
> I don't want to divert /usr/lib/os-release. Let it contain the upstream version for reference. This file does not matter, because /etc/os-release takes precedence over it.
Hopefully it's clear from my comments above, that I find this approach fine.
>>
>>>
>>> I would rather divert '/etc/os-release' in a postprocessing script than `/usr/lib/os-release` because, I see the post-processing more of a local configuration.
>>
>> Diverting /etc/os-release or /usr/lib/os-release depends on what we want to achieve (it would be even better letting the ISAR user decide it):
>
> I don't want to divert /usr/lib/os-release. Let it contain the upstream version for reference. This file does not matter, because /etc/os-release takes precedence over it.
>
>>
>> 1. Keep originating Debian for reference? Then change only /etc/os-release.
>
> Yes.
>
>>
>> 2. Wipe away all information about originating Debian? Then change /usr/lib/os-release.
>
> No.
>
>>
>> /etc/os-release might appear like a local configuration for the ISAR builder, but IMO it's intrinsic information about the distribution being built with ISAR. As the documentation that you are quoting says "defined by the operating system vendor and should generally not be changed by the administrator." Therefore IMO the path /etc/os-release was kept for historical reasons (see footnote 1 of the announcement [1]), but it's an exception to the convention that files under "/etc" are configurations and can be changed.
>
> Well its a separate discussion if Isar is in the role of a distribution vendor or of an administrator.
Is this a philosophical discussion? :-) Am I making a fuss?
>>
>> 1. Keep originating Debian for reference? Then change only /etc/os-release.
>
> Yes.
>
>>
>> 2. Wipe away all information about originating Debian? Then change /usr/lib/os-release.
>
> No.
>
>>
>> /etc/os-release might appear like a local configuration for the ISAR builder, but IMO it's intrinsic information about the distribution being built with ISAR. As the documentation that you are quoting says "defined by the operating system vendor and should generally not be changed by the administrator." Therefore IMO the path /etc/os-release was kept for historical reasons (see footnote 1 of the announcement [1]), but it's an exception to the convention that files under "/etc" are configurations and can be changed.
>
> Well its a separate discussion if Isar is in the role of a distribution vendor or of an administrator.
I might have a different understanding for "admin", but if not distribution builder, I'd say installation builder, but not administrator.
IMO ISAR is a distribution builder and as such a tool for distribution vendors. As a Debian user I find at home with many ISAR images, but not at all with some others.
A distribution according this definition "A Linux distribution may also be described as a particular assortment of application and utility software [...], packaged together with the Linux kernel in such a way that its capabilities meet the needs of many users.", taken from https://en.wikipedia.org/wiki/Linux_distribution
>
> I would say its something in between.
>
>>
>>
>>> For multiple reasons we cannot used a package to write the version information, so that is out of the picture anyway.
>> As said above, if for the reasons you and Henning mention using a package is out of the picture, this is probably the only feasible solution... Except for those people who wrote their own base-files package to resolve this issue, since their package won't work anymore :-) They would have to manage this file using hooks instead of directly declaring the file as managed by their package.
>
> This was why I proposed a apt.conf.d hook. Maybe we could generally do it like this.
No matter how you divert that file (bitbake post processing, apt.conf.d hook,...) the mentioned conflict with vendor-provided base-files packages appear whenever you have a file divert. Unless that vendor-provided base-files package modifies the file diversion being applied by ISAR (I don't know if it's possible) to become the owner of the file.
>>> For multiple reasons we cannot used a package to write the version information, so that is out of the picture anyway.
>> As said above, if for the reasons you and Henning mention using a package is out of the picture, this is probably the only feasible solution... Except for those people who wrote their own base-files package to resolve this issue, since their package won't work anymore :-) They would have to manage this file using hooks instead of directly declaring the file as managed by their package.
>
> This was why I proposed a apt.conf.d hook. Maybe we could generally do it like this.
>
> Some package changes /usr/lib/os-release and a apt.conf.d hook would merge this content with ours into /etc/os-release.
I still don't get the need for such "black magic".
> Some package changes /usr/lib/os-release and a apt.conf.d hook would merge this content with ours into /etc/os-release.
>
>>>
>>> The only other way would be to create a new file just for the image or build version, but that would break compatibility now.
>> Do you mean using a completely different file? Something like /etc/isar-release or /usr/lib/isar-release? If that's what you mean, I wouldn't do it. Not only because of the compatibility issue that you mention, but also because ISAR is de-facto creating a Debian Derivative that information belongs logically into /etc/os-release. Someone finding vanilla Debian inputs in /etc/os-release would expect a vanilla Debian installation, not an ISAR-built distro.
>
> While I agree that Isar in some configurations creates a Debian Derivative, currently it puts image version and build date into /etc/os-release. Which is strictly speaking not the right place for that.
If the image version and build date are relevant to have a clear identification of the OS version, then I disagree. If that's informative and not needed to have a unique identification of the OS version, then I agree that it could be somewhere else (although I still think it's not a wrong place for it).
>>>
>>> The only other way would be to create a new file just for the image or build version, but that would break compatibility now.
>> Do you mean using a completely different file? Something like /etc/isar-release or /usr/lib/isar-release? If that's what you mean, I wouldn't do it. Not only because of the compatibility issue that you mention, but also because ISAR is de-facto creating a Debian Derivative that information belongs logically into /etc/os-release. Someone finding vanilla Debian inputs in /etc/os-release would expect a vanilla Debian installation, not an ISAR-built distro.
>
> While I agree that Isar in some configurations creates a Debian Derivative, currently it puts image version and build date into /etc/os-release. Which is strictly speaking not the right place for that.
>
> It would be better to have something like /etc/base-image-version etc. for that.
>
> regards,
> Claudius
> It would be better to have something like /etc/base-image-version etc. for that.
>
> regards,
> Claudius
vijaikumar....@gmail.com
Feb 5, 2021, 6:55:21 AM2/5/21
to isar-users
Hi All,
After going through this, I feel that we could move the ISAR system details to a separate file like /etc/isar_{version/base/info}.
ISAR, as I see could be used to create a custom distribution or in a simple case just the functions of what an installer would do in other
distribution(bootstrap and prepare a simple bootable image for(*and) install).
If we see the minimal case, it does not make sense to have the ISAR info in /etc/os-release. And probably we should leave the /etc/os-release
alone; at least in the main ISAR repo and leave the custom configuration part to the users. In this way, we are not enforcing any uncommon
behavior (dpkg-divert+hook) that a new user least expects in ISAR in its vanilla form. This leaves more freedom to the downstream users to
choose a method(custom base-files / dpkg-divert / any unofficial way) that best fits their need based on what and where they need the information.
If you have built an image with ISAR, you could find some details about the ISAR system used from the /etc/isar_{version/base/info}. If you want to
add more information to it, well you are free to do so. If you feel the information is crucial, that it needs to be present in /etc/os-release, and
willing to maintain it for your custom distro, well you could do it too. If you feel it is unnecessary, well you could remove it too.
And then comes the discussion of backward compatibility. Probably implement and maintain dpkg-divert approach and provide a way for existing users to switch to them using a flag/feature.
Thanks,
Vijai Kumar K
Silvano Cirujano Cuesta
Feb 5, 2021, 9:57:45 AM2/5/21
to vijaikumar....@gmail.com, isar-users
Hi Vijai,
I like your proposal.
If we need to provide "something" that keeps modifying the /etc/os-release for backwards compatibility, do we want to keep the current "broken" implementation?
If we want to "fix" that implementation, then we can do it in a way that not only provides that backwards compatibility, but also to provides a useful feature. The default would be not touching /etc/os-release and one recipe could provide (when requested with a flag) a package for "managing" /etc/os-release (assuming it can be somehow accomplished with multiconfigs, or be a mutually exclusive feature).
Ciao,
Silvano
> A distribution according this definition "A Linux distribution may also be described as a particular assortment of application and utility software [...], packaged together with the Linux kernel in such a way that its capabilities meet the needs of many users.", taken from https://en.wikipedia.org/wiki/Linux_distribution <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FLinux_distribution&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2210c7432f134485959a08d8c9ccf62a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637481229432997922%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=b7fZq0akmNohTagcefD81VInLbBkAv%2FcjQTLZNKqIjc%3D&reserved=0>
> --
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com <mailto:isar-users+...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/6f4b945f-7763-49ee-a8c5-2e02d450a2e0n%40googlegroups.com <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F6f4b945f-7763-49ee-a8c5-2e02d450a2e0n%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2210c7432f134485959a08d8c9ccf62a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637481229433007915%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PKO1KmREqy6GRr20lbuZ%2FsxeINqLIB86%2Fgfd4QeKlQo%3D&reserved=0>.
I like your proposal.
If we need to provide "something" that keeps modifying the /etc/os-release for backwards compatibility, do we want to keep the current "broken" implementation?
If we want to "fix" that implementation, then we can do it in a way that not only provides that backwards compatibility, but also to provides a useful feature. The default would be not touching /etc/os-release and one recipe could provide (when requested with a flag) a package for "managing" /etc/os-release (assuming it can be somehow accomplished with multiconfigs, or be a mutually exclusive feature).
Ciao,
Silvano
>
> >
> > I would say its something in between.
> >
> >>
> >> [1] https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2F0pointer.de%2Fblog%2Fprojects%2Fos-release&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C4348b0c7cf544189355608d8bec12a38%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637469084129831158%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gobc49NK4Zh02%2BrVOBBU%2BQMg7sAu5XVstVzu39CquOA%3D&reserved=0 <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2F0pointer.de%2Fblog%2Fprojects%2Fos-release&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2210c7432f134485959a08d8c9ccf62a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637481229433007915%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Hhqb%2FyU4z%2BqBlh4Yz2ocfxsQTaRtQotyAEJ3dG64k24%3D&reserved=0>
> >
> > I would say its something in between.
> >
> >>
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com <mailto:isar-users+...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/6f4b945f-7763-49ee-a8c5-2e02d450a2e0n%40googlegroups.com <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F6f4b945f-7763-49ee-a8c5-2e02d450a2e0n%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2210c7432f134485959a08d8c9ccf62a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637481229433007915%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PKO1KmREqy6GRr20lbuZ%2FsxeINqLIB86%2Fgfd4QeKlQo%3D&reserved=0>.
Message has been deleted
vijai kumar
Feb 7, 2021, 4:02:15 AM2/7/21
to Silvano Cirujano Cuesta, isar-users
On Fri, Feb 5, 2021 at 8:27 PM Silvano Cirujano Cuesta <silvano.cir...@siemens.com> wrote:
Hi Vijai,
I like your proposal.
If we need to provide "something" that keeps modifying the /etc/os-release for backwards compatibility, do we want to keep the current "broken" implementation?
I would not suggest we keep the current broken implementation. We should fix it.
If we want to "fix" that implementation, then we can do it in a way that not only provides that backwards compatibility, but also to provides a useful feature. The default would be not touching /etc/os-release and one recipe could provide (when requested with a flag) a package for "managing" /etc/os-release (assuming it can be somehow accomplished with multiconfigs, or be a mutually exclusive feature).
As Claudius previously mentioned, this might break caching. If the information is purely build-system related, should we try to put it in a package? Does this file need updates in the future? Well, from the current standpoint, I don't see it needing updates. The data is only applicable when we build using ISAR. If hypothetically, we decided to recreate the distro with a cached apt-repo generated with ISAR using some other tool, this info is not relevant.
That said, we need to understand from the community, what are the real-life scenarios they are using this ISAR information in /etc/os-release for? I am not personally using that info. If it is being used somewhere, we need to know why? So that we know how costly it is to break backward compatibility. Huge or none at all?
Thanks,
VIjai Kumar K
Silvano Cirujano Cuesta
Feb 8, 2021, 3:51:00 AM2/8/21
to vijai kumar, isar-users
On 07/02/2021 10:02, vijai kumar wrote:
>
>
> On Fri, Feb 5, 2021 at 8:27 PM Silvano Cirujano Cuesta <silvano.cir...@siemens.com <mailto:silvano.cir...@siemens.com>> wrote:
>
> Hi Vijai,
>
> I like your proposal.
>
> If we need to provide "something" that keeps modifying the /etc/os-release for backwards compatibility, do we want to keep the current "broken" implementation?
>
>
> I would not suggest we keep the current broken implementation. We should fix it.
>
>
>
> If we want to "fix" that implementation, then we can do it in a way that not only provides that backwards compatibility, but also to provides a useful feature. The default would be not touching /etc/os-release and one recipe could provide (when requested with a flag) a package for "managing" /etc/os-release (assuming it can be somehow accomplished with multiconfigs, or be a mutually exclusive feature).
>
>
> As Claudius previously mentioned, this might break caching. If the information is purely build-system related, should we try to put it in a package? Does this file need updates in the future? Well, fro"m the current standpoint, I don't see it needing updates. The data is only applicable when we build using ISAR. If hypothetically, we decided to recreate the distro with a cached apt-repo generated with ISAR using some other tool, this info is not relevant.
>
> On Fri, Feb 5, 2021 at 8:27 PM Silvano Cirujano Cuesta <silvano.cir...@siemens.com <mailto:silvano.cir...@siemens.com>> wrote:
>
> Hi Vijai,
>
> I like your proposal.
>
> If we need to provide "something" that keeps modifying the /etc/os-release for backwards compatibility, do we want to keep the current "broken" implementation?
>
>
> I would not suggest we keep the current broken implementation. We should fix it.
>
>
>
> If we want to "fix" that implementation, then we can do it in a way that not only provides that backwards compatibility, but also to provides a useful feature. The default would be not touching /etc/os-release and one recipe could provide (when requested with a flag) a package for "managing" /etc/os-release (assuming it can be somehow accomplished with multiconfigs, or be a mutually exclusive feature).
>
>
You possibly miss my explanation why simply placing that file there on post-processing is an error: any package-update of base-files would overwrite that file with the Debian vanilla one. In order to avoid that file from being changed on package updates we need to create either a file diversion or package replacement (like Ubuntu does).
BTW, a package doing only "dkpg-divert" for /etc/os-release doesn't need to contain any version information. Then the version information can be added on image postprocessing. That would be a compromise, because it wouldn't break any package caching.
I don't know exactly how caching works, but I wonder which caching would be breaking just because a package changes. Cached packages? Cached rootfs? Cached images? I would expect almost all rebuilds to be modifying images. I would also expect most of the rebuilds to be modifying the rootfs. Please correct me, if I'm wrong.
>
> That said, we need to understand from the community, what are the real-life scenarios they are using this ISAR information in /etc/os-release for? I am not personally using that info. If it is being used somewhere, we need to know why? So that we know how costly it is to break backward compatibility. Huge or none at all?
Back to your "real-life scenario" question. We started using it because that what the place to identify which base-system release we hat while integrating other components on the system. Moving that information to a file like "/etc/isar-release" would fix it for us, we would simply have to update our integration to fetch the information from that file.
Here start my personal opinion, nothing to do with my "real-time scenario". ISAR is being used in multiple cases to build Debian-derivates [1] and I would expect a Debian-derivative to provide some hint about it in its /etc/os-release. Therefore I would like to see ISAR providing the means to easily accomplish it. But since I seam to be the only one seeing that need, I might try to convince you with a RFC patch :-)
[1] https://www.debian.org/derivatives/
Silvano
>
> Thanks,
> VIjai Kumar K
>
>
>
> Ciao,
>
> Silvano
>
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FLinux_distribution&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2210c7432f134485959a08d8c9ccf62a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637481229432997922%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=b7fZq0akmNohTagcefD81VInLbBkAv%2FcjQTLZNKqIjc%3D&reserved=0 <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FLinux_distribution&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cc4baff9a162d4f6554ba08d8cb471284%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637482853429368834%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BEXgejPcmGTr32%2BbJ%2BVx7teZ9SPDa5vhC2sprApiCRo%3D&reserved=0>>
> >
> > >
> > > I would say its something in between.
> > >
> > >>
> > >> [1] https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2F0pointer.de%2Fblog%2Fprojects%2Fos-release&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C4348b0c7cf544189355608d8bec12a38%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637469084129831158%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gobc49NK4Zh02%2BrVOBBU%2BQMg7sAu5XVstVzu39CquOA%3D&reserved=0 <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2F0pointer.de%2Fblog%2Fprojects%2Fos-release&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cc4baff9a162d4f6554ba08d8cb471284%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637482853429368834%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Ktv25HHI%2Bf7XOQmsb6ViKvFfDQDzhpzR6NUIly1y8Z4%3D&reserved=0>
> > >
> > > I would say its something in between.
> > >
> > >>
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2F0pointer.de%2Fblog%2Fprojects%2Fos-release&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2210c7432f134485959a08d8c9ccf62a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637481229433007915%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Hhqb%2FyU4z%2BqBlh4Yz2ocfxsQTaRtQotyAEJ3dG64k24%3D&reserved=0 <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2F0pointer.de%2Fblog%2Fprojects%2Fos-release&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cc4baff9a162d4f6554ba08d8cb471284%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637482853429378828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=flCLe1V1QbiuuvdS62vC7qYXUVNeJIv0szecFs9C2jg%3D&reserved=0>>
> > To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/6f4b945f-7763-49ee-a8c5-2e02d450a2e0n%40googlegroups.com <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F6f4b945f-7763-49ee-a8c5-2e02d450a2e0n%2540googlegroups.com&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cc4baff9a162d4f6554ba08d8cb471284%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637482853429388823%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6NhouiZuDTLTf%2FYF5%2BdQHEL%2FBkga3PyysBp3xTUV%2FyI%3D&reserved=0>
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F6f4b945f-7763-49ee-a8c5-2e02d450a2e0n%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2210c7432f134485959a08d8c9ccf62a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637481229433007915%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PKO1KmREqy6GRr20lbuZ%2FsxeINqLIB86%2Fgfd4QeKlQo%3D&reserved=0
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F6f4b945f-7763-49ee-a8c5-2e02d450a2e0n%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cc4baff9a162d4f6554ba08d8cb471284%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637482853429388823%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2FVR4TnQcZEodPPHnwzqHpiuBi4tEwUK3hurROqsMlxI%3D&reserved=0>>.
>
Baurzhan Ismagulov
Feb 8, 2021, 12:20:30 PM2/8/21
to isar-...@googlegroups.com
Hello Silvano,
On Fri, Jan 15, 2021 at 03:26:14PM +0100, Silvano Cirujano Cuesta wrote:
> I've noticed that '/etc/os-release' is being changed on the image in meta/classes/image-postproc-extension.bbclass [1]. What BTW ends up changing '/usr/lib/os-release', since it's only a symlink. But both '/etc/os-release' and '/usr/lib/os-release' are owned by 'base-files'...
>
> An upgrade of 'base-files' would be replacing (silently, since is not marked as a configuration file) '/usr/lib/os-release' with the version of the upstream 'base-files' package and possibly breaking any tools in the system relying on certain values in that file.
>
> Is there a reason that I'm missing for doing so instead of the Debian-way (file diversion with dpkg-divert)? Or any hack that I've overseen that avoids the mentioned breakage?
Thanks for raising this topic. I'd like to address some of the issues.
* I see identification as a downstream issue. However, it would be nice to
provide something by default. Thus we touch os-release.
* Regarding an Isar-specific identification file: This is advised against in
http://0pointer.de/blog/projects/os-release.html.
* Breaking /etc/os-release -> ../usr/lib/os-release is a bug. We should modify
/usr/lib/os-release, in whatever way. We should not break the symlink. I
agree we should fix this.
* Isar was conceived as a rootfs builder that should be able to work without
creating a derivative (remember SLIND). If a downstream feels like it, they
may create a derivative.
* For a given rootfs, the identification information is intrinsically static.
Thus, the best would be to provide it in a static way, e.g. by patching (the
current approach) or providing base-files (I see this as a downstream issue).
* os-release contains static information and doesn't have user-modifiable
contents. Thus, I agree with Debian that this is not a config file.
* Upgrading base-files would overwrite os-release. From Isar point of view, I'm
fine with that; after all, it isn't the original image, it's image + upgrade
-- the identification should be changed as well.
* If a downstream wants to preserve the identification through package
upgrades, they should provide their own base-files. Maybe we could make it
easier for them in stock Isar.
* Since the information is static, dynamic approaches (config files, postinst,
alternatives, diversions) are not necessary. They are more complex than they
seem at the first glance (package upgrade order, package removal order,
downgrades, failed upgrades, idempotency, etc.).
To summarize, I suggest fixing /etc/os-release -> ../usr/lib/os-release
symlink breakage.
If there is an itch for the upgrade problem, I'd be open for the discussion
on providing base-files with Isar.
With kind regards,
Baurzhan.
On Fri, Jan 15, 2021 at 03:26:14PM +0100, Silvano Cirujano Cuesta wrote:
> I've noticed that '/etc/os-release' is being changed on the image in meta/classes/image-postproc-extension.bbclass [1]. What BTW ends up changing '/usr/lib/os-release', since it's only a symlink. But both '/etc/os-release' and '/usr/lib/os-release' are owned by 'base-files'...
>
> An upgrade of 'base-files' would be replacing (silently, since is not marked as a configuration file) '/usr/lib/os-release' with the version of the upstream 'base-files' package and possibly breaking any tools in the system relying on certain values in that file.
>
> Is there a reason that I'm missing for doing so instead of the Debian-way (file diversion with dpkg-divert)? Or any hack that I've overseen that avoids the mentioned breakage?
* I see identification as a downstream issue. However, it would be nice to
provide something by default. Thus we touch os-release.
* Regarding an Isar-specific identification file: This is advised against in
http://0pointer.de/blog/projects/os-release.html.
* Breaking /etc/os-release -> ../usr/lib/os-release is a bug. We should modify
/usr/lib/os-release, in whatever way. We should not break the symlink. I
agree we should fix this.
* Isar was conceived as a rootfs builder that should be able to work without
creating a derivative (remember SLIND). If a downstream feels like it, they
may create a derivative.
* For a given rootfs, the identification information is intrinsically static.
Thus, the best would be to provide it in a static way, e.g. by patching (the
current approach) or providing base-files (I see this as a downstream issue).
* os-release contains static information and doesn't have user-modifiable
contents. Thus, I agree with Debian that this is not a config file.
* Upgrading base-files would overwrite os-release. From Isar point of view, I'm
fine with that; after all, it isn't the original image, it's image + upgrade
-- the identification should be changed as well.
* If a downstream wants to preserve the identification through package
upgrades, they should provide their own base-files. Maybe we could make it
easier for them in stock Isar.
* Since the information is static, dynamic approaches (config files, postinst,
alternatives, diversions) are not necessary. They are more complex than they
seem at the first glance (package upgrade order, package removal order,
downgrades, failed upgrades, idempotency, etc.).
To summarize, I suggest fixing /etc/os-release -> ../usr/lib/os-release
symlink breakage.
If there is an itch for the upgrade problem, I'd be open for the discussion
on providing base-files with Isar.
With kind regards,
Baurzhan.
vijai kumar
Feb 9, 2021, 1:02:43 AM2/9/21
to Silvano Cirujano Cuesta, isar-users
On Mon, Feb 8, 2021 at 2:20 PM Silvano Cirujano Cuesta
<silvano.cir...@siemens.com> wrote:
>
>
> On 07/02/2021 10:02, vijai kumar wrote:
> >
> >
> > On Fri, Feb 5, 2021 at 8:27 PM Silvano Cirujano Cuesta <silvano.cir...@siemens.com <mailto:silvano.cir...@siemens.com>> wrote:
> >
> > Hi Vijai,
> >
> > I like your proposal.
> >
> > If we need to provide "something" that keeps modifying the /etc/os-release for backwards compatibility, do we want to keep the current "broken" implementation?
> >
> >
> > I would not suggest we keep the current broken implementation. We should fix it.
> >
> >
> >
> > If we want to "fix" that implementation, then we can do it in a way that not only provides that backwards compatibility, but also to provides a useful feature. The default would be not touching /etc/os-release and one recipe could provide (when requested with a flag) a package for "managing" /etc/os-release (assuming it can be somehow accomplished with multiconfigs, or be a mutually exclusive feature).
> >
> >
> > As Claudius previously mentioned, this might break caching. If the information is purely build-system related, should we try to put it in a package? Does this file need updates in the future? Well, fro"m the current standpoint, I don't see it needing updates. The data is only applicable when we build using ISAR. If hypothetically, we decided to recreate the distro with a cached apt-repo generated with ISAR using some other tool, this info is not relevant.
>
> You possibly miss my explanation why simply placing that file there on post-processing is an error: any package-update of base-files would overwrite that file with the Debian vanilla one. In order to avoid that file from being changed on package updates we need to create either a file diversion or package replacement (like Ubuntu does).
Ah, In the above I was merely talking about the information (i.e the
<silvano.cir...@siemens.com> wrote:
>
>
> On 07/02/2021 10:02, vijai kumar wrote:
> >
> >
> > On Fri, Feb 5, 2021 at 8:27 PM Silvano Cirujano Cuesta <silvano.cir...@siemens.com <mailto:silvano.cir...@siemens.com>> wrote:
> >
> > Hi Vijai,
> >
> > I like your proposal.
> >
> > If we need to provide "something" that keeps modifying the /etc/os-release for backwards compatibility, do we want to keep the current "broken" implementation?
> >
> >
> > I would not suggest we keep the current broken implementation. We should fix it.
> >
> >
> >
> > If we want to "fix" that implementation, then we can do it in a way that not only provides that backwards compatibility, but also to provides a useful feature. The default would be not touching /etc/os-release and one recipe could provide (when requested with a flag) a package for "managing" /etc/os-release (assuming it can be somehow accomplished with multiconfigs, or be a mutually exclusive feature).
> >
> >
> > As Claudius previously mentioned, this might break caching. If the information is purely build-system related, should we try to put it in a package? Does this file need updates in the future? Well, fro"m the current standpoint, I don't see it needing updates. The data is only applicable when we build using ISAR. If hypothetically, we decided to recreate the distro with a cached apt-repo generated with ISAR using some other tool, this info is not relevant.
>
> You possibly miss my explanation why simply placing that file there on post-processing is an error: any package-update of base-files would overwrite that file with the Debian vanilla one. In order to avoid that file from being changed on package updates we need to create either a file diversion or package replacement (like Ubuntu does).
ISAR version, commit info etc.) regardless of where it is put. I do
agree that the current way of changing the content of /etc/os-release
is an error, since now we know that it belongs to base-files and its
update makes that information disappear.
>
> BTW, a package doing only "dkpg-divert" for /etc/os-release doesn't need to contain any version information. Then the version information can be added on image postprocessing. That would be a compromise, because it wouldn't break any package caching.
>
> I don't know exactly how caching works, but I wonder which caching would be breaking just because a package changes. Cached packages? Cached rootfs? Cached images? I would expect almost all rebuilds to be modifying images. I would also expect most of the rebuilds to be modifying the rootfs. Please correct me, if I'm wrong.
>
> >
> > That said, we need to understand from the community, what are the real-life scenarios they are using this ISAR information in /etc/os-release for? I am not personally using that info. If it is being used somewhere, we need to know why? So that we know how costly it is to break backward compatibility. Huge or none at all?
>
> I'm sort of part of that community :-) I get here from a project where we are using ISAR. A provider of us identified the above mentioned issue and fixed it on one of their layers replacing base-files with their own version, what I consider the right fix, but still a fix of something broken in ISAR. But being myself also a (small) contributor of ISAR I decided to address the issue upstream.
>
> Back to your "real-life scenario" question. We started using it because that what the place to identify which base-system release we hat while integrating other components on the system. Moving that information to a file like "/etc/isar-release" would fix it for us, we would simply have to update our integration to fetch the information from that file.
>
> Here start my personal opinion, nothing to do with my "real-time scenario". ISAR is being used in multiple cases to build Debian-derivates [1] and I would expect a Debian-derivative to provide some hint about it in its /etc/os-release. Therefore I would like to see ISAR providing the means to easily accomplish it. But since I seam to be the only one seeing that need, I might try to convince you with a RFC patch :-)
there is no hard and fast rule which describes what is "OS
information". It could be accomplished by modifying base-files. I
would recommend ISAR support something similar to it instead of
bringing in a new package to handle /etc/os-release, which by the way
is redundant.
Thanks,
Vijai Kumar K
Baurzhan Ismagulov
Feb 10, 2021, 4:22:52 AM2/10/21
to isar-users
On Sun, Feb 07, 2021 at 12:31:23AM -0800, vijaikumar....@gmail.com wrote:
> > If we need to provide "something" that keeps modifying the /etc/os-release
> > for backwards compatibility, do we want to keep the current "broken"
> > implementation?
>
> I would not suggest keeping it as it is. We know it is broken. Better to
> > If we need to provide "something" that keeps modifying the /etc/os-release
> > for backwards compatibility, do we want to keep the current "broken"
> > implementation?
>
> fix it.
I suggest to patch /usr/lib/os-release as the first step to preserve the
/etc/os-release symlink (we could provide a patch), then see whether we want to
optionally provide base-files in Isar.
> > If we want to "fix" that implementation, then we can do it in a way that
> > not only provides that backwards compatibility, but also to provides a
> > useful feature. The default would be not touching /etc/os-release and one
> > recipe could provide (when requested with a flag) a package for "managing"
> > /etc/os-release (assuming it can be somehow accomplished with multiconfigs,
> > or be a mutually exclusive feature).
> >
>
> As Claudius previously mentioned, this might break caching. If the
> information is purely build-system related, should we try to put it in a
> standpoint, I don't see it needing updates. The data is only applicable
> when we build using ISAR. If hypothetically, we decided to recreate the
> distro with the cached apt-repo generated with ISAR using some other tool,
> when we build using ISAR. If hypothetically, we decided to recreate the
> this info is not relevant.
Under "caching" I understand base-apt. Providing base-files in Isar shouldn't
break base-apt. Isar-generated base-files would land in isar-apt. Updating that
file again and again with every new build is a normal project lifecycle.
> That said, we need to understand from the community, what are the real-life
> scenarios they are using this ISAR information in /etc/os-release for? I am
> not personally using that info. If it is being used somewhere, we need to
> know why? So that we know how costly it is to break backward compatibility.
> Huge or none at all?
On Tue, Feb 09, 2021 at 11:32:30AM +0530, vijai kumar wrote:
> I do
> agree that the current way of changing the content of /etc/os-release
> is an error, since now we know that it belongs to base-files and its
> update makes that information disappear.
being a bug or not depends IMHO on whether a downstream declares itself a
distro or not. If they don't, let Debian overwrite it -- it isn't an
Isar-generated image anymore. If they do, they should provide their own
base-files.
> Agreed. /etc/os-release is supposed to be extendable. And looks like
> there is no hard and fast rule which describes what is "OS
> information". It could be accomplished by modifying base-files. I
> would recommend ISAR support something similar to it instead of
> bringing in a new package to handle /etc/os-release, which by the way
> is redundant.
modified package would be cleaner than patching the rootfs. That said, my itch
isn't currently that strong to provide a patch :) .
With kind regards,
Baurzhan.
vijaikumar....@gmail.com
Feb 11, 2021, 12:54:39 AM2/11/21
to isar-users
Hi Baurzhan,
Since ISAR is providing that information by default, and people use it based
on upstream trust, we should probably have that information in a way that it
is usable even after an upgrade. Or not provide it at all. I am more inclined to
base-files approach here since we know that /etc/os-release is maintained by it.
Since providing another file is not the recommended way, as per Debian, we
should probably provide the isar information in /etc/os-release through custom
base-files in upstream isar for reference.
Do you feel it is a strong enough reason to have custom base-files in ISAR?
If so, I could try and send a patch for RFC sometime later.
Thanks,
Vijai Kumar K
Baurzhan Ismagulov
Feb 11, 2021, 3:49:35 AM2/11/21
to isar-users
On Wed, Feb 10, 2021 at 09:54:39PM -0800, vijaikumar....@gmail.com wrote:
> Since ISAR is providing that information by default, and people use it based
> on upstream trust, we should probably have that information in a way that it
> is usable even after an upgrade. Or not provide it at all. I am more
> inclined to
> base-files approach here since we know that /etc/os-release is maintained
> by it.
> Since providing another file is not the recommended way, as per Debian, we
> should probably provide the isar information in /etc/os-release through
> custom
> base-files in upstream isar for reference.
>
> Do you feel it is a strong enough reason to have custom base-files in ISAR?
Clean or nothing -- fair enough. I don't have strong feelings that we should
> Since ISAR is providing that information by default, and people use it based
> on upstream trust, we should probably have that information in a way that it
> is usable even after an upgrade. Or not provide it at all. I am more
> inclined to
> base-files approach here since we know that /etc/os-release is maintained
> by it.
> Since providing another file is not the recommended way, as per Debian, we
> should probably provide the isar information in /etc/os-release through
> custom
> base-files in upstream isar for reference.
>
> Do you feel it is a strong enough reason to have custom base-files in ISAR?
provide that information at all :) , but I admit it is handy after we have it.
As long as Isar doesn't become a distro, I'm fine with providing base-files as
it cleans things up, especially long-term (updates).
With kind regards,
Baurzhan.
vijaikumar....@gmail.com
Feb 11, 2021, 5:34:00 AM2/11/21
to isar-users
Ok. In that case, I will try preparing a patch and send it for review sometime
soon.
Thanks,
Vijai Kumar K
With kind regards,
Baurzhan.
Reply all
Reply to author
Forward
0 new messages