FDT format support for the sw-description file

[Paul HENRYS d'AUBIGNY]

Hello all,

I was wondering how relevant could it be to be able to write sw-description files with the device-tree syntax (DTS). The cpio archive for SWUpdate would then embed a DTB describing the images and upgrade information which could be parsed by SWUpdate.

Currently, the sw-description can be written with libconfig and json formats.

The idea behind this would be to be able to embed a sw-description as a device tree in U-Boot binman device tree, so that binman could be used to generate a SWUpdate image and its dependencies (e.g. FIT) at once. Having a common format based on FDT would allow binman to eventually parse the sw-description information to add certain data such as the sha256 of images generated by binman.

Would adding FDT format for the sw-description make sense? Any concern about such a format for the sw-description file?

Thanks.
Paul

[Stefano Babic]

Hi Paul,

On 26.06.24 18:20, Paul HENRYS d'AUBIGNY wrote:
> Hello all,
>
> I was wondering how relevant could it be to be able to write
> sw-description files with the device-tree syntax (DTS). The cpio archive
> for SWUpdate would then embed a DTB describing the images and upgrade
> information which could be parsed by SWUpdate.
> Currently, the sw-description can be written with libconfig and json
> formats.

Well, it will be a third parser for SWUpdate. The question is if it is
needed.

> The idea behind this would be to be able to embed a sw-description as a
> device tree in U-Boot binman device tree, so that binman could be used
> to generate a SWUpdate image and its dependencies (e.g. FIT) at once.
> Having a common format based on FDT would allow binman to eventually
> parse the sw-description information to add certain data such as the
> sha256 of images generated by binman.
>

Well, first I take the chance to explain why DTS was not taken. Even if
I worked a lot with U-Boot, I think that binding SWUpdate with U-Boot is
not a good idea. SWUpdate must support multiple bootloaders, and they
share nothing with U-Boot. And binman itself is not a self running
project, but part of U-Boot. This leads that projects using another
bootloader have dependency with U-Boot, that makes no sense.

Another point is that DTS is, at the end, the language for hardware
description in kernel. If kernel will need some important changes in
DTS, it will be done - without taking into account backward
compatibility. It is not an issue in kernel: a new kernel requiring new
version of DTS is delivered together with DTB.

But in case of SWUpdate, we need to guarantee bakcward compatibility.
Even last release of SWUpdate can install an older SWU (there are just a
couple of exceptions). This is something I saw recently, and a new
SWUpdate can still install a SWU thought for a 2016.10. There is no way
for SWUpdate to guarantee such kind of compatibility with DTS: kernel
will always have priority in such cases.

Another point is if it is possible to convert sw-description from / to
DTS. This is supported today: a libconfig sw-description can be
converted via a tool to JSON, and both sw-descriptions work flawlessly.
With DTS, even this tool should be implemented to have the feature.

It looks to me that support for DTS is just to reuse part of binman -
but well, I can also say that binman can be extended to supportJSON or
libconfig. I do not know which is the added value to use binman, the SWU
is already generated via meta-swupdate (for OE) or SWUgenerator, and of
course sha256 are already generated. So I do not see the advantage to
use binman - and binman should also pack rootfs and other artifacts.
This can work in OE onlz if binman will be extracted from U-Boot and
runs as own project, but it does not seem the case.

> Would adding FDT format for the sw-description make sense?

Please go ahead to explain the advantages, because it seems to me just
rewriting in a different language to have at the end the same features.

> Any concern
> about such a format for the sw-description file?
>

Best regards,
Stefano Babic

[Paul HENRYS d'AUBIGNY]

Hi Stefano,

Thanks for your answer.

Le mer. 26 juin 2024 à 20:44, Stefano Babic
<stefan...@swupdate.org> a écrit :

>
> Hi Paul,
>
> On 26.06.24 18:20, Paul HENRYS d'AUBIGNY wrote:
> > Hello all,
> >
> > I was wondering how relevant could it be to be able to write
> > sw-description files with the device-tree syntax (DTS). The cpio archive
> > for SWUpdate would then embed a DTB describing the images and upgrade
> > information which could be parsed by SWUpdate.
> > Currently, the sw-description can be written with libconfig and json
> > formats.
>
> Well, it will be a third parser for SWUpdate. The question is if it is
> needed.
>
> > The idea behind this would be to be able to embed a sw-description as a
> > device tree in U-Boot binman device tree, so that binman could be used
> > to generate a SWUpdate image and its dependencies (e.g. FIT) at once.
> > Having a common format based on FDT would allow binman to eventually
> > parse the sw-description information to add certain data such as the
> > sha256 of images generated by binman.
> >
>
> Well, first I take the chance to explain why DTS was not taken. Even if
> I worked a lot with U-Boot, I think that binding SWUpdate with U-Boot is
> not a good idea. SWUpdate must support multiple bootloaders, and they
> share nothing with U-Boot. And binman itself is not a self running
> project, but part of U-Boot. This leads that projects using another
> bootloader have dependency with U-Boot, that makes no sense.
>

I agree that SWUpdate should not be bound to U-Boot but I don't think
that would be the case.
The idea would be to provide another mean to generate SWUpdate images.
Since binman can be used to generate several image formats, I was
thinking it could be relevant to also have it being able to generate
SWUpdate images.
Because binman uses FDT as an input to get the information to generate
images, it could allow binman to be able to parse the sw-description
file if required. That's where I mentioned the sha256 calculation
since this value could not be known before all other images are
generated by binman.
DTS format also allows the use of macros and includes and thus it is
possible to have configurable DTS files compiled in one DTS provided
as input to binman. This could also allow the use of template DTS. But
I think configurability and templates files are already supported by
meta_swupdate as far as I could see but for Yocto buildsystem.

> Another point is that DTS is, at the end, the language for hardware
> description in kernel. If kernel will need some important changes in
> DTS, it will be done - without taking into account backward
> compatibility. It is not an issue in kernel: a new kernel requiring new
> version of DTS is delivered together with DTB.
>
> But in case of SWUpdate, we need to guarantee bakcward compatibility.
> Even last release of SWUpdate can install an older SWU (there are just a
> couple of exceptions). This is something I saw recently, and a new
> SWUpdate can still install a SWU thought for a 2016.10. There is no way
> for SWUpdate to guarantee such kind of compatibility with DTS: kernel
> will always have priority in such cases.
>
> Another point is if it is possible to convert sw-description from / to
> DTS. This is supported today: a libconfig sw-description can be
> converted via a tool to JSON, and both sw-descriptions work flawlessly.
> With DTS, even this tool should be implemented to have the feature.
>

That would indeed be extra work.

> It looks to me that support for DTS is just to reuse part of binman -
> but well, I can also say that binman can be extended to supportJSON or
> libconfig. I do not know which is the added value to use binman, the SWU
> is already generated via meta-swupdate (for OE) or SWUgenerator, and of
> course sha256 are already generated. So I do not see the advantage to
> use binman - and binman should also pack rootfs and other artifacts.
> This can work in OE onlz if binman will be extracted from U-Boot and
> runs as own project, but it does not seem the case.
>

Correct if my understanding is wrong, meta-swupdate provides the
support of templates and configurability but it's only for Yocto
buildsystem. SWUgenerator also provides configurability providing a
config file with variables but there is no template support.
I am looking for the best way to have templates that can be used to
generate a sw-description and swupdate image and avoid having too big
sw-description files.
This being said, I need to think about it but this config file with
SWUgenerator could be sufficient.
By the way, I do agree that having binman in U-Boot is a limitation
today to make a more generic use of it.

> > Would adding FDT format for the sw-description make sense?
>
> Please go ahead to explain the advantages, because it seems to me just
> rewriting in a different language to have at the end the same features.
>

As I wrote above but maybe that is not a valid reason, I like the DTS
approach which makes it possible to split the sw-description in sub
DTS files included in the "main" sw-description file.

[Stefano Babic]

Hi Paul,

But the construct is quite forced: because binman uses FDT as input,
consequence is to use FDT in SWUpdate. But again, binman could then be
extended to understand libconfig...

> DTS format also allows the use of macros and includes and thus it is
> possible to have configurable DTS files compiled in one DTS provided
> as input to binman.

includes is something missing here, sure. They are managed by the dtc
compiler itself. One further advantage is that DTS are compiled, and the
DTB is smaller than source. But well, this is not very relevant because
a SWU is nowadays big enough to make the size of sw-description negligible.

> This could also allow the use of template DTS. But
> I think configurability and templates files are already supported by
> meta_swupdate as far as I could see but for Yocto buildsystem.

Yes

Right, as the name suggest, it is a layer for OE.

> SWUgenerator also provides configurability providing a
> config file with variables but there is no template support.
> I am looking for the best way to have templates that can be used to
> generate a sw-description and swupdate image and avoid having too big
> sw-description files.

SWUgenerator is thought to be used for all that is not OE, so Buildroot
and Debian for example. So instead of adding a new language for the
parser, it makes more sense for me to add the missing features to the
current generators. The config file with vars is thought to help to have
templates: you have just one sw-description and multiple config files
for any single build, and this matches what is done in meta-swupdate.

> This being said, I need to think about it but this config file with
> SWUgenerator could be sufficient.
> By the way, I do agree that having binman in U-Boot is a limitation
> today to make a more generic use of it.
>

Yes

>>> Would adding FDT format for the sw-description make sense?
>>
>> Please go ahead to explain the advantages, because it seems to me just
>> rewriting in a different language to have at the end the same features.
>>
> As I wrote above but maybe that is not a valid reason, I like the DTS
> approach which makes it possible to split the sw-description in sub
> DTS files included in the "main" sw-description file.

Then it seems the real missing part is having includes in
sw-description. Instead of adding a new format, it is maybe worth to add
the missing features to the current generators.

Best regards,
Stefano Babic

[Stefano Babic]

Hi Paul,

please never drop the ML from your answers - thanks.

On 01.07.24 09:48, Paul HENRYS d'AUBIGNY wrote:
> Hi Stefano,
>

> Le jeu. 27 juin 2024 à 08:39, Stefano Babic

> Indeed and it actually seems libconfig itself already supports
> including files using "@include" directive. I will take a deeper look
> at SWUgenerator as it looks like the way to go.

In any case, it is required that just one single sw-description file is
packed into the SWU, because this is what is signed and verified by
SWUpdate. SWUgenerator reads and writes back sw-description, so this is
maybe something (I haven't tested) working out of the box, while
meta-swupdate can't support it.

Best regards,
Stefano Babic


> Thanks for your feedback and advise.