Delta updates with hawkbit?

[mattwo...@gmail.com]

Hi all,

Its not clear to me if delta updates with hawkbit is possible. I
found this article which seems to imply no due to authorization header
requirements for the management API:
https://www.thegoodpenguin.co.uk/blog/delta-ota-update-with-swupdate/

Has anyone made delta updates work with hawkbit?

Thanks, Matt.

[Stefano Babic]

Hi Matt,

(maybe added article's author in discussion):

Hawkbit splits between the fleet management service and the artifact
repository. File repository on Hawkbit Server itself is just the default
way for the repositories.

Frankly speaking, I find tricky the way explained by the author to use
the management interface. Please check Hawkbit's concepts and
description of management interface in Hawkbit's documentation, for
example here:

https://www.eclipse.org/hawkbit/ui/

A device is never supposed to communicate with Hawkbit via the
Management Interface, but it should use the DDI interface, and this is
what is implemented in SWUpdate. SWUpdate won't support the
authorization header for Management Interface because it should never
use it to communicate with the server.

As concept for delta update, SWUpdate requires the SWU and a URL for
each artifact that requires delta update, in format of ZCK file. In the
simplest case this is just one file (rootfs), but it can be many files
(for example with containers). URLs can belong to different repositories
/ different servers. The URLs are defined at build time, because they
are part of the SWU and then signed. On the other side, Hawkbit adds
dynamically the artifacts into its database with a software module ID,
that cannot be known at build time, and the URL contains this ID. We
cannot exclude even that rule / way to set the ID could change in
future, just the API is guaranteed by Hawkbit's developers to be
maintained. This makes dangerous to bind these URLs and the way to get
them from what Hawkbit is doing.

So from start, the repository to store the ZCK artifacts is not bound to
Hawkbit, and can be any reachable URL (supporting byte request range, of
course). Nobody prohibits to set up such as server on the same machine
of Hawkbit, but this is generally not requested.

The Hawkbit Server is often located on the cloud whose costs depend on
the generated traffic, and moving the artifact repos outside allows to
find for a competitor with the cheapest solution. That means it is
possible from release to release to put the artifacts on another server,
if this reduces costs, but older releases could still be deployed. With
delta update, the traffic generated on the Hawkbit server can become
negligible, because it could just contain more or less meta data (ZCH
headers) information.

It is not clear to me what is the intention and possible security issues
mentioned by author with " we located it on publicly accessible server –
but of course this isn’t recommended for production due to security
concerns.". The Hawkbit itself is a "public" server as well, and you
know that you can set up a reverse proxy if you want to limit the
access. Nevertheless, the server is not in SWUpdate's trust of chain. It
could be compromised, too, without compromising the device. SWUpdate
will detect any manipulation, and this is for the ZCK (delta) files,
too, even if it is not deployed with SWU.

After these considerations, I do not say it could not be possible to
bind in some way the repo with Hawkbit, but this should be done
exclusively using the DDI interface, for example by retrieving and
parsing the list of software modules and in one of them URLs are
provided, but this was up now never asked. Splitting the artifact repos
from the main fleet management server seems a much more flexible solution.

Best regards,
Stefano

--
=====================================================================
DENX Software Engineering GmbH, Managing Director: Erika Unter
HRB 165235 Munich, Office: Kirchenstr.5, 82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de
=====================================================================

[Andrew Murray]

Hi Stefano, Matt,

On Sat, 4 Feb 2023 at 11:14, Stefano Babic <sba...@denx.de> wrote:
>
> Hi Matt,
>
> (maybe added article's author in discussion):

Thanks for adding me in :)

>
> On 03.02.23 23:09, mattwo...@gmail.com wrote:
> > Hi all,
> >
> > Its not clear to me if delta updates with hawkbit is possible. I
> > found this article which seems to imply no due to authorization header
> > requirements for the management API:
> > https://www.thegoodpenguin.co.uk/blog/delta-ota-update-with-swupdate/
> >
> > Has anyone made delta updates work with hawkbit?
>
> Hawkbit splits between the fleet management service and the artifact
> repository. File repository on Hawkbit Server itself is just the default
> way for the repositories.
>
> Frankly speaking, I find tricky the way explained by the author to use
> the management interface. Please check Hawkbit's concepts and
> description of management interface in Hawkbit's documentation, for
> example here:
>
> https://www.eclipse.org/hawkbit/ui/
>
> A device is never supposed to communicate with Hawkbit via the
> Management Interface, but it should use the DDI interface, and this is
> what is implemented in SWUpdate. SWUpdate won't support the
> authorization header for Management Interface because it should never
> use it to communicate with the server.

For context, when we were testing delta updates with Hawkbit we stored
the zck on a 'publicly accessible server' - by which we meant just a
plain old HTTP server that was accessible locally by the device and
without any authentication - such that we could hard code it's URL in
the url parameter of sw-description file.

We identified that when you upload artifacts via the Hawkbit UI they
were accessible via the RESTful management API, hence the suggestion
in the blog post that this could be considered for use. But as you and
the documentation points out this isn't intended for consumption by
the device and requires HTTP Basic Authentication which isn't
supported by SWupdate. (But we didn't understand all of this at the
time of writing).

>
> As concept for delta update, SWUpdate requires the SWU and a URL for
> each artifact that requires delta update, in format of ZCK file. In the
> simplest case this is just one file (rootfs), but it can be many files
> (for example with containers). URLs can belong to different repositories
> / different servers. The URLs are defined at build time, because they
> are part of the SWU and then signed. On the other side, Hawkbit adds
> dynamically the artifacts into its database with a software module ID,
> that cannot be known at build time, and the URL contains this ID. We
> cannot exclude even that rule / way to set the ID could change in
> future, just the API is guaranteed by Hawkbit's developers to be
> maintained. This makes dangerous to bind these URLs and the way to get
> them from what Hawkbit is doing.

You are referring to the URL's exchanged across DDI right? Which
currently look a bit like this?

/DEFAULT/controller/v1/site/softwaremodules/3/artifacts/image.swu

>
> So from start, the repository to store the ZCK artifacts is not bound to
> Hawkbit, and can be any reachable URL (supporting byte request range, of
> course). Nobody prohibits to set up such as server on the same machine
> of Hawkbit, but this is generally not requested.

If someone wanted to put these artifacts on a generic server, does
SWUpdate provide any support for authentication/authorisation? I.e. I
can imagine people may want this in an S3 bucket, etc?

I think it's a reasonable assumption to want to put any ZCK artifacts
alongside the SWUs within the control of Hawkbit.

>
> The Hawkbit Server is often located on the cloud whose costs depend on
> the generated traffic, and moving the artifact repos outside allows to
> find for a competitor with the cheapest solution. That means it is
> possible from release to release to put the artifacts on another server,
> if this reduces costs, but older releases could still be deployed. With
> delta update, the traffic generated on the Hawkbit server can become
> negligible, because it could just contain more or less meta data (ZCH
> headers) information.
>
> It is not clear to me what is the intention and possible security issues
> mentioned by author with " we located it on publicly accessible server –
> but of course this isn’t recommended for production due to security
> concerns.". The Hawkbit itself is a "public" server as well, and you
> know that you can set up a reverse proxy if you want to limit the
> access. Nevertheless, the server is not in SWUpdate's trust of chain. It
> could be compromised, too, without compromising the device. SWUpdate
> will detect any manipulation, and this is for the ZCK (delta) files,
> too, even if it is not deployed with SWU.

The blog post wasn't clear here - by security concerns I meant you may
not want your binaries on a server that anyone can download via a
public URL without authentication - which was the case for our
testing.

>
> After these considerations, I do not say it could not be possible to
> bind in some way the repo with Hawkbit, but this should be done
> exclusively using the DDI interface, for example by retrieving and
> parsing the list of software modules and in one of them URLs are
> provided, but this was up now never asked. Splitting the artifact repos
> from the main fleet management server seems a much more flexible solution.

After writing the post, we did make further progress such that we were
able to download the zck via the DDI. Our approach looked like this:

- Make a note of the URL of the artifact downloaded in
server_process_update_artifact (e.g. url/rootfs.ext4) - thus allowing
us to infer the URL to the location of the directory (software module)
containing the SWU and any other artifacts.
- Insider the delta handler (delta_retrieve_attributes) if the URL if
the zck file is a specific identifier/token, then we substitute it for
the URL we previously made note of, but with the additional .zck
suffix (e.g. url/rootfs.ext4.zck)
- This allows us to store the ext4.zck in Hawkbit along side the .ext4
and obtain a URL for it (as part of the same software module).

However the delta_downloader.c doesn't support passing an
Authorisation token - so we needed to add that (i.e.
channel_data_defaults.auth_token), along with a new chunks_downloader
module settings for providing the gateway token.

And finally, we noticed that the Hawkbit server doesn't conform to the
multi-part RFC which says "The boundary must be followed immediately
either by another CRLF and the header fields for the next part, or by
two CRLFs, in which case there are no header fields for the next part
(and it is therefore assumed to be of Content-Type text/plain)" - this
causes multipart handling to fail - we made a fix for that. Also
SWUpdate assumes that the multi-part boundary separator is lowercase,
however Hawkbit outputs an upper case one! (Happy to share any of
those fixes).

We got it to work, but with the limited time we had the solution
wasn't elegant or generic and it makes many assumptions.

Going forward, I think there are some questions open with regards to
configuration:

- How does the user specify in the sw-description the location of the
zck that is relative to the location of the swu obtained via the DDI?
- How does the user specify authorization tokens for it - presumably
use the same one used for the DDI?
- When the zck is not located on a hawkbit server, how does the user
specify authorization information?

Thanks,

Andrew Murray

[Stefano Babic]

Hi Andrew,

On 04.02.23 17:04, Andrew Murray wrote:
> Hi Stefano, Matt,
>
> On Sat, 4 Feb 2023 at 11:14, Stefano Babic <sba...@denx.de> wrote:
>>
>> Hi Matt,
>>
>> (maybe added article's author in discussion):
>
> Thanks for adding me in :)

You're welcome :-)

Ok, understood. The point is not so clear in the article.

>
> We identified that when you upload artifacts via the Hawkbit UI they
> were accessible via the RESTful management API, hence the suggestion
> in the blog post that this could be considered for use. But as you and
> the documentation points out this isn't intended for consumption by
> the device and requires HTTP Basic Authentication which isn't
> supported by SWupdate. (But we didn't understand all of this at the
> time of writing).

Ok

>
>
>>
>> As concept for delta update, SWUpdate requires the SWU and a URL for
>> each artifact that requires delta update, in format of ZCK file. In the
>> simplest case this is just one file (rootfs), but it can be many files
>> (for example with containers). URLs can belong to different repositories
>> / different servers. The URLs are defined at build time, because they
>> are part of the SWU and then signed. On the other side, Hawkbit adds
>> dynamically the artifacts into its database with a software module ID,
>> that cannot be known at build time, and the URL contains this ID. We
>> cannot exclude even that rule / way to set the ID could change in
>> future, just the API is guaranteed by Hawkbit's developers to be
>> maintained. This makes dangerous to bind these URLs and the way to get
>> them from what Hawkbit is doing.
>
> You are referring to the URL's exchanged across DDI right? Which
> currently look a bit like this?
>
> /DEFAULT/controller/v1/site/softwaremodules/3/artifacts/image.swu

Right.

>
>
>>
>> So from start, the repository to store the ZCK artifacts is not bound to
>> Hawkbit, and can be any reachable URL (supporting byte request range, of
>> course). Nobody prohibits to set up such as server on the same machine
>> of Hawkbit, but this is generally not requested.
>
> If someone wanted to put these artifacts on a generic server, does
> SWUpdate provide any support for authentication/authorisation?

This can be done via reverse proxy, see previous Matt's thread. SWUpdate
passes to libcurl key and certificate for the connection if provided.
This is also the way preferred in Hawkbit (token and in the specific
case gateway token are considered for development / testing) to
authenticate the device. So or there is noauthentication at all (plug
and play mode), or device should received own key and cetificate for the
connection.

> I.e. I
> can imagine people may want this in an S3 bucket, etc?
>

Well, Hawkbit supports artifacts on S3 bucket, see hawkbit extensions
for artifactory.

But really people want to update with and without the Hawkbit server.
SUpdate makes this independently from the way you update, and the same
SWU (with or without delta) can be stored on Hawkbit, pushed to the
device via the Webserver or locally via USB storage, or whatever.

> I think it's a reasonable assumption to want to put any ZCK artifacts
> alongside the SWUs within the control of Hawkbit.

As I said, there are extensions, and specific for S3 bucket and
Microsoft Azure. But people want to use any kind of cloud service, and
to integrate this in Hawkbit means that a new extension must be written
(and integrated into Hawkbit project).

>
>
>>
>> The Hawkbit Server is often located on the cloud whose costs depend on
>> the generated traffic, and moving the artifact repos outside allows to
>> find for a competitor with the cheapest solution. That means it is
>> possible from release to release to put the artifacts on another server,
>> if this reduces costs, but older releases could still be deployed. With
>> delta update, the traffic generated on the Hawkbit server can become
>> negligible, because it could just contain more or less meta data (ZCH
>> headers) information.
>>
>> It is not clear to me what is the intention and possible security issues
>> mentioned by author with " we located it on publicly accessible server –
>> but of course this isn’t recommended for production due to security
>> concerns.". The Hawkbit itself is a "public" server as well, and you
>> know that you can set up a reverse proxy if you want to limit the
>> access. Nevertheless, the server is not in SWUpdate's trust of chain. It
>> could be compromised, too, without compromising the device. SWUpdate
>> will detect any manipulation, and this is for the ZCK (delta) files,
>> too, even if it is not deployed with SWU.
>
> The blog post wasn't clear here

Maybe time to update the post ;-)

> - by security concerns I meant you may
> not want your binaries on a server that anyone can download via a
> public URL without authentication - which was the case for our
> testing.

Ok, this is clear now. I will say that this is generally not the main
concern (yes, it is requested, too, but with lower priority). The main
request is that a not verified SWU is not allowed to be installed on the
device. The SWU generally can be read, and the Hawkbit Server could be
compromised, too. If there are concerns to make the SWU public, the SWU
itself should be encrypted (and SWUpdate supports this).

And as I explained above, customers have often more ways to update the
devices - fleet management is one case, but the same device could be
updated in another way, even offline via USB pen. SWUpdate is
transparent how the software is reaching the device.

>
>
>>
>> After these considerations, I do not say it could not be possible to
>> bind in some way the repo with Hawkbit, but this should be done
>> exclusively using the DDI interface, for example by retrieving and
>> parsing the list of software modules and in one of them URLs are
>> provided, but this was up now never asked. Splitting the artifact repos
>> from the main fleet management server seems a much more flexible solution.
>
> After writing the post, we did make further progress such that we were
> able to download the zck via the DDI. Our approach looked like this:
>
> - Make a note of the URL of the artifact downloaded in
> server_process_update_artifact (e.g. url/rootfs.ext4) - thus allowing
> us to infer the URL to the location of the directory (software module)
> containing the SWU and any other artifacts.

ok - but this makes probably the SWU not working outside Hawkbit.
Nevertheless, I agree it is a use case.

> - Insider the delta handler (delta_retrieve_attributes) if the URL if
> the zck file is a specific identifier/token, then we substitute it for
> the URL we previously made note of, but with the additional .zck
> suffix (e.g. url/rootfs.ext4.zck)

Probably I prefer that this happens independently from the delta
handler. The delta handler receives a URL via a "property" - if URL
should be replaced, this should happen when sw-description is parsed and
/ or suitable Lua extension.

> - This allows us to store the ext4.zck in Hawkbit along side the .ext4
> and obtain a URL for it (as part of the same software module).
>

Ok, I understand that URL should be replaced in some way, it is to find
which is the best way. My concern is that if the URL is provided as URL
of an additional file of the same software module, this is not signed
and verified. Nothing terrible, SWUpdate is still able to recognize a
malformed or modified ZCK file, but maybe not the best.

> However the delta_downloader.c doesn't support passing an
> Authorisation token - so we needed to add that (i.e.
> channel_data_defaults.auth_token),

Right, it was never asked. And again, the delta handler is thought to
work with any server supporting byte range request, not only hawkbit.
But the connection can be verified if private key / certificate are
passed for the SSL connection.

> along with a new chunks_downloader
> module settings for providing the gateway token.
>
> And finally, we noticed that the Hawkbit server doesn't conform to the
> multi-part RFC which says "The boundary must be followed immediately
> either by another CRLF and the header fields for the next part, or by
> two CRLFs, in which case there are no header fields for the next part
> (and it is therefore assumed to be of Content-Type text/plain)" - this
> causes multipart handling to fail - we made a fix for that.

Fine - does it mean you push the fix to the Hawkbit project to be
integrated into mainline ?

> Also
> SWUpdate assumes that the multi-part boundary separator is lowercase,
> however Hawkbit outputs an upper case one! (Happy to share any of
> those fixes).

Please share.

>
> We got it to work, but with the limited time we had the solution
> wasn't elegant or generic and it makes many assumptions.
>
> Going forward,

Right - there are space for improvement.

> I think there are some questions open with regards to
> configuration:
>
> - How does the user specify in the sw-description the location of the
> zck that is relative to the location of the swu obtained via the DDI?

My current use cases and customer's request was to split the artifact
repository from the Hawkbit server, and the URL is fixed and defined at
build time.

If companies will require this, a way should be found, just probably
made generic instead of the way you described before. But I agree, the
information is coming from DDI as "links" in the deployment info answer.

> - How does the user specify authorization tokens for it - presumably
> use the same one used for the DDI?

Yes

> - When the zck is not located on a hawkbit server, how does the user
> specify authorization information?

Key and certificate, no authorization token is currently provided.

[Matt Wood]

Hi Andrew and Stefano,

thanks so much for all the explanations.

I believe I have a solution for the artifact storage in hawkbit without using the management API but I'm running into an error when suricatta picks up the swu.

Its complaining about a syntax error in the sw-description and that the JSON file is corrupted.

Does anyone have a full sw-description file that has been tested they can share?

Thanks, Matt.

[Stefano Babic]

On 07.02.23 20:58, Matt Wood wrote:
> Hi Andrew and Stefano,
>
> thanks so much for all the explanations.
>
> I believe I have a solution for the artifact storage in hawkbit without using the management API

Ok - can you explaining us what you did ?

>but I'm running into an error when suricatta picks up the swu.
>
> Its complaining about a syntax error in the sw-description and that the JSON file is corrupted.

A syntax error is a syntax error - that means that any software loading
the file and using libconfig will complain. The "JSON file is corrupted"
is just because SWUpdate tries to run any suitable parser (libconfig ==>
JSON), when the first fails, and of course your sw-description is not in
JSON format. The error is in your sw-description (parenthesis, maybe ?),
you can post it for review.

>
> Does anyone have a full sw-description file that has been tested they can share?
>

It does not help, your sw-description is buggy, it is sometimes tricky
to find it. libconfig just reports a "syntax error" and the line where
it is found, but no further reasons.

Best regards,
Stefano Babic

[Matt Wood]

On 2/8/23 07:15, Stefano Babic wrote:
> On 07.02.23 20:58, Matt Wood wrote:
>> Hi Andrew and Stefano,
>>
>> thanks so much for all the explanations.
>>
>> I believe I have a solution for the artifact storage in hawkbit without using the management API
>
> Ok - can you explaining us what you did ?

As soon as I confirm things work I'll post my findings.

>
>> but I'm running into an error when suricatta picks up the swu.
>>
>> Its complaining about a syntax error in the sw-description and that the JSON file is corrupted.
>
> A syntax error is a syntax error - that means that any software loading the file and using libconfig will complain. The "JSON file is corrupted" is just because SWUpdate tries to run any suitable parser (libconfig ==> JSON), when the first fails, and of course your sw-description is not in JSON format. The error is in your sw-description (parenthesis, maybe ?), you can post it for review.
>

This is my sw-descriptin file:

{
filename = "delta.header";
type = "delta";
device = "/dev/mmcblk0p2";
properties: {
url = "https://example.com/DEFAULT/controller/v1/sama5d27-wlsom1-ek-sd/softwaremodules/6/artifacts/delta.zck";
chain = "raw";
source = "/dev/mmcblk0p3";
};
}

And the command to re-pack:

ls -f sw-description delta.header emmcsetup.lua | cpio -ov -H crc > ../delta.swu

Thanks, Matt.

[Matt Wood]

Updated sw-description below. It still fails.

On 2/8/23 09:33, Matt Wood wrote:
>
>
> On 2/8/23 07:15, Stefano Babic wrote:
>> On 07.02.23 20:58, Matt Wood wrote:
>>> Hi Andrew and Stefano,
>>>
>>> thanks so much for all the explanations.
>>>
>>> I believe I have a solution for the artifact storage in hawkbit without using the management API
>>
>> Ok - can you explaining us what you did ?
>
> As soon as I confirm things work I'll post my findings.
>>
>>> but I'm running into an error when suricatta picks up the swu.
>>>
>>> Its complaining about a syntax error in the sw-description and that the JSON file is corrupted.
>>
>> A syntax error is a syntax error - that means that any software loading the file and using libconfig will complain. The "JSON file is corrupted" is just because SWUpdate tries to run any suitable parser (libconfig ==> JSON), when the first fails, and of course your sw-description is not in JSON format. The error is in your sw-description (parenthesis, maybe ?), you can post it for review.
>>
>

software =
{
version = "0.1.0";
description = "Delta Update Test";

hardware-compatibility: [ "1.0" ];

[Stefano Babic]

On 08.02.23 15:33, Matt Wood wrote:
>
>
> On 2/8/23 07:15, Stefano Babic wrote:
>> On 07.02.23 20:58, Matt Wood wrote:
>>> Hi Andrew and Stefano,
>>>
>>> thanks so much for all the explanations.
>>>
>>> I believe I have a solution for the artifact storage in hawkbit without using the management API
>>
>> Ok - can you explaining us what you did ?
>
> As soon as I confirm things work I'll post my findings.
>>
>>> but I'm running into an error when suricatta picks up the swu.
>>>
>>> Its complaining about a syntax error in the sw-description and that the JSON file is corrupted.
>>
>> A syntax error is a syntax error - that means that any software loading the file and using libconfig will complain. The "JSON file is corrupted" is just because SWUpdate tries to run any suitable parser (libconfig ==> JSON), when the first fails, and of course your sw-description is not in JSON format. The error is in your sw-description (parenthesis, maybe ?), you can post it for review.
>>
>
> This is my sw-descriptin file:
>
> {
> filename = "delta.header";
> type = "delta";
> device = "/dev/mmcblk0p2";
> properties: {
> url = "https://example.com/DEFAULT/controller/v1/sama5d27-wlsom1-ek-sd/softwaremodules/6/artifacts/delta.zck";
> chain = "raw";
> source = "/dev/mmcblk0p3";
> };
> }
>

Well, I guess this is just a part of it, but if I write the whole file as:

software = {

hardware-compatibility = ["1.0"];
version = "1.0";
images: (

{
filename = "delta.header";
type = "delta";
device = "/dev/mmcblk0p2";
properties: {
url =
"https://example.com/DEFAULT/controller/v1/sama5d27-wlsom1-ek-sd/softwaremodules/6/artifacts/delta.zck";
chain = "raw";
source = "/dev/mmcblk0p3";
}

})
}

And I test in dry-run:

swupdate -l 5 -Hpc:1.0 --dry-run -i /tmp/delta.swu

File is parsed without errors:

> And the command to re-pack:
>
> ls -f sw-description delta.header emmcsetup.lua | cpio -ov -H crc > ../delta.swu
>
> Thanks, Matt.

Best regards,
Stefano

[Matt Wood]

Hi Stefano,

Strange, I tried that same command and got the errors below. I'm using the 2022.12 tag from github.

[ERROR] : SWUPDATE failed [0] ERROR parser.c : parser : 961 : Found nothing to install
[DEBUG] : SWUPDATE running : [parse_json] : Parsing config file /tmp/sw-description
[ERROR] : SWUPDATE failed [0] ERROR parser.c : parse_json : 1054 : JSON File corrupted
[ERROR] : SWUPDATE failed [0] ERROR parser.c : parse : 164 : no parser available to parse sw-description!
[ERROR] : SWUPDATE failed [0] ERROR stream_interface.c : extract_files : 182 : Compatible SW not found
swupdate_image_write failed: Connection reset by peer
[ERROR] : SWUPDATE failed [0] ERROR install_from_file.c : endupdate : 55 : SWUpdate *failed* !

The first error is interesting. I added a package to the build to ensure there was a filesystem difference.

Any other thoughts on where the problem might be?

Thanks, Matt.

[mattwo...@gmail.com]

Hi Stefano and Andrew,

I was able to get swupdate to recognize the delta swu. It was indeed
an issue with my sw-description file.

But, I'm still not able to successfully update from a delta file.

For testing I added iperf3 package to my build, and the output of
zck_cmp_uncomp shows:
Total to be reused : 704309573
Total to be downloaded : 2766422

But when I run the update it fails and says I ran out of space, see below.

Its not clear to me why this is happening. Can you provide some insight?

Thanks, Matt

[TRACE] : SWUPDATE running : [zck_log_toswupdate] : (finish_chunk)
Index 2424 digest
1ba1caa4138fb76dd1aa54ea8dfa14db58314cb5f09a4be81b487b16f4638dec
digest uncomp 1ba1caa4138fb76dd1aa54ea8dfa14db58314cb5f09a4be81b487b16f4638dec
[ERROR] : SWUPDATE failed [0] ERROR : (write_data) Error writing data:
No space left on device
[ERROR] : SWUPDATE failed [0] ERROR : ZCK returns Error writing data:
No space left on device

[WARN ] : SWUPDATE running : [install_delta] : ZCK Header form
/dev/mmcblk0p3 cannot be created, fallback to full download
[INFO ] : SWUPDATE running : [get_total_size] : Total bytes to be
reused : 0

[INFO ] : SWUPDATE running : [get_total_size] : Total bytes to be
downloaded : 56328473

[INFO ] : SWUPDATE running : [install_delta] : Size of artifact to be
installed : 722872320
[TRACE] : SWUPDATE running : [install_single_image] : Found installer
for stream delta.header raw
[TRACE] : SWUPDATE running : [trigger_download] : Range request :
509556-56838028
Child 286(chunks_downloader) killed by signal 11

[MANUEL DURAN]

Hello everyone,

I have some errors like you are talking about:

[TRACE] : SWUPDATE running :  [add_properties] : Found properties for output.zck .header:
[TRACE] : SWUPDATE running :  [add_properties_cb] :             Property url: http://192.168.1.2/output.zck
[TRACE] : SWUPDATE running :  [add_properties_cb] :             Property chain: raw
[TRACE] : SWUPDATE running :  [add_properties_cb] :             Property source: /dev/mmcblk0p3
[TRACE] : SWUPDATE running :  [add_properties_cb] :             Property zcklogl evel: error
[TRACE] : SWUPDATE running :  [_parse_images] : Found Image: output.zck.header in device : /dev/mmcblk0p2 for handler delta
[ERROR] : SWUPDATE failed [0] ERROR : feature 'delta' required for 'output.zck.header' in sw-description is absent!
[ERROR] : SWUPDATE failed [0] ERROR : Compatible SW not found
[ERROR] : SWUPDATE failed [1] Image invalid or corrupted. Not installing ...

I followed documentation:

my sw-description file includes something like this:

images: (
                  {
                              filename = "output.zck.header";      
                              device = "/dev/mmcblk0p2";
                              type = "delta";
                                        properties: {
                                        url = "http://192.168.1.2/output.zck";

                                        chain = "raw";
                                        source = "/dev/mmcblk0p3";

                                        zckloglevel = "error";
                                       /* debug-chunks = "true"; */
                          };
                   }
      );

I add my outpuz.zck file into my http server. (output.zck is the core-image-minimal which I divided with ZCHUNK)

I also configured CONFIG_DELTA=y and ZSTD...=y in my defconfig file. But I always get the error:      delta required for output.zck header in sw-description is absent!

I tried to download an .swu file from my http server in order to test download funcionality and it works properly.

can anyone help me with this error?

Thanks!!

Manuel

[Stefano Babic]

On 14.02.23 12:27, MANUEL DURAN wrote:
> Hello everyone,
>
> I have some errors like you are talking about:
>
> [TRACE] : SWUPDATE running :  [add_properties] : Found properties for
> output.zck .header:
> [TRACE] : SWUPDATE running :  [add_properties_cb] :             Property
> url: http://192.168.1.2/output.zck
> [TRACE] : SWUPDATE running :  [add_properties_cb] :             Property
> chain: raw
> [TRACE] : SWUPDATE running :  [add_properties_cb] :             Property
> source: /dev/mmcblk0p3
> [TRACE] : SWUPDATE running :  [add_properties_cb] :             Property
> zcklogl evel: error
> [TRACE] : SWUPDATE running :  [_parse_images] : Found Image:
> output.zck.header in device : /dev/mmcblk0p2 for handler delta

> *[ERROR] : SWUPDATE failed [0] ERROR : feature 'delta' required for
> 'output.zck.header' in sw-description is absent!*

> [ERROR] : SWUPDATE failed [0] ERROR : Compatible SW not found
> [ERROR] : SWUPDATE failed [1] Image invalid or corrupted. Not installing ...
>
> I followed documentation:
>
> my sw-description file includes something like this:
>
> images: (
>                   {
>                               filename = "output.zck.header";
>                               device = "/dev/mmcblk0p2";

> *  type = "delta";*

>                                         properties: {
>                                         url =
> "http://192.168.1.2/output.zck";
>                                         chain = "raw";
>                                         source = "/dev/mmcblk0p3";
>                                         zckloglevel = "error";
>                                        /* debug-chunks = "true"; */
>                           };
>                    }
>       );
> I add my outpuz.zck file into my http server. (output.zck is the
> core-image-minimal which I divided with ZCHUNK)
>
> I also configured CONFIG_DELTA=y and ZSTD...=y in my defconfig file. But
> I always get the error:      delta required for output.zck header in
> sw-description is absent!
>
> I tried to download an .swu file from my http server in order to test
> download funcionality and it works properly.
>
> can anyone help me with this error?

Missing CONFIG_DELTA when you build swupdate.

Best regards,
Stefano Babic

> "https://example.com/DEFAULT/controller/v1/sama5d27-wlsom1-ek-sd/softwaremodules/6/artifacts/delta.zck <https://example.com/DEFAULT/controller/v1/sama5d27-wlsom1-ek-sd/softwaremodules/6/artifacts/delta.zck>";

> > >> chain = "raw";
> > >> source = "/dev/mmcblk0p3";
> > >> };
> > >> }
> > >>
> > >
> > > Well, I guess this is just a part of it, but if I write the
> whole file as:
> > >
> > > software = {
> > >
> > > hardware-compatibility = ["1.0"];
> > > version = "1.0";
> > > images: (
> > > {
> > > filename = "delta.header";
> > > type = "delta";
> > > device = "/dev/mmcblk0p2";
> > > properties: {
> > > url =

> "https://example.com/DEFAULT/controller/v1/sama5d27-wlsom1-ek-sd/softwaremodules/6/artifacts/delta.zck <https://example.com/DEFAULT/controller/v1/sama5d27-wlsom1-ek-sd/softwaremodules/6/artifacts/delta.zck>";

> https://www.thegoodpenguin.co.uk/blog/delta-ota-update-with-swupdate/ <https://www.thegoodpenguin.co.uk/blog/delta-ota-update-with-swupdate/>

> --
> You received this message because you are subscribed to the Google
> Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to swupdate+u...@googlegroups.com
> <mailto:swupdate+u...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/swupdate/c4d4e619-dc5f-47b4-a9da-c373cb70148an%40googlegroups.com <https://groups.google.com/d/msgid/swupdate/c4d4e619-dc5f-47b4-a9da-c373cb70148an%40googlegroups.com?utm_medium=email&utm_source=footer>.

[MANUEL DURAN]

Hello everyone!

Thanks Stefano for your reply. Now it works!

But I have a really  important question:

OK, I have a rootfs loaded in partition2. From this, I want to "delta-update" the rootfs of the partition three with another very similar, that is in a valid http server. There are differences such as hostname, and other minimal things.... This is only to test delta update.

Ok, when I run in my shell: swupdate -i file.swu -e stable,copyX, I always get  this:

[TRACE] : SWUPDATE running :  [install_delta] : ZCK Header read successfully from SWU, creating header from /dev/mmcblk0p3

[ERROR] : SWUPDATE failed [0] ERROR : ZCK returns Error writing data: No space left on device

 [WARN ] : SWUPDATE running :  [install_delta] : ZCK Header form /dev/mmcblk0p3 cannot be created, fallback to full download

It is always downloading all the chunks

If i try df -h,     I can see there are a lot of free space in both partitions, more than 500MB, and both rootfs weight 40MB. So please, can someone tell me what is going on here?

I am really confused because I know that I have a lot of free space, so why swupdate is always downloading all the chunks?

Thank so much.

Manuel