ext4.gz image

[Jasper Vanlerberghe]

First of all, thanks for the great work on swupdate.

 

The last week I have been running into an issue, that I cannot seem to figure out.

I have been creating swu update-packages with yocto. Those contain next to the sw-description file.

• A script that (preupdate) symlinks a partition to where the update needs to go
• A raw image: ext4.gz that gets written to the symlink

 

This works flawless.

 

In a next step I was trying to add updates to the ext4.gz by:

• Unpacking the swu (cpio)
• Gunzipping the ext4.gz
• Loop mounting the ext4
• Adding files to the mount (untarring a package)
• Unmounting the loop mount
• Gzip the ext4 file
• Repack in a cpio

 

That works when I tried it for adding some files to e.g. /opt but not when I added some files to /usr/lib.

During the update process I get this error, which is not very descriptive and I have trouble tracking down the root cause. Could you maybe shed some light on that?

               

[TRACE] : SWUPDATE running :  [parse_hw_compatibility] : Accepted Hw Revision : 1.0

[TRACE] : SWUPDATE running :  [_parse_images] : Found compressed Image rootfs 4.0.0.3 image.ext4.gz in device : /dev/update for handler raw

[TRACE] : SWUPDATE running :  [_parse_scripts] : Found Script: update.sh

[TRACE] : SWUPDATE running :  [parse] : Number of found artifacts: 1

[TRACE] : SWUPDATE running :  [parse] : Number of scripts: 1

[TRACE] : SWUPDATE running :  [parse] : Number of steps to be run: 3

[TRACE] : SWUPDATE running :  [check_hw_compatibility] : Hardware proc Revision: 1.0

[TRACE] : SWUPDATE running :  [check_hw_compatibility] : Hardware compatibility verified

[DEBUG] : SWUPDATE running :  [preupdatecmd] : Running Pre-update command

[TRACE] : SWUPDATE running :  [extract_files] : Found file

[TRACE] : SWUPDATE running :  [extract_files] :         filename update.sh

[TRACE] : SWUPDATE running :  [extract_files] :         size 1558 required

[TRACE] : SWUPDATE running :  [extract_files] : Found file

[TRACE] : SWUPDATE running :  [extract_files] :         filename image.ext4.gz

[TRACE] : SWUPDATE running :  [extract_files] :         size 302957947 required

[TRACE] : SWUPDATE running :  [extract_padding] : Expecting up to 512 padding bytes at end-of-file

[TRACE] : SWUPDATE running :  [network_initializer] : Valid image found: copying to FLASH

swupdate_image_write failed: Connection reset by peer

[ERROR] : SWUPDATE failed [0] ERROR install_from_file.c : endupdate : 55 : SWUpdate *failed* !

[TRACE] : SWUPDATE running :  [unlink_sockets] : unlink socket /tmp/swupdateprog

[TRACE] : SWUPDATE running :  [unlink_sockets] : unlink socket /tmp/sockinstctrl

 

I checked and yocto uses mkfs.ext4 and reference to a folder to create the ext4 file. After that they run fsck.ext4.
I tried running fsck.ext4 to check for errors and fix them before repacking the files in the swu, but that did not solve anything.

 

The ext4 I imagine is just a stream of raw bits and I imagine swupdate doing something equivalent to dd. So I really cannot wrap my head around why the folder in the seems to matter.

Also I double checked the ext4 file size, that is exactly the same for working swu’s and non-working ones.

 

I am kind of running out of ideas for this one.

[Stefano Babic]

Hi Jasper,

On 5/19/25 16:04, Jasper Vanlerberghe wrote:
> First of all, thanks for the great work on swupdate.
>
> The last week I have been running into an issue, that I cannot seem to
> figure out.
>
> I have been creating swu update-packages with yocto. Those contain next
> to the sw-description file.
>

> * A script that (preupdate) symlinks a partition to where the update
> needs to go

...and please check in the doc my slides about "Best practise", and the
point "#5. Don’t abuse shell scripts" and, more important, "#10 Do not
trust vendor !". It is clear you copied this from a vendor.

> * A raw image: ext4.gz that gets written to the symlink
>
> This works flawless.
>

..but not in streaming mode...

> In a next step I was trying to add updates to the ext4.gz by:
>

> * Unpacking the swu (cpio)
> * Gunzipping the ext4.gz
> * Loop mounting the ext4
> * Adding files to the mount (untarring a package)
> * Unmounting the loop mount
> * Gzip the ext4 file
> * Repack in a cpio

>
> That works when I tried it for adding some files to e.g. /opt but not
> when I added some files to /usr/lib.
>
> During the update process I get this error, which is not very
> descriptive

.
...because you have started SWUpdate with restricted TRACES. Add -l 5 to
the command line. Nevertheless:

This seems to me a clear reason: image cannot be written.

>
> [ERROR] : SWUPDATE failed [0] ERROR install_from_file.c : endupdate :
> 55 : SWUpdate *failed* !
>
> [TRACE] : SWUPDATE running :  [unlink_sockets] : unlink socket /tmp/
> swupdateprog
>
> [TRACE] : SWUPDATE running :  [unlink_sockets] : unlink socket /tmp/

> sockinstctrl

>
> I checked and yocto uses mkfs.ext4 and reference to a folder to create
> the ext4 file. After that they run fsck.ext4.
> I tried running fsck.ext4 to check for errors and fix them before
> repacking the files in the swu, but that did not solve anything.
>
> The ext4 I imagine is just a stream of raw bits and I imagine swupdate
> doing something equivalent to dd. So I really cannot wrap my head around
> why the folder in the seems to matter.
>
> Also I double checked the ext4 file size, that is exactly the same for
> working swu’s and non-working ones.

How can be the same if you add maybe libraries to /usr/lib ?

>
> I am kind of running out of ideas for this one.

The simplest reason is often the right one: you are breaking through the
partition. The maxium size is defined when the partition table is
created. And, if you have use a wic without redefining the partition
size, the partition as default is 30% bigger than the rootfs, and you
can quickly after a while get into this kind of trouble.

Nevertheless, add trace verbosity, and you could get more information,
if you think the partition is big enough for your new ext4.

Best regards,
Stefano Babic

[Jasper Vanlerberghe]

Thanks for the quick response.

On Monday, May 19, 2025 at 4:23:50 PM UTC+2 Stefano Babic wrote:

Hi Jasper,

On 5/19/25 16:04, Jasper Vanlerberghe wrote:
> First of all, thanks for the great work on swupdate.
>
> The last week I have been running into an issue, that I cannot seem to
> figure out.
>
> I have been creating swu update-packages with yocto. Those contain next
> to the sw-description file.
>
> * A script that (preupdate) symlinks a partition to where the update
> needs to go

...and please check in the doc my slides about "Best practise", and the
point "#5. Don’t abuse shell scripts" and, more important, "#10 Do not
trust vendor !". It is clear you copied this from a vendor.

Are you referring to https://sbabic.github.io/swupdate/swupdate-best-practise.html

I do not find your exact reference.

 

> * A raw image: ext4.gz that gets written to the symlink
>
> This works flawless.
>

..but not in streaming mode...

What do you refer to with streaming mode? using an image of type raw?

 

> In a next step I was trying to add updates to the ext4.gz by:
>
> * Unpacking the swu (cpio)
> * Gunzipping the ext4.gz
> * Loop mounting the ext4
> * Adding files to the mount (untarring a package)
> * Unmounting the loop mount
> * Gzip the ext4 file
> * Repack in a cpio
>
> That works when I tried it for adding some files to e.g. /opt but not
> when I added some files to /usr/lib.
>
> During the update process I get this error, which is not very
> descriptive

.
...because you have started SWUpdate with restricted TRACES. Add -l 5 to
the command line. Nevertheless:

I am testing using: swupdate -l 5 -i /run/media/sda1/image.swu 

It is a dummy file with the text "this is a testfile" added to either of the locations. Not really a significant amount of data

>
> I am kind of running out of ideas for this one.

The simplest reason is often the right one: you are breaking through the
partition. The maxium size is defined when the partition table is
created. And, if you have use a wic without redefining the partition
size, the partition as default is 30% bigger than the rootfs, and you
can quickly after a while get into this kind of trouble.

Nevertheless, add trace verbosity, and you could get more information,
if you think the partition is big enough for your new ext4.

If I unpack the swu and gunzip the ext4.gz in both cases (working and non-working swu) the file size is 5905580032

My partitions look like this (I have an A and a B)

Number  Start         End           Size         Type      File system  Flags
 1      1048576B      135266303B    134217728B   primary   fat32        boot, lba
 2      138412032B    6043992063B   5905580032B  primary   ext4
 3      6043992064B   11949572095B  5905580032B  primary   ext4
 4      11953765376B  15758000127B  3804234752B  extended               lba
 5      11953766400B  13027508223B  1073741824B  logical   ext4
 6      13031702528B  15758000127B  2726297600B  logical   ext4

 

Best regards,
Stefano Babic