Inquiry Regarding SWU Format Handling for Large Images (10GB+)

[kaijing wang]

Dear SWUpdate Development Team,
We are encountering file size limitations when using SWUpdate for OTA updates, as the CPIO archive format appears to have constraints with very large files (10GB+).
Could you please advise if there are recommended solutions for this scenario?
Environment Details:
SWUpdate Version: v2022.05
We appreciate your professional guidance on this matter.
Best regards,
Kaijing

[Stefano Babic]

Hi Kaijing,

On 5/9/25 10:31, kaijing wang wrote:
> Dear SWUpdate Development Team,
> We are encountering file size limitations when using SWUpdate for OTA
> updates, as the CPIO archive format appears to have constraints with
> very large files (10GB+).

Ouch...

The maximum size with CPIO is 4GB.

> Could you please advise if there are recommended solutions for this
> scenario?
> Environment Details:
> SWUpdate Version: v2022.05

It is known and I have listed this on the improvements" site (feature
waiting for sponsor).

https://github.com/sbabic/swupdate/blob/master/doc/source/improvement_proposals.rst#support-files-bigger-than-4gb

The right way to address this is to implement a new format into SWUpdate
able to remove this limitation (but without adding complexity). I have
some solutions to explore.

The only way to work-around this is to split the artifact in chunks
smaller than 4GB and set the "offset" attribute into sw-description.

> We appreciate your professional guidance on this matter.

Best regards,

Stefano Babic

[Nicholas Clark]

Would it make sense to add a kconfig option that switches the archive format from cpio to pax? Pax is well-supported by libarchive, provides the same benefits as cpio, and can handle very large files.

If you're interested in the feature, I could try an implementation.

[Stefano Babic]

Hi Nicholas,

On 11/7/25 04:40, Nicholas Clark wrote:
> Would it make sense to add a kconfig option that switches the archive
> format from cpio to pax?

A different format can be surely added, but I am against a Kconfig that
defines it at compile time. If a new format is introduced, SWUpdate must
be able to detect it at runtime. Else we create backward incompatibilities.

> Pax is well-supported by libarchive, provides
> the same benefits as cpio, and can handle very large files.

pax is already supported by the archive handler, but unpacking the
artifacts is not done via libarchive but in native code. Same thing
should be done again.

Regarding the format, SWUpdate does not use at all most of the
attributes that each format header has. So it is even to think about if
a full features format is required when SWUpdate just needs to extract
the file and know its size.

Best regards,
Stefano Babic

>
> If you're interested in the feature, I could try an implementation.
>
> On Friday, May 9, 2025 at 2:05:57 AM UTC-7 Stefano Babic wrote:
>
> Hi Kaijing,
>
> On 5/9/25 10:31, kaijing wang wrote:
> > Dear SWUpdate Development Team,
> > We are encountering file size limitations when using SWUpdate for
> OTA
> > updates, as the CPIO archive format appears to have constraints with
> > very large files (10GB+).
>
> Ouch...
>
> The maximum size with CPIO is 4GB.
>
> > Could you please advise if there are recommended solutions for this
> > scenario?
> > Environment Details:
> > SWUpdate Version: v2022.05
>
> It is known and I have listed this on the improvements" site (feature
> waiting for sponsor).
>
> https://github.com/sbabic/swupdate/blob/master/doc/source/

> improvement_proposals.rst#support-files-bigger-than-4gb <https://
> github.com/sbabic/swupdate/blob/master/doc/source/
> improvement_proposals.rst#support-files-bigger-than-4gb>

>
> The right way to address this is to implement a new format into
> SWUpdate
> able to remove this limitation (but without adding complexity). I have
> some solutions to explore.
>
> The only way to work-around this is to split the artifact in chunks
> smaller than 4GB and set the "offset" attribute into sw-description.
>
> > We appreciate your professional guidance on this matter.
>
> Best regards,
> Stefano Babic
>

> --
> You received this message because you are subscribed to the Google
> Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to swupdate+u...@googlegroups.com
> <mailto:swupdate+u...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/
> swupdate/e5f179c1-82ad-4eec-ac2a-c6dca65cc98en%40googlegroups.com
> <https://groups.google.com/d/msgid/swupdate/e5f179c1-82ad-4eec-ac2a-
> c6dca65cc98en%40googlegroups.com?utm_medium=email&utm_source=footer>.