GBytes to a GInputStream

[Richard Hughes]

Hi all,

I've just merged a mega-patch (289 files changed, 3807 insertions,
3192 deletions!) to fwupd git master that changes the firmware loading
interface from a GBytes to a GInputStream. This means we no longer
load the entire firmware blob into memory as the first step and
instead provide the plugins a stream that they can read specific
blocks or memory at specific addresses. This means several things:

* We can now deploy large firmware binaries on machines with limited
RAM -- which affects updating things like 5G modems on low-end
ChromeBooks, and also MTD partitions on resource constrained BMC
devices!

* We read (and seek!) the file descriptor passed to the daemon, which
means we only have to open() the archive in client, and the daemon
will decompress and stream blob payloads out of the archive (for
non-compressed sections) without any copying at all!

This means deploying a 2GiB firmware using the daemon has reduced the
RSS requirement from ~2.1GiB to... ~8MiB. You also need the latest
versions of libjcat and passim (released just today) to see this
result, otherwise we fall back to loading the entire archive in
various places.

I've migrated all the libfwupdplugin helper code, and 68 of the
affected vendor plugins. I've been able to test any hardware I have
here, and also all the devices that have USB emulation data as device
tests, but I *may* have introduced a regression somewhere. If I've
broken something I'm obviously happy to fix it!

Please could you test fwupd git main branch with your devices in the
next 4 weeks and report back any problems.

As a side note, I also branched 1_9_X well before any of these
disruptive changes, and I think the 1.9.x releases are a much better
idea for stable distros. It'll be some time (months?) before we do a
2.0.x release as we also broke ABI and all this new code needs a
significant amount of real-world testing.

Richard.