Fastboot behaviour with sparse images

[Maxime Ripard]

Hi,

I'm currently writing the support in U-Boot for NAND-backed devices
using fastboot [1], and that work derived a bit to supporting the
sparse images.

For "regular" images that are being stored, we expect a pair of
download and flash commands. Simple.

Things start to get a bit more complex with sparse images that have
been split because of a max-download-size lower than the actual image
size.

Here, from what I could gather from various random blog posts, the
fastboot client implementation and dumping a few USB sessions, the
client simply creates several download / flash pairs, always on the
same partition, without any way to distinct that from several
subsequent writes issued by the user.

So, I'm guessing that the expectation is that the bootloader
implementation should store the last offset it wrote to, and simple
resume from there if the partition names in the flash commands are the
same, which would prevent two subsequent write on the same partition
by any client. Am I right?

A related question is when should we erase the NAND partition? Only
when doing fastboot erase, or also when doing fastboot write (which,
combined with the issue raised above, would also mean that we don't
want to do an erase on the whole partition everytime there's a flash
command on it).

Thanks!
Maxime

1: http://lists.denx.de/pipermail/u-boot/2015-August/226053.html

--
Maxime Ripard, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Maxime Ripard]

Hi Tom,

On Wed, Oct 21, 2015 at 10:09:28AM -0400, Tom Rini wrote:

> I think for this last question, some experimentation with the existing
> tools might be required. As there's no required explicit erase for MMC,
> I think it might make sense to say you erase nand up front and then
> write as anything else starts getting really tricky and we're just
> second-guessing the user.

Actually, the only FS the fastboot tool seems to be doing it for the
moment are ext4 and F2FS. It can probably be extended to UBI and raw
partitions, but that won't fix the tools that are bundled by the
distros at the moment.

So I guess we can always erase it now using the session counter: if we
are writing the first chunk, erase the whole partition, if we're not,
then simply flash it at the previous offset.

How does it sound?
Maxime

[Maxime Ripard]

On Thu, Oct 22, 2015 at 07:22:54AM -0400, Tom Rini wrote:

> Sounds workable but testing with the existing tools will be the key and
> the hard part here :(

Well, if we always erase when we write, the worst case scenario would
be one erase too many. It doesn't sound that bad, or hard.

[Colin Cross]

On Monday, October 12, 2015 at 6:43:37 AM UTC-7, Maxime Ripard wrote:

Hi,

I'm currently writing the support in U-Boot for NAND-backed devices
using fastboot [1], and that work derived a bit to supporting the
sparse images.

For "regular" images that are being stored, we expect a pair of
download and flash commands. Simple.

Things start to get a bit more complex with sparse images that have
been split because of a max-download-size lower than the actual image
size.

Here, from what I could gather from various random blog posts, the
fastboot client implementation and dumping a few USB sessions, the
client simply creates several download / flash pairs, always on the
same partition, without any way to distinct that from several
subsequent writes issued by the user.

So, I'm guessing that the expectation is that the bootloader
implementation should store the last offset it wrote to, and simple
resume from there if the partition names in the flash commands are the
same, which would prevent two subsequent write on the same partition
by any client. Am I right?

No, each blob passed to the bootloader will begin with a sparse "skip" chunk that will seek to the correct place to resume writing.  The bootloader shouldn't need to store any metadata across commands.  Just read in the blob from the data command, then write it out using a port of the Apache-licensed libsparse during the flash command.

 

A related question is when should we erase the NAND partition? Only
when doing fastboot erase, or also when doing fastboot write (which,
combined with the issue raised above, would also mean that we don't
want to do an erase on the whole partition everytime there's a flash
command on it).

Fastboot should send an erase command before every sequence of writes.  Erase the whole partition on the erase command, and don't erase anything on the flash command.

[Maxime Ripard]

Hi Colin,

Oh, so that's how it works. Great. I guess however that you still need
to scan out the area you skip for bad blocks to account them in the
offset calculation as well then (when you're using NAND).

> > A related question is when should we erase the NAND partition? Only
> > when doing fastboot erase, or also when doing fastboot write (which,
> > combined with the issue raised above, would also mean that we don't
> > want to do an erase on the whole partition everytime there's a flash
> > command on it).
>
> Fastboot should send an erase command before every sequence of writes.
> Erase the whole partition on the erase command, and don't erase anything
> on the flash command.

Ack.

Thanks!
Maxime