[PATCH] expand-on-first-boot: support resizing a btrfs root

[Joe MacDonald]

resize2fs fails if the build sets the root filesystem for an image to
btrfs though the partition is expanded properly. Since the rest of the
script works as intended, simply checking the last partition type with
lsblk and adding `btrfs resize` instaed of `resize2fs` in the appropriate
circumstance allows everything to function as expected.

Signed-off-by: Joe MacDonald <joe_ma...@mentor.com>
---
.../files/expand-last-partition.sh | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 7175dfd..23cfd86 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -36,8 +36,15 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
# Inform the kernel about the partitioning change
partx -u "${LAST_PART}"

-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
-
-resize2fs "${LAST_PART}"
+# Determine the filesystem type and perform the appropriate resize function
+ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
+case ${ROOTFS_TYPE} in
+ ext* )
+ # Do not fail resize2fs if no mtab entry is found, e.g.,
+ # when using systemd mount units.
+ export EXT2FS_NO_MTAB_OK=1
+ resize2fs "${LAST_PART}"
+ ;;
+ btrfs ) btrfs filesystem resize max / ;;
+ * ) exit 1 ;;
+esac
--
2.30.2

[Jan Kiszka]

On 22.10.21 20:01, Joe MacDonald wrote:
> resize2fs fails if the build sets the root filesystem for an image to
> btrfs though the partition is expanded properly. Since the rest of the
> script works as intended, simply checking the last partition type with
> lsblk and adding `btrfs resize` instaed of `resize2fs` in the appropriate

typo: instead

Could we reformat this to use multiple line, like the block above?

> + * ) exit 1 ;;
> +esac
>

Good extension!

Jan

--
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

[Joe MacDonald]

[Re: [PATCH] expand-on-first-boot: support resizing a btrfs root] On 21.10.22 (Fri 20:19) Jan Kiszka wrote:

> On 22.10.21 20:01, Joe MacDonald wrote:
> > resize2fs fails if the build sets the root filesystem for an image to
> > btrfs though the partition is expanded properly. Since the rest of the
> > script works as intended, simply checking the last partition type with
> > lsblk and adding `btrfs resize` instaed of `resize2fs` in the appropriate
>
> typo: instead

Oop, fixed. Thanks.

Yes, good suggestion. v2 also includes an informational message on the
fall-through condition rather than silently exiting with a failure.

-Joe.

>
> > + * ) exit 1 ;;
> > +esac
> >
>
> Good extension!
>
> Jan
>
> --
> Siemens AG, T RDA IOT
> Corporate Competence Center Embedded Linux

--
-Joe MacDonald.
Linux Architect | Mentor® A Siemens Business
:wq

[Joe MacDonald]

resize2fs fails if the build sets the root filesystem for an image to
btrfs though the partition is expanded properly. Since the rest of the
script works as intended, simply checking the last partition type with

lsblk and adding `btrfs resize` instead of `resize2fs` in the appropriate

circumstance allows everything to function as expected.

Signed-off-by: Joe MacDonald <joe_ma...@mentor.com>
---

.../files/expand-last-partition.sh | 22 ++++++++++++++-----
1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 7175dfd..6975d9c 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \

# Inform the kernel about the partitioning change
partx -u "${LAST_PART}"

-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
-
-resize2fs "${LAST_PART}"
+# Determine the filesystem type and perform the appropriate resize function
+ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
+case ${ROOTFS_TYPE} in
+ ext* )
+ # Do not fail resize2fs if no mtab entry is found, e.g.,
+ # when using systemd mount units.
+ export EXT2FS_NO_MTAB_OK=1
+ resize2fs "${LAST_PART}"
+ ;;
+ btrfs )

+ btrfs filesystem resize max /
+ ;;
+ * )
+ echo "Unrecognized filesystem type ${ROOTFS_TYPE} - no resize performed"
+ exit 1
+ ;;
+esac
--
2.30.2

[Henning Schild]

Yes, good addition! But it would be even better if we had it running in
CI. We could switch one target over to btrfs by providing an example
wks file, one that boots an image in qemu.
That way we would have a btrfs example in the tree and get confident
that wic will do things as i would hope for.

Am Fri, 22 Oct 2021 15:50:47 -0400
schrieb Joe MacDonald <joe_ma...@mentor.com>:

Need to do something to "DEBIAN_DEPENDS", maybe "e2fsprogs" ->
"e2fsprogs | btrfs-progs".

[Henning Schild]

Am Mon, 25 Oct 2021 10:10:54 +0200
schrieb Henning Schild <henning...@siemens.com>:

It is in fact more complicated than that. The two "progs" should not
both be depended on, because we would install too much stuff. We can
not use "|" because we will not know which one we get. Say one was
installed for any other reason, the one we need for rootfs might not
get pulled in.
Recommends would be better, but isar will not pull them. (still would
be an idea to be "correct")

We could also spit out two packages "expand-on-first-boot-ext|btrfs"
but that does not scale well for the next fs to cover.

So i would say "Recommends: e2fsprogs, btrfs-progs" and make that
script "command -v" and cry. Maybe keeping e2fsprogs in a first patch,
and removing it from Depends in a second interface-breaking patch.

regards,
Henning

[Henning Schild]

Hey Joe,

will you follow up with a v3? We are having another patch for
expand-on-first-boot coming up and discussions on how we could maybe
eventually switch to systemd-repart.

https://groups.google.com/g/isar-users/c/p5QADbRC3EA

Henning

Am Fri, 22 Oct 2021 15:50:47 -0400
schrieb Joe MacDonald <joe_ma...@mentor.com>:

[Jan Kiszka]

On 22.04.22 09:57, Henning Schild wrote:
> Hey Joe,
>
> will you follow up with a v3? We are having another patch for
> expand-on-first-boot coming up and discussions on how we could maybe
> eventually switch to systemd-repart.

systemd-growfs. Only this would be relevant for adding more fs types
like here. repart has nothing to do with that.

Jan

--
Siemens AG, Technology
Competence Center Embedded Linux

[Henning Schild]

It all looks like we want to revert the systemd-growfs patch and go
back to using plain tools for each partition type. So i looked at this
again and will likely propose another round, keeping the author but
changing it quite a bit.

Am Fri, 22 Oct 2021 15:50:47 -0400
schrieb Joe MacDonald <joe_ma...@mentor.com>:

> resize2fs fails if the build sets the root filesystem for an image to
> btrfs though the partition is expanded properly. Since the rest of the
> script works as intended, simply checking the last partition type with
> lsblk and adding `btrfs resize` instead of `resize2fs` in the
> appropriate circumstance allows everything to function as expected.

It it not about "rootfs" it is about the last partition.

> Signed-off-by: Joe MacDonald <joe_ma...@mentor.com>
> ---
> .../files/expand-last-partition.sh | 22
> ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
>
> diff --git
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> index 7175dfd..6975d9c 100755 ---
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> +++
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> @@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform
> the kernel about the partitioning change partx -u "${LAST_PART}" -#
> Do not fail resize2fs if no mtab entry is found, e.g., -# when using
> systemd mount units. -export EXT2FS_NO_MTAB_OK=1
> -
> -resize2fs "${LAST_PART}"
> +# Determine the filesystem type and perform the appropriate resize
> function +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
> +case ${ROOTFS_TYPE} in

not rootfs but last partition

> + ext* )

indentation is inconsistent with the rest of the script

> + # Do not fail resize2fs if no mtab entry is found, e.g.,
> + # when using systemd mount units.
> + export EXT2FS_NO_MTAB_OK=1
> + resize2fs "${LAST_PART}"
> + ;;
> + btrfs )
> + btrfs filesystem resize max /

Here it is again not about root. We will need the temporary mount bits
from the systemd-growfs patch and recycle them here. Because btrfs
wants to be mounted and we happen to know how that works from the other
patch.

Henning

[Roberto A. Foglietta]

On Fri, 9 Dec 2022 at 16:40, Henning Schild <henning...@siemens.com> wrote:
>
> It all looks like we want to revert the systemd-growfs patch and go
> back to using plain tools for each partition type. So i looked at this
> again and will likely propose another round, keeping the author but
> changing it quite a bit.
>
> Am Fri, 22 Oct 2021 15:50:47 -0400
> schrieb Joe MacDonald <joe_ma...@mentor.com>:
>
> > resize2fs fails if the build sets the root filesystem for an image to
> > btrfs though the partition is expanded properly. Since the rest of the
> > script works as intended, simply checking the last partition type with
> > lsblk and adding `btrfs resize` instead of `resize2fs` in the
> > appropriate circumstance allows everything to function as expected.
>
> It it not about "rootfs" it is about the last partition.

This is the code you need to integrate btrfs filesystem

case $(lsblk -fno FSTYPE "${LAST_PART}") in
ext4) resize2fs "${LAST_PART}"
;;
btrfs) mkdir -p /tmp/btrfs
mount "${LAST_PART}" /tmp/btrfs
btrfs filesystem resize max /tmp/btrfs
umount /tmp/btrfs && rmdir /tmp/btrfs
;;
esac

In attachment the patch against the current next branch.
It has been sent by elastic mail but went in moderation, as usual.

Best regards, R-

[Henning Schild]

Am Sat, 10 Dec 2022 04:28:48 +0100
schrieb "Roberto A. Foglietta" <roberto....@gmail.com>:

> On Fri, 9 Dec 2022 at 16:40, Henning Schild
> <henning...@siemens.com> wrote:
> >
> > It all looks like we want to revert the systemd-growfs patch and go
> > back to using plain tools for each partition type. So i looked at
> > this again and will likely propose another round, keeping the
> > author but changing it quite a bit.
> >
> > Am Fri, 22 Oct 2021 15:50:47 -0400
> > schrieb Joe MacDonald <joe_ma...@mentor.com>:
> >
> > > resize2fs fails if the build sets the root filesystem for an
> > > image to btrfs though the partition is expanded properly. Since
> > > the rest of the script works as intended, simply checking the
> > > last partition type with lsblk and adding `btrfs resize` instead
> > > of `resize2fs` in the appropriate circumstance allows everything
> > > to function as expected.
> >
> > It it not about "rootfs" it is about the last partition.
>
> This is the code you need to integrate btrfs filesystem
>
> case $(lsblk -fno FSTYPE "${LAST_PART}") in
> ext4) resize2fs "${LAST_PART}"
> ;;
> btrfs) mkdir -p /tmp/btrfs
> mount "${LAST_PART}" /tmp/btrfs
> btrfs filesystem resize max /tmp/btrfs
> umount /tmp/btrfs && rmdir /tmp/btrfs
> ;;
> esac

I have an updated version of Joes patch looking very similar but still
needing some more testing. Feel free to review that once it comes.

As for this one, it is a clumsy duplicate and i will not even review.

Henning

[Roberto A. Foglietta]

On Sat, 10 Dec 2022 at 17:29, Henning Schild <henning...@siemens.com> wrote:

> As for this one, it is a clumsy duplicate and I will not even review.

The ancestor of this my "clumsy duplicate" works decently since September 30th

https://groups.google.com/g/isar-users/c/dxVJqRvnWjw/m/Y5g5mbIPBQAJ

while the alternative based on systemd-resizefs shown to be not viable
but was hanging on because Stretch support which is going to be
dismissed because EOL since July 2020

Please Henning, play nicely ;-)

[Henning Schild]

Am Fri, 22 Oct 2021 15:50:47 -0400
schrieb Joe MacDonald <joe_ma...@mentor.com>:

> resize2fs fails if the build sets the root filesystem for an image to
> btrfs though the partition is expanded properly. Since the rest of the
> script works as intended, simply checking the last partition type with
> lsblk and adding `btrfs resize` instead of `resize2fs` in the
> appropriate circumstance allows everything to function as expected.
>
> Signed-off-by: Joe MacDonald <joe_ma...@mentor.com>
> ---
> .../files/expand-last-partition.sh | 22
> ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
>
> diff --git
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> index 7175dfd..6975d9c 100755 ---
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> +++
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> @@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform
> the kernel about the partitioning change partx -u "${LAST_PART}" -#
> Do not fail resize2fs if no mtab entry is found, e.g., -# when using
> systemd mount units. -export EXT2FS_NO_MTAB_OK=1
> -
> -resize2fs "${LAST_PART}"
> +# Determine the filesystem type and perform the appropriate resize
> function +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})

Calling "lsblk" suffers from the same udev race we have seen with
systemd-growfs ... only very subtle where we get like "" ... no
filesystem.

It seems to help to mount here and findmnt the mntpoint, we need the
mounting for btrfs anyhow.

Also lsblk can return multiple filesystems, like luks or lvm layers
underneath. Where findmnt only returns the top.

Joe, i had to rewrite your patch considerably. So i think i will change
the authorship to myself. I would really like to give you the credits
but am afraid to give you blame ... this whole expand story seems never
ending ...
So the patch will mention you, but likely not as author.

Henning

[Roberto A. Foglietta]

On Mon, 12 Dec 2022 at 16:24, Henning Schild <henning...@siemens.com> wrote:

>
> Joe, i had to rewrite your patch considerably. So i think i will change
> the authorship to myself. I would really like to give you the credits
> but am afraid to give you blame ... this whole expand story seems never
> ending ...
> So the patch will mention you, but likely not as author.
>

Thank you in advance for the mention. I wish to point out to you the
fact that supporting the btrfs is completely another issue than
triggering the udev to link the device and frankly speaking I do not
see the reason to keep the two things together. So, my ancestor patch
and the following development correctly address the btrfs support but
inherit the problem of failing when "${LAST_PART}" == "" which is a
completely another story because it depends on udev. Also the code to
trigger udev in order to create the devices links has been presented
here from another project of mine. So, the mention makes sense.

As a mere recap, this story should never have started at all. Because
there were two separate problems, no one of them was my duty to solve.
Moreover, the team got stuck into the idea of supporting every
filesystem on every platform: wrong approach. First separate legacy
from development, second fix a problem at time, third follows a
continuous improvement and continuous delivery model.

The btrfs support was not mandatory but suggested for the future due
to its intrinsic characteristics and the performance improvements
delivered into kernel v6.1. Your mandatory problem was to deal with
udev on a specific hardware platform and this was confused with a ext4
resizing failure - sometimes fails sometimes not depending the udev
timings (as far as I can remember) - so the btrfs got into the scene
as a suggestion to discard a software failure in resizing. If both
fail the problem is somewhere, in fact.

> but am afraid to give you blame

The only blame you can give to me is not being in charge of this at no
any time but trying to help without even having a piece of hardware to
make a test. In fact, the udev tests/code I did - I did on my
Christmas present for my nephew!

Best regards, R-

[Moessbauer, Felix]

One problem remains: Where to mount.
If we mount on /mnt this will not work on an RO rootfs out of the box.
We currently have this issue in CIP core.

We could mount below /tmp which should be writable, but I don't know
which side effects this as the tmpfs itself will be mounted / created
by systemd.

Felix

[Roberto A. Foglietta]

On Tue, 13 Dec 2022 at 03:10, Moessbauer, Felix
<felix.mo...@siemens.com> wrote:

>
> One problem remains: Where to mount.
> If we mount on /mnt this will not work on an RO rootfs out of the box.
> We currently have this issue in CIP core.
>
> We could mount below /tmp which should be writable, but I don't know
> which side effects this as the tmpfs itself will be mounted / created
> by systemd.
>

I have updated my patch to use /dev/shm but the path could be modified
setting a variable into the running environment.
The patch has been sent with git send-email using elastic smtp, so it
went into moderation. Here attached for evaluation.

Best regards, -R