[PATCH v4 0/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[Tobias Schmidl]

Diff to v1:
- Rewording on patch message ("root partition" --> "last partition",
added note to systemd-dependency and its impact)
- Added systemd-version dependency
- Removed runtime if case that allowed for non-systemd systems

Diff to v2:
- Version bump to v1.2
- Removed e2fsprogs as dependency

Diff to v3:
- Omit the explicit call to systemd-growfs if a fstab option has been
found
- Be able to deal with both mounted and unmounted mountpoints

Tobias Schmidl (1):
expand-on-first-boot: Switch from resize2fs to systemd-growfs

...oot_1.1.bb => expand-on-first-boot_1.2.bb} | 5 +++--
.../files/expand-last-partition.sh | 21 +++++++++++++++++--
2 files changed, 22 insertions(+), 4 deletions(-)
rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (78%)

--
2.36.1

[Tobias Schmidl]

We want to be more versatile in our approach of resizing the last
partition. Therefore we switch from resize2fs to systemd-growfs.

This allows for ext4, btrfs, xfs, and dm-crypt partitions to be resized.

Since systemd-growfs landed in v236, this obsoletes expand-on-first-boot
on stretch (v232).

Signed-off-by: Tobias Schmidl <tobias...@siemens.com>
---

...oot_1.1.bb => expand-on-first-boot_1.2.bb} | 5 +++--
.../files/expand-last-partition.sh | 21 +++++++++++++++++--
2 files changed, 22 insertions(+), 4 deletions(-)
rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (78%)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
similarity index 78%
rename from meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
rename to meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
index 1703a64..48d30d3 100644
--- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
+++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
@@ -1,15 +1,16 @@
# Resize last partition to full medium size on fist boot
#
# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
#
# SPDX-License-Identifier: MIT

inherit dpkg-raw

DESCRIPTION = "This service grows the last partition to the full medium during first boot"
+MAINTAINER = "isar-users <isar-...@googlegroups.com>"

-DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk, util-linux"
+DEBIAN_DEPENDS = "systemd (>=236), sed, grep, coreutils, mount, fdisk, util-linux"

SRC_URI = " \
file://expand-on-first-boot.service \
diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index c0edde7..1743890 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -3,7 +3,7 @@
# Resize last partition to full medium size
#
# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
#
# SPDX-License-Identifier: MIT

@@ -45,4 +45,21 @@ partx -u "${LAST_PART}"
# when using systemd mount units.
export EXT2FS_NO_MTAB_OK=1

-resize2fs "${LAST_PART}"
+if grep -q x-systemd.growfs /etc/fstab; then
+ echo "Found x-systemd.growfs option in /etc/fstab, won't call it explicitly." >&2
+ exit 0
+fi
+
+MOUNT_POINT=$(findmnt -o target -n "${LAST_PART}")
+UNMOUNT_AFTERWORDS=0
+if [ -z "${MOUNT_POINT}" ]; then
+ MOUNT_POINT=$(findmnt -o target -n --fstab "{$LAST_PART}")
+ if [ -z "${MOUNT_POINT}" ]; then
+ echo "Cannot find mount point for ${LAST_PART}" >&2
+ exit 1
+ else
+ UNMOUNT_AFTERWORDS=1
+ fi
+fi
+/lib/systemd/systemd-growfs "${MOUNT_POINT}"
+[ $UNMOUNT_AFTERWORDS -eq 1 ] && umount "${MOUNT_POINT}"
--
2.36.1

[Jan Kiszka]

afterwards ;)

> +if [ -z "${MOUNT_POINT}" ]; then
> + MOUNT_POINT=$(findmnt -o target -n --fstab "{$LAST_PART}")
> + if [ -z "${MOUNT_POINT}" ]; then
> + echo "Cannot find mount point for ${LAST_PART}" >&2
> + exit 1
> + else
> + UNMOUNT_AFTERWORDS=1
> + fi
> +fi
> +/lib/systemd/systemd-growfs "${MOUNT_POINT}"
> +[ $UNMOUNT_AFTERWORDS -eq 1 ] && umount "${MOUNT_POINT}"

Explicit exit 0, or "if then"? Otherwise, the script may return with an
error when the condition is false.

Looks good to me otherwise.

Jan

--
Siemens AG, Technology
Competence Center Embedded Linux

[Henning Schild]

Am Wed, 29 Jun 2022 14:01:44 +0200
schrieb Tobias Schmidl <tobias...@siemens.com>:

where did the mount go? i would assume a mount to be here somewhere
which happens in the context where we remember that we have to umount

Here a "noauto" in the fstab line of that last partition should help to
put that script into the role of the mounter and actually test that
path.

So the problem is that the partition needs to be mounted to get
resized. And now we have the problem to sync up with other mounters and
potentially having to umount ... not as easy as it sounds. We need to
get the order right and know things like "noauto" or other fun
exceptions.
I suggest to try and --bind mount it to /tmp/partition-to-expand ... if
we are lucky we can grow it when mounted twice and unconditionally
mount/umount without ever having to findmnt. But i am kind of afraid of
an EBUSY ...

Henning

[Henning Schild]

Am Wed, 29 Jun 2022 14:01:44 +0200
schrieb Tobias Schmidl <tobias...@siemens.com>:

I guess we might need some comments here. In the other mail you wrote
that the fstab magic string did not work as expected, but i guess that
was the attempt to smuggle it in without actually having it in fstab.

Does the combination of expand-on-first-boot (in the proposed version)
work together with the fstab string set on the last partition?
I would assume it would not because the service is executed too late,
or it might work on the second boot ... It is probably good to bail
when finding that string in fstab, but for people reading the code they
might think that they could as well write it into fstab to reach a fs
growing that way.

Henning

[Tobias Schmidl]

Diff to v1:
- Rewording on patch message ("root partition" --> "last partition",
added note to systemd-dependency and its impact)
- Added systemd-version dependency
- Removed runtime if case that allowed for non-systemd systems

Diff to v2:
- Version bump to v1.2
- Removed e2fsprogs as dependency

Diff to v3:
- Omit the explicit call to systemd-growfs if a fstab option has been
found
- Be able to deal with both mounted and unmounted mountpoints

Diff to v4:
- Mount the last partition unconditionally to a tmp dir and let
systemd-growfs operate there.

Tobias Schmidl (1):
expand-on-first-boot: Switch from resize2fs to systemd-growfs

...oot_1.1.bb => expand-on-first-boot_1.2.bb} | 5 +++--

.../files/expand-last-partition.sh | 19 +++++++++++++++++--
2 files changed, 20 insertions(+), 4 deletions(-)

rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (78%)

--
2.36.1

[Tobias Schmidl]

We want to be more versatile in our approach of resizing the last
partition. Therefore we switch from resize2fs to systemd-growfs.

This allows for ext4, btrfs, xfs, and dm-crypt partitions to be resized.

Since systemd-growfs landed in v236, this obsoletes expand-on-first-boot
on stretch (v232).

Signed-off-by: Tobias Schmidl <tobias...@siemens.com>
---

...oot_1.1.bb => expand-on-first-boot_1.2.bb} | 5 +++--
.../files/expand-last-partition.sh | 19 +++++++++++++++++--
2 files changed, 20 insertions(+), 4 deletions(-)
rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (78%)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
similarity index 78%
rename from meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
rename to meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
index 1703a64..48d30d3 100644
--- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
+++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
@@ -1,15 +1,16 @@
# Resize last partition to full medium size on fist boot
#
# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
#
# SPDX-License-Identifier: MIT

inherit dpkg-raw

DESCRIPTION = "This service grows the last partition to the full medium during first boot"
+MAINTAINER = "isar-users <isar-...@googlegroups.com>"

-DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk, util-linux"
+DEBIAN_DEPENDS = "systemd (>=236), sed, grep, coreutils, mount, fdisk, util-linux"

SRC_URI = " \
file://expand-on-first-boot.service \
diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh

index c0edde7..31f1ae3 100755

--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -3,7 +3,7 @@
# Resize last partition to full medium size
#
# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
#
# SPDX-License-Identifier: MIT

@@ -45,4 +45,19 @@ partx -u "${LAST_PART}"

# when using systemd mount units.
export EXT2FS_NO_MTAB_OK=1

-resize2fs "${LAST_PART}"
+if grep -q x-systemd.growfs /etc/fstab; then
+ echo "Found x-systemd.growfs option in /etc/fstab, won't call it explicitly." >&2
+ exit 0
+fi

+
+# mount $LAST_PART out of tree, so we won't conflict with other mounts
+MOUNT_POINT=$(mktemp -d -p /mnt "$(basename "$0").XXXXXXXXXX")
+if [ ! -d "${MOUNT_POINT}" ]; then
+ echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2
+ exit 1
+fi
+
+mount "${LAST_PART}" "${MOUNT_POINT}"
+/lib/systemd/systemd-growfs "${MOUNT_POINT}"
+umount "${MOUNT_POINT}"
+rmdir "${MOUNT_POINT}"
--
2.36.1

[Henning Schild]

Am Thu, 30 Jun 2022 15:55:31 +0200
schrieb Tobias Schmidl <tobias...@siemens.com>:

Cool, seems that "unconditional mount/umount" trick indeed works. LGTM

Henning

[Henning Schild]

ich sortiere so diffs immer andersrum, die neuste oben so wie man ein
changelog halt schreibt, ist aber geschmackssache

bei der Serie hättest du Jan mit draufnehmen können, der hat nen review
gemacht und damit quasi "subscribed"

Aber ja, Kleinigkeiten.

Henning

Am Thu, 30 Jun 2022 15:55:30 +0200
schrieb Tobias Schmidl <tobias...@siemens.com>:

[Joe MacDonald]

I've been largely silent on these threads so far but I have been watching
the progress. This version looks good to me and will work well, I think,
for my longer-term goals as well. The switch to systemd-growfs looks like
the correct future-proof approach, so thanks for that. :-)

[Re: [PATCH v5 0/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs] On 22.06.30 (Thu 17:05) Henning Schild wrote:

> ich sortiere so diffs immer andersrum, die neuste oben so wie man ein
> changelog halt schreibt, ist aber geschmackssache

I also prefer to have the 'diff history' comments in a most-recent-first
format, as in changelogs, but since the list here is short it's still easy
to find the relevant information quickly.

--
-Joe MacDonald.
Linux Architect | Siemens Digital Industries Software
:wq

[Joe MacDonald]

Only one minor observation that I should have made sooner (so if there are
other changes that might warrant a v6, great, but otherwise I don't think
it's necessary).

This error message may be a bit more helpful if it stated the impact of
this problem. Something like this, instead:

echo "Cannot create temporary mount point ${MOUNT_POINT}, ${LAST_PART} will not be resized." >&2

-J.

> + exit 1
> +fi
> +
> +mount "${LAST_PART}" "${MOUNT_POINT}"
> +/lib/systemd/systemd-growfs "${MOUNT_POINT}"
> +umount "${MOUNT_POINT}"
> +rmdir "${MOUNT_POINT}"
> --
> 2.36.1
>

[Henning Schild]

Am Thu, 30 Jun 2022 12:41:25 -0400
schrieb Joe MacDonald <Joe_Ma...@mentor.com>:

The script failing always has that result. In fact you end up in a
weird state where the partition has been grown and the filesystem has
not. Not sure this is needed.

Joe did you get a chance to try that all on your btrfs use-case? I know
Tobias tested that but getting a confirm that it works in other layers
would be cool. But we do not need such a confirm to continue, we assume
it will work in layers.

Henning

[Henning Schild]

Ooops, the "reply-all" while wanting "off-list" mistake again ;)

At least i did not say anything which should not have been said in
public.

Henning

Am Thu, 30 Jun 2022 12:36:21 -0400
schrieb Joe MacDonald <Joe_Ma...@mentor.com>:

[Joe MacDonald]

That's a good point. Actually, ideally we would roll back to the previous
(probably consistent) state, but that's going to be a difficult thing to
do safely, for sure!

> Joe did you get a chance to try that all on your btrfs use-case? I know
> Tobias tested that but getting a confirm that it works in other layers
> would be cool. But we do not need such a confirm to continue, we assume
> it will work in layers.

I did not yet, but I keep intending to, it's just a matter of finding time
to. Having watched the progression of the work, I would be very surprised
if it does not, but I'll follow up here as soon as I get a chance to
verify this one way or the other.

-J.

>
> Henning
>
> > -J.
> >
> > > + exit 1
> > > +fi
> > > +
> > > +mount "${LAST_PART}" "${MOUNT_POINT}"
> > > +/lib/systemd/systemd-growfs "${MOUNT_POINT}"
> > > +umount "${MOUNT_POINT}"
> > > +rmdir "${MOUNT_POINT}"
> > > --
> > > 2.36.1
> > >
> >
>

[Schmidl, Tobias]

Hi all,

What is the status of this patch? Are there any impediments?

Kind regards,

Tobias

[Anton Mikanovich]

20.07.2022 15:38, Schmidl, Tobias wrote:
> What is the status of this patch? Are there any impediments?
>
> Kind regards,
>
> Tobias
>

Hello Tobias,

To have systemd-growfs merged to next we need to implement backward
compatibility with stretch. This can be done inside the recipe, or maybe
even
with splitting recipes for stretch and others.