[PATCH v6 0/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[Henning Schild]

Diff to v5:
- keep fallback path for debian stretch simply using old code and rdeps

Original Authors log:

Diff to v1:
- Rewording on patch message ("root partition" --> "last partition",
added note to systemd-dependency and its impact)
- Added systemd-version dependency
- Removed runtime if case that allowed for non-systemd systems

Diff to v2:
- Version bump to v1.2
- Removed e2fsprogs as dependency

Diff to v3:
- Omit the explicit call to systemd-growfs if a fstab option has been
found
- Be able to deal with both mounted and unmounted mountpoints

Diff to v4:
- Mount the last partition unconditionally to a tmp dir and let
systemd-growfs operate there.

Tobias Schmidl (1):
expand-on-first-boot: Switch from resize2fs to systemd-growfs

...oot_1.1.bb => expand-on-first-boot_1.2.bb} | 3 +-
.../files/expand-last-partition.sh | 31 ++++++++++++++++---
2 files changed, 28 insertions(+), 6 deletions(-)
rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (88%)

--
2.37.4

[Henning Schild]

From: Tobias Schmidl <tobias...@siemens.com>

We want to be more versatile in our approach of resizing the last
partition. Therefore we switch from resize2fs to systemd-growfs.

This allows for ext4, btrfs, xfs, and dm-crypt partitions to be resized.

Signed-off-by: Tobias Schmidl <tobias...@siemens.com>
Signed-off-by: Henning Schild <henning...@siemens.com>
---

...oot_1.1.bb => expand-on-first-boot_1.2.bb} | 3 +-
.../files/expand-last-partition.sh | 31 ++++++++++++++++---
2 files changed, 28 insertions(+), 6 deletions(-)
rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (88%)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
similarity index 88%
rename from meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
rename to meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
index 1703a648ca27..8d11aef00927 100644
--- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
+++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
@@ -1,13 +1,14 @@
# Resize last partition to full medium size on fist boot
#
# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
#
# SPDX-License-Identifier: MIT

inherit dpkg-raw

DESCRIPTION = "This service grows the last partition to the full medium during first boot"
+MAINTAINER = "isar-users <isar-...@googlegroups.com>"

DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk, util-linux"

diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 759ff8b61ccd..046a1394d213 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -3,7 +3,7 @@
# Resize last partition to full medium size
#
# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
#
# SPDX-License-Identifier: MIT

@@ -57,8 +57,29 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
# Inform the kernel about the partitioning change
partx -u "${LAST_PART}"

-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
+# this is for debian stretch or systemd < 236
+if [ ! -x /lib/systemd/systemd-growfs ]; then
+ # Do not fail resize2fs if no mtab entry is found, e.g.,
+ # when using systemd mount units.
+ export EXT2FS_NO_MTAB_OK=1

-resize2fs "${LAST_PART}"
+ resize2fs "${LAST_PART}"
+ exit 0
+if
+
+if grep -q x-systemd.growfs /etc/fstab; then
+ echo "Found x-systemd.growfs option in /etc/fstab, won't call it explicitly." >&2
+ exit 0
+fi
+
+# mount $LAST_PART out of tree, so we won't conflict with other mounts
+MOUNT_POINT=$(mktemp -d -p /mnt "$(basename "$0").XXXXXXXXXX")
+if [ ! -d "${MOUNT_POINT}" ]; then
+ echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2
+ exit 1
+fi
+
+mount "${LAST_PART}" "${MOUNT_POINT}"
+/lib/systemd/systemd-growfs "${MOUNT_POINT}"
+umount "${MOUNT_POINT}"
+rmdir "${MOUNT_POINT}"
--
2.37.4

[Henning Schild]

I am taking this over from Tobias. The last remaining point was keeping
the support for stretch, which i implemented here.

Henning

Am Tue, 25 Oct 2022 15:47:19 +0200
schrieb Henning Schild <henning...@siemens.com>:

[Bezdeka, Florian]

On Tue, 2022-10-25 at 15:49 +0200, Henning Schild wrote:
> I am taking this over from Tobias. The last remaining point was keeping
> the support for stretch, which i implemented here.

There were complains on the list that we have a possible race-window
here. A /data partition was unmounted after the resize took place.

That is likely not directly related to this patch, as this is "not in
yet" but I assume we still would have the same race.

See [1].

[1] https://groups.google.com/g/isar-users/c/o-iZbN7hPRw

[Henning Schild]

Am Tue, 25 Oct 2022 15:55:00 +0200
schrieb "Bezdeka, Florian (T CED SES-DE)" <florian...@siemens.com>:

> On Tue, 2022-10-25 at 15:49 +0200, Henning Schild wrote:
> > I am taking this over from Tobias. The last remaining point was
> > keeping the support for stretch, which i implemented here.
>
> There were complains on the list that we have a possible race-window
> here. A /data partition was unmounted after the resize took place.
>
> That is likely not directly related to this patch, as this is "not in
> yet" but I assume we still would have the same race.
>
> See [1].

I would hope that the race will be gone when doing things with systemd,
but i do not know. In the stretch case it might be still in, but i
really do not care ... we should drop stretch from Isar or maintain it
on the best effort basis suggested here. Make sure it still works ...
but not much more.

I have read that thread and pointed out there that systemd resize will
be coming, and that people can try it already to see if it solves their
problem. Will take Raphael in Cc.

Raphael please have a look at the following patch, it might just solve
your resize race.

https://patchwork.isar-build.org/project/isar/patch/20221025134720.225...@siemens.com/

Henning

[Henning Schild]

Am Tue, 25 Oct 2022 15:47:20 +0200
schrieb Henning Schild <henning...@siemens.com>:

syntax error ... stay tuned ...

[Henning Schild]

Diff to v6:
- fix shell script syntax error
- include a patch to actually make the package installable on stretch

Diff to v5:
- keep fallback path for debian stretch simply using old code and rdeps

Original Authors log:

Diff to v1:
- Rewording on patch message ("root partition" --> "last partition",
added note to systemd-dependency and its impact)
- Added systemd-version dependency
- Removed runtime if case that allowed for non-systemd systems

Diff to v2:
- Version bump to v1.2
- Removed e2fsprogs as dependency

Diff to v3:
- Omit the explicit call to systemd-growfs if a fstab option has been
found
- Be able to deal with both mounted and unmounted mountpoints

Diff to v4:
- Mount the last partition unconditionally to a tmp dir and let
systemd-growfs operate there.

Henning Schild (1):
expand-on-first-boot: fix runtime deps for debian stretch

Tobias Schmidl (1):
expand-on-first-boot: Switch from resize2fs to systemd-growfs

...oot_1.1.bb => expand-on-first-boot_1.2.bb} | 5 +--
.../files/expand-last-partition.sh | 31 ++++++++++++++++---
2 files changed, 29 insertions(+), 7 deletions(-)
rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (82%)

--
2.37.4

[Henning Schild]

In debian stretch sfdisk was part of util-linux and not the new package
fdisk. Change the DEBIAN_DEPENDS to allow installation on stretch again.

Fixes: a298cccda57f ("expand-on-first-boot: List fdisk as explicit dependency")

Signed-off-by: Henning Schild <henning...@siemens.com>
---

.../expand-on-first-boot/expand-on-first-boot_1.1.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
index 1703a648ca27..630e8d2cc510 100644
--- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
+++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
@@ -9,7 +9,7 @@ inherit dpkg-raw

DESCRIPTION = "This service grows the last partition to the full medium during first boot"

-DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk, util-linux"
+DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux"

SRC_URI = " \
file://expand-on-first-boot.service \
--
2.37.4

[Henning Schild]

From: Tobias Schmidl <tobias...@siemens.com>

We want to be more versatile in our approach of resizing the last
partition. Therefore we switch from resize2fs to systemd-growfs.

This allows for ext4, btrfs, xfs, and dm-crypt partitions to be resized.

Signed-off-by: Tobias Schmidl <tobias...@siemens.com>

Signed-off-by: Henning Schild <henning...@siemens.com>
---

...oot_1.1.bb => expand-on-first-boot_1.2.bb} | 3 +-

.../files/expand-last-partition.sh | 31 ++++++++++++++++---
2 files changed, 28 insertions(+), 6 deletions(-)
rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (89%)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
similarity index 89%

rename from meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
rename to meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb

index 630e8d2cc510..0996000c3ce0 100644

--- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
+++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
@@ -1,13 +1,14 @@
# Resize last partition to full medium size on fist boot
#
# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
#
# SPDX-License-Identifier: MIT

inherit dpkg-raw

DESCRIPTION = "This service grows the last partition to the full medium during first boot"

+MAINTAINER = "isar-users <isar-...@googlegroups.com>"

DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux"

diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 759ff8b61ccd..57055ccdcbd8 100755

--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -3,7 +3,7 @@
# Resize last partition to full medium size
#
# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
#
# SPDX-License-Identifier: MIT

@@ -57,8 +57,29 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
# Inform the kernel about the partitioning change
partx -u "${LAST_PART}"

-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
+# this is for debian stretch or systemd < 236
+if [ ! -x /lib/systemd/systemd-growfs ]; then
+ # Do not fail resize2fs if no mtab entry is found, e.g.,
+ # when using systemd mount units.
+ export EXT2FS_NO_MTAB_OK=1

-resize2fs "${LAST_PART}"
+ resize2fs "${LAST_PART}"
+ exit 0

+fi

+
+if grep -q x-systemd.growfs /etc/fstab; then
+ echo "Found x-systemd.growfs option in /etc/fstab, won't call it explicitly." >&2
+ exit 0
+fi

+
+# mount $LAST_PART out of tree, so we won't conflict with other mounts
+MOUNT_POINT=$(mktemp -d -p /mnt "$(basename "$0").XXXXXXXXXX")
+if [ ! -d "${MOUNT_POINT}" ]; then
+ echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2
+ exit 1
+fi
+
+mount "${LAST_PART}" "${MOUNT_POINT}"
+/lib/systemd/systemd-growfs "${MOUNT_POINT}"
+umount "${MOUNT_POINT}"
+rmdir "${MOUNT_POINT}"

--
2.37.4

[Henning Schild]

I ran a set of manual tests on that. stretch worked and did resize, on
bullseye i tried ext4 and btrfs, worked fine.

Henning

Am Wed, 26 Oct 2022 10:20:59 +0200
schrieb Henning Schild <henning...@siemens.com>:

[Anton Mikanovich]

26.10.2022 11:23, Henning Schild wrote:
> I ran a set of manual tests on that. stretch worked and did resize, on
> bullseye i tried ext4 and btrfs, worked fine.
>
> Henning

Great, thanks for testing. We will also check it and I hope merge soon.

If speaking about stretch itself, we should decide a date of dropping it to
allow downstreams still use it move to the next distro.

[Raphael Lisicki]

On 25.10.22 16:31, Henning Schild wrote:
>
> I have read that thread and pointed out there that systemd resize will
> be coming, and that people can try it already to see if it solves their
> problem. Will take Raphael in Cc.
>
> Raphael please have a look at the following patch, it might just solve
> your resize race.
>

I did not manage to test this yet, but I found quite some documentation
by systemd on the very same issue [1] and its shortcomings. I would be
moderately confident that systemd resize and systemd-udev will not clash.

The mentioned udevadm lock command is not available with debian bullseye.



[1] https://systemd.io/BLOCK_DEVICE_LOCKING/


best regards
raphael

[Anton Mikanovich]

Applied to next, thanks.

[Henning Schild]

Am Wed, 26 Oct 2022 10:21:01 +0200
schrieb Henning Schild <henning...@siemens.com>:

We have some layers that make parts of the filesystem structure
read-only. While this patch will be reverted the code will come back
with another patch i will propose.

I will switch from -p /mnt to -p "", which essentially means going to
/tmp or allowing people to set TMPDIR

Henning