[PATCH 0/4] Update WIC to the latest revision

Anton Mikanovich

unread,

May 8, 2024, 10:38:14 AMMay 8

to isar-...@googlegroups.com, Anton Mikanovich

This patchset updates WIC libs to the latest OE-core revision (version
5.0.0, codename Scarthgap).
Commit a4237f08 ("wic/plugins/images/direct: Allow changes in fstab on rootfs")
was reapplied because it is needed for some Isar targets.
There is also a new fix to keep compatibility with Debian Buster and
Ubuntu Focal which prevent those distros from having full identifier
reproducibility during image prepare in WIC.

Anton Mikanovich (4):
Revert "wic/plugins/images/direct: Allow changes in fstab on rootfs"
wic: Update to the latest revision
Revert "Revert "wic/plugins/images/direct: Allow changes in fstab on
rootfs""
wic: Restore compatibility with old distros

.../lib/wic/canned-wks/efi-bootdisk.wks.in | 2 +-
.../lib/wic/canned-wks/qemux86-directdisk.wks | 2 +-
scripts/lib/wic/filemap.py | 7 +
scripts/lib/wic/help.py | 2 +-
scripts/lib/wic/ksparser.py | 6 +-
scripts/lib/wic/misc.py | 2 +-
scripts/lib/wic/partition.py | 47 +++++-
scripts/lib/wic/plugins/imager/direct.py | 134 ++++++++++++++----
scripts/lib/wic/plugins/source/bootimg-efi.py | 108 +++++++++++---
.../wic/plugins/source/bootimg-partition.py | 9 +-
.../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-
scripts/lib/wic/plugins/source/empty.py | 59 +++++++-
scripts/lib/wic/plugins/source/rawcopy.py | 9 +-
scripts/lib/wic/plugins/source/rootfs.py | 4 +-
14 files changed, 336 insertions(+), 61 deletions(-)

--
2.34.1

Anton Mikanovich

unread,

May 8, 2024, 10:38:15 AMMay 8

to isar-...@googlegroups.com, Anton Mikanovich

This reverts commit a4237f08e67ad677e26d47a64d68e32417545a14.
---
scripts/lib/wic/plugins/imager/direct.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index c44159b2..da483dae 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
updated = False
for part in self.parts:
if not part.realnum or not part.mountpoint \
- or not part.mountpoint.startswith('/'):
+ or part.mountpoint == "/" or not part.mountpoint.startswith('/'):
continue

if part.use_uuid:
--
2.34.1

Anton Mikanovich

unread,

May 8, 2024, 10:38:16 AMMay 8

to isar-...@googlegroups.com, Anton Mikanovich

OE-core Revision: 7ef767d84d56b25498e45db83bb8f9d9caebeaf9.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

scripts/lib/wic/partition.py | 47 ++++++-
scripts/lib/wic/plugins/imager/direct.py | 122 ++++++++++++++----
scripts/lib/wic/plugins/source/bootimg-efi.py | 108 +++++++++++++---

.../wic/plugins/source/bootimg-partition.py | 9 +-
.../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-

scripts/lib/wic/plugins/source/empty.py | 59 ++++++++-

scripts/lib/wic/plugins/source/rawcopy.py | 9 +-
scripts/lib/wic/plugins/source/rootfs.py | 4 +-

14 files changed, 324 insertions(+), 61 deletions(-)

diff --git a/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in b/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in
index 7300e65e..2fd286ff 100644
--- a/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in
+++ b/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in
@@ -1,3 +1,3 @@
bootloader --ptable gpt
-part /boot --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/boot --fstype=vfat --label boot --active --align 1024 --use-uuid --overhead-factor 1.0
+part /boot --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/boot --fstype=vfat --label boot --active --align 1024 --use-uuid --overhead-factor 1.1
part / --source rootfs --fstype=ext4 --label root --align 1024 --exclude-path boot/
diff --git a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks b/scripts/lib/wic/canned-wks/qemux86-directdisk.wks
index 22b45217..80899761 100644
--- a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks
+++ b/scripts/lib/wic/canned-wks/qemux86-directdisk.wks
@@ -4,5 +4,5 @@

include common.wks.inc

-bootloader --timeout=0 --append="rw oprofile.timer=1 rootfstype=ext4 "
+bootloader --timeout=0 --append="rw oprofile.timer=1 rootfstype=ext4 console=tty console=ttyS0 "

diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 4d9da281..85b39d5d 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -46,6 +46,13 @@ def get_block_size(file_obj):
bsize = stat.st_blksize
else:
raise IOError("Unable to determine block size")
+
+ # The logic in this script only supports a maximum of a 4KB
+ # block size
+ max_block_size = 4 * 1024
+ if bsize > max_block_size:
+ bsize = max_block_size
+
return bsize

class ErrorNotSupp(Exception):
diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index 73e3380c..163535e4 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -1118,7 +1118,7 @@ COMMAND:
TOPIC:
overview - Presents an overall overview of Wic
plugins - Presents an overview and API for Wic plugins
- kickstart - Presents a Wic kicstart file reference
+ kickstart - Presents a Wic kickstart file reference

Examples:
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index a49b7b97..7ef3dc83 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -159,7 +159,7 @@ class KickStart():
part.add_argument('--fstype', default='vfat',
choices=('ext2', 'ext3', 'ext4', 'btrfs',
'squashfs', 'vfat', 'msdos', 'erofs',
- 'swap'))
+ 'swap', 'none'))
part.add_argument('--mkfs-extraopts', default='')
part.add_argument('--label')
part.add_argument('--use-label', action='store_true')
@@ -171,6 +171,7 @@ class KickStart():
part.add_argument('--rootfs-dir')
part.add_argument('--type', default='primary',
choices = ('primary', 'logical'))
+ part.add_argument('--hidden', action='store_true')

# --size and --fixed-size cannot be specified together; options
# ----extra-space and --overhead-factor should also raise a parser
@@ -187,11 +188,12 @@ class KickStart():
part.add_argument('--uuid')
part.add_argument('--fsuuid')
part.add_argument('--no-fstab-update', action='store_true')
+ part.add_argument('--mbr', action='store_true')

bootloader = subparsers.add_parser('bootloader')
bootloader.add_argument('--append')
bootloader.add_argument('--configfile')
- bootloader.add_argument('--ptable', choices=('msdos', 'gpt'),
+ bootloader.add_argument('--ptable', choices=('msdos', 'gpt', 'gpt-hybrid'),
default='msdos')
bootloader.add_argument('--timeout', type=int)
bootloader.add_argument('--source')
diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 2b90821b..1a7c140f 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -25,7 +25,7 @@ from wic import WicError
logger = logging.getLogger('wic')

# executable -> recipe pairs for exec_native_cmd
-NATIVE_RECIPES = {"bmaptool": "bmap-tools",
+NATIVE_RECIPES = {"bmaptool": "bmaptool",
"dumpe2fs": "e2fsprogs",
"grub-mkimage": "grub-efi",
"isohybrid": "syslinux",
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index e50871b8..795707ec 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -59,6 +59,8 @@ class Partition():
self.updated_fstab_path = None
self.has_fstab = False
self.update_fstab_in_rootfs = False
+ self.hidden = args.hidden
+ self.mbr = args.mbr

self.lineno = lineno
self.source_file = ""
@@ -133,6 +135,8 @@ class Partition():
self.update_fstab_in_rootfs = True

if not self.source:
+ if self.fstype == "none" or self.no_table:
+ return
if not self.size and not self.fixed_size:
raise WicError("The %s partition has a size of zero. Please "
"specify a non-zero --size/--fixed-size for that "
@@ -280,6 +284,20 @@ class Partition():

extraopts = self.mkfs_extraopts or "-F -i 8192"

+ if os.getenv('SOURCE_DATE_EPOCH'):
+ sde_time = int(os.getenv('SOURCE_DATE_EPOCH'))
+ if pseudo:
+ pseudo = "export E2FSPROGS_FAKE_TIME=%s;%s " % (sde_time, pseudo)
+ else:
+ pseudo = "export E2FSPROGS_FAKE_TIME=%s; " % sde_time
+
+ # Set hash_seed to generate deterministic directory indexes
+ namespace = uuid.UUID("e7429877-e7b3-4a68-a5c9-2f2fdf33d460")
+ if self.fsuuid:
+ namespace = uuid.UUID(self.fsuuid)
+ hash_seed = str(uuid.uuid5(namespace, str(sde_time)))
+ extraopts += " -E hash_seed=%s" % hash_seed
+
label_str = ""
if self.label:
label_str = "-L %s" % self.label
@@ -300,6 +318,30 @@ class Partition():
mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)

+ if os.getenv('SOURCE_DATE_EPOCH'):
+ sde_time = hex(int(os.getenv('SOURCE_DATE_EPOCH')))
+ debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
+ files = []
+ for root, dirs, others in os.walk(rootfs_dir):
+ base = root.replace(rootfs_dir, "").rstrip(os.sep)
+ files += [ "/" if base == "" else base ]
+ files += [ base + "/" + n for n in dirs + others ]
+ with open(debugfs_script_path, "w") as f:
+ f.write("set_current_time %s\n" % (sde_time))
+ if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
+ f.write("set_inode_field /etc/fstab mtime %s\n" % (sde_time))
+ f.write("set_inode_field /etc/fstab mtime_extra 0\n")
+ for file in set(files):
+ for time in ["atime", "ctime", "crtime"]:
+ f.write("set_inode_field \"%s\" %s %s\n" % (file, time, sde_time))
+ f.write("set_inode_field \"%s\" %s_extra 0\n" % (file, time))
+ for time in ["wtime", "mkfs_time", "lastcheck"]:
+ f.write("set_super_value %s %s\n" % (time, sde_time))
+ for time in ["mtime", "first_error_time", "last_error_time"]:
+ f.write("set_super_value %s 0\n" % (time))
+ debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
+ exec_native_cmd(debugfs_cmd, native_sysroot)
+
self.check_for_Y2038_problem(rootfs, native_sysroot)

def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
@@ -353,7 +395,7 @@ class Partition():
exec_native_cmd(mcopy_cmd, native_sysroot)

if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
- mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
+ mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
exec_native_cmd(mcopy_cmd, native_sysroot)

chmod_cmd = "chmod 644 %s" % rootfs
@@ -381,6 +423,9 @@ class Partition():
(extraopts, self.fsuuid, rootfs, rootfs_dir)
exec_native_cmd(erofs_cmd, native_sysroot, pseudo=pseudo)

+ def prepare_empty_partition_none(self, rootfs, oe_builddir, native_sysroot):
+ pass
+
def prepare_empty_partition_ext(self, rootfs, oe_builddir,
native_sysroot):
"""
diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index da483dae..a1d15265 100644

--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
updated = False
for part in self.parts:
if not part.realnum or not part.mountpoint \

- or part.mountpoint == "/" or not part.mountpoint.startswith('/'):
+ or part.mountpoint == "/" or not (part.mountpoint.startswith('/') or part.mountpoint == "swap"):
continue

if part.use_uuid:
@@ -149,6 +149,9 @@ class DirectPlugin(ImagerPlugin):
self.updated_fstab_path = os.path.join(self.workdir, "fstab")
with open(self.updated_fstab_path, "w") as f:
f.writelines(fstab_lines)
+ if os.getenv('SOURCE_DATE_EPOCH'):
+ fstab_time = int(os.getenv('SOURCE_DATE_EPOCH'))
+ os.utime(self.updated_fstab_path, (fstab_time, fstab_time))

def _full_path(self, path, name, extention):
""" Construct full file path to a file we generate. """
@@ -310,7 +313,10 @@ class PartitionedImage():
# all partitions (in bytes)
self.ptable_format = ptable_format # Partition table format
# Disk system identifier
- self.identifier = random.SystemRandom().randint(1, 0xffffffff)
+ if os.getenv('SOURCE_DATE_EPOCH'):
+ self.identifier = random.Random(int(os.getenv('SOURCE_DATE_EPOCH'))).randint(1, 0xffffffff)
+ else:
+ self.identifier = random.SystemRandom().randint(1, 0xffffffff)

self.partitions = partitions
self.partimages = []
@@ -336,7 +342,7 @@ class PartitionedImage():
# generate parition and filesystem UUIDs
for part in self.partitions:
if not part.uuid and part.use_uuid:
- if self.ptable_format == 'gpt':
+ if self.ptable_format in ('gpt', 'gpt-hybrid'):
part.uuid = str(uuid.uuid4())
else: # msdos partition table
part.uuid = '%08x-%02d' % (self.identifier, part.realnum)
@@ -392,6 +398,10 @@ class PartitionedImage():
raise WicError("setting custom partition type is not " \
"implemented for msdos partitions")

+ if part.mbr and self.ptable_format != 'gpt-hybrid':
+ raise WicError("Partition may only be included in MBR with " \
+ "a gpt-hybrid partition table")
+
# Get the disk where the partition is located
self.numpart += 1
if not part.no_table:
@@ -400,7 +410,7 @@ class PartitionedImage():
if self.numpart == 1:
if self.ptable_format == "msdos":
overhead = MBR_OVERHEAD
- elif self.ptable_format == "gpt":
+ elif self.ptable_format in ("gpt", "gpt-hybrid"):
overhead = GPT_OVERHEAD

# Skip one sector required for the partitioning scheme overhead
@@ -484,7 +494,7 @@ class PartitionedImage():
# Once all the partitions have been layed out, we can calculate the
# minumim disk size
self.min_size = self.offset
- if self.ptable_format == "gpt":
+ if self.ptable_format in ("gpt", "gpt-hybrid"):
self.min_size += GPT_OVERHEAD

self.min_size *= self.sector_size
@@ -505,22 +515,49 @@ class PartitionedImage():

return exec_native_cmd(cmd, self.native_sysroot)

+ def _write_identifier(self, device, identifier):
+ logger.debug("Set disk identifier %x", identifier)
+ with open(device, 'r+b') as img:
+ img.seek(0x1B8)
+ img.write(identifier.to_bytes(4, 'little'))
+
+ def _make_disk(self, device, ptable_format, min_size):
+ logger.debug("Creating sparse file %s", device)
+ with open(device, 'w') as sparse:
+ os.ftruncate(sparse.fileno(), min_size)
+
+ logger.debug("Initializing partition table for %s", device)
+ exec_native_cmd("parted -s %s mklabel %s" % (device, ptable_format),
+ self.native_sysroot)
+
+ def _write_disk_guid(self):
+ if self.ptable_format in ('gpt', 'gpt-hybrid'):
+ if os.getenv('SOURCE_DATE_EPOCH'):
+ self.disk_guid = uuid.UUID(int=int(os.getenv('SOURCE_DATE_EPOCH')))
+ else:
+ self.disk_guid = uuid.uuid4()
+
+ logger.debug("Set disk guid %s", self.disk_guid)
+ sfdisk_cmd = "sfdisk --disk-id %s %s" % (self.path, self.disk_guid)
+ exec_native_cmd(sfdisk_cmd, self.native_sysroot)
+
def create(self):
- logger.debug("Creating sparse file %s", self.path)
- with open(self.path, 'w') as sparse:
- os.ftruncate(sparse.fileno(), self.min_size)
+ self._make_disk(self.path,
+ "gpt" if self.ptable_format == "gpt-hybrid" else self.ptable_format,
+ self.min_size)

- logger.debug("Initializing partition table for %s", self.path)
- exec_native_cmd("parted -s %s mklabel %s" %
- (self.path, self.ptable_format), self.native_sysroot)
+ self._write_identifier(self.path, self.identifier)
+ self._write_disk_guid()

- logger.debug("Set disk identifier %x", self.identifier)
- with open(self.path, 'r+b') as img:
- img.seek(0x1B8)
- img.write(self.identifier.to_bytes(4, 'little'))
+ if self.ptable_format == "gpt-hybrid":
+ mbr_path = self.path + ".mbr"
+ self._make_disk(mbr_path, "msdos", self.min_size)
+ self._write_identifier(mbr_path, self.identifier)

logger.debug("Creating partitions")

+ hybrid_mbr_part_num = 0
+
for part in self.partitions:
if part.num == 0:
continue
@@ -565,11 +602,19 @@ class PartitionedImage():
self._create_partition(self.path, part.type,
parted_fs_type, part.start, part.size_sec)

- if part.part_name:
+ if self.ptable_format == "gpt-hybrid" and part.mbr:
+ hybrid_mbr_part_num += 1
+ if hybrid_mbr_part_num > 4:
+ raise WicError("Extended MBR partitions are not supported in hybrid MBR")
+ self._create_partition(mbr_path, "primary",
+ parted_fs_type, part.start, part.size_sec)
+
+ if self.ptable_format in ("gpt", "gpt-hybrid") and (part.part_name or part.label):
+ partition_label = part.part_name if part.part_name else part.label
logger.debug("partition %d: set name to %s",
- part.num, part.part_name)
+ part.num, partition_label)
exec_native_cmd("sgdisk --change-name=%d:%s %s" % \
- (part.num, part.part_name,
+ (part.num, partition_label,
self.path), self.native_sysroot)

if part.part_type:
@@ -579,32 +624,55 @@ class PartitionedImage():
(part.num, part.part_type,
self.path), self.native_sysroot)

- if part.uuid and self.ptable_format == "gpt":
+ if part.uuid and self.ptable_format in ("gpt", "gpt-hybrid"):
logger.debug("partition %d: set UUID to %s",
part.num, part.uuid)
exec_native_cmd("sgdisk --partition-guid=%d:%s %s" % \
(part.num, part.uuid, self.path),
self.native_sysroot)

- if part.label and self.ptable_format == "gpt":
- logger.debug("partition %d: set name to %s",
- part.num, part.label)
- exec_native_cmd("parted -s %s name %d %s" % \
- (self.path, part.num, part.label),
- self.native_sysroot)
-
if part.active:
- flag_name = "legacy_boot" if self.ptable_format == 'gpt' else "boot"
+ flag_name = "legacy_boot" if self.ptable_format in ('gpt', 'gpt-hybrid') else "boot"
logger.debug("Set '%s' flag for partition '%s' on disk '%s'",
flag_name, part.num, self.path)
exec_native_cmd("parted -s %s set %d %s on" % \
(self.path, part.num, flag_name),
self.native_sysroot)
+ if self.ptable_format == 'gpt-hybrid' and part.mbr:
+ exec_native_cmd("parted -s %s set %d %s on" % \
+ (mbr_path, hybrid_mbr_part_num, "boot"),
+ self.native_sysroot)
if part.system_id:
exec_native_cmd("sfdisk --part-type %s %s %s" % \
(self.path, part.num, part.system_id),
self.native_sysroot)

+ if part.hidden and self.ptable_format == "gpt":
+ logger.debug("Set hidden attribute for partition '%s' on disk '%s'",
+ part.num, self.path)
+ exec_native_cmd("sfdisk --part-attrs %s %s RequiredPartition" % \
+ (self.path, part.num),
+ self.native_sysroot)
+
+ if self.ptable_format == "gpt-hybrid":
+ # Write a protective GPT partition
+ hybrid_mbr_part_num += 1
+ if hybrid_mbr_part_num > 4:
+ raise WicError("Extended MBR partitions are not supported in hybrid MBR")
+
+ # parted cannot directly create a protective GPT partition, so
+ # create with an arbitrary type, then change it to the correct type
+ # with sfdisk
+ self._create_partition(mbr_path, "primary", "fat32", 1, GPT_OVERHEAD)
+ exec_native_cmd("sfdisk --part-type %s %d 0xee" % (mbr_path, hybrid_mbr_part_num),
+ self.native_sysroot)
+
+ # Copy hybrid MBR
+ with open(mbr_path, "rb") as mbr_file:
+ with open(self.path, "r+b") as image_file:
+ mbr = mbr_file.read(512)
+ image_file.write(mbr)
+
def cleanup(self):
pass

diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 634a808d..13a9cddf 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -220,6 +220,8 @@ class BootimgEFIPlugin(SourcePlugin):
cls.do_configure_grubefi(hdddir, creator, cr_workdir, source_params)
elif source_params['loader'] == 'systemd-boot':
cls.do_configure_systemdboot(hdddir, creator, cr_workdir, source_params)
+ elif source_params['loader'] == 'uefi-kernel':
+ pass
else:
raise WicError("unrecognized bootimg-efi loader: %s" % source_params['loader'])
except KeyError:
@@ -330,40 +332,78 @@ class BootimgEFIPlugin(SourcePlugin):
shutil.copyfileobj(in_file, initrd)
initrd.close()

+ # Searched by systemd-boot:
+ # https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images
+ install_cmd = "install -d %s/EFI/Linux" % hdddir
+ exec_cmd(install_cmd)
+
+ staging_dir_host = get_bitbake_var("STAGING_DIR_HOST")
+ target_sys = get_bitbake_var("TARGET_SYS")
+
+ objdump_cmd = "%s-objdump" % target_sys
+ objdump_cmd += " -p %s" % efi_stub
+ objdump_cmd += " | awk '{ if ($1 == \"SectionAlignment\"){print $2} }'"
+
+ ret, align_str = exec_native_cmd(objdump_cmd, native_sysroot)
+ align = int(align_str, 16)
+
+ objdump_cmd = "%s-objdump" % target_sys
+ objdump_cmd += " -h %s | tail -2" % efi_stub
+ ret, output = exec_native_cmd(objdump_cmd, native_sysroot)
+
+ offset = int(output.split()[2], 16) + int(output.split()[3], 16)
+
+ osrel_off = offset + align - offset % align
+ osrel_path = "%s/usr/lib/os-release" % staging_dir_host
+ osrel_sz = os.stat(osrel_path).st_size
+
+ cmdline_off = osrel_off + osrel_sz
+ cmdline_off = cmdline_off + align - cmdline_off % align
+ cmdline_sz = os.stat(cmdline.name).st_size
+
+ dtb_off = cmdline_off + cmdline_sz
+ dtb_off = dtb_off + align - dtb_off % align
+
dtb = source_params.get('dtb')
if dtb:
if ';' in dtb:
raise WicError("Only one DTB supported, exiting")
- dtb_params = '--add-section .dtb=%s/%s --change-section-vma .dtb=0x40000' % \
- (deploy_dir, dtb)
+ dtb_path = "%s/%s" % (deploy_dir, dtb)
+ dtb_params = '--add-section .dtb=%s --change-section-vma .dtb=0x%x' % \
+ (dtb_path, dtb_off)
+ linux_off = dtb_off + os.stat(dtb_path).st_size
+ linux_off = linux_off + align - linux_off % align
else:
dtb_params = ''
+ linux_off = dtb_off

- # Searched by systemd-boot:
- # https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images
- install_cmd = "install -d %s/EFI/Linux" % hdddir
- exec_cmd(install_cmd)
+ linux_path = "%s/%s" % (staging_kernel_dir, kernel)
+ linux_sz = os.stat(linux_path).st_size

- staging_dir_host = get_bitbake_var("STAGING_DIR_HOST")
- target_sys = get_bitbake_var("TARGET_SYS")
+ initrd_off = linux_off + linux_sz
+ initrd_off = initrd_off + align - initrd_off % align

# https://www.freedesktop.org/software/systemd/man/systemd-stub.html
objcopy_cmd = "%s-objcopy" % target_sys
- objcopy_cmd += " --add-section .osrel=%s/usr/lib/os-release" % staging_dir_host
- objcopy_cmd += " --change-section-vma .osrel=0x20000"
+ objcopy_cmd += " --enable-deterministic-archives"
+ objcopy_cmd += " --preserve-dates"
+ objcopy_cmd += " --add-section .osrel=%s" % osrel_path
+ objcopy_cmd += " --change-section-vma .osrel=0x%x" % osrel_off
objcopy_cmd += " --add-section .cmdline=%s" % cmdline.name
- objcopy_cmd += " --change-section-vma .cmdline=0x30000"
+ objcopy_cmd += " --change-section-vma .cmdline=0x%x" % cmdline_off
objcopy_cmd += dtb_params
- objcopy_cmd += " --add-section .linux=%s/%s" % (staging_kernel_dir, kernel)
- objcopy_cmd += " --change-section-vma .linux=0x2000000"
+ objcopy_cmd += " --add-section .linux=%s" % linux_path
+ objcopy_cmd += " --change-section-vma .linux=0x%x" % linux_off
objcopy_cmd += " --add-section .initrd=%s" % initrd.name
- objcopy_cmd += " --change-section-vma .initrd=0x3000000"
+ objcopy_cmd += " --change-section-vma .initrd=0x%x" % initrd_off
objcopy_cmd += " %s %s/EFI/Linux/linux.efi" % (efi_stub, hdddir)
+
exec_native_cmd(objcopy_cmd, native_sysroot)
else:
- install_cmd = "install -m 0644 %s/%s %s/%s" % \
- (staging_kernel_dir, kernel, hdddir, kernel)
- exec_cmd(install_cmd)
+ if source_params.get('install-kernel-into-boot-dir') != 'false':
+ install_cmd = "install -m 0644 %s/%s %s/%s" % \
+ (staging_kernel_dir, kernel, hdddir, kernel)
+ exec_cmd(install_cmd)

if get_bitbake_var("IMAGE_EFI_BOOT_FILES"):
for src_path, dst_path in cls.install_task:
@@ -385,6 +425,28 @@ class BootimgEFIPlugin(SourcePlugin):
for mod in [x for x in os.listdir(kernel_dir) if x.startswith("systemd-")]:
cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, mod[8:])
exec_cmd(cp_cmd, True)
+ elif source_params['loader'] == 'uefi-kernel':
+ kernel = get_bitbake_var("KERNEL_IMAGETYPE")
+ if not kernel:
+ raise WicError("Empty KERNEL_IMAGETYPE %s\n" % target)
+ target = get_bitbake_var("TARGET_SYS")
+ if not target:
+ raise WicError("Unknown arch (TARGET_SYS) %s\n" % target)
+
+ if re.match("x86_64", target):
+ kernel_efi_image = "bootx64.efi"
+ elif re.match('i.86', target):
+ kernel_efi_image = "bootia32.efi"
+ elif re.match('aarch64', target):
+ kernel_efi_image = "bootaa64.efi"
+ elif re.match('arm', target):
+ kernel_efi_image = "bootarm.efi"
+ else:
+ raise WicError("UEFI stub kernel is incompatible with target %s" % target)
+
+ for mod in [x for x in os.listdir(kernel_dir) if x.startswith(kernel)]:
+ cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, kernel_efi_image)
+ exec_cmd(cp_cmd, True)
else:
raise WicError("unrecognized bootimg-efi loader: %s" %
source_params['loader'])
@@ -396,6 +458,11 @@ class BootimgEFIPlugin(SourcePlugin):
cp_cmd = "cp %s %s/" % (startup, hdddir)
exec_cmd(cp_cmd, True)

+ for paths in part.include_path or []:
+ for path in paths:
+ cp_cmd = "cp -r %s %s/" % (path, hdddir)
+ exec_cmd(cp_cmd, True)
+
du_cmd = "du -bks %s" % hdddir
out = exec_cmd(du_cmd)
blocks = int(out.split()[0])
@@ -410,6 +477,13 @@ class BootimgEFIPlugin(SourcePlugin):
logger.debug("Added %d extra blocks to %s to get to %d total blocks",
extra_blocks, part.mountpoint, blocks)

+ # required for compatibility with certain devices expecting file system
+ # block count to be equal to partition block count
+ if blocks < part.fixed_size:
+ blocks = part.fixed_size
+ logger.debug("Overriding %s to %d total blocks for compatibility",
+ part.mountpoint, blocks)
+
# dosfs image, created by mkdosfs
bootimg = "%s/boot.img" % cr_workdir

diff --git a/scripts/lib/wic/plugins/source/bootimg-partition.py b/scripts/lib/wic/plugins/source/bootimg-partition.py
index 5dbe2558..94183174 100644
--- a/scripts/lib/wic/plugins/source/bootimg-partition.py
+++ b/scripts/lib/wic/plugins/source/bootimg-partition.py
@@ -1,4 +1,6 @@
#
+# Imported from openembedded-core
+#
# SPDX-License-Identifier: GPL-2.0-only
#
# DESCRIPTION
@@ -30,6 +32,7 @@ class BootimgPartitionPlugin(SourcePlugin):
"""

name = 'bootimg-partition'
+ image_boot_files_var_name = 'IMAGE_BOOT_FILES'

@classmethod
def do_configure_partition(cls, part, source_params, cr, cr_workdir,
@@ -54,12 +57,12 @@ class BootimgPartitionPlugin(SourcePlugin):
else:
var = ""

- boot_files = get_bitbake_var("IMAGE_BOOT_FILES" + var)
+ boot_files = get_bitbake_var(cls.image_boot_files_var_name + var)
if boot_files is not None:
break

if boot_files is None:
- raise WicError('No boot files defined, IMAGE_BOOT_FILES unset for entry #%d' % part.lineno)
+ raise WicError('No boot files defined, %s unset for entry #%d' % (cls.image_boot_files_var_name, part.lineno))

logger.debug('Boot files: %s', boot_files)

@@ -110,7 +113,7 @@ class BootimgPartitionPlugin(SourcePlugin):
# Use a custom configuration for extlinux.conf
extlinux_conf = custom_cfg
logger.debug("Using custom configuration file "
- "%s for extlinux.cfg", configfile)
+ "%s for extlinux.conf", configfile)
else:
raise WicError("configfile is specified but failed to "
"get it from %s." % configfile)
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index 32e47f18..a207a835 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -122,7 +122,7 @@ class BootimgPcbiosPlugin(SourcePlugin):
syslinux_conf += "DEFAULT boot\n"
syslinux_conf += "LABEL boot\n"

- kernel = "/vmlinuz"
+ kernel = "/" + get_bitbake_var("KERNEL_IMAGETYPE")
syslinux_conf += "KERNEL " + kernel + "\n"

syslinux_conf += "APPEND label=boot root=%s %s\n" % \
@@ -155,8 +155,8 @@ class BootimgPcbiosPlugin(SourcePlugin):
kernel = "%s-%s.bin" % \
(get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME"))

- cmds = ("install -m 0644 %s/%s %s/vmlinuz" %
- (staging_kernel_dir, kernel, hdddir),
+ cmds = ("install -m 0644 %s/%s %s/%s" %
+ (staging_kernel_dir, kernel, hdddir, get_bitbake_var("KERNEL_IMAGETYPE")),
"install -m 444 %s/syslinux/ldlinux.sys %s/ldlinux.sys" %
(bootimg_dir, hdddir),
"install -m 0644 %s/syslinux/vesamenu.c32 %s/vesamenu.c32" %
diff --git a/scripts/lib/wic/plugins/source/empty.py b/scripts/lib/wic/plugins/source/empty.py
index 041617d6..c1903ca5 100644
--- a/scripts/lib/wic/plugins/source/empty.py
+++ b/scripts/lib/wic/plugins/source/empty.py
@@ -1,4 +1,6 @@
#
+# Imported from openembedded-core
+#
# SPDX-License-Identifier: MIT
#

@@ -7,9 +9,19 @@
# To use it you must pass "empty" as argument for the "--source" parameter in
# the wks file. For example:
# part foo --source empty --ondisk sda --size="1024" --align 1024
+#
+# The plugin supports writing zeros to the start of the
+# partition. This is useful to overwrite old content like
+# filesystem signatures which may be re-recognized otherwise.
+# This feature can be enabled with
+# '--sourceparams="[fill|size=<N>[S|s|K|k|M|G]][,][bs=<N>[S|s|K|k|M|G]]"'
+# Conflicting or missing options throw errors.

import logging
+import os

+from wic import WicError
+from wic.ksparser import sizetype
from wic.pluginbase import SourcePlugin

logger = logging.getLogger('wic')
@@ -17,6 +29,16 @@ logger = logging.getLogger('wic')
class EmptyPartitionPlugin(SourcePlugin):
"""
Populate unformatted empty partition.
+
+ The following sourceparams are supported:
+ - fill
+ Fill the entire partition with zeros. Requires '--fixed-size' option
+ to be set.
+ - size=<N>[S|s|K|k|M|G]
+ Set the first N bytes of the partition to zero. Default unit is 'K'.
+ - bs=<N>[S|s|K|k|M|G]
+ Write at most N bytes at a time during source file creation.
+ Defaults to '1M'. Default unit is 'K'.
"""

name = 'empty'
@@ -29,4 +51,39 @@ class EmptyPartitionPlugin(SourcePlugin):
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
"""
- return
+ get_byte_count = sizetype('K', True)
+ size = 0
+
+ if 'fill' in source_params and 'size' in source_params:
+ raise WicError("Conflicting source parameters 'fill' and 'size' specified, exiting.")
+
+ # Set the size of the zeros to be written to the partition
+ if 'fill' in source_params:
+ if part.fixed_size == 0:
+ raise WicError("Source parameter 'fill' only works with the '--fixed-size' option, exiting.")
+ size = get_byte_count(part.fixed_size)
+ elif 'size' in source_params:
+ size = get_byte_count(source_params['size'])
+
+ if size == 0:
+ # Nothing to do, create empty partition
+ return
+
+ if 'bs' in source_params:
+ bs = get_byte_count(source_params['bs'])
+ else:
+ bs = get_byte_count('1M')
+
+ # Create a binary file of the requested size filled with zeros
+ source_file = os.path.join(cr_workdir, 'empty-plugin-zeros%s.bin' % part.lineno)
+ if not os.path.exists(os.path.dirname(source_file)):
+ os.makedirs(os.path.dirname(source_file))
+
+ quotient, remainder = divmod(size, bs)
+ with open(source_file, 'wb') as file:
+ for _ in range(quotient):
+ file.write(bytearray(bs))
+ file.write(bytearray(remainder))
+
+ part.size = (size + 1024 - 1) // 1024 # size in KB rounded up
+ part.source_file = source_file
diff --git a/scripts/lib/wic/plugins/source/rawcopy.py b/scripts/lib/wic/plugins/source/rawcopy.py
index 7c90cd3c..73017e86 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -1,4 +1,6 @@
#
+# Imported from openembedded-core
+#
# SPDX-License-Identifier: GPL-2.0-only
#

@@ -23,6 +25,10 @@ class RawCopyPlugin(SourcePlugin):

@staticmethod
def do_image_label(fstype, dst, label):
+ # don't create label when fstype is none
+ if fstype == 'none':
+ return
+
if fstype.startswith('ext'):
cmd = 'tune2fs -L %s %s' % (label, dst)
elif fstype in ('msdos', 'vfat'):
@@ -52,7 +58,8 @@ class RawCopyPlugin(SourcePlugin):
decompressor = {
".bz2": "bzip2",
".gz": "gzip",
- ".xz": "xz"
+ ".xz": "xz",
+ ".zst": "zstd -f",
}.get(extension)
if not decompressor:
raise WicError("Not supported compressor filename extension: %s" % extension)
diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
index fc06312e..c990143c 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -43,7 +43,7 @@ class RootfsPlugin(SourcePlugin):
# directory, or modify a directory outside OpenEmbedded).
full_path = os.path.realpath(os.path.join(rootfs_dir, path))
if not full_path.startswith(os.path.realpath(rootfs_dir)):
- logger.error("%s: Must point inside the rootfs:" % (cmd, path))
+ logger.error("%s: Must point inside the rootfs: %s" % (cmd, path))
sys.exit(1)

return full_path
@@ -224,7 +224,7 @@ class RootfsPlugin(SourcePlugin):
if part.update_fstab_in_rootfs and part.has_fstab and not part.no_fstab_update:
fstab_path = os.path.join(new_rootfs, "etc/fstab")
# Assume that fstab should always be owned by root with fixed permissions
- install_cmd = "install -m 0644 %s %s" % (part.updated_fstab_path, fstab_path)
+ install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path)
if new_pseudo:
pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
else:
--
2.34.1

Anton Mikanovich

unread,

May 8, 2024, 10:38:17 AMMay 8

to isar-...@googlegroups.com, Anton Mikanovich

After update of WIC it requires sfdisk to support --disk-id command,
which is false for Debian Buster and Ubuntu Focal.
Do not try to set disk identifier for those distro versions to keep
them buildable but with lack of reproducibility.

This hack should be removed after deprecation of Buster and Focal.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

scripts/lib/wic/plugins/imager/direct.py | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index 33a869ec..a65ae36a 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -530,6 +530,13 @@ class PartitionedImage():

exec_native_cmd("parted -s %s mklabel %s" % (device, ptable_format),

self.native_sysroot)

+ def _sfdisk_supports_disk_id(self):
+ ret, help_out = exec_native_cmd("sfdisk --help", self.native_sysroot)
+ if 'disk-id' not in help_out:
+ logger.warn("Current sfdisk version DOES NOT support changing disk ID")
+ return False
+ return True
+
def _write_disk_guid(self):

if self.ptable_format in ('gpt', 'gpt-hybrid'):

if os.getenv('SOURCE_DATE_EPOCH'):
@@ -537,6 +544,11 @@ class PartitionedImage():
else:
self.disk_guid = uuid.uuid4()

+ # Backport compatibility for Debian Buster and Ubuntu Focal
+ if not self._sfdisk_supports_disk_id():
+ logger.warn("Disk identifier can't be set, reproducibility is broken!")
+ return

+
logger.debug("Set disk guid %s", self.disk_guid)

sfdisk_cmd = "sfdisk --disk-id %s %s" % (self.path, self.disk_guid)

exec_native_cmd(sfdisk_cmd, self.native_sysroot)
--
2.34.1

Anton Mikanovich

unread,

May 8, 2024, 10:38:17 AMMay 8

to isar-...@googlegroups.com, Anton Mikanovich

Commit a4237f08 was reverted in oe-core, but still needs in Isar.
Revert it back until other implementation will be done.

This reverts commit 045e0da5ef147d3e9e0dc4ea8db95c0746984efa.

---
scripts/lib/wic/plugins/imager/direct.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index a1d15265..33a869ec 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py

@@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
updated = False
for part in self.parts:
if not part.realnum or not part.mountpoint \

- or part.mountpoint == "/" or not (part.mountpoint.startswith('/') or part.mountpoint == "swap"):
+ or not (part.mountpoint.startswith('/') or part.mountpoint == "swap"):
continue

if part.use_uuid:
--
2.34.1

Anton Mikanovich

unread,

May 8, 2024, 10:46:48 AMMay 8

to isar-...@googlegroups.com, Felix Moessbauer, Tobias Schmidl, Henning Schild, Baurzhan Ismagulov

08/05/2024 17:38, Anton Mikanovich wrote:
> Commit a4237f08 was reverted in oe-core, but still needs in Isar.
> Revert it back until other implementation will be done.
>
> This reverts commit 045e0da5ef147d3e9e0dc4ea8db95c0746984efa.

I'm not sure about the correct way of handling '/' exclusion
re-introduced in a new WIC revision, so if anyone have an opinion
please feel free to share it.

Anton Mikanovich

unread,

May 17, 2024, 7:44:46 AM (12 days ago) May 17

to isar-...@googlegroups.com, Anton Mikanovich

This patchset updates WIC libs to the latest OE-core revision (version
5.0.0, codename Scarthgap).
Commit a4237f08 ("wic/plugins/images/direct: Allow changes in fstab on rootfs")
was reapplied because it is needed for some Isar targets.
There is also a new fix to keep compatibility with Debian Buster and
Ubuntu Focal which prevent those distros from having full identifier
reproducibility during image prepare in WIC.

---
Changes since v1:
- Add missing signed-offs

Anton Mikanovich (4):
Revert "wic/plugins/images/direct: Allow changes in fstab on rootfs"
wic: Update to the latest revision
Revert "Revert "wic/plugins/images/direct: Allow changes in fstab on
rootfs""
wic: Restore compatibility with old distros

scripts/lib/wic/partition.py | 47 +++++-
scripts/lib/wic/plugins/imager/direct.py | 134 ++++++++++++++----
scripts/lib/wic/plugins/source/bootimg-efi.py | 108 +++++++++++---

.../wic/plugins/source/bootimg-partition.py | 9 +-
.../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-

scripts/lib/wic/plugins/source/empty.py | 59 +++++++-

scripts/lib/wic/plugins/source/rawcopy.py | 9 +-
scripts/lib/wic/plugins/source/rootfs.py | 4 +-

Anton Mikanovich

unread,

May 17, 2024, 7:44:47 AM (12 days ago) May 17

to isar-...@googlegroups.com, Anton Mikanovich

This change was reverted in oe-core.

This reverts commit a4237f08e67ad677e26d47a64d68e32417545a14.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>

---
scripts/lib/wic/plugins/imager/direct.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py

index c44159b2..da483dae 100644

--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
updated = False
for part in self.parts:
if not part.realnum or not part.mountpoint \

- or not part.mountpoint.startswith('/'):
+ or part.mountpoint == "/" or not part.mountpoint.startswith('/'):

Anton Mikanovich

unread,

May 17, 2024, 7:44:49 AM (12 days ago) May 17

to isar-...@googlegroups.com, Anton Mikanovich

Commit a4237f08 was reverted in oe-core, but still needs in Isar.
Revert it back until other implementation will be done.

This reverts commit 045e0da5ef147d3e9e0dc4ea8db95c0746984efa.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---
scripts/lib/wic/plugins/imager/direct.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py

index a1d15265..33a869ec 100644

--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
updated = False
for part in self.parts:
if not part.realnum or not part.mountpoint \

- or part.mountpoint == "/" or not (part.mountpoint.startswith('/') or part.mountpoint == "swap"):

+ or not (part.mountpoint.startswith('/') or part.mountpoint == "swap"):

Anton Mikanovich

unread,

May 17, 2024, 7:44:49 AM (12 days ago) May 17

to isar-...@googlegroups.com, Anton Mikanovich

OE-core Revision: 7ef767d84d56b25498e45db83bb8f9d9caebeaf9.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

scripts/lib/wic/partition.py | 47 ++++++-
scripts/lib/wic/plugins/imager/direct.py | 122 ++++++++++++++----
scripts/lib/wic/plugins/source/bootimg-efi.py | 108 +++++++++++++---

.../wic/plugins/source/bootimg-partition.py | 9 +-
.../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-

scripts/lib/wic/plugins/source/empty.py | 59 ++++++++-

scripts/lib/wic/plugins/source/rawcopy.py | 9 +-
scripts/lib/wic/plugins/source/rootfs.py | 4 +-

diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index da483dae..a1d15265 100644

--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
updated = False
for part in self.parts:
if not part.realnum or not part.mountpoint \

+ logger.debug("Set disk guid %s", self.disk_guid)

Anton Mikanovich

unread,

May 17, 2024, 7:44:51 AM (12 days ago) May 17

to isar-...@googlegroups.com, Anton Mikanovich

After update of WIC it requires sfdisk to support --disk-id command,
which is false for Debian Buster and Ubuntu Focal.
Do not try to set disk identifier for those distro versions to keep
them buildable but with lack of reproducibility.

This hack should be removed after deprecation of Buster and Focal.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

scripts/lib/wic/plugins/imager/direct.py | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index 33a869ec..a65ae36a 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py

@@ -530,6 +530,13 @@ class PartitionedImage():

exec_native_cmd("parted -s %s mklabel %s" % (device, ptable_format),

self.native_sysroot)

+ def _sfdisk_supports_disk_id(self):
+ ret, help_out = exec_native_cmd("sfdisk --help", self.native_sysroot)
+ if 'disk-id' not in help_out:
+ logger.warn("Current sfdisk version DOES NOT support changing disk ID")
+ return False
+ return True
+
def _write_disk_guid(self):

if self.ptable_format in ('gpt', 'gpt-hybrid'):

if os.getenv('SOURCE_DATE_EPOCH'):
@@ -537,6 +544,11 @@ class PartitionedImage():
else:
self.disk_guid = uuid.uuid4()

+ # Backport compatibility for Debian Buster and Ubuntu Focal
+ if not self._sfdisk_supports_disk_id():
+ logger.warn("Disk identifier can't be set, reproducibility is broken!")
+ return

+
logger.debug("Set disk guid %s", self.disk_guid)

sfdisk_cmd = "sfdisk --disk-id %s %s" % (self.path, self.disk_guid)

exec_native_cmd(sfdisk_cmd, self.native_sysroot)
--
2.34.1

Anton Mikanovich

unread,

May 17, 2024, 8:49:37 AM (11 days ago) May 17

to isar-...@googlegroups.com, Felix Moessbauer, Tobias Schmidl, Henning Schild, Kiszka, Jan, Baurzhan Ismagulov

17/05/2024 14:44, Anton Mikanovich wrote:
> This patchset updates WIC libs to the latest OE-core revision (version
> 5.0.0, codename Scarthgap).
> Commit a4237f08 ("wic/plugins/images/direct: Allow changes in fstab on rootfs")
> was reapplied because it is needed for some Isar targets.
> There is also a new fix to keep compatibility with Debian Buster and
> Ubuntu Focal which prevent those distros from having full identifier
> reproducibility during image prepare in WIC.

We are going to merge this series 'as-is' (including the revert of
'/' exclusion fix revert) soon. We see [1];

Felix, are you planning to address the issue upstream?

[1]
https://patchwork.yoctoproject.org/project/oe-core/cover/20230105125223.39821...@siemens.com/

Anton Mikanovich

unread,

May 21, 2024, 7:07:44 AM (8 days ago) May 21

to isar-...@googlegroups.com

17/05/2024 14:44, Anton Mikanovich wrote:

> This patchset updates WIC libs to the latest OE-core revision (version
> 5.0.0, codename Scarthgap).
> Commit a4237f08 ("wic/plugins/images/direct: Allow changes in fstab on rootfs")
> was reapplied because it is needed for some Isar targets.
> There is also a new fix to keep compatibility with Debian Buster and
> Ubuntu Focal which prevent those distros from having full identifier
> reproducibility during image prepare in WIC.

Applied to next.

MOESSBAUER, Felix

unread,

May 23, 2024, 9:15:05 AM (5 days ago) May 23

to ami...@ilbers.de, isar-...@googlegroups.com, i...@ilbers.de, Kiszka, Jan

On Fri, 2024-05-17 at 15:49 +0300, Anton Mikanovich wrote:
> 17/05/2024 14:44, Anton Mikanovich wrote:
> > This patchset updates WIC libs to the latest OE-core revision
> > (version
> > 5.0.0, codename Scarthgap).
> > Commit a4237f08 ("wic/plugins/images/direct: Allow changes in fstab
> > on rootfs")
> > was reapplied because it is needed for some Isar targets.
> > There is also a new fix to keep compatibility with Debian Buster
> > and
> > Ubuntu Focal which prevent those distros from having full
> > identifier
> > reproducibility during image prepare in WIC.
>
> We are going to merge this series 'as-is' (including the revert of
> '/' exclusion fix revert) soon. We see [1];
>
> Felix, are you planning to address the issue upstream?

I'll give it a try. However there was some back-and-forth on that
topic, including a revert of one of our patches from colleagues which
are no longer at Siemens.

Felix

>
> [1]
> https://patchwork.yoctoproject.org/project/oe-core/cover/20230105125223.39821...@siemens.com/
>

--
Siemens AG, Technology
Linux Expert Center

Reply all

Reply to author

Forward