ROOTFS_SIZE changes between multiple builds
Adithya....@toshiba-tsip.com
Hi All,
I have been recently working on reproducible builds for a downstream project (isar-cip-core).
On that effort, I came across a behavior with the ROOTFS_SIZE variable. Sometimes I see that when I build 2 images to check for reproducibility, the value of the ROOTFS_SIZE variable changes between builds. The variation usually only by a few KB's and this behavior is not seen quite often either. At the moment I am not sure about the cause of this problem, Hence posting here seeking suggestions from the rest of you.
I tried checking the rootfs contents, but the contents are identical despite the difference in the size of the rootfs directory.
I am trying to understand this for the following reasons:
1. In the case where no size parameter is set for a partition in the wks file, wic estimates the rootfs size from the rootfs directory in WORKDIR, See [1]. When the rootfs size is different for the 2 builds, the resulting filesystem images are also different
(in the case that I encountered, the 2 ext4 filesystem images are off by a few blocks).
2. While creating an ext4 filesystem images with the IMAGE_CMD of isar, the filesystem images are created based the ROOTFS_SIZE. In the scenario where ROOTFS_SIZE are the different, the resultant filesystem images are also non-reproducible. See [2]
With regards to point 2 above, that maybe a different topic as there are other problems in achieving reproducibility of ext4 filesystem images, like support for file timestamp clamping to SOURCE_DATE_EPOCH was only recently added to mkfs.ext4 (v1.47.1)
Would like to understand if anyone here has come across such a scenario / any pointers on how to get to the root cause?
[1]
https://github.com/ilbers/isar/blob/master/scripts/lib/wic/partition.py#L242
[2]
https://github.com/ilbers/isar/blob/master/meta/classes/imagetypes.bbclass#L19
Thanks and Regards,
Adithya Balakumar
Adithya....@toshiba-tsip.com
I tried understanding the root cause of this issue, When there is a difference in the ROOTFS_SIZE, I see that some of the files in the rootfs tree has different number of blocks allocated to the file.
I say "some" files because the files/directories that are affected are not the same in every build. For instance, I am attaching an example below where the man8 directory using different number of blocks despite having the same size according to the stat command
Build1:
host:~/work/isar-cip-core$ stat build/tmp/work/cip-core-bookworm-armhf/cip-core-image-bbb/1.0-r0/rootfs/usr/share/man/man8
File: build3/tmp/work/cip-core-bookworm-armhf/cip-core-image-bbb/1.0-r0/rootfs/usr/share/man/man8
Size: 24576 Blocks: 56 IO Block: 4096 directory
Device: 802h/2050d Inode: 26428993 Links: 2
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2024-07-15 15:29:02.560481148 +0530
Modify: 2024-06-24 11:13:41.000000000 +0530
Change: 2024-07-15 15:29:01.376474194 +0530
Birth: -
Build2:
host:~/work/isar-cip-core$ stat build2/tmp/work/cip-core-bookworm-armhf/cip-core-image-bbb/1.0-r0/rootfs/usr/share/man/man8
File: build4/tmp/work/cip-core-bookworm-armhf/cip-core-image-bbb/1.0-r0/rootfs/usr/share/man/man8
Size: 24576 Blocks: 48 IO Block: 4096 directory
Device: 802h/2050d Inode: 26027389 Links: 2
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2024-07-15 15:55:17.458599845 +0530
Modify: 2024-06-24 11:13:41.000000000 +0530
Change: 2024-07-15 15:55:16.270589701 +0530
Birth: -
Using the "du" command to measure the size of the rootfs directory in such a case, yields different values for the ROOTFS_SIZE.
I noticed that, openembedded core uses a custom function to measure the directory size, See [1] and [2]. Using the same function in isar seems to soves this problem based on my tests. Any thoughts?
[1] https://github.com/openembedded/openembedded-core/blob/master/meta/classes-recipe/image.bbclass#L559
[2] https://github.com/openembedded/openembedded-core/blob/master/meta/lib/oe/utils.py#L502