[Sbcl-devel] sbcl reproducible builds

[Bernhard M. Wiedemann]

Hi,

I have once again had a look into sbcl reproducible builds and found
that there are 2 hard problems left:

contrib/*.fasl files contain a timestamp (seconds since 1900-01-01) in 4
places. Such values could come from time.lisp's get-universal-time


--- od -tx1 RPMS.1/usr/lib/sbcl/contrib/sb-aclrepl.fasl
+++ od -tx1 RPMS.2/usr/lib/sbcl/contrib/sb-aclrepl.fasl
@@ -18,8 +18,8 @@
0000420 4f 4d 2d 53 45 4c 46 3b 43 4f 4e 54 52 49 42 3b
0000440 53 42 2d 41 43 4c 52 45 50 4c 3b 4d 4f 44 55 4c
0000460 45 2d 53 45 54 55 50 2e 4c 49 53 50 2e 4e 45 57
-0000500 45 53 54 05 22 f1 f0 d0 e8 00 00 00 00 02 7c 04
-0000520 00 22 f1 f0 d0 e8 00 00 00 00 5c 00 05 02 6e 05
+0000500 45 53 54 05 22 4c f1 d0 e8 00 00 00 00 02 7c 04
+0000520 00 22 4c f1 d0 e8 00 00 00 00 5c 00 05 02 6e 05
0000540 30 04 05 01 40 05 23 21 2f 68 6f 6d 65 2f 61 62
0000560 75 69 6c 64 2f 72 70 6d 62 75 69 6c 64 2f 42 55
0000600 49 4c 44 2f 73 62 63 6c 2d 32 2e 33 2e 39 2f 73
@@ -15630,7 +15630,7 @@
0750320 53 45 4c 46 3b 43 4f 4e 54 52 49 42 3b 53 42 2d
0750340 41 43 4c 52 45 50 4c 3b 4d 4f 44 55 4c 45 2d 50
0750360 52 4f 56 49 44 45 2e 4c 49 53 50 2e 4e 45 57 45
-0750400 53 54 05 22 f1 f0 d0 e8 00 00 00 00 02 7c 53 04
+0750400 53 54 05 22 4c f1 d0 e8 00 00 00 00 02 7c 53 04
0750420 53 42 2d 43 54 1d 01 54 4f 50 2d 4c 45 56 45 4c
0750440 2d 46 4f 52 4d 4d 0f 50 52 4f 56 49 44 45 64 0a
0750460 53 42 2d 41 43 4c 52 45 50 4c 05 82 05 82 05 02
@@ -15649,7 +15649,7 @@
0751000 8b ec b8 00 00 00 00 ff d0 48 0f 42 e3 ba 17 01
0751020 00 50 c9 f8 c3 00 00 00 00 00 00 00 00 00 00 00
0751040 00 00 00 00 10 00 00 00 2f 00 08 00 01 04 13 15
-0751060 00 01 04 14 38 00 04 00 22 f1 f0 d0 e8 00 00 00
+0751060 00 01 04 14 38 00 04 00 22 4c f1 d0 e8 00 00 00
0751100 00 2b 01 9d 00 05 02 6e 05 30 04 05 01 04 18 0e
0751120 12 01 40 19


My reproducer, that showed that it is only timestamps with resolution of
1 second is

for n in $(seq 20) ; do
rm ./obj/sbcl-home/contrib/sb-sprof.fasl && date +%s &&
./make-target-contrib.sh >/dev/null && md5sum
./obj/sbcl-home/contrib/sb-sprof.fasl
done



and there are 3+2+2 random bytes in sbcl.core, that could come from
uninitialized memory, timestamps or other sources.

--- od -tx1 RPMS.1/usr/lib/sbcl/sbcl.core
+++ od -tx1 RPMS.2/usr/lib/sbcl/sbcl.core
@@ -1652755,10 +1652755,10 @@
152136240 6b 32 32 53 00 00 00 00 00 00 00 00 00 00 00 00
152136260 e3 cc 87 00 10 00 00 00 2f 25 a2 ff 0f 00 0c 00
152136300 2d 05 00 06 00 00 00 00 ac a5 c2 50 13 f2 91 77
-152136320 d6 ea 06 00 00 00 00 00 00 00 00 00 00 00 00 00
+152136320 de 04 07 00 00 00 00 00 00 00 00 00 00 00 00 00
152136340 53 95 00 00 10 00 00 00 2f 2a a2 ff 0f 00 0e 00
152136360 2d 05 00 06 00 00 00 00 34 b8 90 d8 68 f5 b2 7e
-152136400 d8 fc 06 00 00 00 00 00 00 00 00 00 00 00 00 00
+152136400 96 f9 06 00 00 00 00 00 00 00 00 00 00 00 00 00
152136420 93 95 00 00 10 00 00 00 4f 2a a2 ff 0f 00 0e 00
152136440 2d 05 00 06 00 00 00 00 56 bf e9 31 c3 62 58 6d
152136460 a3 12 01 00 10 00 00 00 00 00 00 00 00 00 00 00
@@ -1652876,7 +1652876,7 @@
152142060 17 01 00 50 00 00 00 00 00 00 00 00 00 00 00 00
152142100 b3 01 88 00 10 00 00 00 af e2 a2 ff 0f 00 06 00
152142120 2d 05 00 06 00 00 00 00 68 db 20 0e d0 ff 3b 6e
-152142140 c0 bc a2 37 00 00 00 00 00 00 00 00 00 00 00 00
+152142140 20 bd a2 37 00 00 00 00 00 00 00 00 00 00 00 00
152142160 13 96 00 00 10 00 00 00 df f0 a2 ff 0f 00 0c 00
152142200 2d 05 00 06 00 00 00 00 a6 e9 8f 04 aa 50 d1 7f
152142220 17 01 00 50 00 00 00 00 00 00 00 00 00 00 00 00


We also carry a downstream patch for 2 years against embedding
hostname+date:
https://code.opensuse.org/package/sbcl/blob/master/f/sbcl-use-SOURCE_DATE_EPOCH-for-build-id.patch

Hints for debugging and patches welcome.

Ciao
Bernhard M.


_______________________________________________
Sbcl-devel mailing list
Sbcl-...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-devel

[Stas Boukarev]

Is SOURCE_DATE_EPOCH some standard variable?

[Bernhard M. Wiedemann]

On 11/10/2023 16.36, Stas Boukarev wrote:
> Is SOURCE_DATE_EPOCH some standard variable?

Yes, the definition is at
https://reproducible-builds.org/specs/source-date-epoch/
and it is already used by gcc, cmake, rpm and many more places.