Re: Electron Reproducibility Last Mile - LC_UUID on macOS

[Mark Mentovai]

Nico’s suspicion about LC_UUID’s purpose is correct. We don’t want to remove the UUID at all. It will absolutely break crash reporting. The UUID is used on the crash server to match debug information and the original code module up with the crash dump.

The UUIDs also have local value, as they prevent debuggers from incorrectly using debug info that doesn’t match. I’m heavily inclined against -no_uuid anywhere.

LC_UUID is supposed to be deterministic. See ld64 src/ld/OutputFile.cpp ld::tool::OutputFile::computeContentUUID, starting at line 3459. The UUID is based on a MD5 hash of what are supposed to be stable parts of the output. A random UUID is only used in ld::tool::OutputFile::writeOutputFile (line 3693) when the (undocumented) -random_uuid command argument is used (src/ld/Options.cpp Options::parse, line 3354).

There may be a bug in ld64 that causes the UUID to not be deterministic even when the rest of the linker output is, but you’d need to verify this. In the past, the linker parallelized certain operations and didn’t sort their outputs stably, which led to non-deterministic output, but the cases I reported were all fixed. (There may be more, though.)

In summary:

ld64 should already be doing what the end of Nico’s comment says is the ideal goal. I’m still rooting for a Mach-O lld, though.

On Wed, Apr 8, 2020 at 11:59 AM Nico Weber <tha...@chromium.org> wrote:

Hi Andrea,

thanks for your work on this, this is really cool.

I believe (but I'm not sure) that the UUID is used to link a binary to its dSYM, so I'd guess we need it on the crash server side if enable_dsyms is true at least.

In other cases: Are there drawbacks to just passing -no_uuid to ld64 so that it's not written in the first place, instead of stripping it out later?

For a better longer-term plan, see the end of https://bugs.chromium.org/p/chromium/issues/detail?id=1068970#c3 :)

Nico

On Wed, Apr 8, 2020 at 9:14 AM Andrea Brancaleoni @ Doyensec <and...@doyensec.com> wrote:

Hi there, 

As part of a joint effort with the electronjs core maintainers, in the previous months

we tried to enable full reproducible builds for electron binaries.

While this is mostly true for Windows target where an user has a already the ability to verify

The path from source to binary code (see verify.bat https://github.com/electron/electron/pull/22969).

This is still a work in progress for Linux and macOS (https://github.com/electron/electron/pulls/thypon).

While trying to obtain reproducible builds on macOS I discovered that electron binaries

Are containing a 16 byte field (LC_UUID) which is different on remote server and local builds,

While the rest of the binary is having similar content.

Electron dev team has some concern on removing LC_UUID, in particular in relation of breakpad usage

Of this field https://github.com/google/breakpad/blob/master/src/common/mac/file_id.cc#L79 .

I personally don’t think it will change anything since it will fallback

to use uuid generation already available in other systems (Windows and Linux).

Since it is touching such a sensitive part of electron, in particular crash reporting we would like to hear back from

Chromium dev team.

In particular:

- Do you think is reasonable to remove this particular field enabling `-no_uuid` during stripping?

- Do you think is feasible to merge this patch in chromium mainline https://github.com/electron/electron/pull/22958/files/4e2cf14846360fb4267d6e55b47b7e50d374ce2b#diff-bcaca759051dbf33bd71e2e286ccc221 ?

- Anything else worth knowing? Do you have additional concerns on the topic?

Cheers, 
Andrea Brancaleoni @ Doyensec 

[Nico Weber]

[Andrea Brancaleoni @ Doyensec]