Linux aarch64 binaries for SDK lag several versions behind

[Brion Vibber]

Hi all!

I noticed that while there is now an arm64/aarch64 Linux binary available for emsdk, it's several versions behind:

# macOS

% ./emsdk install latest
Resolving SDK alias 'latest' to '3.1.30'
Resolving SDK version '3.1.30' to 'sdk-releases-dc1fdcfd3f5b9d29cb1ebdf15e6e845bef9b0cc1-64bit'

# Linux on same machine

$ ./emsdk install latest
warning: 'latest' on arm64-linux may be slightly behind other architectures
Resolving SDK alias 'latest-arm64-linux' to '3.1.21'

What can people do as interested developers to help automate this process so that aarch64 Linux versions come out at the same time as x86_64 ones? Is there a need for hardware? Software? Developer time? Money? It'd be nice for this to be totally transparent to the developer installing the SDK, without requiring a lengthy local compilation of LLVM to get current versions.

Thanks!

-- brion vibber

Software Architect

Wikimedia Foundation

[Sam Clegg]

I agree it would be nice to make this more automatic but we haven't found an easy way to do this yet.

Currently the process is that the arm64 binaries get built somewhere in the amazon cloud (thanks Michael for setting this up) and then that sends an email to me when it's done.  I then have to manually download the archive and push it up to the emsdk google cloud bucket where emsdk pulls from.   I then need to create a PR to bump the version in emsdk.

Because this is a manual process with several steps (and honestly because we don't have too many arm64 users yet) I sometimes take a while to get around to doing it.

If you can think of any way to automate this or speed this up that would be great. 

(It looks like the builder has stopped running recently.. the last email I got was for 3.1.24 back in October.  Michael, do you have time to look into that proximate problem).

cheers,

sam

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYTnFj6U%2Bws8MSZJ98Pq%2B2pC_WN7abhNpH4N9huUOF5fX7w%40mail.gmail.com.

[Sam Clegg]

Thinking about this just now I did think of one way to take myself out of the loop each update:  We could have emsdk point directly at amazon S3 bucket for the arm64 binaries.. then it would just be a question of someone creating a PR to emsdk for each release (any interested party such as yourself could then do that part).

cheer,

sam

[Brion Vibber]

On Tue, Jan 17, 2023 at 1:49 PM 'Sam Clegg' via emscripten-discuss <emscripte...@googlegroups.com> wrote:

Thinking about this just now I did think of one way to take myself out of the loop each update:  We could have emsdk point directly at amazon S3 bucket for the arm64 binaries.. then it would just be a question of someone creating a PR to emsdk for each release (any interested party such as yourself could then do that part).

Is there a reason it's not part of the main build process and started/committed by the same people at the same time?

 

-- brion

[Sam Clegg]

The main builder run on chromium infrastructure here: https://ci.chromium.org/p/emscripten-releases/g/main/console.

There are not currently any arm64 builders there. We just have manOS, linux, and windows builders that run on x86_64.  We could look into cross building in the same way we do for macOS arm64? 

 

 

-- brion

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYTm7BmeMocruQSo0SNFwo_WQ7ibTSo7xQs_N%3Dva782B_4Q%40mail.gmail.com.

[Brion Vibber]

On Tue, Jan 17, 2023 at 2:21 PM 'Sam Clegg' via emscripten-discuss <emscripte...@googlegroups.com> wrote:

The main builder run on chromium infrastructure here: https://ci.chromium.org/p/emscripten-releases/g/main/console.

There are not currently any arm64 builders there. We just have manOS, linux, and windows builders that run on x86_64.

That makes sense!

 

  We could look into cross building in the same way we do for macOS arm64? 

Yes, this ought to work fairly well. I know arm linux is still kinda niche, but I think it's only getting bigger -- and because Linux is big in tooling like Docker, there's speed benefits to avoiding x86_64 emulation on Linux even on arm64 Macs. :)

I'd be happy to put in the work to modify the builder so an x86_64 Linux host can produce an arm64 and armv7 build -- that's a one-time cost and then we can get automated builds from there on.

Unlike macOS we don't get a cross-arch system image "for free" with the system compiler, so this means:

* grab suitable linux system images with headers and libraries to link against

* make sure builder can pass the appropriate cross-compile setup

* if it's necessary to run tests on the resulting binaries, set up to run them in emulation via qemu-aarch64

Emulated tests would be relatively slow, but the building will be the bulk of time and will run native.

I'm uncertain which overarchiving build scripts are in use for building the main binaries, if that's in emsdk or over in binaryen or a bit of both?

-- brion

 

 

 

-- brion

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYTm7BmeMocruQSo0SNFwo_WQ7ibTSo7xQs_N%3Dva782B_4Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_D%3DX1Lye7xWTz50QH%2BVE3UpfNuKGkj3SW5-AnF93p4mg%40mail.gmail.com.

[Sam Clegg]

On Tue, Jan 17, 2023 at 2:38 PM Brion Vibber <br...@pobox.com> wrote:

On Tue, Jan 17, 2023 at 2:21 PM 'Sam Clegg' via emscripten-discuss <emscripte...@googlegroups.com> wrote:

The main builder run on chromium infrastructure here: https://ci.chromium.org/p/emscripten-releases/g/main/console.

There are not currently any arm64 builders there. We just have manOS, linux, and windows builders that run on x86_64.

That makes sense!

 

  We could look into cross building in the same way we do for macOS arm64? 

Yes, this ought to work fairly well. I know arm linux is still kinda niche, but I think it's only getting bigger -- and because Linux is big in tooling like Docker, there's speed benefits to avoiding x86_64 emulation on Linux even on arm64 Macs. :)

I'd be happy to put in the work to modify the builder so an x86_64 Linux host can produce an arm64 and armv7 build -- that's a one-time cost and then we can get automated builds from there on.

Unlike macOS we don't get a cross-arch system image "for free" with the system compiler, so this means:

* grab suitable linux system images with headers and libraries to link against

* make sure builder can pass the appropriate cross-compile setup

* if it's necessary to run tests on the resulting binaries, set up to run them in emulation via qemu-aarch64

Emulated tests would be relatively slow, but the building will be the bulk of time and will run native.

I'm uncertain which overarchiving build scripts are in use for building the main binaries, if that's in emsdk or over in binaryen or a bit of both?

The build script that build all the pre-built binaries is here:  https://chromium.googlesource.com/emscripten-releases/+/refs/heads/main/src/build.py

-- brion

 

 

 

-- brion

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYTm7BmeMocruQSo0SNFwo_WQ7ibTSo7xQs_N%3Dva782B_4Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_D%3DX1Lye7xWTz50QH%2BVE3UpfNuKGkj3SW5-AnF93p4mg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYTkeEns3QUUF%2BpxvDp9nzc78rdUNwbUXMd6dOEZy%2Bx0iMQ%40mail.gmail.com.

[Brion Vibber]

On Tue, Jan 17, 2023 at 2:48 PM 'Sam Clegg' via emscripten-discuss <emscripte...@googlegroups.com> wrote:

The build script that build all the pre-built binaries is here:  https://chromium.googlesource.com/emscripten-releases/+/refs/heads/main/src/build.py

Perfect, I should be able to work with this. :) I'll poke at it for a few days and see what I can get running in terms of adding a Linux cross-compile mode.

Thanks!

-- brion

[Brion Vibber]

On Tue, Jan 17, 2023 at 2:48 PM 'Sam Clegg' via emscripten-discuss <emscripte...@googlegroups.com> wrote:

The build script that build all the pre-built binaries is here:  https://chromium.googlesource.com/emscripten-releases/+/refs/heads/main/src/build.py

I've got it building for x86_64 Linux as-is, but before I go diving into the cross-compilation I want to check if it's supposed to be failing a lot of the llvm tests, or if perhaps there's something wrong with my setup...

./src/build.py --sync-include cmake,deps,nodejs,sysroot

./src/build.py --build-include libcxx,llvm-test-depends,llvm,binaryen,emscripten,archive

./src/build.py --test-include llvmtest

...

Testing Time: 197.78s

  Passed: 294

  Failed:  43

Thanks!

-- brion

[Sam Clegg]

There are two types of llvm tests, the unit tests.  They show up on the waterfall as "LLVM regression" and "LLVM testsuite": https://ci.chromium.org/ui/p/emscripten-releases/builders/ci/linux-test-suites/b8791586887468760305/overview

It does look like the testsuite has 43 failures, but the build stop is not failing: https://logs.chromium.org/logs/emscripten-releases/buildbucket/cr-buildbucket/8791586887468760305/+/u/LLVM_testsuite/stdout

So it looks like those are expected/known failures.

  Failed:  43

Thanks!

-- brion

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYT%3DxSzv3%3D-56BJpGejt%3DxLyayo3EsauqunOG6BCu5Bat6g%40mail.gmail.com.