Disabling V8 snapshots for ARM cross-compiles

[Thomas Preston]

Hi,

We have noticed that the latest version of Chromium makes the v8 snapshot build mandatory.

We were turning this off for building under Buildroot (for linux, targetting 32-bit ARM), because Buildroot has no concept of multilib, and host packages can't be installed with 32-bit memory architectures.

Is there any way to continue to disable v8 snapshot building? If not, where can we find out more about building ARM 32-bit V8 snapshots on x64 hosts?

Many thanks,

Thomas

[Jakob Gruber]

Unfortunately there's currently no easy way to do this. A 64-bit mksnapshot binary (currently) cannot create 32-bit snapshots.

Your best bet is to build snapshot_blob.bin and embedded.S elsewhere (e.g. on the host system or in a VM). The final build could then use these generated files and skip the mksnapshot step. Would that be an option for you?

See also the Known Issues section here. 

Note there's no technical reason why mksnapshot cannot support 64-to-32 cross-builds, so it may become possible in the future. There are no concrete plans at the moment though (you could file bug at crbug.com/v8/new).

 

Many thanks,

Thomas

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups "v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/v8-users/c805e785-68d0-44bd-b207-e2848a9c3c76%40googlegroups.com.

[Thomas Preston]

On Tuesday, 7 January 2020 09:14:36 UTC, Jakob Gruber wrote:

On Mon, Jan 6, 2020 at 6:30 PM Thomas Preston <thomas...@codethink.co.uk> wrote:

Hi,

We have noticed that the latest version of Chromium makes the v8 snapshot build mandatory.

We were turning this off for building under Buildroot (for linux, targetting 32-bit ARM), because Buildroot has no concept of multilib, and host packages can't be installed with 32-bit memory architectures.

Is there any way to continue to disable v8 snapshot building? If not, where can we find out more about building ARM 32-bit V8 snapshots on x64 hosts?

Unfortunately there's currently no easy way to do this. A 64-bit mksnapshot binary (currently) cannot create 32-bit snapshots.

Your best bet is to build snapshot_blob.bin and embedded.S elsewhere (e.g. on the host system or in a VM). The final build could then use these generated files and skip the mksnapshot step. Would that be an option for you?

See also the Known Issues section here. 

That's useful, thanks. It looks like checking-in pre-built 32-bit artifacts is the way to go.

Is it possible to just run mksnapshot? Maybe we can just tell ninja to build `obj/v8/v8_snapshot/embedded.o`

Full-Chromium requires a 64-bit build environment https://chromium.googlesource.com/chromium/src/+/master/docs/linux/build_instructions.md

Alternatively, are there any pre-built artifacts we can just download? This would be ideal, as we wouldn't have to check in large files to Buildroot.

[Jakob Gruber]

On Tue, Jan 7, 2020 at 3:13 PM Thomas Preston <thomas....@codethink.co.uk> wrote:

On Tuesday, 7 January 2020 09:14:36 UTC, Jakob Gruber wrote:

On Mon, Jan 6, 2020 at 6:30 PM Thomas Preston <thomas...@codethink.co.uk> wrote:

Hi,

We have noticed that the latest version of Chromium makes the v8 snapshot build mandatory.

We were turning this off for building under Buildroot (for linux, targetting 32-bit ARM), because Buildroot has no concept of multilib, and host packages can't be installed with 32-bit memory architectures.

Is there any way to continue to disable v8 snapshot building? If not, where can we find out more about building ARM 32-bit V8 snapshots on x64 hosts?

Unfortunately there's currently no easy way to do this. A 64-bit mksnapshot binary (currently) cannot create 32-bit snapshots.

Your best bet is to build snapshot_blob.bin and embedded.S elsewhere (e.g. on the host system or in a VM). The final build could then use these generated files and skip the mksnapshot step. Would that be an option for you?

See also the Known Issues section here. 

That's useful, thanks. It looks like checking-in pre-built 32-bit artifacts is the way to go.

Is it possible to just run mksnapshot? Maybe we can just tell ninja to build `obj/v8/v8_snapshot/embedded.o`

Yes that's certainly possible. Try the run_mksnapshot target. Make sure to use the same git hash and build config though between the mksnapshot run and the final V8 build!

 

Full-Chromium requires a 64-bit build environment https://chromium.googlesource.com/chromium/src/+/master/docs/linux/build_instructions.md 

Alternatively, are there any pre-built artifacts we can just download? This would be ideal, as we wouldn't have to check in large files to Buildroot.

No, sorry. I do like the idea, but we don't have anything like that right now.

 

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups "v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/v8-users/d6503022-b79e-47bc-8fad-4207b6a69b21%40googlegroups.com.

[Thomas Preston]

On Tuesday, 7 January 2020 14:21:25 UTC, Jakob Gruber wrote:

On Tue, Jan 7, 2020 at 3:13 PM Thomas Preston <thomas...@codethink.co.uk> wrote:

On Tuesday, 7 January 2020 09:14:36 UTC, Jakob Gruber wrote:

On Mon, Jan 6, 2020 at 6:30 PM Thomas Preston <thomas...@codethink.co.uk> wrote:

Hi,

We have noticed that the latest version of Chromium makes the v8 snapshot build mandatory.

We were turning this off for building under Buildroot (for linux, targetting 32-bit ARM), because Buildroot has no concept of multilib, and host packages can't be installed with 32-bit memory architectures.

Is there any way to continue to disable v8 snapshot building? If not, where can we find out more about building ARM 32-bit V8 snapshots on x64 hosts?

Unfortunately there's currently no easy way to do this. A 64-bit mksnapshot binary (currently) cannot create 32-bit snapshots.

Your best bet is to build snapshot_blob.bin and embedded.S elsewhere (e.g. on the host system or in a VM). The final build could then use these generated files and skip the mksnapshot step. Would that be an option for you?

See also the Known Issues section here. 

That's useful, thanks. It looks like checking-in pre-built 32-bit artifacts is the way to go.

Is it possible to just run mksnapshot? Maybe we can just tell ninja to build `obj/v8/v8_snapshot/embedded.o`

Yes that's certainly possible. Try the run_mksnapshot target. Make sure to use the same git hash and build config though between the mksnapshot run and the final V8 build!

Has anyone tried this? We are able to create a 32-bit snapshot_blob.bin and embedded.S, but we can't inject it into our 64-bit-host build because ninja wants to recreate the artifacts.

I understand ninja uses timestamps to figure out which files are dirty, but I'm not sure which files specifically I need to touch to trick ninja into using my snapshot_blob.bin and embedded.S.

I've tried touching the following combinations:

- snapshot_blob.bin, gen/v8/embedded.S, obj/v8/run_mksnapshot_default.stamp

- All dirty files output from `ninja -n -d explain -C out/Release run_mksnapshot_default`

[Thomas Preston]

On Tuesday, 28 January 2020 11:14:12 UTC, Thomas Preston wrote:

On Tuesday, 7 January 2020 14:21:25 UTC, Jakob Gruber wrote:

On Tue, Jan 7, 2020 at 3:13 PM Thomas Preston <thomas...@codethink.co.uk> wrote:

On Tuesday, 7 January 2020 09:14:36 UTC, Jakob Gruber wrote:

On Mon, Jan 6, 2020 at 6:30 PM Thomas Preston <thomas...@codethink.co.uk> wrote:

Hi,

We have noticed that the latest version of Chromium makes the v8 snapshot build mandatory.

We were turning this off for building under Buildroot (for linux, targetting 32-bit ARM), because Buildroot has no concept of multilib, and host packages can't be installed with 32-bit memory architectures.

Is there any way to continue to disable v8 snapshot building? If not, where can we find out more about building ARM 32-bit V8 snapshots on x64 hosts?

Unfortunately there's currently no easy way to do this. A 64-bit mksnapshot binary (currently) cannot create 32-bit snapshots.

Your best bet is to build snapshot_blob.bin and embedded.S elsewhere (e.g. on the host system or in a VM). The final build could then use these generated files and skip the mksnapshot step. Would that be an option for you?

See also the Known Issues section here. 

That's useful, thanks. It looks like checking-in pre-built 32-bit artifacts is the way to go.

Is it possible to just run mksnapshot? Maybe we can just tell ninja to build `obj/v8/v8_snapshot/embedded.o`

Yes that's certainly possible. Try the run_mksnapshot target. Make sure to use the same git hash and build config though between the mksnapshot run and the final V8 build!

Has anyone tried this? We are able to create a 32-bit snapshot_blob.bin and embedded.S, but we can't inject it into our 64-bit-host build because ninja wants to recreate the artifacts.

We ended up solving this by letting the run_mksnapshot_default target finish, then overwriting snapshot_blob.bin and embedded.S with the pre-built 32-bit versions. The build actually errors when trying to build another target, which uses embedded.S.

[Thomas Preston]

On Wednesday, 29 January 2020 09:37:48 UTC, Thomas Preston wrote:

We ended up solving this by letting the run_mksnapshot_default target finish, then overwriting snapshot_blob.bin and embedded.S with the pre-built 32-bit versions. The build actually errors when trying to build another target, which uses embedded.S.

This is the error when we don't replace the snapshot_blob.bin and embedded.S with 32-bit versions:

.quad V8FooBar
      ^ unsupported relocation on symbol

[Jakob Gruber]

It sounds like we could improve support for this case by providing a gn argument for prebuilt `snapshot_blob.bin` and `embedded.S` artifacts. If provided, we could skip building mksnapshot entirely. Would that help?

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups "v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/v8-users/a8c43168-2ff4-4a6a-bead-850a6e43fa30%40googlegroups.com.

[Thomas Preston]

Yes, that would save some time building. Please let me know if there's any update on this. Thank you

On Wednesday, 8 April 2020 15:52:08 UTC+1, Jakob Gruber wrote:

It sounds like we could improve support for this case by providing a gn argument for prebuilt `snapshot_blob.bin` and `embedded.S` artifacts. If provided, we could skip building mksnapshot entirely. Would that help?

On Wed, Jan 29, 2020 at 10:44 AM Thomas Preston <thomas...@codethink.co.uk> wrote:

On Wednesday, 29 January 2020 09:37:48 UTC, Thomas Preston wrote:

We ended up solving this by letting the run_mksnapshot_default target finish, then overwriting snapshot_blob.bin and embedded.S with the pre-built 32-bit versions. The build actually errors when trying to build another target, which uses embedded.S.

This is the error when we don't replace the snapshot_blob.bin and embedded.S with 32-bit versions:

.quad V8FooBar
      ^ unsupported relocation on symbol

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups "v8-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to v8-u...@googlegroups.com.

[Jakob Gruber]

Filed https://crbug.com/v8/10414 to track this.

To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/v8-users/3955444e-46df-4639-9a3d-8c3405259fb6%40googlegroups.com.

[Patrick De Bella]

Hi Thomas,

Can you please provide some details on how you created the 32-bit versions of embedded.S and snapshot_blob.bin?  I'm new to Chromium, GN and Ninja and having some issues with this. 

Patrick

[Thomas Preston]

Hi Patrick,

In order to build 32-bit Chromium in Buildroot, we had to isolate the whole thing in a 32-bit container, then run:

buildroot$ HOST_CFLAGS="-O2 -m32" linux32 make libcef

The 32-bit container is i386 ubuntu:19.10 described by a Dockerfile very similar to the Buildroot testing Dockerfile:

--- support/docker/Dockerfile   2020-04-29 10:13:47.654241375 +0100
+++ package/libcef/v8_snapshot/arm/Dockerfile   2020-04-29 10:15:30.339897520 +0100
@@ -1,41 +1,33 @@
-# This Dockerfile generates the docker image that gets used by Gitlab CI
-# To build it (YYYYMMDD.HHMM is the current date and time in UTC):
-#   sudo docker build -t buildroot/base:YYYYMMDD.HHMM support/docker
-#   sudo docker push buildroot/base:YYYYMMDD.HHMM
-
-# We use a specific tag for the base image *and* the corresponding date
-# for the repository., so do not forget to update the apt-sources.list
-# file that is shipped next to this Dockerfile.
-FROM debian:stretch-20171210
-
-LABEL maintainer="Buildroot mailing list <buil...@buildroot.org>" \
-      vendor="Buildroot" \
-description="Container with everything needed to run Buildroot"
+# 32-bit Dockerfile which mimics support/docker/Dockerfile
+FROM i386/ubuntu:19.10

 # Setup environment
 ENV DEBIAN_FRONTEND noninteractive

 # This repository can be a bit slow at times. Don't panic...
-COPY apt-sources.list /etc/apt/sources.list
+#COPY apt-sources.list /etc/apt/sources.list

 # The container has no package lists, so need to update first
 RUN dpkg --add-architecture i386 && \
     apt-get update -y
 RUN apt-get install -y --no-install-recommends \
+        automake \
         bc \
         build-essential \
         bzr \
         ca-certificates \
+        clang \
         cmake \
         cpio \
         cvs \
         file \
-        g++-multilib \
+        g++ \
         git \
-        libc6:i386 \
+        libc6 \
         libncurses5-dev \
         locales \
         mercurial \
+        ninja-build \
         python-flake8 \
         python-nose2 \
         python-pexpect \
@@ -47,6 +39,7 @@
         rsync \
         subversion \
         unzip \
+        vim \
         wget \
         && \
     apt-get -y autoremove && \

Without using Buildroot, you could probably build Chromium itself with something similar. You'll need a 32-bit version of gn too.

Maybe it's possible to build Chromium in a 32-bit VM too.

I hope that helps!

Thomas

[Patrick De Bella]

Hey Thomas,

Thank you for your quick reply.  I appreciate you sharing that information with me.  I assume that once we have the 32-bit embedded.S and snapshot_blob.bin we should be good until we switch Chromum/CEF versions.  Hopefully they fix this and we don't have to jump through these hoops in the future.

Patrick

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups "v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/v8-users/9c4ef4e5-a36e-4864-ab87-15153ad30680%40googlegroups.com.

[Thomas Preston]

On Monday, 4 May 2020 16:44:40 UTC+1, Patrick De Bella wrote:

I assume that once we have the 32-bit embedded.S and snapshot_blob.bin we should be good until we switch Chromum/CEF versions.

That's right, you only have to build them once. We use these overwrite the broken 64-bit binaries created when targeting 32-bit arm on an 64-bit build machine.