Cross-building with differing byte order

[Ben Hutchings]

I'm currently working with a patch series that restores ppc32 support to V8 and Node, which I want to get upstream. I need to be able to cross-build for ppc32 from an x86 system (little-endian to big-endian).

This patch series was originally developed for Node v12.16.1 with V8 v7.8(ish), and was built with V8 snapshots disabled as they generally did not work in cross-builds. However, since snapshots are always enabled now, I need to make them work for this cross-build configuration.

I've made a start on this, but it looks like a lot of the (de)serialisation code will need to be changed to allow for differing byte order between build- and run-time. (Word size seems to be less of an issue, so long as it's possible to run 32-bit programs on the build machine.)

Questions:

1. Is it expected that snapshots should generally work in cross-builds?
2. If they are expected to work, will it be acceptable to make a lot of changes to (de)serialisation to make them work with differing byte order?
3. If they are not expected to work, is there an escape hatch that still allows them to be disabled in cross-builds?
4. Has anyone else worked on this problem, or otherwise have some pointers for how to handle it?

Ben.

[Jakob Gruber]

Hi Ben,

sorry for the late reply, I was (and still am) out of office.

On Mon, Dec 7, 2020 at 3:24 PM Ben Hutchings <ben.hu...@essensium.com> wrote:

I'm currently working with a patch series that restores ppc32 support to V8 and Node, which I want to get upstream. I need to be able to cross-build for ppc32 from an x86 system (little-endian to big-endian).

This patch series was originally developed for Node v12.16.1 with V8 v7.8(ish), and was built with V8 snapshots disabled as they generally did not work in cross-builds. However, since snapshots are always enabled now, I need to make them work for this cross-build configuration.

I've made a start on this, but it looks like a lot of the (de)serialisation code will need to be changed to allow for differing byte order between build- and run-time. (Word size seems to be less of an issue, so long as it's possible to run 32-bit programs on the build machine.)

Questions:

1. Is it expected that snapshots should generally work in cross-builds?

Yes, with restrictions. For example, host- and target bitness must match, both must be 32- or 64-bit. I haven't thought much about byte order in this context, but it seems possible. Have you tried building the x86 mksnapshot as big-endian? Mksnapshot doesn't run any JS code and thus doesn't use the JS heap; all it needs to do is create and serialize it properly. That seems like a good first path to explore.

There are some older, slightly related docs here: Windows cross-compilation, cross-compilation and mksnapshot.

1. If they are expected to work, will it be acceptable to make a lot of changes to (de)serialisation to make them work with differing byte order?

Yes, changes to support cross-byte-order builds would be acceptable in general. I am hoping that their scope would remain small though. 

1. If they are not expected to work, is there an escape hatch that still allows them to be disabled in cross-builds?
2. Has anyone else worked on this problem, or otherwise have some pointers for how to handle it?

Ben.

--
--
v8-dev mailing list
v8-...@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/v8-dev/ee4bfba2-cee9-4302-96d8-79686ba8c95bn%40googlegroups.com.

[Ben Hutchings]

On Thu, 31 Dec 2020 at 09:00, Jakob Gruber <jgr...@chromium.org> wrote:

Hi Ben,

sorry for the late reply, I was (and still am) out of office.

On Mon, Dec 7, 2020 at 3:24 PM Ben Hutchings <ben.hu...@essensium.com> wrote:

[...]

Questions:

1. Is it expected that snapshots should generally work in cross-builds?

Yes, with restrictions. For example, host- and target bitness must match, both must be 32- or 64-bit. I haven't thought much about byte order in this context, but it seems possible. Have you tried building the x86 mksnapshot as big-endian?

I think my cross-builds were running mksnapshot with a big-endian target specified, but I would have to check that to be certain. But currently most object fields are copied directly to/from snapshots with no concern for byte order, so I don't think that had much effect.

 

Mksnapshot doesn't run any JS code and thus doesn't use the JS heap; all it needs to do is create and serialize it properly. That seems like a good first path to explore.

There are some older, slightly related docs here: Windows cross-compilation, cross-compilation and mksnapshot.

Thanks, this explains some things I was confused about.

1. If they are expected to work, will it be acceptable to make a lot of changes to (de)serialisation to make them work with differing byte order?

Yes, changes to support cross-byte-order builds would be acceptable in general. I am hoping that their scope would remain small though. 

[...]

I'm afraid they're currently quite extensive, as I needed to extend object body descriptor classes to support iteration over all object fields (not just slots).

Ben.