go1.5 cross-compilation ARM (raspberry pi B+)

[Brandon Thomson]

Hey gophers,

I am testing cross-compilation with go1.5 beta 2 from 

  FreeBSD 10.1-RELEASE amd64

  to

  FreeBSD 10.1-RELEASE armv5 (for Raspberry Pi B+, custom kernel)

The command produces an ARM binary after few seconds:

    % env GOCHAR=5 GOARM=5 GOARCH=arm /usr/local/go1.5/bin/go build -o xpi-www
    % file xpi-www
    xpi-www: ELF 32-bit LSB executable, ARM, version 1 (FreeBSD), statically linked, not stripped

However, when run on Pi, binary exits with:

    exited on signal 4 (core dumped)

Curious if anyone has gotten this working, or if I should report a bug?

Also, I noticed go env does not show any GOARM= or GOCHAR= even though I have set it. Not sure if that is related?

[Dave Cheney]

You probably want to set GOARM=6 for the Raspberry Pi, and GOARM=7 for the RPi2.

I think the missing value is GOOS=linux, as you can see from the example, the binary produced is still targeting FreeBSD. I'd try

env GOOS=linux GOARCH=arm GOARM=6 go build $CMD

[Brad Fitzpatrick]

It sounds like he runs FreeBSD on his Raspberry Pi, so GOOS=linux omission was intentional.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave Cheney]

I am sorry, I misunderstood. If twiddling GOARM doesn't help, please gdb the binary and tell me what it thinks signal 4 is. I'm suspecting SIGILL.

[Brandon Thomson]

Hey guys, thanks for your hard work and contributions to Go. Yes, it was SIGILL.

I produced binaries for different arm versions:

    % env GOARM=5 GOARCH=arm /usr/local/go1.5/bin/go build -o g5
    % env GOARM=6 GOARCH=arm /usr/local/go1.5/bin/go build -o g6
    % env GOARM=7 GOARCH=arm /usr/local/go1.5/bin/go build -o g7

Same results for all:

    # g5
    Illegal instruction (core dumped)
    # g6
    Illegal instruction
    # g7
    Illegal instruction

(I am not sure why it is not saying signal 4 anymore... maybe because gdb is installed now)

gdb output (I have also copied /usr/local/go1.5/src/runtime/asm_arm.s to the Pi so debugging output would appear):

    # echo "run" > /tmp/c

    # gdb -batch -x /tmp/c /usr/bin/g5
    [GDB will not be able to debug user-mode threads: Shared object "libthread_db.so" not found, required by "gdb"]
    warning: shared library handler failed to enable breakpoint
    
    Program received signal SIGILL, Illegal instruction.
    runtime.armPublicationBarrier () at /usr/local/go1.5/src/runtime/asm_arm.s:747
    warning: Source file is more recent than executable.
    
    747             WORD $0xf57ff05e        // DMB ST
    Current language:  auto; currently minimal
    


    # gdb -batch -x /tmp/c /usr/bin/g6
    [GDB will not be able to debug user-mode threads: Shared object "libthread_db.so" not found, required by "gdb"]
    warning: shared library handler failed to enable breakpoint
    
    Program received signal SIGILL, Illegal instruction.
    runtime.armPublicationBarrier () at /usr/local/go1.5/src/runtime/asm_arm.s:747
    warning: Source file is more recent than executable.
    
    747             WORD $0xf57ff05e        // DMB ST
    Current language:  auto; currently minimal
    


    # gdb -batch -x /tmp/c /usr/bin/g7
    [GDB will not be able to debug user-mode threads: Shared object "libthread_db.so" not found, required by "gdb"]
    warning: shared library handler failed to enable breakpoint
    
    Program received signal SIGILL, Illegal instruction.
    runtime.check () at /usr/local/go1.5/src/runtime/runtime1.go:146
    146     /usr/local/go1.5/src/runtime/runtime1.go: No such file or directory.
            in /usr/local/go1.5/src/runtime/runtime1.go
    Current language:  auto; currently minimal

[Fredrik Ehnbom]

Untested, but try replacing src/runtime/sys_freebsd_arm.s:385:

// TODO: this is only valid for ARMv7+

TEXT ·publicationBarrier(SB),NOSPLIT,$-4-0

B runtime·armPublicationBarrier(SB)

with:

// TODO: this is only valid for ARMv??

TEXT ·publicationBarrier(SB),NOSPLIT,$-4-0

WORD $0xee070fba // mcr p15, 0, r0, c7, c10, 5

RET

[Brandon Thomson]

Thank you, now GOARM=5 and GOARM=6 binaries are working (only tested "hello world" and http.Get so far)

[Brad Fitzpatrick]

[+austin]

Austin, this is probably why Linux is broken on the dashboard too?

[Austin Clements]

Linux takes a completely different code path for this, so this isn't what's breaking the Linux builders.

My understanding is that Go doesn't officially support FreeBSD on earlier versions of ARM, but I could be mistaken.

Memory barriers on ARM are a mess. You're right that this is the right instruction on ARMv6, but it will crash on ARMv5, will take hundreds of cycles to do nothing on single core ARMv6 (very common), and I'm not sure it's supported on ARMv7. Because of how complex this is, the Linux kernel actually provides user space with the appropriate code to implement a memory barrier on the current architecture (even detecting the current architecture is a pain!). This is what we use on Linux to implement the publication barrier. Do you know if FreeBSD provides something similar?

[Fredrik Ehnbom]

It's not obvious to me from https://github.com/freebsd/freebsd/search?p=1&q=dmb. From what I could see it appears they use "dmb", "mcr p15, 0, %0, c7, c10, 5" and "mcr p15, 0, %0, c7, c10, 4" for 7, 6 and 5 respectively with a compile time flag selecting which to use.

/f

[Austin Clements]

Yes, I think you're right that FreeBSD doesn't abstract the memory barrier for user space.

If we intend to support FreeBSD/ARMv6 (which is not something I'm taking as a given, since any port requires continued resources to maintain and increases the cost of certain types of changes to the runtime and compilers), the runtime may need to detect the current architecture and select the appropriate memory barrier. I don't think it's enough to base this on GOARM because you could compile for ARMv5, which has no barriers because it's always uniprocessor, but run on an SMP ARMv6 or ARMv7.

[Michael Hudson-Doyle]

Looks like freebsd/armv6 is not working currently:

mwhudson@glamdring:go$ grep -i publication --context=1
src/runtime/sys_freebsd_arm.s

// TODO: this is only valid for ARMv7+
TEXT ·publicationBarrier(SB),NOSPLIT,$-4-0
B runtime·armPublicationBarrier(SB)

sys_linux_arm.s uses a kernel helper for this:

mwhudson@glamdring:go$ grep -i publication --context=1
src/runtime/sys_linux_arm.s

TEXT publicationBarrier<>(SB),NOSPLIT,$0
// void __kuser_memory_barrier(void);
--

TEXT ·publicationBarrier(SB),NOSPLIT,$0
BL publicationBarrier<>(SB)
RET

Does FreeBSD have anything similar?

Cheers,
mwh

[Dave Cheney]

We only support freebsd/arm on armv6 and above (and so does freebsd itself)

If this workaround can be used for just freebsd_arm (travelling ATM) then it should be OK.

[Austin Clements]

If this also works on ARMv7, I'm okay with this workaround, though it should have an accompanying 1.6 issue for trying to use DMB on FreeBSD/ARMv7+.