panic on arm cross compile

740 views
Skip to first unread message

jonathan...@gmail.com

unread,
Feb 26, 2015, 10:29:09 PM2/26/15
to golan...@googlegroups.com
Hello there! 
I was hoping someone might point me in the right direction for things to check about a simple hello world that is not running on an arm device.

Device is pretty custom. I am running the following:

sudo CC_FOR_TARGET=/home/me/Downloads/armv7-marvell-linux-gnueabi-softfp_i686/bin/arm-marvell-linux-gnueabi-gcc-4.6.4 GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 ./make.bash --no-clean

which succeeds. Then to build the helloworld that looks like:

package main
import "fmt"
func main() {
    fmt.Println("hello")
}

GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 go build test.go

file test yeilds: test: ELF 32-bit LSB  executable, ARM, EABI5 version 1 (SYSV), statically linked, not stripped

cat /proc/cpuinfo looks like:
Processor : Marvell PJ4Bv7 Processor rev 1 (v7l)
BogoMIPS : 1196.85
Features : swp half thumb fastmult vfp edsp vfpv3 vfpv3d16 tls 
CPU implementer : 0x56
CPU architecture: 7
CPU variant : 0x1
CPU part : 0x581
CPU revision : 1

Hardware : Marvell Armada-370
Revision : 0000
Serial : 0000000000000000

but when I run it on the device the following happens:

unexpected fault address 0xa8e24
fatal error: fault
[signal 0xb code=0x2 addr=0xa8e24 pc=0xa8e24]

goroutine 1 [running, locked to thread]:
runtime.gothrow(0xcc9f8, 0x5)
/usr/local/go/src/runtime/panic.go:503 +0x84 fp=0x10335eb4 sp=0x10335ea8
runtime.sigpanic()
/usr/local/go/src/runtime/sigpanic_unix.go:29 +0x2a4 fp=0x10335edc sp=0x10335eb4
sync/atomic.LoadUint32(0x126b78, 0x0)
?:0 fp=0x10335ee0 sp=0x10335ee0
syscall.Getenv(0xb, 0x1030a060, 0x72ccc, 0xcecf0, 0x8)
/usr/local/go/src/syscall/env_unix.go:72 +0x54 fp=0x10335f0c sp=0x10335ee0
syscall.Getenv(0x10335f60, 0x30315500, 0x1030a060, 0x5e064, 0x10338010)
/usr/local/go/src/syscall/env_unix.go:72 +0x54 fp=0x10335f38 sp=0x10335f0c

goroutine 2 [runnable]:
runtime.forcegchelper()
/usr/local/go/src/runtime/proc.go:90
runtime.goexit()
/usr/local/go/src/runtime/asm_arm.s:1322 +0x4

goroutine 3 [runnable]:
runtime.bgsweep()
/usr/local/go/src/runtime/mgc0.go:82
runtime.goexit()
/usr/local/go/src/runtime/asm_arm.s:1322 +0x4


Of course I have tried GOARM 5, 6 and 7.. for both the go src build and test.go build.

Thanks !!

Dave Cheney

unread,
Feb 27, 2015, 12:24:19 AM2/27/15
to golan...@googlegroups.com, jonathan...@gmail.com
If your host has more than 128mb of ram can you try compiling locally. You can grab precompiled tarballs from http://dave.cheney.net/unofficial-arm-tarballs. The installation instructions are identical to the official tarballs from the website.

jonathan...@gmail.com

unread,
Feb 27, 2015, 7:35:08 PM2/27/15
to golan...@googlegroups.com, jonathan...@gmail.com
Unfortunately our devices have virtually zero space left on their flash drives where the OS is mounted. Any ideas on why the cross compile isn't working? Or perhaps some more information I can gather that might help? I am a bit clueless in this realm :(

Dave Cheney

unread,
Feb 27, 2015, 7:58:22 PM2/27/15
to golan...@googlegroups.com, jonathan...@gmail.com
Can you make the binary available somewhere, I'll test it on my machine.

What kernel is the machine running ? 

jonathan...@gmail.com

unread,
Feb 27, 2015, 8:17:56 PM2/27/15
to golan...@googlegroups.com, jonathan...@gmail.com
Binary is available at http://temp-host.com/download.php?file=br91ln
cat /proc/version gives Linux version 3.2.40 (kman@kmachine) (gcc version 4.6.4 (Linaro GCC branch-4.6.4. Marvell GCC Dev 201310-2126.3d181f66 64K MAXPAGESIZE ALIGN) ) #1 Wed Feb 11 16:59:39 CST 2015
These boxes are pretty custom so I wouldn't be surprised if there are kernel changes.

Thanks I really appreciate the help! I love golang in my own projects and there are some perfect use cases at the company I am with, so naturally trying to get signoff. Hopefully I can add into the success stories and get some open source work going too!

Dave Cheney

unread,
Feb 27, 2015, 8:45:32 PM2/27/15
to golan...@googlegroups.com, jonathan...@gmail.com
That file worked on an armv7 host.

I suggest downloading the 1.4.2 tarball mentioned above, extracting the bin/go binary and running that on your host. If that doesn't work you may have to provide more information about the customisations to your kernel.

jonathan...@gmail.com

unread,
Mar 2, 2015, 3:19:28 PM3/2/15
to golan...@googlegroups.com, jonathan...@gmail.com
Hey Dave, I tried both tarballs extracted to the device on some removable storage, and both give not found errors. I'll see if I can't get more information on the kernel, but I was hoping to run go on multiple devices and they are all different.. most ARM though.
Most are some form of busybox on top of debian or some such thing. Simple C programs compile with the toolchain and run without any issues.

Dave Cheney

unread,
Mar 2, 2015, 3:35:50 PM3/2/15
to golan...@googlegroups.com, jonathan...@gmail.com
From your original message it looked like you were trying to cross compile with cgo enabled. There is a long answer to why this is difficult that other may give, so i'll just simplify it to, please don't try. 

I think if you build a new go install from source (it's literally 2 commands), the env GOARCH=arm GOOS=linux ./make.bash --no-clean to create your cross compile env, you'll have better results.

jonathan...@gmail.com

unread,
Mar 2, 2015, 5:04:42 PM3/2/15
to golan...@googlegroups.com, jonathan...@gmail.com
Is there anyway I can do this on my machine and not the arm devices? The devices have zero dev tools available and all we have available for them are the toolchains. Perhaps I am missing something

jonathan...@gmail.com

unread,
Mar 2, 2015, 7:28:57 PM3/2/15
to golan...@googlegroups.com, jonathan...@gmail.com
Looks like I get further in the init on this particular device if I use GOARM=5 as seen below:
unexpected fault address 0xaa2a0
fatal error: fault
[signal 0xb code=0x2 addr=0xaa2a0 pc=0xaa2a0]

goroutine 1 [running, locked to thread]:
runtime.gothrow(0xd29f8, 0x5)
/usr/local/go/src/runtime/panic.go:503 +0x84 fp=0x10335f1c sp=0x10335f10
runtime.sigpanic()
/usr/local/go/src/runtime/sigpanic_unix.go:29 +0x2a4 fp=0x10335f44 sp=0x10335f1c
time.init()
/usr/local/go/src/time/zoneinfo_unix.go:84 fp=0x10335f48 sp=0x10335f48
fmt.init()
/usr/local/go/src/fmt/scan.go:1169 +0x64 fp=0x10335f94 sp=0x10335f48
fmt.init()
/usr/local/go/src/fmt/scan.go:1169 +0x64 fp=0x10335fe0 sp=0x10335f94
runtime.main()
/usr/local/go/src/runtime/proc.go:46 +0x88 fp=0x10336004 sp=0x10335fe0

goroutine 2 [runnable]:
runtime.forcegchelper()
/usr/local/go/src/runtime/proc.go:90
runtime.goexit()
/usr/local/go/src/runtime/asm_arm.s:1322 +0x4

goroutine 3 [runnable]:
runtime.bgsweep()
/usr/local/go/src/runtime/mgc0.go:82
runtime.goexit()
/usr/local/go/src/runtime/asm_arm.s:1322 +0x4



Does this have something to do with it being an ARM7 device with soft floats?

minux

unread,
Mar 2, 2015, 7:58:48 PM3/2/15
to jonathan...@gmail.com, golang-nuts
according to cpuinfo, your device does have vfpv3, so you can use GOARM=7.
The fact that your OS is softfloat shouldn't affect this.

On Mon, Mar 2, 2015 at 7:28 PM, <jonathan...@gmail.com> wrote:
Looks like I get further in the init on this particular device if I use GOARM=5 as seen below:
unexpected fault address 0xaa2a0
fatal error: fault
[signal 0xb code=0x2 addr=0xaa2a0 pc=0xaa2a0]
this segmentation fault is interesting, notice that addr == pc == 0xaa2a0.

what's the instruction at 0xaa2a0? Also, try running the program with GOTRACEBACK=2. 

jonathan...@gmail.com

unread,
Mar 2, 2015, 8:39:27 PM3/2/15
to golan...@googlegroups.com, jonathan...@gmail.com
When I use GOARM=7 I get the panic always in a different place than GOARM=5 which was the last I posted. GOARM=7 looks like the first post:
unexpected fault address 0xa8e24
fatal error
: fault
[signal 0xb code=0x2 addr=0xa8e24 pc=0xa8e24]



goroutine
1 [running, locked to thread]:

runtime
.gothrow(0xcc9f8, 0x5)
 
/usr/local/go/src/runtime/panic.go:503 +0x84 fp=0x10335eb4 sp=0x10335ea8
runtime
.sigpanic()
 
/usr/local/go/src/runtime/sigpanic_unix.go:29 +0x2a4 fp=0x10335edc sp=0x10335eb4
sync
/atomic.LoadUint32(0x126b78, 0x0)
 
?:0 fp=0x10335ee0 sp=0x10335ee0

syscall
.Getenv(0xd, 0x1030a060, 0x72ccc, 0xcecf0, 0x8)

 
/usr/local/go/src/syscall/env_unix.go:72 +0x54 fp=0x10335f0c sp=0x10335ee0
syscall
.Getenv(0x10335f60, 0x30315500, 0x1030a060, 0x5e064, 0x10338010)
 
/usr/local/go/src/syscall/env_unix.go:72 +0x54 fp=0x10335f38 sp=0x10335f0c



goroutine
2 [runnable]:
runtime
.forcegchelper()
 
/usr/local/go/src/runtime/proc.go:90 fp=0x103187ec sp=0x103187ec
runtime
.goexit()
 
/usr/local/go/src/runtime/asm_arm.s:1322 +0x4 fp=0x103187ec sp=0x103187ec
created
by runtime.init·4
 
/usr/local/go/src/runtime/proc.go:87 +0x34


goroutine
3 [runnable]:
runtime
.bgsweep()
 
/usr/local/go/src/runtime/mgc0.go:82 fp=0x1031bfec sp=0x1031bfec
runtime
.goexit()
 
/usr/local/go/src/runtime/asm_arm.s:1322 +0x4 fp=0x1031bfec sp=0x1031bfec
created
by gc
 
/usr/local/go/src/runtime/mgc0.c:1386




And if I use objdump -m i386 then the relevant section around 0xa8e24 looks like:
000a8dfc <sync/atomic.AddUint32>:
   a8dfc
: 04 e0                 add    $0xe0,%al
   a8dfe
: 2d e5 08 20 9d       sub    $0x9d2008e5,%eax
   a8e03
: e5 0c                 in     $0xc,%eax
   a8e05
: 40                   inc    %eax
   a8e06
: 9d                   popf  
   a8e07
: e5 00                 in     $0x0,%eax
   a8e09
: 00 92 e5 00 10 a0     add    %dl,-0x5fefff1b(%edx)
   a8e0f
: e1 04                 loope  a8e15 <sync/atomic.AddUint32+0x19>
   a8e11
: 10 81 e0 e1 ff ff     adc    %al,-0x1e20(%ecx)
   a8e17
: eb fa                 jmp    a8e13 <sync/atomic.AddUint32+0x17>
   a8e19
: ff                   (bad)  
   a8e1a
: ff                   (bad)  
   a8e1b
: 3a 10                 cmp    (%eax),%dl
   a8e1d
: 10 8d e5 04 f0 9d     adc    %cl,-0x620ffb1b(%ebp)
   a8e23
: e4 04                 in     $0x4,%al


000a8e24 <sync/atomic.LoadUint32>:
   a8e24
: 04 e0                 add    $0xe0,%al
   a8e26
: 2d e5 08 20 9d       sub    $0x9d2008e5,%eax
   a8e2b
: e5 00                 in     $0x0,%eax
   a8e2d
: 00 92 e5 00 10 a0     add    %dl,-0x5fefff1b(%edx)
   a8e33
: e1 d9                 loope  a8e0e <sync/atomic.AddUint32+0x12>
   a8e35
: ff                   (bad)  
   a8e36
: ff                   (bad)  
   a8e37
: eb fb                 jmp    a8e34 <sync/atomic.LoadUint32+0x10>
   a8e39
: ff                   (bad)  
   a8e3a
: ff                   (bad)  
   a8e3b
: 3a 0c 10             cmp    (%eax,%edx,1),%cl
   a8e3e
: 8d                   (bad)  
   a8e3f
: e5 04                 in     $0x4,%eax
   a8e41
: f0 9d                 lock popf
   a8e43
: e4 f6                 in     $0xf6,%al


000a8e44 <sync/atomic.LoadUintptr>:
   a8e44
: f6 ff                 idiv   %bh
   a8e46
: ff                   (bad)  
   a8e47
: ea ff ff ff ea 04 e0 ljmp   $0xe004,$0xeaffffff

What does that mean that that spot in memory doesn't contain that function? (this is not my area of expertise :D)?

minux

unread,
Mar 2, 2015, 8:55:13 PM3/2/15
to jonathan...@gmail.com, golang-nuts
On Mon, Mar 2, 2015 at 8:39 PM, <jonathan...@gmail.com> wrote:
When I use GOARM=7 I get the panic always in a different place than GOARM=5 which was the last I posted. GOARM=7 looks like the first post:
unexpected fault address 0xa8e24
fatal error
: fault
[signal 0xb code=0x2 addr=0xa8e24 pc=0xa8e24]
The crash has nothing to do with GOARM values. notice that addr is equal to pc again.
what's your kernel version? Try running the program with $GOTRACEBACK=2, e.g.

GOTRACEBACK=2 ./theprogram
the binary is for ARM, so you need an arm disassembler. If you don't have one, you can
upload the binary somewhere, and I can take a look.

I really don't understand how that's possible (segfaulting in .text section).
Something is weird with the environment.

jonathan...@gmail.com

unread,
Mar 2, 2015, 9:15:26 PM3/2/15
to golan...@googlegroups.com, jonathan...@gmail.com
Binary using GOARM=5 is at http://temp-host.com/download.php?file=hp61ll

When running with GOTRACEBACK=2 I don't see but a couple "created by" lines:
unexpected fault address 0xaa2a0
fatal error: fault
[signal 0xb code=0x2 addr=0xaa2a0 pc=0xaa2a0]

goroutine 1 [running, locked to thread]:
runtime.gothrow(0xd29f8, 0x5)
/usr/local/go/src/runtime/panic.go:503 +0x84 fp=0x10335f1c sp=0x10335f10
runtime.sigpanic()
/usr/local/go/src/runtime/sigpanic_unix.go:29 +0x2a4 fp=0x10335f44 sp=0x10335f1c
time.init()
/usr/local/go/src/time/zoneinfo_unix.go:84 fp=0x10335f48 sp=0x10335f48
fmt.init()
/usr/local/go/src/fmt/scan.go:1169 +0x64 fp=0x10335f94 sp=0x10335f48
fmt.init()
/usr/local/go/src/fmt/scan.go:1169 +0x64 fp=0x10335fe0 sp=0x10335f94
runtime.main()
/usr/local/go/src/runtime/proc.go:46 +0x88 fp=0x10336004 sp=0x10335fe0

goroutine 2 [runnable]:
runtime.forcegchelper()
/usr/local/go/src/runtime/proc.go:90 fp=0x103187ec sp=0x103187ec
runtime.goexit()
/usr/local/go/src/runtime/asm_arm.s:1322 +0x4 fp=0x103187ec sp=0x103187ec
created by runtime.init·4
/usr/local/go/src/runtime/proc.go:87 +0x34

goroutine 3 [runnable]:
runtime.bgsweep()
/usr/local/go/src/runtime/mgc0.go:82 fp=0x1031bfec sp=0x1031bfec
runtime.goexit()
/usr/local/go/src/runtime/asm_arm.s:1322 +0x4 fp=0x1031bfec sp=0x1031bfec
created by gc
/usr/local/go/src/runtime/mgc0.c:1386

minux

unread,
Mar 2, 2015, 9:48:28 PM3/2/15
to jonathan...@gmail.com, golang-nuts
On Mon, Mar 2, 2015 at 9:14 PM, <jonathan...@gmail.com> wrote:
Binary using GOARM=5 is at http://temp-host.com/download.php?file=hp61ll

When running with GOTRACEBACK=2 I don't see but a couple "created by" lines:
unexpected fault address 0xaa2a0
fatal error: fault
[signal 0xb code=0x2 addr=0xaa2a0 pc=0xaa2a0]
code=0x2 is SEGV_ACCERR, which means the address is not mapped for execution.

  [ 1] .text             PROGBITS        00010c00 000c00 09d7ec 00  AX  0   0  4

  Type             Offset        VirtAddr         PhysAddr   FileSiz    MemSiz  Flg Align
  LOAD           0x000000 0x00010000 0x00010000 0x9e3ec 0x9e3ec R E 0x1000

The address 0xaa2a0 should definitely be mapped for execution.

disassembly showed that 0xaa2a0 is the very first instruction of the time.init function.
000aa2a0 <time.init>:
   aa2a0:       e59a1008        ldr     r1, [sl, #8]
   aa2a4:       e15d0001        cmp     sp, r1
   aa2a8:       91a0300e        movls   r3, lr
   aa2ac:       9bfe7d38        blls    49794 <runtime.morestack_noctxt>

I really don't understand what's happening here. Something is preventing the OS from
mapping that page for execution.

jonathan...@gmail.com

unread,
Mar 2, 2015, 10:53:39 PM3/2/15
to golan...@googlegroups.com, jonathan...@gmail.com
I know these devices have page size changes. And the /proc/version file shows:
Linux version 3.10.39 (kman@kmachine) (gcc version 4.6.4 (Linaro GCC branch-4.6.4. Marvell GCC Dev 201310-2126.3d181f66 64K MAXPAGESIZE ALIGN) ) #26 SMP ...

which has the word align in it... :( I'll see if I can't get some details from the vendors.

minux

unread,
Mar 2, 2015, 11:01:55 PM3/2/15
to jonathan...@gmail.com, golang-nuts
On Mon, Mar 2, 2015 at 10:53 PM, <jonathan...@gmail.com> wrote:
I know these devices have page size changes. And the /proc/version file shows:
Linux version 3.10.39 (kman@kmachine) (gcc version 4.6.4 (Linaro GCC branch-4.6.4. Marvell GCC Dev 201310-2126.3d181f66 64K MAXPAGESIZE ALIGN) ) #26 SMP ...

which has the word align in it... :( I'll see if I can't get some details from the vendors.
OK. That explains it. Assume the page size is 64KB, there is page permission conflict 
in the last page of .text segment and the start of .rodata segment.

  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  PHDR           0x000034 0x00010034 0x00010034 0x000c0 0x000c0 R   0x1000
  LOAD           0x000000 0x00010000 0x00010000 0x9e3ec 0x9e3ec R E 0x1000
  LOAD           0x09f000 0x000af000 0x000af000 0x75e11 0x75e11 R   0x1000
  LOAD           0x115000 0x00125000 0x00125000 0x05388 0x14ac8 RW  0x1000
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x4
  LOOS+5041580   0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4

The .text segment ends at 0x9efff, and the next read-only segment starts at
0x9f000, and those are in the same 64KB page. Normally, the kernel should
either reject such binaries or set the permission of conflicting page to the larger
set (R + E in this case), but somehow your kernel sets the permission to the
smaller one (R only).

I think it's a kernel bug, but beware that we've never tested linux/arm on systems
non-4KB page size systems, there might be other problems after fixing the kernel.

jonathan...@gmail.com

unread,
Mar 3, 2015, 1:57:26 PM3/3/15
to golan...@googlegroups.com, jonathan...@gmail.com
Ah thanks for looking, but this is bad news! :( Looking at some of the C/C++ projects around it looks like on most devices they compile with -fPIC. Would that help here? Could I add it into the cross compile?
I tried just using
CC_FOR_TARGET="<toolchain gcc> -fPIC" ... ./make.bash
but get the same results.

jonathan...@gmail.com

unread,
Mar 4, 2015, 3:44:13 PM3/4/15
to golan...@googlegroups.com, jonathan...@gmail.com
I am just shooting in the dark here but is it perhaps within go like it maybe was for powerpc? https://github.com/golang/go/commit/f9fdc887ae71ebcf26c980f2f15ace2efec94881


Reaaaally hoping I won't be going back to javaland !

Dave Cheney

unread,
Mar 4, 2015, 4:49:57 PM3/4/15
to golan...@googlegroups.com
Email me off list, I'll send you some patches to try

minux

unread,
Mar 4, 2015, 5:12:05 PM3/4/15
to jonathan...@gmail.com, golang-nuts
On Wed, Mar 4, 2015 at 3:43 PM, <jonathan...@gmail.com> wrote:
I am just shooting in the dark here but is it perhaps within go like it maybe was for powerpc? https://github.com/golang/go/commit/f9fdc887ae71ebcf26c980f2f15ace2efec94881


Reaaaally hoping I won't be going back to javaland !

Just setting the PageSize in runtime won't help in this case. Some cmd/ld changes
are needed to round the section sizes to multiples of 64K.

Aram Hăvărneanu

unread,
Mar 4, 2015, 5:36:03 PM3/4/15
to minux, jonathan...@gmail.com, golang-nuts
We really need builders that use 64k pages.

--
Aram Hăvărneanu

jonathan...@gmail.com

unread,
Mar 6, 2015, 1:31:00 PM3/6/15
to golan...@googlegroups.com, mi...@golang.org, jonathan...@gmail.com
So is this a fix needed in go or in our kernel? If go, is anyone interested in looking at this? If in the kernel, how do I get it to show up through a C program?

Aram Hăvărneanu

unread,
Mar 7, 2015, 2:24:03 PM3/7/15
to jonathan...@gmail.com, golang-nuts, minux
In Go. You can work around it using a kernel with 4k pages, but
presumably you have a reason to use a kernel with 64k pages.

--
Aram Hăvărneanu

jonathan...@gmail.com

unread,
Mar 12, 2015, 4:33:45 PM3/12/15
to golan...@googlegroups.com, jonathan...@gmail.com, mi...@golang.org
Yes we use 64k pages for other performance reasons and certainly can't be changed.
So how do I wind up the interest and get more eyes on this? :D Are these kind of things a priority for the go team or do I need to put some serious time into brushing up with C?

Aram Hăvărneanu

unread,
Mar 12, 2015, 5:04:56 PM3/12/15
to jonathan...@gmail.com, golang-nuts, minux
I will deal with it as it's affecting arm64 as well.

Unfortunately I can't promise a timeframe, but it will get fixed.

--
Aram Hăvărneanu

Aram Hăvărneanu

unread,
Mar 12, 2015, 5:05:29 PM3/12/15
to jonathan...@gmail.com, golang-nuts, minux
Also, if you want to help you don't need to brush up with C, as the
runtime is all in Go and assembly now, no more C code in the tree.

--
Aram Hăvărneanu

jonathan...@gmail.com

unread,
Mar 12, 2015, 7:57:27 PM3/12/15
to golan...@googlegroups.com, jonathan...@gmail.com, mi...@golang.org
Thanks Aram that's great news! I will start looking into the runtime once 1.5 is out for sure. Is there a ticket I can follow progress on for this issue?

jonathan...@gmail.com

unread,
Mar 13, 2015, 2:01:23 PM3/13/15
to golan...@googlegroups.com, jonathan...@gmail.com, mi...@golang.org
Hey Aram, just wanted to shoot you what one of our kernel guys said to see if it jives with the changes that will be made.

The kernel is set to a 64K page size for performance reasons.  In order to accomplish this, we had to change the linker so that the elf tables of all binaries are 64K aligned.  So, binaries will run on a 4K kernel, but 4K aligned binaries from a standard distro (actually they are 32K aligned), will not run.

That doesn't put a wrench in the works does it? 

On Thursday, March 12, 2015 at 2:05:29 PM UTC-7, Aram Hăvărneanu wrote:

Dave Cheney

unread,
Mar 13, 2015, 3:56:40 PM3/13/15
to golan...@googlegroups.com
Yes, quite a lot

minux

unread,
Mar 13, 2015, 5:17:14 PM3/13/15
to jonathan...@gmail.com, golang-nuts
I think we can just increase the section alignment to something even larger,
say 1MB. i will send a patch.

For the runtime, I'd prefer that _PhysPageSize be made a variable and
automatically detected. Non-4k page size systems are increasingly popular,
and Go's tradition is to the correct thing automatically.

Dave Cheney

unread,
Mar 13, 2015, 5:53:59 PM3/13/15
to golan...@googlegroups.com
I thanks minux for the offer of a patch. Jonathan, I can make you up a few test binaries to try.

Minux, I've been thinking that we can recover the page size from auxv. This bit isnt hard, but it would mean that _PageSize would become non constant on linux platforms and this may have some unexpected effects on the efficiency of the gc.

minux

unread,
Mar 13, 2015, 6:26:32 PM3/13/15
to jonathan...@gmail.com, golang-nuts
The simplest fix is to add -ldflags "-R 65536" to cmd/go command line
to build your binaries.

However, that will increase the size of the binary slightly. And I'm
looking for a solution that will set the INITRND value to a larger value
without impacting the size of the executable so that we can set that
to a much larger value (e.g. 4MB) and Go binary will be runnable on
machine with page size as large as 4MB.

jonathan...@gmail.com

unread,
Mar 16, 2015, 2:00:43 PM3/16/15
to golan...@googlegroups.com, jonathan...@gmail.com
Well that totally fixes everything! I even tried a little example that utilizes cgo to make sure that works too.
I very much appreciate this workaround, and may I ask where if anywhere the -R is documented so I can better arm myself in the future?

minux

unread,
Mar 16, 2015, 4:18:36 PM3/16/15
to jonathan...@gmail.com, golang-nuts
On Mon, Mar 16, 2015 at 2:00 PM, <jonathan...@gmail.com> wrote:
Well that totally fixes everything! I even tried a little example that utilizes cgo to make sure that works too.
I very much appreciate this workaround, and may I ask where if anywhere the -R is documented so I can better arm myself in the future?

It's only documented in the go tool 5l output.... As I've said, for Go 1.5, I want to have
a better solution for this problem so that the binary just works.

I've filed #10180 for this. the linker docs gets worse on the tip, and that's #10179.

Dave Cheney

unread,
Mar 16, 2015, 6:01:19 PM3/16/15
to golan...@googlegroups.com
Great news. Is this with the other 64k patch I sent you?

jonathan...@gmail.com

unread,
Mar 16, 2015, 7:14:59 PM3/16/15
to golan...@googlegroups.com
This was with a vanilla checkout of go1.4.2, thanks for the patch though!

jonathan...@gmail.com

unread,
Mar 31, 2015, 8:59:27 PM3/31/15
to golan...@googlegroups.com, jonathan...@gmail.com
Hey minux,
Trying to call some of the image package functions, I am getting "cannot map pages in arena address space". Does this have anything to do with using -ldflags "-R 65536"? Granted the box doesn't have a lot of memory, but shouldn't it start paging when it runs out?

jonathan...@gmail.com

unread,
Mar 31, 2015, 9:59:30 PM3/31/15
to golan...@googlegroups.com, jonathan...@gmail.com
Tried changing all ulimits to unlimited with no success. The program is just decoding a jpeg, that's the failing line with jpeg.Decode().

minux

unread,
Mar 31, 2015, 10:17:03 PM3/31/15
to jonathan...@gmail.com, golang-nuts
On Tue, Mar 31, 2015 at 8:59 PM, <jonathan...@gmail.com> wrote:
Hey minux,
Trying to call some of the image package functions, I am getting "cannot map pages in arena address space". Does this have anything to do with using -ldflags "-R 65536"? Granted the box doesn't have a lot of memory, but shouldn't it start paging when it runs out?

I shouldn't have anything to do with -ldflags "-R 65536".

Please strace the process to see what does the failing mmap syscall
(should be the last one) returns, perhaps you need Dave's patch to
set _PhysPageSize to 65536 in the runtime (if the error is EINVAL).

jonathan...@gmail.com

unread,
Mar 31, 2015, 10:33:46 PM3/31/15
to golan...@googlegroups.com, jonathan...@gmail.com
strace does show mmap2 EINVAL, and a very long list of other stuff :D I'll try with Dave's patch in just a bit.

Dave Cheney

unread,
Mar 31, 2015, 10:45:45 PM3/31/15
to golan...@googlegroups.com
Page size will be an issue, can you show that mmap strace results?

jonathan...@gmail.com

unread,
Mar 31, 2015, 10:53:31 PM3/31/15
to golan...@googlegroups.com, jonathan...@gmail.com
Minux, Dave, it does work with the -ldflags "-R 65536" and having done the make.bash with the  _PhysPageSize set to 65536. Thanks you guys rock! Will this be included in 1.5?

Dave the patch also had one other thing which you said was disabling the scavenger, what is that and what would it affect? (thinking potential other failures cropping up here)

jonathan...@gmail.com

unread,
Apr 23, 2015, 9:08:23 PM4/23/15
to golan...@googlegroups.com, jonathan...@gmail.com
I have not had any more issues with our weird devices on ARM for go1.4, that's good :) Sorry to beat a dead horse here but will this needing to change PhysPageSize and give the linker option be fixed for go1.5? I don't see any references to PhysPageSize in gotip or any PageSize reference that has a meaningful number, so I am thinking the answer is yes.

Thanks!

minux

unread,
Apr 23, 2015, 9:13:59 PM4/23/15
to jonathan...@gmail.com, golang-nuts


On Apr 23, 2015 21:08, <jonathan...@gmail.com> wrote:
>
> I have not had any more issues with our weird devices on ARM for go1.4, that's good :) Sorry to beat a dead horse here but will this needing to change PhysPageSize and give the linker option be fixed for go1.5? I don't see any references to PhysPageSize in gotip or any PageSize reference that has a meaningful number, so I am thinking the answer is yes.

There is issue 10180 for the non-4K page size problem. I intend to get it fixed for Go 1.5.

Reply all
Reply to author
Forward
0 new messages