compile time error vs runtime crash for same array

[arthurwil...@gmail.com]

On a 64bit Mac, this code:

package main

var X [^uint(0)>>14]byte

func main() {

}

produces a compile time error:

main.X: symbol too large (1125899906842623 bytes > 2000000000 bytes)

But this compiles and crashes at runtime. 

package main

func main() {

var X [^uint(0) >> 14]byte

_ = X

}

runtime: out of memory: cannot allocate 1125899906842624-byte block (3997696 in use)

fatal error: out of memory

goroutine 1 [running]:

runtime.throw({0x1061335, 0x11})

        /usr/local/go/src/runtime/panic.go:1198 +0x71 fp=0xc000042658 sp=0xc000042628 pc=0x102b031

runtime.(*mcache).allocLarge(0x100a734, 0x3ffffffffffff, 0x11, 0x1)

        /usr/local/go/src/runtime/mcache.go:229 +0x22e fp=0xc0000426b8 sp=0xc000042658 pc=0x101086e

runtime.mallocgc(0x3ffffffffffff, 0x1059300, 0x1)

        /usr/local/go/src/runtime/malloc.go:1082 +0x5c5 fp=0xc000042738 sp=0xc0000426b8 pc=0x100a645

runtime.newobject(0x1058a00)

        /usr/local/go/src/runtime/malloc.go:1228 +0x27 fp=0xc000042760 sp=0xc000042738 pc=0x100aa87

main.main()

        /Users/billmorgan/git/tmp/main.go:5 +0x25 fp=0xc000042780 sp=0xc000042760 pc=0x1054c65

runtime.main()

        /usr/local/go/src/runtime/proc.go:255 +0x227 fp=0xc0000427e0 sp=0xc000042780 pc=0x102d6c7

runtime.goexit()

        /usr/local/go/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc0000427e8 sp=0xc0000427e0 pc=0x1051fa1

Why does the 2nd program succeed compilation of the same array that failed compilation in the 1st program? 

It was known ahead of time that the allocation would fail so I think the 2nd program should have failed to compile too.

[Sean Liao]

global variables are stored in there data section of the compiled binary, the linker imposes a 2GB size limit
the array in main can be allocated at runtime, and given enough memory, it could succeed

[arthurwil...@gmail.com]

On Saturday, November 13, 2021 at 12:48:41 PM UTC-6 seank...@gmail.com wrote:

global variables are stored in there data section of the compiled binary, the linker imposes a 2GB size limit
the array in main can be allocated at runtime, and given enough memory, it could succeed

Why does the linker impose a 2GB limit on .bss data but not heap memory? Is there a limit on heap variables?  

[Jan Mercl]

The linker knows nothing about heap. That's a runtime only thing.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/5841fddd-8bf6-4e4e-b27c-7e74c8917516n%40googlegroups.com.

[tapi...@gmail.com]

It is 1 << 50 is the limit on heap. The following code doesn't compile:

package main

func main() {
    var X [1 << 50]byte // type [1125899906842624]byte larger than address space
    _ = X
}

But with 1 << 50 - 1, it compiles:

package main

func main() {
    var X [1 << 50 - 1]byte // runtime: out of memory: cannot allocate 1125899906842624-byte block (3997696 in use)
    _ = X
}

It is weird that there are several different runtime error messages:

package main

func main() {
    var X [1 << 46]byte // fatal error: runtime: out of memory
    _ = X
}

package main

func main() {
    var X [1 << 47]byte // runtime: out of memory: cannot allocate 140737488355328-byte block (3997696 in use)\nfatal error: out of memory
    _ = X
}

So it looks there are at least 3 thresholds are set in the code.

[tapi...@gmail.com]

There is an issue for this: https://github.com/golang/go/issues/17378