How to Create Dynamically Linked Go Programs Redux

[jlfo...@berkeley.edu]

Now that Go 1.21 has been released, I've returned to trying to figure out how to 

dynamically link a Go program. Back in January I posted the results of my first attempt

with an earlier version of Go, which was:

1) Building a Go shared library by running

        go install -buildmode=shared std

as root works fine.

2) Building a dynamically linked Go executable as a non-privileged user by adding

        -linkshared

to "go build" fails with lots of file access permission errors because
the go build tool tries to write to the Go shared library, which a non-privileged
user can't do.

3) Building a dynamically link Go executable as root works but a new version
of the Go shared library gets made in the process. This makes this command
take much longer than it should. Plus, having to be root is a non-starter.

I started looking at what "go build" is doing by adding the "-x" option.
I was able to figure out how to build and link the following program

package main
func main() {
}

using a shared library. This was quite an accomplishment. But, then I tried making it into
a "Hello, world!" program.

package main
import "fmt"
func main() {
    fmt.Println("Hello, world!\n")
}

This also compiles and links, but running it results in

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7fe02814e593]

goroutine 1 [running]:
os.(*File).write(...)
        /usr/local/go/src/os/file_posix.go:46
os.(*File).Write(0x0, {0xc0003a6000?, 0xf, 0x7fe028059f25?})
        /usr/local/go/src/os/file.go:183 +0x53
fmt.Fprintln({0x202f88, 0x0}, {0xc00035af20, 0x1, 0x1})
        /usr/local/go/src/fmt/print.go:305 +0x6f
fmt.Println(...)
        /usr/local/go/src/fmt/print.go:314

To compile the program I ran

WORK=/tmp/go-build3183434751
/usr/local/go/pkg/tool/linux_amd64/compile -p main -complete -installsuffix dynlink -goversion go1.21.0 -c=3 -dynlink -linkshared -nolocalimports -importcfg $WORK/b001/importcfg  ./file1.go

and to link it I ran

WORK=/tmp/go-build3183434751
/usr/local/go/pkg/tool/linux_amd64/link -o a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -linkshared -w file1.o

$WORK/b001/importcfg is
packagefile fmt=/usr/local/go/pkg/linux_amd64_dynlink/fmt.a
packagefile runtime=/usr/local/go/pkg/linux_amd64_dynlink/runtime.a
packagefile runtime/cgo=/usr/local/go/pkg/linux_amd64_dynlink/runtime/cgo.a

$WORK/b001/importcfg.link is many lines like
packagefile fmt=/usr/local/go/pkg/linux_amd64_dynlink/fmt.a
packageshlib fmt=/usr/local/go/pkg/linux_amd64_dynlink/libstd.so

Both these files were created when I ran the regular "go build -linkedshare" command.
I have to admit that I don't really understand what these files should contain,
and I wouldn't be surprised if this is what's causing my problem.

Any suggestions for what I'm doing wrong?

(go version go1.21.0 linux/amd64)

Cordially,

Jon Forrest

[Ian Lance Taylor]

Thanks. This looks like a bug. Somehow we must not be testing quite
this case in our -buildmode=shared testsuite.

I opened https://go.dev/issue/61973.

Ian

[jlfo...@berkeley.edu]

Thanks.

There's a more fundamental problem, though. Try running

go build -linkshared hello.go

as a non-privileged user. You'll get a bunch of permission denied messages. (I had mentioned this

back in January). This is why I'm fooling around running the compiler and linker manually.

I suggest you add this to your test suite.

Cordially,

Jon Forrest

[Ian Lance Taylor]

On Sat, Aug 12, 2023 at 10:27 AM jlfo...@berkeley.edu
<jlfo...@berkeley.edu> wrote:
>
> Thanks.
>
> There's a more fundamental problem, though. Try running
>
> go build -linkshared hello.go
>
> as a non-privileged user. You'll get a bunch of permission denied messages. (I had mentioned this
> back in January). This is why I'm fooling around running the compiler and linker manually.
>
> I suggest you add this to your test suite.

To be clear, I ran all the commands I listed as a non-privileged user.

It's not clear why you are running any of the commands as root.

Ian

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b644a419-b3a7-4021-9e1c-b8954e07000cn%40googlegroups.com.

[jlfo...@berkeley.edu]

Believe me, it's not something I want to do. But, it's the only way that

building a Go program with dynamic linking will work.

Here's what I mean:

% go build -linkshared file1.go
internal/abi: go build internal/abi: copying /home/jonf/.cache/go-build/06/0628acf114fffa5bc27f637027df97345926fcd84fc223b7965e9a29abc9a2fc-d: open /usr/local/go/pkg/linux_amd64_dynlink/internal/abi.a: permission denied
runtime/internal/math: go build runtime/internal/math: copying /home/jonf/.cache/go-build/c5/c50274855cf2c87ce7f9579d93ecbd945ce2a2fc71064b9baaf8c7fa9bc1fac9-d: open /usr/local/go/pkg/linux_amd64_dynlink/runtime/internal/math.a: permission denied
runtime/internal/sys: go build runtime/internal/sys: copying /home/jonf/.cache/go-build/59/59f10e728f337902b572dc91cc0fa5babba58786fd6b3855a08c37de6826f987-d: open /usr/local/go/pkg/linux_amd64_dynlink/runtime/internal/sys.a: permission denied
internal/bytealg: go build internal/bytealg: copying /home/jonf/.cache/go-build/e5/e55aca54d406715f95c5cdda5149a53efe34e68759cdeef7658c207f69380f6e-d: open /usr/local/go/pkg/linux_amd64_dynlink/internal/bytealg.a: permission denied
math: go build math: copying /home/jonf/.cache/go-build/66/66ea7240358ed8a641ab6dcd3ce290a817f633146c68b5cf98e56c2b35e5fa9c-d: open /usr/local/go/pkg/linux_amd64_dynlink/math.a: permission denied
image/color/palette: go build image/color/palette: copying /home/jonf/.cache/go-build/42/4250f9f38ffaf1e12f6a1e123bbb5e9323aebdcdf4dd65e7bbb4fb700763c5a7-d: open /usr/local/go/pkg/linux_amd64_dynlink/image/color/palette.a: permission denied
crypto/internal/boring/bcache: go build crypto/internal/boring/bcache: copying /home/jonf/.cache/go-build/8b/8b887de65adc0f4d5c4ecd00452f5c26f909f2a22bde7195b185ad4121060ac3-d: open /usr/local/go/pkg/linux_amd64_dynlink/crypto/internal/boring/bcache.a: permission denied
slices: go build slices: copying /home/jonf/.cache/go-build/09/09413e8fd6fd3d702b748ad7da013e04bca8ab23206c0cf1c815be383ec5f14f-d: open /usr/local/go/pkg/linux_amd64_dynlink/slices.a: permission denied

Do you perhaps have write permission to /usr/local/go?

Then there's also the question of why building a Go program should write to

/usr/local/go/pkg/linux_amd64_dynlink.

Jon