Question about adding new feature to go (FreeBSD/netlink) and compliation

[Martin Stiemerling]

Hi all,

FreeBSD has since a while also support for the netlink facility, similar to Linux. For Linux there is support in go via the syscall pkg for working with Linux's netlink, but not yet for FreeBSD

I have started to add the FreeBSD part to go's syscall package and fixed some errors, but run into troubles when compiling go. Though my approach is by now copying a lot form syscall_linux, as a first naive approach. :-)

When using make.bash these steps work:

Building Go cmd/dist using /usr/local/go122. (go1.22.6 freebsd/amd64)
Building Go toolchain1 using /usr/local/go122.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1

However, after "Building Go toolchain2 using go_bootstrap and Go toolchain1" this error turns out:

"go: cannot find GOROOT directory: /home/mls/goworks
go tool dist: FAILED: /home/mls/goworks/pkg/tool/freebsd_amd64/go_bootstrap install -pgo=off cmd/asm cmd/cgo cmd/compile cmd/link cmd/preprofile: exit status 2"

The changed go code is available here: 

https://code.fbi.h-da.de/m.stiemerling/go/-/tree/freebsd-netlink

Compiling directly from go's git master branch succeeds, so I have managed to break something in my part. However,  I have not found, other than make.bash -v, to get any further info or error message what is wrong.

Does anybody have a hint to get more or better infos during the build process why it fails?

go env below!

Thanks,

  Martin

GO111MODULE='on'
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/mls/.cache/go-build'
GOENV='/home/mls/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='freebsd'
GOINSECURE=''
GOMODCACHE='/home/mls/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='freebsd'
GOPATH='/home/mls/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go122'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go122/pkg/tool/freebsd_amd64'
GOVCS=''
GOVERSION='go1.22.6'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='cc'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/home/mls/goworks/src/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3067957388=/tmp/go-build -gno-record-gcc-switches'

[Ian Lance Taylor]

On Tue, Sep 3, 2024 at 9:03 AM Martin Stiemerling
<mstiem...@gmail.com> wrote:
>
> FreeBSD has since a while also support for the netlink facility, similar to Linux. For Linux there is support in go via the syscall pkg for working with Linux's netlink, but not yet for FreeBSD
>
> I have started to add the FreeBSD part to go's syscall package and fixed some errors, but run into troubles when compiling go. Though my approach is by now copying a lot form syscall_linux, as a first naive approach. :-)
>
> When using make.bash these steps work:
> Building Go cmd/dist using /usr/local/go122. (go1.22.6 freebsd/amd64)
> Building Go toolchain1 using /usr/local/go122.
> Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1
>
> However, after "Building Go toolchain2 using go_bootstrap and Go toolchain1" this error turns out:
> "go: cannot find GOROOT directory: /home/mls/goworks
> go tool dist: FAILED: /home/mls/goworks/pkg/tool/freebsd_amd64/go_bootstrap install -pgo=off cmd/asm cmd/cgo cmd/compile cmd/link cmd/preprofile: exit status 2"

Where does the directory /home/mls/goworks come from? Does it exist?

Do you have GOROOT set in the environment?

Ian

[Martin Stiemerling]

/home/mls/goworks exists and under /home/mls/goworks/src is the go source code.

>
> Do you have GOROOT set in the environment?

Nope, no environment set, as described under https://go.dev/doc/install/source#environment.

Interestingly does the whole process just run through, with the same setting, i.e., no GOROOT set, without my modifications (master branch).

I guess something goes wrong and the binaries under /home/mls/goworks/bin are not generated.

Thus the assumed GOROOT of the build process, i.e., /home/mls/goworks and $GOROOT/bin, doesn’t work.

However, I cannot see any error message or warning...

Thanks,

Martin

[Ian Lance Taylor]

On Tue, Sep 3, 2024 at 10:07 AM Martin Stiemerling

Thanks. My conclusion is that this must have something to do with
your changes. I saw that you posted your complete tree, but is there
an easy way to see only what you changed?

Ian

[Martin Stiemerling]

Here you go:
https://code.fbi.h-da.de/m.stiemerling/go/-/compare/master...freebsd-netlink?from_project_id=36647&straight=false





>
> Ian

[Ian Lance Taylor]

On Tue, Sep 3, 2024 at 10:25 AM Martin Stiemerling

Thanks. I'm sorry, I don't see anything there that could be causing
the problems you describe. I don't know what is happening.

Ian

[Martin Stiemerling]

Thanks a lot!

I am also clueless, especially as there is no error message etc telling why, or where, it is breaking.

Martin

[Mark Geisert]

Maybe a red herring, but you do have

GOROOT='/usr/local/go122'

in the output of go env in your original post.

..mark

[Martin Stiemerling]

> Am 04.09.2024 um 06:58 schrieb Mark Geisert <mark.a....@gmail.com>:
>
> Maybe a red herring, but you do have
> GOROOT='/usr/local/go122'
> in the output of go env in your original post.

This is set automatically by go itself. I’ve also stumbled across this and double-checked if GOROOT is set by my or the shell. However, this isn’t the case.

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/6b004c80-7b34-4a01-b486-0ce5f8bc7a5cn%40googlegroups.com.

[Martin Stiemerling]

Hi again,

I just wanted to get back to this and probably ask somebody out of the go core team, if you guys have recently used the mkall.sh script under src/syscall for freebsd/amd64?

I guess my problems start already there, as running this on a fresh master already and building go it starts complaining :-)

Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
/home/mls/goworks/src/syscall/ztypes_freebsd_amd64.go:285:15: undefined: _cgopackage
/home/mls/goworks/src/syscall/syscall_freebsd_amd64.go:18:4: k.Ident undefined (type *Kevent_t has no field or method Ident)
/home/mls/goworks/src/syscall/syscall_freebsd_amd64.go:19:4: k.Filter undefined (type *Kevent_t has no field or method Filter)
/home/mls/goworks/src/syscall/syscall_freebsd_amd64.go:20:4: k.Flags undefined (type *Kevent_t has no field or method Flags)

This is due to
type Kevent_t C.struct_kevent_freebsd11

not having a counterpart, as it it should be
type Kevent_t C.struct_freebsd11_kevent

As there is only
/usr/include/sys/event.h:struct freebsd11_kevent

Anyhow, I am be completely wrong.

However, after changing to type Kevent_t C.struct_freebsd11_kevent, the build process stops after "Building Go toolchain2 using go_bootstrap and Go toolchain1“.


Thanks in advance,

Martin

[Martin Stiemerling]

Sorry, again, but my best guess is now that the reference is still FreeBSD 12 and I am just to far ahead with anything beyond FreeBSD 12…as we are at 14.1 and counting.

This is kind of mentioned in this issue: https://github.com/golang/go/issues/54355

„FreeBSD 12.4 reaches its EOL date on 31st December. Hopefully, switching the reference branch to stable/13 in the New Year will also pick up the constants for netlink which was merged into stable/13 and is present in FreeBSD 13.2 and later.“

[Ian Lance Taylor]

On Fri, Sep 6, 2024 at 7:42 AM Martin Stiemerling
<mstiem...@gmail.com> wrote:
>
> I just wanted to get back to this and probably ask somebody out of the go core team, if you guys have recently used the mkall.sh script under src/syscall for freebsd/amd64?

No, we haven't. The syscall package is all-but-frozen. Only
unavoidable changes are made to it. Those changes are generally made
manually.

Most programs should be using the golang.org/x/sys packages instead.
Those regeneration scripts are in better shape.

Ian