If Go is using libc instead of syscalls on macOS now, why is it not shown via otool -L?

[ivan.m...@gmail.com]

Hello,

I read the following from one of the contributors:

> On Solaris (and Windows), and more recently in macOS as well we link with libc (or equivalent).

> Go used to do raw system calls on macOS, and binaries were occasionally broken by kernel updates. Now Go uses libc on macOS.

I just checked syscall/zsyscall_darwin_amd64.go, and indeed in Go 1.10 we had

        Syscall(SYS_WRITE, uintptr(fd), uintptr(_p0), uintptr(len(p)))

now we have

        syscall(funcPC(libc_write_trampoline), uintptr(fd), uintptr(_p0), uintptr(len(p)))

But when I run otool -L on the binary to see if /usr/lib/libSystem.B.dylib is linked, I get nothing.

Is it because Go uses its own linker that's not supported by macOS tools?

P.S. Are there plans to do the same transition for Linux for consistency and stability?

Thanks.

-Ivan

[howar...@gmail.com]

I believe that by default, Go programs are statically linked.

From the man page for otool:

-L     Display the names and version numbers of  the  shared  libraries
              that  the  object file uses, as well as the shared library ID if
              the file is a shared library.

This is listing what is dynamically linked, not statically linked, so it does not show up.

[andrey mirtchovski]

macOS doesn't support static linking user binaries. in fact I do see
libSystem linked for each go binary on my Mojave system, including the
simplest non-outputting hello world:

$ cat > t.go
package main; func main(){}
$ go build t.go && otool -L t
t: /usr/lib/libSystem.B.dylib (compatibility version 0.0.0, current
version 0.0.0)
$

[Ian Lance Taylor]

On Fri, May 3, 2019 at 11:38 AM <ivan.m...@gmail.com> wrote:
>
> I read the following from one of the contributors:
>
> > On Solaris (and Windows), and more recently in macOS as well we link with libc (or equivalent).
> > Go used to do raw system calls on macOS, and binaries were occasionally broken by kernel updates. Now Go uses libc on macOS.
>
> I just checked syscall/zsyscall_darwin_amd64.go, and indeed in Go 1.10 we had
>
> Syscall(SYS_WRITE, uintptr(fd), uintptr(_p0), uintptr(len(p)))
>
> now we have
>
> syscall(funcPC(libc_write_trampoline), uintptr(fd), uintptr(_p0), uintptr(len(p)))
>
> But when I run otool -L on the binary to see if /usr/lib/libSystem.B.dylib is linked, I get nothing.
>
> Is it because Go uses its own linker that's not supported by macOS tools?

Seems like it should work. Are you sure you are using Go 1.12?

> P.S. Are there plans to do the same transition for Linux for consistency and stability?

No. The Linux kernel, fortunately, supports static linking and
provides a stable syscall API. There is no reason to make this change
on GNU/Linux.

Ian

[ivan.m...@gmail.com]

Indeed, it works with Go 1.12. I have 3 different Go versions, and my aliasing was messed up. Sorry.

> No.  The Linux kernel, fortunately, supports static linking and provides a stable syscall API.  There is no reason to make this change on GNU/Linux. 

Good to know, thanks. So Linux, BSD, Plan9 are now the only platforms using raw syscalls. 

Last question: did performance drop after the switch? There must be benchmarks.

[Ian Lance Taylor]

Yes, performance dropped, but not by a lot. I forget how much and I
couldn't quickly find the information. khr@ might know.

We more or less had to do it despite the performance loss. See
https://golang.org/issue/17490.

Ian

[ivan.m...@gmail.com]

I know that calling C functions via cgo is relatively expensive because of the goroutines, so there must be some optimizations/tricks to make all these libc calls fast.

[Ian Lance Taylor]

On Fri, May 3, 2019 at 3:36 PM <ivan.m...@gmail.com> wrote:
>
> I know that calling C functions via cgo is relatively expensive because of the goroutines, so there must be some optimizations/tricks to make all these libc calls fast.

Yes, this only uses part of the cgo call path, not all of it. In
particular there's no need to notify the scheduler about these calls
(beyond what was already done before this change), just as we don't
need to notify the scheduler about all system calls.

Ian

> On Saturday, May 4, 2019 at 12:21:27 AM UTC+2, ivan.m...@gmail.com wrote:
>>
>> Indeed, it works with Go 1.12. I have 3 different Go versions, and my aliasing was messed up. Sorry.
>>
>> > No. The Linux kernel, fortunately, supports static linking and provides a stable syscall API. There is no reason to make this change on GNU/Linux.
>>
>> Good to know, thanks. So Linux, BSD, Plan9 are now the only platforms using raw syscalls.
>>
>> Last question: did performance drop after the switch? There must be benchmarks.
>

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[keith....@gmail.com]

I don't have hard numbers on slowdowns. It will depend a lot on the binary in question.

At the time I checked the compiler and linker and it did not slow those down (to the accuracy of our benchmarks). The linker particularly is fairly read syscall intensive.

We've seen benchmarks that have significant slowdowns though. There was a factor of 4 slowdown on 1.12 for a (IMO a not very realistic) http benchmark. This wasn't conclusively pinned on the libSystem changes, but it seems likely.

> To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

[Ivan Medoedov]

Thanks very much. This is all very interesting.

You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/uX8eUeyuuAY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.