Prevent RTS from creating an OS thread

822 views
Skip to first unread message

Martynas Pumputis

unread,
Jun 28, 2016, 9:33:40 PM6/28/16
to golan...@googlegroups.com
Hi list,

Recently I've bumped into a problem when using the Linux network
namespaces in Go.

In my program I have the following function:

func do() {
runtime.LockOSThread()
defer runtime.UnlockOSThread()
netns.Set(containerNs)
defer netns.Set(hostNs)
doSomeStuff()
}

where netns.Set [1] changes the network namespace (does the setns(2)
syscall via syscall.Syscall). The problem with this function is that
during the execution of doSomeStuff() or defer netns.Set(hostNs) the
Go runtime might create a new OS thread (e.g. due to blocking) which
will reside in the same namespace as parent (CLONE_NEWNET is not set
when doing clone(2)). The newly created thread might be used for
scheduling other go-routines which will end up running in the "wrong"
namespace. As a consequence, the go-routines might not be able to
access some netdevs available within the host network namespace or
previously created sockets.

One workaround to this problem is to shell out a process (e.g. via
exec.Command) which would set the namespace and do doSomeStuff().
However, this approach is sort of ugly, because a) we create
unnecessary process; b) doSomeStuff() cannot be implemented in Go
without special hacks [2].

Considering a vast adoption of Go within containers software, it's a
bit odd that the runtime does not provide any means to address the
problem. Maybe I'm missing something?

[1]: https://github.com/vishvananda/netns
[2]: https://github.com/opencontainers/runc/tree/master/libcontainer/nsenter

Best,
Martynas

Ian Lance Taylor

unread,
Jun 29, 2016, 10:41:15 PM6/29/16
to Martynas Pumputis, golang-nuts
You aren't missing anything. Doing this correctly requires runtime
support, and that support does not exist. It's not even obvious how
to write it, at least not to me.

Ian

Martynas Pumputis

unread,
Jun 30, 2016, 9:10:41 AM6/30/16
to Ian Lance Taylor, golang-nuts
I see :-/

One very fragile workaround I've had in mind is to replace any
occurrence of syscall.Syscall by syscall.Rawsyscall within call graph
of doSomeStuff(). However, the Go scheduler is not truly cooperative,
so unexpected preemption can happen during the execution (e.g. due to
a stack growth) which might result in creation of a new OS thread (M).

Alex Bligh

unread,
Jul 1, 2016, 2:35:02 PM7/1/16
to Martynas Pumputis, Alex Bligh, Ian Lance Taylor, golang-nuts

> On 30 Jun 2016, at 14:10, Martynas Pumputis <mart...@gmail.com> wrote:
>
> I see :-/
>
> One very fragile workaround I've had in mind is to replace any
> occurrence of syscall.Syscall by syscall.Rawsyscall within call graph
> of doSomeStuff(). However, the Go scheduler is not truly cooperative,
> so unexpected preemption can happen during the execution (e.g. due to
> a stack growth) which might result in creation of a new OS thread (M).

I've done this in C by creating a new thread and using the method
you describe. Would an alternative approach be to fork() in go
(look at github.com/sevlyar/go-daemon for instance), having previously
set up some IPC, and have your forked process (also in go, and in
the same source code) perform the things that need to be done in
the separate namespace? If you are always using the same alternate
namespace, you then only have IPC overhead, and if you are using
different namespaces each time, it costs a fork() but not an exec().

Alex
> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

--
Alex Bligh




Matt Harden

unread,
Jul 2, 2016, 12:23:40 AM7/2/16
to Alex Bligh, Martynas Pumputis, Ian Lance Taylor, golang-nuts
Forking is not safe in Go either.

Tamás Gulácsi

unread,
Jul 2, 2016, 2:59:36 AM7/2/16
to golang-nuts
What about os/exec with a C shim to setup what needs to be done and start the Go program, passing on given FDs for IPC?

Alex Bligh

unread,
Jul 2, 2016, 5:33:10 AM7/2/16
to Matt Harden, Alex Bligh, Martynas Pumputis, Ian Lance Taylor, golang-nuts

On 2 Jul 2016, at 05:23, Matt Harden <matt....@gmail.com> wrote:

> Forking is not safe in Go either.

Why? Let's assume one knows what one is doing and doesn't try to use channels etc.

--
Alex Bligh




Ian Lance Taylor

unread,
Jul 2, 2016, 1:14:49 PM7/2/16
to Alex Bligh, golang-nuts, Matt Harden, Martynas Pumputis

In general you can't use fork in a multi-threaded program, because a different thread might be holding, say, the malloc lock during the fork.  The result would be that the child process can not allocate any memory.  The only time fork is safe is when it is immediately followed by exec.

Ian

Reply all
Reply to author
Forward
0 new messages