Prevent RTS from creating an OS thread

[Martynas Pumputis]

Hi list,

Recently I've bumped into a problem when using the Linux network
namespaces in Go.

In my program I have the following function:

func do() {
runtime.LockOSThread()
defer runtime.UnlockOSThread()
netns.Set(containerNs)
defer netns.Set(hostNs)
doSomeStuff()
}

where netns.Set [1] changes the network namespace (does the setns(2)
syscall via syscall.Syscall). The problem with this function is that
during the execution of doSomeStuff() or defer netns.Set(hostNs) the
Go runtime might create a new OS thread (e.g. due to blocking) which
will reside in the same namespace as parent (CLONE_NEWNET is not set
when doing clone(2)). The newly created thread might be used for
scheduling other go-routines which will end up running in the "wrong"
namespace. As a consequence, the go-routines might not be able to
access some netdevs available within the host network namespace or
previously created sockets.

One workaround to this problem is to shell out a process (e.g. via
exec.Command) which would set the namespace and do doSomeStuff().
However, this approach is sort of ugly, because a) we create
unnecessary process; b) doSomeStuff() cannot be implemented in Go
without special hacks [2].

Considering a vast adoption of Go within containers software, it's a
bit odd that the runtime does not provide any means to address the
problem. Maybe I'm missing something?

[1]: https://github.com/vishvananda/netns
[2]: https://github.com/opencontainers/runc/tree/master/libcontainer/nsenter

Best,
Martynas

[Ian Lance Taylor]

You aren't missing anything. Doing this correctly requires runtime
support, and that support does not exist. It's not even obvious how
to write it, at least not to me.

Ian

[Martynas Pumputis]

I see :-/

One very fragile workaround I've had in mind is to replace any
occurrence of syscall.Syscall by syscall.Rawsyscall within call graph
of doSomeStuff(). However, the Go scheduler is not truly cooperative,
so unexpected preemption can happen during the execution (e.g. due to
a stack growth) which might result in creation of a new OS thread (M).

[Alex Bligh]

> On 30 Jun 2016, at 14:10, Martynas Pumputis <mart...@gmail.com> wrote:
>
> I see :-/
>
> One very fragile workaround I've had in mind is to replace any
> occurrence of syscall.Syscall by syscall.Rawsyscall within call graph
> of doSomeStuff(). However, the Go scheduler is not truly cooperative,
> so unexpected preemption can happen during the execution (e.g. due to
> a stack growth) which might result in creation of a new OS thread (M).

I've done this in C by creating a new thread and using the method
you describe. Would an alternative approach be to fork() in go
(look at github.com/sevlyar/go-daemon for instance), having previously
set up some IPC, and have your forked process (also in go, and in
the same source code) perform the things that need to be done in
the separate namespace? If you are always using the same alternate
namespace, you then only have IPC overhead, and if you are using
different namespaces each time, it costs a fork() but not an exec().

Alex

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

--
Alex Bligh

[Matt Harden]

Forking is not safe in Go either.

[Tamás Gulácsi]

What about os/exec with a C shim to setup what needs to be done and start the Go program, passing on given FDs for IPC?

[Alex Bligh]

On 2 Jul 2016, at 05:23, Matt Harden <matt....@gmail.com> wrote:

> Forking is not safe in Go either.

Why? Let's assume one knows what one is doing and doesn't try to use channels etc.

--
Alex Bligh

[Ian Lance Taylor]

In general you can't use fork in a multi-threaded program, because a different thread might be holding, say, the malloc lock during the fork.  The result would be that the child process can not allocate any memory.  The only time fork is safe is when it is immediately followed by exec.

Ian