Proposal: runtime.RunWithMainThreadOnly()

[Richard Gooch]

  Hi, all. There are a number of system calls which change the state of the calling thread. Examples include setpriority(2) and setns(2). In the C world, it's easy to make such changes prior to starting any threads, and thus all threads inherit the state of the initial (main) thread. In this way, one can change the state of the entire process, which is usually what is desired.

In Go, this is not possible, since by the time the main() function is called, the runtime has created multiple OS threads. Thus, these calls change the state of the OS thread that happened to be running the main goroutine at the time. At any time the main goroutine can hop between OS threads, with the result that the process state appears to switch back and forth randomly.

My current work-around for this is the double-exec trick: I call runtime.LockOSThread(), make my state-changing system calls and then re-exec with syscall.Exec(), passing an extra command-line flag to disable the state-change+exec. This is a bit horrible. I get away with it because my application only needs to make these changes at startup. If I needed to effect a process-wide state change later on - say based on a decision that happens sometime later - I'd basically be stuck, because I can't afford to throw away all the internal state that's been built up.

I propose a new function: runtime.RunWithMainThreadOnly(). This would allow you to run code with only the main (initial) OS thread: all the other OS threads would be destroyed. Once this code section completes, the runtime would be able to create OS threads again, which would now inherit any state changes.

This function could either be called with a function to call, and when that function returns, things go back to normal. Alternatively, one would pass a bool to enable/disable the main-thread-only mode. Personally, I prefer the former interface, as it seems cleaner and less prone to people forgetting to switch back to normal mode. Also, for simple state change code, one could use a closure, which would be nice.

Thoughts?

Regards,

Richard....

[minux]

On Tue, Jan 26, 2016 at 1:31 AM, Richard Gooch <rg+go...@safe-mbox.com> wrote:

I propose a new function: runtime.RunWithMainThreadOnly(). This would allow you to run code with only the main (initial) OS thread: all the other OS threads would be destroyed. Once this code section completes, the runtime would be able to create OS threads again, which would now inherit any state changes.

Fixing the underlying problem is not that hard, and I'd want

a solution that will make syscall.Setuid work rather than

introducing some new runtime API to workaround the problem.

[Dave Cheney]

I'm not in favour of this, Go programmers are isolated from threads by
design. I understand that there are unfortunate realities of
interaction with operating systems, open gl libraries, etc, that make
a mockery of this ideal, but they are unfortunate exceptions, not
justifications for an invasive change like this.

There are also other aspects to consider.

- This feature feels very linux specific, how do these concepts map
onto other platforms that Go supports like windows or iOS?
- Goroutines are hosted on threads not just for user process, but also
runtime processes like garbage collection, finalisation, the
background scavenger, timers, network poller, and so on. Leaking a
thread with modified properties into this mix would be a disaster, so
something akin to LockOSThread would still be needed to make sure that
thread was fenced off.

You've gone into some detail about how this would be implemented, but
I haven't seen justification for why you need it. If this feature was
implemented, how would you use it ?

Thanks

Dave

> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Richard Gooch]

If the relevant functions in the syscall package had "all process" counterparts, then my needs would be satisfied. Examples:
- Set*id()
- Unshare()

Either something like SetpriorityFullProcess() and UnshareFullProcess(), or a syscall/process package which has the same function signatures but applies the change to the whole process.

Note that - under the covers - the code will need to destroy all but the main/initial OS thread while performing the change. For Unshare() there is no other way to do this: one cannot simply call unshare(2) for all OS threads: each OS thread would end up in a different namespace.

Regards,

Richard....

[Richard Gooch]

On Monday, 25 January 2016 22:39:15 UTC-8, Dave Cheney wrote:

I'm not in favour of this, Go programmers are isolated from threads by
design. I understand that there are unfortunate realities of
interaction with operating systems, open gl libraries, etc, that make
a mockery of this ideal, but they are unfortunate exceptions, not
justifications for an invasive change like this.

There are also other aspects to consider.

- This feature feels very linux specific, how do these concepts map
onto other platforms that Go supports like windows or iOS?
- Goroutines are hosted on threads not just for user process, but also
runtime processes like garbage collection, finalisation, the
background scavenger, timers, network poller, and so on. Leaking a
thread with modified properties into this mix would be a disaster, so
something akin to LockOSThread would still be needed to make sure that
thread was fenced off.

My proposal doesn't "leak" a thread with modified properties. It allows one to change the entire process state. That's clearly a good thing. Right now I can only achieve that with the double-exec trick.

In fact, the current state of Go is that calling a number of the functions in the syscall package drops/leaks an OS thread with different properties than the rest. So, right now, the disaster you speak of is the behaviour that Go provides. I am proposing a way to fix this.

As for this being Linux-specific: I'm not sure what the semantics of some of these system calls is in a threaded environment with other OSes. However, if there were "full process" variants of these system calls (which under the hood destroyed all but the main/initial OS thread while making the changes), that would serve my needs as well.

You've gone into some detail about how this would be implemented, but
I haven't seen justification for why you need it. If this feature was
implemented, how would you use it ?

I gave a couple of examples. The Setpriority() case should be self-explanatory. I want to change the priority for the entire process.

The other example (I mentioned setns(2) but more correctly it's syscall.Unshare() that I'm using). I have code that needs to operate in a separate namespace than the rest of the system. One use-case is creating a separate mount namespace and then adding mounts within that namespace. I don't want to tamper with the global mount namespace and I want to ensure that the mounts are unmounted and the namespace cleaned up if the process dies/exits.
 

[Russ Cox]

On Tue, Jan 26, 2016 at 1:31 AM, Richard Gooch <rg+go...@safe-mbox.com> wrote:

  Hi, all. There are a number of system calls which change the state of the calling thread. Examples include setpriority(2) and setns(2). In the C world, it's easy to make such changes prior to starting any threads, and thus all threads inherit the state of the initial (main) thread. In this way, one can change the state of the entire process, which is usually what is desired.

In Go, this is not possible, since by the time the main() function is called, the runtime has created multiple OS threads. Thus, these calls change the state of the OS thread that happened to be running the main goroutine at the time. At any time the main goroutine can hop between OS threads, with the result that the process state appears to switch back and forth randomly.

My current work-around for this is the double-exec trick: I call runtime.LockOSThread(), make my state-changing system calls and then re-exec with syscall.Exec(), passing an extra command-line flag to disable the state-change+exec. This is a bit horrible. I get away with it because my application only needs to make these changes at startup. If I needed to effect a process-wide state change later on - say based on a decision that happens sometime later - I'd basically be stuck, because I can't afford to throw away all the internal state that's been built up.

I propose a new function: runtime.RunWithMainThreadOnly(). This would allow you to run code with only the main (initial) OS thread: all the other OS threads would be destroyed. Once this code section completes, the runtime would be able to create OS threads again, which would now inherit any state changes.

What if there is a thread blocked in a system call that might not ever finish? Does RunWithMainThreadOnly just hang forever waiting?

Russ

[Richard Gooch]

There are two workable approaches I can see for threads blocked in system calls. One is to wait forever. The other is to mark that thread for deletion upon exit from the system call. I'm inclined towards the latter.

Simply ignoring blocked threads is not an option since if/when it returns from the system call, it will be different from the rest of the threads. Trying to make the thread like all the others after returning from the system call would be complex and fragile, as one would need to maintain a sequenced list of operations to perform.

Regards,

Richard....

[Ian Lance Taylor]

If we can make this work, I don't see a clear need to add a new
runtime function. We ought to be able to make the syscall functions
do the right thing using entirely internal mechanisms. Am I missing
something that makes the explicit function necessary?

I hadn't thought of this approach for https://golang.org/issue/1435;
it would be nice if we could make it work.

Ian

[Richard Gooch]

Provided that all the syscall functions that mutate only the thread state have full-process variants, then I think there is no great need to expose the runWithMainThreadOnly() function. It still might be useful for people who want to call a system call which does not yet have an entry function in the syscall package, as it would allow them to make progress without waiting for some future Go release.
 

I hadn't thought of this approach for https://golang.org/issue/1435;
it would be nice if we could make it work.

My. What a long, painful thread. 5 years without a fix. :cry:

Regards,

Richard....

[Richard Gooch]

Converted to an issue so this doesn't get dropped: https://github.com/golang/go/issues/14163