Why golang doesn't have the `fork` syscall ?

[fmar...@gmail.com]

Hello,

I'm pretty new to go in general and I've doing some investigations around it. 

I was wondering what is the technical reason for not having direct support for doing a `Fork` sys call. I saw that there is an API called `ForkAndExec` which would the `fork` but immediately will replace the current process with the new one using `exec`. 

I could see that doing forks is not the go way of handling parallelism, maybe that is the reason for not having the api ?

Just curious! 

Thanks before hand!!

[Brian Hatfield]

Searching my go-nuts history, I found a thread that pointed here: https://stackoverflow.com/questions/28370646/how-do-i-fork-a-go-process/28371586#28371586

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/54baf176-2123-4166-a9a1-6f1d14c9ecb8%40googlegroups.com.

[Jan Mercl]

On Fri, Jul 26, 2019 at 8:15 PM <fmar...@gmail.com> wrote:

> I was wondering what is the technical reason for not having direct support for doing a `Fork` sys call.

Unix supports forking of only single threaded processes.

[Martín Fernández]

That would make total sense! Thanks for quick reply!

[Robert Engels]

-----Original Message-----
>From: Jan Mercl <0xj...@gmail.com>
>Sent: Jul 26, 2019 1:37 PM
>To: fmar...@gmail.com
>Cc: golang-dev <golan...@googlegroups.com>
>Subject: Re: [golang-dev] Why golang doesn't have the `fork` syscall ?
>
>On Fri, Jul 26, 2019 at 8:15 PM <fmar...@gmail.com> wrote:
>
>> I was wondering what is the technical reason for not having direct support for doing a `Fork` sys call.
>
>Unix supports forking of only single threaded processes.
>

>--
>You received this message because you are subscribed to the Google Groups "golang-dev" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.

>To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CAA40n-WS%2Bw43f-DvX0PXXrzmMiaTbSUKzXBO%3DGM2%2BwYi5RDjPQ%40mail.gmail.com.

That is not true. Fork is valid for multi-threaded processes. The process will be duplicated but on the calling thread of fork() will have its execution context continued in the forked process.

Additionally, some variants including Solaris (and POSIX) have a forkall() which duplicates all threads.

[robert engels]

*only

The Oracle docs are incorrect about POSIX, as their docs state:

The addition of the forkall() function to the standard was considered and rejected. The forkall() function lets all the threads in the parent be duplicated in the child. This essentially duplicates the state of the parent in the child. This allows threads in the child to continue processing and allows locks and the state to be preserved without explicit pthread_atfork() code. The calling process has to ensure that the threads processing state that is shared between the parent and child (that is, file descriptors or MAP_SHARED memory) behaves properly after forkall(). For example, if a thread is reading a file descriptor in the parent when forkall() is called, then two threads (one in the parent and one in the child) are reading the file descriptor after the forkall(). If this is not desired behavior, the parent process has to synchronize with such threads before calling forkall().

The Oracle docs talk about fork() in conjunction with multi-threaded processes in quite a bit of detail at https://docs.oracle.com/cd/E19120-01/open.solaris/816-5137/gen-1/index.html and the concerns with forkall()

and many OS’s implement it, including Solaris, UniwWare, FreeBSD.

[Martín Fernández]

So, if the kernel supports fork on multithreaded processes, why forking would still be a technical problem for the go runtime ?

Maybe the limitation of starting only the thread that called fork could be an issue ? The GC would probably be running another thread that will never get started ?

```

The child process is created with a single thread—the one that

          called fork().  The entire virtual address space of the parent is

          replicated in the child, including the states of mutexes,

          condition variables, and other pthreads objects; the use of

          pthread_atfork(3) may be helpful for dealing with problems that

          this can cause.
``` 

On Friday, July 26, 2019 at 8:28:54 PM UTC-3, robert engels wrote:

*only

The Oracle docs are incorrect about POSIX, as their docs state:

The addition of the forkall() function to the standard was considered and rejected. The forkall() function lets all the threads in the parent be duplicated in the child. This essentially duplicates the state of the parent in the child. This allows threads in the child to continue processing and allows locks and the state to be preserved without explicit pthread_atfork() code. The calling process has to ensure that the threads processing state that is shared between the parent and child (that is, file descriptors or MAP_SHARED memory) behaves properly after forkall(). For example, if a thread is reading a file descriptor in the parent when forkall() is called, then two threads (one in the parent and one in the child) are reading the file descriptor after the forkall(). If this is not desired behavior, the parent process has to synchronize with such threads before calling forkall().

The Oracle docs talk about fork() in conjunction with multi-threaded processes in quite a bit of detail at https://docs.oracle.com/cd/E19120-01/open.solaris/816-5137/gen-1/index.html and the concerns with forkall()

and many OS’s implement it, including Solaris, UniwWare, FreeBSD.

On Jul 26, 2019, at 2:17 PM, Robert Engels <ren...@ix.netcom.com> wrote:

-----Original Message-----

From: Jan Mercl <0xj...@gmail.com>
Sent: Jul 26, 2019 1:37 PM
To: fmar...@gmail.com
Cc: golang-dev <golan...@googlegroups.com>
Subject: Re: [golang-dev] Why golang doesn't have the `fork` syscall ?

On Fri, Jul 26, 2019 at 8:15 PM <fmar...@gmail.com> wrote:

I was wondering what is the technical reason for not having direct support for doing a `Fork` sys call.

Unix supports forking of only single threaded processes.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CAA40n-WS%2Bw43f-DvX0PXXrzmMiaTbSUKzXBO%3DGM2%2BwYi5RDjPQ%40mail.gmail.com.

[andrey mirtchovski]

> The Oracle docs are incorrect about POSIX, as their docs state

Can you do a "fork" without exec in Java? I've always wanted to find
out. I don't mean the Oracle version, I mean all of them.

[Martín Fernández]

Kind of related but today we talked about jruby (Ruby on the jvm) implementation of fork. Doesn't seem to be supported. 

Martín

[andrey mirtchovski]

Kind of related but today we talked about jruby (Ruby on the jvm) implementation of fork. Doesn't seem to be supported. 

fork() just by itself is unsafe on any unix-like system nowadays. for example, here's the CAVEATS section from fork(3) on a modern macOS system:

 CAVEATS

     There are limits to what you can do in the child process.  To be totally safe you should restrict your-
     self to only executing async-signal safe operations until such time as one of the exec functions is
     called.  All APIs, including global data symbols, in any framework or library should be assumed to be
     unsafe after a fork() unless explicitly documented to be safe or async-signal safe.  If you need to use
     these frameworks in the child process, you must exec.  In this situation it is reasonable to exec your-
     self.

[Florian Weimer]

* andrey mirtchovski:

>> The Oracle docs are incorrect about POSIX, as their docs state
>
> Can you do a "fork" without exec in Java? I've always wanted to find
> out. I don't mean the Oracle version, I mean all of them.

No, Java doesn't support fork in that way. In the Hotspot
implementation, there is no way to resurrect its internal threads. The
Z garbage collector also used shared alias mappings for the heap, which
would not be duplicated by a fork or forkall call, so parent and child
would share the Java heap, which would not work.

Thanks,
Florian