Limiting File Descriptors on Forked Linux cmd.exec

[Jeff Stein]

I'm struggling to understand if I'm able to do something.

In my very odd use case we are writing a websever that handles connections via a forked process.

I have a listener process that listens for TCP connections.

So each net.Conn that comes in we pull off its file descriptor:

fd, err := conn.(*net.TCPConn).File()

duplicate that file descriptor and then fork off a process passing in that file descriptor.

In my forked handler I'll reconstruct the HTTP connection and "do stuff".

The concern I'm having is that it appears when I fork a process I inherit all of the parent file descriptors so if I have say 5 incoming connections and then I fork my child process technically could write to a different connection.

I've played around with the various options:

cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: false,}

and using cmd.ExtraFiles

No matter what I do I seem unable to limit the sub process to ONLY using the specific File Descriptor I want it to have access to.

I believe this is doable in C - but I'm not sure if I can do this in GoLang as-is without mods .

Thanks!!!

[Ian Lance Taylor]

What you are describing shouldn't happen. A child process should only
get the file descriptors explicitly passed via the os/exec.Cmd fields
Stdin, Stdout, Stderr, and ExtraFiles. So tell us more: OS and
version of Go, and what is showing you that all file descriptors are
being passed down to the child.

Ian

[Robert Engels]

The could be calling fork() as in the system call - which copies all file descriptors but I didn’t think Go processes could fork.

Seems you would need to remap stdin and stdout in the fork to do anything useful.

This sounds very PHP - what goes around comes around.

> On Mar 1, 2024, at 8:01 PM, Ian Lance Taylor <ia...@golang.org> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUb27YBCyE52QisHLyB9XPPpEycMxt4FrFJogGsFMiemQ%40mail.gmail.com.

[Ian Lance Taylor]

On Fri, Mar 1, 2024 at 6:17 PM Robert Engels <ren...@ix.netcom.com> wrote:
>
> The could be calling fork() as in the system call - which copies all file descriptors but I didn’t think Go processes could fork.
>
> Seems you would need to remap stdin and stdout in the fork to do anything useful.
>
> This sounds very PHP - what goes around comes around.

Good point, I am assuming that the OP is using the os/exec package to
start up a new copy of the process.

A simple fork without an exec can't work in Go, or in any
multi-threaded program.

Ian

[Jeff Stein]

OP here -> I'm going to put together some test apps - toss them on GitHub and make sure I actually know what I'm talking about :) 

[Jeff Stein]

I think I may have discovered the issue.

I made a sys call to duplicate the file descriptor

dupFd, err := syscall.Dup(int(file.Fd()))
if err != nil {
log.Printf("Error duplicating file descriptor: %v", err)
return 0, ""
}

which likely reset the FD_CLOEXEC Flag:

(from Advanced Programming in the Unix Environment) - section 3.12:

The new file descriptor returned by dup is guaranteed to be the lowest-numbered available file descriptor. With dup2, we specify the value of the new descriptor with the fd2 argument. If fd2 is already open, it is first closed. If fd equals fd2, then dup2 returns fd2 without closing it. Otherwise, the FD_CLOEXEC file descriptor flag is cleared for fd2, so that fd2 is left open if the process calls exec.

Does this sound like what may have been happening?

[Jeff Stein]

It appears this was in fact the issue.

I added some code to print out the `FD_CLOEXEC` state. Files called via syscall.Dupe2 end up with a FD_CLOEXEC=0 whereas anything passed via ExtraFiles has a FDCLOEXEC=1.

Whoever said "This is impossible" thanks - you spurred me towards a wonderful discovery about what NOT to do :) 

[Jeff Stein]

And looking through the source code on os.exec_linux I think I've also validated this as well:

if fd[i] == i {
// dup2(i, i) won't clear close-on-exec flag on Linux,
// probably not elsewhere either.
_, _, err1 = RawSyscall(fcntl64Syscall, uintptr(fd[i]), F_SETFD, 0)
if err1 != 0 {
goto childerror
}
continue
}