Exposing the runtime poller to work with generic file descriptors

[elli...@gmail.com]

While I was doing research on how to implement timeouts on a user library for an AF_CAN Linux socket, I came across several threads and issues where people have been asking about exposing the runtime poller for user-level libraries.

The first request for this feature that I found was way back in 2011 from Albert Strasheim:
https://groups.google.com/forum/#!msg/golang-dev/4NndYA4QoGY/0wkmgI9KcU8J

It was then brought up in issues in 2013 by Dmitry Vyukov and lexprfuncall:
https://github.com/golang/go/issues/6817
https://github.com/golang/go/issues/6222

There is another nice discussion in 2014, from a request from Quoc-Viet Nguyen:
http://grokbase.com/t/gg/golang-nuts/14at84gaby/go-nuts-fd-polling-with-syscall-select

In this discussion, Nick Patavalis shows a user-level library that implements a poller outside of the runtime poller and discusses the shortcomings of the current "idiomatic" way of doing a blocking read in a separate go routine with a timeout select that will close the file descriptor.

There are also several newer issues and discussions regarding this as well:
https://github.com/golang/go/issues/10565
https://github.com/golang/go/issues/15021
https://groups.google.com/forum/#!msg/golang-nuts/Wdpj20r360A/2gvP9q6hCwAJ

The point is, it seems that many people have independently come up with use cases where it would make a lot of sense to have a user facing API for the runtime poller, not only for efficiency but also for just having robust and correct implementations.

Is this something that could be pushed for Go 1.9? Should issue 15021 be used for driving this forward?

I have a concern that the original issue proposed there is only about non-standard sockets (which is indeed my current use-case), but generic file descriptor polling can be used for all sorts of other IPC mechanisms and it seems a shame to artificially limit the solution.

It seems like several people have tried to drive this issue before in the past 5 years but without much luck. What would be the best course of action for resolving these issues for Go 1.9?

[Brad Fitzpatrick]

You could propose a minimal but tolerable API that would only pollute, say, the syscall package (ignoring that it's frozen), the runtime package, or a sub-package of the runtime package (but not the net package), and show that it addresses a handful of outstanding feature requests or bug reports.

I don't believe there's been such a proposal yet.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[elli...@gmail.com]

The original request was for "os/poll", I think that would be best, or "runtime/poll".  Agreed that "net" makes no sense, because it actually has nothing inherently to do with networking. I would imagine an API something like this.

type Descriptor struct {
    // contains filtered or unexported fields
}

const (
    Read = 0
    Write = 1
)

func Open(fd int) (*Descriptor, error)
func (d *Descriptor) Close() error
// Design Question: Should Open/Close be used, or Register/DeRegister (the file descriptor would already be allocated, and would not actually be closed with Close), however Open/Close seem to be the standard way of doing it.

func (d *Descriptor) Read(p []byte) (n int, err error)
func (d *Descriptor) Write(b []byte) (n int, err error)
func (d *Descriptor) SetDeadline(t time.Time) error
func (d *Descriptor) SetReadDeadline(t time.Time) error
func (d *Descriptor) SetWriteDeadline(t time.Time) error

// QUESTION: Should LocalAddr() and RemoteAddr() be added for net.Conn compatibility?

func (d *Descriptor) Wait(mode int) error
// mode => poll.Read, poll.Write, or poll.Read | poll.Write

func (d *Descriptor) Cancel() error
// concurrently cancel a blocking Read, Write, or Wait call.

From my list, this directly addresses the first 4 links, as they are all more or less are directly asking for this package.

Issues 10565, 15021, would be addressed indirectly, creating a net.Conn or net.PacketConn by wrapping a Descriptor would be trivial.

Similarily, this would solve my problem with creating a net.Conn from the SocketCAN type that was to x/sys/unix in issue 16188.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.

[Brad Fitzpatrick]

The best place to discuss this is probably on a specific bug.

You might want to write a proposal doc, per the proposal process.

(Side note: Go uses uintptr as the type for fds, not int, for compatibility with Windows)

To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.

[elli...@gmail.com]

Alright, I will write up a proposal.

I was confused about what should be used as the file descriptor type, I saw that os.File.Fd() returns a uintptr, but all functions in the Linux syscall and x/sys/unix package use int, I didn't realize the discrepancy with the windows implementation. Since one of reasons for using the runtime poller is cross-platform support I will switch it to a uintptr.

[elli...@gmail.com]

Added a first version of the proposal:

https://github.com/golang/go/issues/18507