Poll and Select syscalls

[Manlio Perillo]

Regards.

For a project I'm working on I need a simple level-triggered poller, since I only need to poll a single file descriptor from a single goroutine.

I was planning to use Poll, however it is not available.  I'm using Select, however the FD_SET, FD_CLR and FD_ISSET functions are missing, and this means that I have to implement them for each supported platform.

I would like to avoid to use Epoll, since it is not designed to work with very few file descriptors.

What is the reason why Poll was not added? It it not even present in the new x/sys package.

Thanks   Manlio Perillo

[Guillaume J. Charmes]

You can take a look at http://github.com/creack/goselect. It is a work in progress, but it gives you the idea. It supports almost all platforms.

-- 

Guillaume J. Charmes about.me/guillaumecharmes

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ian Taylor]

The usual approach in Go is to use a simple goroutine that reads from
the file and writes to a channel. Then you can use the select
statement to check the channel when appropriate. It's unusual to want
to use the poll or select system calls since the language gives you
other ways to do the same thing.

Ian

[Manlio Perillo]

Il giorno martedì 18 novembre 2014 18:10:19 UTC+1, Ian Lance Taylor ha scritto:

On Tue, Nov 18, 2014 at 8:10 AM, Manlio Perillo
<manlio....@gmail.com> wrote:
>
> For a project I'm working on I need a simple level-triggered poller, since I
> only need to poll a single file descriptor from a single goroutine.

> [...] 

The usual approach in Go is to use a simple goroutine that reads from
the file and writes to a channel.  Then you can use the select
statement to check the channel when appropriate.  It's unusual to want
to use the poll or select system calls since the language gives you
other ways to do the same thing.

I'm tring use both poll and go select, and then send the data to a channel.

Without poll I have to handle the timeout with extra code, and I may have to use two goroutines (one for reading and the other for writing).

Regards  Manlio

[Manlio Perillo]

Il giorno martedì 18 novembre 2014 17:56:58 UTC+1, Guillaume J. Charmes ha scritto:

You can take a look at http://github.com/creack/goselect. It is a work in progress, but it gives you the idea. It supports almost all platforms.

Thanks.

I was using a timeout of type int, thus causing another portability issue when converting to the type of Timeval.Usec.

Using time.Duration solved the problem.

I have also redefined FD_SET and FD_ISSET to work on, at leasy, linux 386 and amd64.

I suspect that using poll is much more portable.

Regards  Manlio

[Manlio Perillo]

Il giorno martedì 18 novembre 2014 17:56:58 UTC+1, Guillaume J. Charmes ha scritto:

You can take a look at http://github.com/creack/goselect. It is a work in progress, but it gives you the idea. It supports almost all platforms.

Note that POSIX IMHO requires that NFDBITS is equal to CHAR_BIT * sizeof(long), and CHAR_BIT is always 8 on POSIX.

Unfortunately, in Go you can't take the size of a type, as a constant expression.

I suggest you to update fdset_32.go , replacing
NFDBITS = 4 * 8
with
NFDBITS = 8 * 4
and adding a comment

Another solution is to use cgo:

/*

int NFDBITS = 8 * (int) sizeof(long);

*/

import "C"

var (

    NFDBITS = C.NFDBITS

)

Regards  Manlio Perillo

[Manlio Perillo]

Il giorno giovedì 20 novembre 2014 12:42:47 UTC+1, Manlio Perillo ha scritto:

Il giorno martedì 18 novembre 2014 17:56:58 UTC+1, Guillaume J. Charmes ha scritto:

You can take a look at http://github.com/creack/goselect. It is a work in progress, but it gives you the idea. It supports almost all platforms.

Note that POSIX IMHO requires that NFDBITS is equal to CHAR_BIT * sizeof(long), and CHAR_BIT is always 8 on POSIX.

Unfortunately, in Go you can't take the size of a type, as a constant expression.

My bad.

Single Unix Specification Issue 7 has removed the requirement for the fd_set structure to have a member

    long fds_bits[]

> [...]

Regards   Manlio

[Guillaume J. Charmes]

You can take a look at fdset_doc.go, there is the original code for the different os/arch, they all do `#define __NFDBITS       (sizeof(__fd_mask) * 8)`.

-- 

Guillaume J. Charmes about.me/guillaumecharmes

--

[Manlio Perillo]

Il giorno giovedì 20 novembre 2014 14:54:25 UTC+1, Guillaume J. Charmes ha scritto:

You can take a look at fdset_doc.go, there is the original code for the different os/arch, they all do `#define __NFDBITS       (sizeof(__fd_mask) * 8)`.

-- 

I reimplemented the select wrapper:

http://play.golang.org/p/kl2OnlCfhQ

The code should be portable on all platforms supported by Go, but I have only checked on:

* Linux amd64 using native int (64bits) for the bit set

* Linux amd64 using int32 for the bit set

Can you check if it works on the other platforms?

Thanks   Manlio Perillo

[Guillaume J. Charmes]

Interesting implementation, but I'd rather stay away from unsafe. Would be interesting to benchmark though.

This will work *only* on linux. It will not compile on other platform because of the syscall.Select prototype. Only linux return two values. All others return only one.

You talk about solaris, however, there is no `syscall.Select` nor `syscall.SYS_SELECT` for Solaris in Go (even though there is a `syscall.FdSet`). If you find a way, I'd be interested.

You talk about the netbsd limitation to 256 fds, however, the code does not enforce it. It also does not uses an int32. I updated my version with the int32, will work on the 256 limit later.

I don't think it is possible to deal with this without build tags. Any suggestions?

Minor detail questions/remarks:

- Most function in the Go stdlin that deal with Fds take and return uintptr, why use int?

- Why declare a variable for the mask?

- I know that we are dealing with C consts, but Go does not like allcap/underscore ;)

-- 

Guillaume J. Charmes about.me/guillaumecharmes

--

[Matt Harden]

Please listen to Ian and use goroutines; don't use poll() or select() directly. Your code will be better for it.

[Manlio Perillo]

Il giorno giovedì 20 novembre 2014 21:15:09 UTC+1, Guillaume J. Charmes ha scritto:

Interesting implementation, but I'd rather stay away from unsafe. Would be interesting to benchmark though.

I expect the benchmark to show the same results.

 

This will work *only* on linux. It will not compile on other platform because of the syscall.Select prototype. Only linux return two values. All others return only one.

Thanks, I did not notice this.

Select sure is a bad guy!

I hope Poll is better.

 

You talk about solaris, however, there is no `syscall.Select` nor `syscall.SYS_SELECT` for Solaris in Go (even though there is a `syscall.FdSet`). If you find a way, I'd be interested.

Thanks again.

I only looked at the ztypes_xxx.go source, and assumed there was a Select syscall.

 

You talk about the netbsd limitation to 256 fds, however, the code does not enforce it.

The kernel should return an error if more than 256 fds are used.

It also does not uses an int32.

Since it is a bit set, it should be the same if you set single bits using an []int32 or []int64,

as long as you use the correct value for FD_BITS.

However I don't deal often with low level bits, so I'm not 100% sure it is correct.

I have verified it to work correctly on linux amd64.

I updated my version with the int32, will work on the 256 limit later.

I don't think it is possible to deal with this without build tags. Any suggestions?

I use the unsafe pointer cast in order to avoid build tags.

However now I think it is better, for my case, to just use epoll for linux or poll via libc.

I understimated the portability issues with syscall package.

 

Minor detail questions/remarks:

- Most function in the Go stdlin that deal with Fds take and return uintptr, why use int?

Because  the syscalls dealing for fds use int.  In the io package the file struct use an int to store the fd.

uintptr is used for the external interface, and in my case I adapted the code from my project, where the API is only used internally.

By the way: what is the reason why os package use uintptr instead of uint?

- Why declare a variable for the mask?

It is more easy to read, for me; and a good compiler should optimize it away.

 

- I know that we are dealing with C consts, but Go does not like allcap/underscore ;)

It was only a draft, and my brain is still not used to Go conventions for constants :)

> [...]

Regards   Manlio Perillo