golang poll/epoll/select

[Vasiliy Tolstov]

Hello, again! If i need to poll some fd to changes, what function i need to use ?

--

Vasiliy Tolstov,

Clodo.ru

e-mail: v.to...@selfip.ru

jabber: va...@selfip.ru

[André Moraes]

> Hello, again! If i need to poll some fd to changes, what function i need to
> use ?

In most cases you should not need to poll explicity.
Write a goroutine that blocks while reading the contents of your fd
(file, network, etc...), and then send the data read across a channel.

Under the hood Go will take care of the poll for you.

But if you really need to poll explicitly, I can't help you since I
never had to do that.

--
André Moraes
http://andredevchannel.blogspot.com/

[Vasiliy Tolstov]

2011/9/22 André Moraes <and...@gmail.com>

In most cases you should not need to poll explicity.
Write a goroutine that blocks while reading the contents of your fd
(file, network, etc...), and then send the data read across a channel.

Under the hood Go will take care of the poll for you.

But if you really need to poll explicitly, I can't help you since I
never had to do that.

Hm... Can somebody from golang team tell me, what is the best way to provide this (example written in C):

xs_watch() // create a hook, that watch to specific path to cheange

epoll_wait() //wait to event on fd that returned by watch

xs_read_watch() //read watched data that appeared

xs_unwatch() //delete watch and stops recive notify

As i see, i need to create goroutine in xs_watch(worker), that runs worker function with read, then some data appeared, goroutine read and do some stuff, xs_unwatch destroy groutine and stops watching... Right? 

[André Moraes]

import os

func readFile(f *os.File, _data chan string, _close chan int) {
// read the file contents with f.Read
// process it
// send in the channel.
_data <- "Sample string. You should send your data here"
// when you are done
// send something on the close channel
_close <- 0
}

func main() {

f, err := os.Open("/my/file/path")
if err != nil {
panic(err) // error while opening the file.
}

dataCh := make(chan string) // unbuffered channel.
closeCh := make(chan int) // unbuffered channel.

go readFile(f, dataCh, closeCh) // dont block while reading the file.

for {
select { // this select the statement that has data on the channel.
case s:=<-dataCh
print(s) // print some data read from the file
case <-closeCh // EOF or other error
break
}
}

}

2011/9/22 Vasiliy Tolstov <v.to...@selfip.ru>:

--
André Moraes
http://andredevchannel.blogspot.com/

[msou...@gmail.com]

Unfortunately I don't think this works if you want to do something like poll stdin, skipping EOFs in a non-busy-waiting pattern.

A simple poll() in C works fine for this, but I can't figure out how do this this in Go because it does not provide a poll.

Maybe I'm missing something.

Mike

[Ian Lance Taylor]

On Fri, Jun 24, 2016 at 2:44 PM, <msou...@gmail.com> wrote:
> Unfortunately I don't think this works if you want to do something like poll
> stdin, skipping EOFs in a non-busy-waiting pattern.
>
> A simple poll() in C works fine for this, but I can't figure out how do this
> this in Go because it does not provide a poll.

In Go you normally simply start a goroutine that reads from Stdin and
sends the data over a channel.

Goroutines are cheap.

Ian

[graha...@gmail.com]

If you have a specific case that isn't covered by blocking in a go-routine, you can always use the syscall's directly, with a very similar API to what you would do in C.

For example here's the epoll ones in stdlib: https://golang.org/pkg/syscall/#EpollCreate

New syscall wrappers are in the x/sys repo: https://godoc.org/golang.org/x/sys/unix

[Michael Soulier]

On Friday, June 24, 2016 at 6:59:28 PM UTC-4, Ian Lance Taylor wrote:

In Go you normally simply start a goroutine that reads from Stdin and
sends the data over a channel.

Goroutines are cheap.

Sure, but when you read and get an EOF you return immediately, so the goroutine would be busy waiting when there's nothing to read, would it not?

Mike 

[Michael Soulier]

I thought I got select working but now it's returning immediately even without any input to stdin.

    // loop forever - we expect to be killed with a SIGTERM or SIGINT

    for {

        logger.Debug("going into select on stdin")

        var r_fdset syscall.FdSet

        for i := 0; i < 16; i++ {

            r_fdset.Bits[i] = 0

        }

        r_fdset.Bits[0] = 1

        selerr := syscall.Select(1, &r_fdset, nil, nil, nil)

        if selerr != nil {

            logger.Warning(selerr)

        }

So I'm doing something wrong. 

[Michael Soulier]

On Friday, June 24, 2016 at 7:10:55 PM UTC-4, graha...@gmail.com wrote:

If you have a specific case that isn't covered by blocking in a go-routine, you can always use the syscall's directly, with a very similar API to what you would do in C.

For example here's the epoll ones in stdlib: https://golang.org/pkg/syscall/#EpollCreate

epoll isn't helpful on Darwin I'm afraid. Select works but I only just figured out how to manually do an FD_SET by accessing the byte array directly.

I'm amazed that Select was implemented, but not the other supporting functions. That's kind of lame, isn't it?

Mike 

[Jesse McNelis]

On 25 Jun 2016 11:08 p.m., "Michael Soulier" <msou...@gmail.com> wrote:
>
> Sure, but when you read and get an EOF you return immediately, so the goroutine would be busy waiting when there's nothing to read, would it not?
>

You'll only get an EOF if the file descriptor has been closed, if it's closed then you're not going to be able to read anything more anyway.

What are you trying to do?

[Michael Soulier]

On Saturday, June 25, 2016 at 11:31:57 AM UTC-4, Jessta wrote:

You'll only get an EOF if the file descriptor has been closed, if it's closed then you're not going to be able to read anything more anyway.

What are you trying to do?

I'm trying to write a replacement for svlogd from the runit suite for my own purposes, and as an exercise to learn Go. To that end, the logger must come up and wait on stdin for a service to begin writing to it. Initially there will not be one. Looking at the svlogd source, the author uses poll on stdin to block until a writer was available.

I have tried to do this with select but once the writer comes up, select is returning immediately even if the service is not writing anything.

func select_stdin() {

    var r_fdset syscall.FdSet

    for i := 0; i < 16; i++ {

        r_fdset.Bits[i] = 0

    }

    r_fdset.Bits[0] = 1

    _, selerr := syscall.Select(1, &r_fdset, nil, nil, nil)

    if selerr != nil {

        logger.Warning(selerr)

    }

}

I'm curious as to what I'm doing wrong with select here, and if it's possible to do this with a goroutine like you describe.

Mike

[Janne Snabb]

On 2016-06-25 22:59, Michael Soulier wrote:
> I'm curious as to what I'm doing wrong with select here, and if it's
> possible to do this with a goroutine like you describe.

You should not be using select in the first place. You are making things
complicated for no reason whatsoever. (If I understand your intention
correctly.)

You should just read from os.Stdin. It will block until there is
something to read (unless you did something special to set it in
non-blocking mode). Check for EOF to know when to quit reading. Use a
goroutine if you need to do something else while the read is blocked. If
you do not need to do something else, you do not need a goroutine. Just
a plain and simple read loop.

Janne Snabb
sn...@epipe.com

[Michael Soulier]

On Saturday, June 25, 2016 at 4:19:34 PM UTC-4, Janne Snabb wrote:

You should not be using select in the first place. You are making things
complicated for no reason whatsoever. (If I understand your intention
correctly.)

You should just read from os.Stdin. It will block until there is
something to read (unless you did something special to set it in
non-blocking mode). Check for EOF to know when to quit reading. Use a
goroutine if you need to do something else while the read is blocked. If
you do not need to do something else, you do not need a goroutine. Just
a plain and simple read loop.

Unfortunately not. runsv starts the logger and connects the service's stdout to the logger's stdin. It opens this pipe even if the service isn't up yet, so when you read from stdin, it immediately returns with an EOF, and does not block. That's why svlogd uses poll on stdin.

If I were simply accepting input from a piped shell command, you'd be right. I wish I were.

Mike 

[Dave Cheney]

That doesn't sound right. Reading from a pipe should only get EOF when the other side of the pipe is closed.

[Michael Soulier]

On Saturday, June 25, 2016 at 8:53:53 PM UTC-4, Michael Soulier wrote:

Unfortunately not. runsv starts the logger and connects the service's stdout to the logger's stdin. It opens this pipe even if the service isn't up yet, so when you read from stdin, it immediately returns with an EOF, and does not block. That's why svlogd uses poll on stdin.

If I were simply accepting input from a piped shell command, you'd be right. I wish I were.

Hmm. Maybe I misunderstand how runsv connects the two. A simple shell test seems to behave more as expected. I'll need to dig.

Mike 

[Michael Soulier]

On Saturday, June 25, 2016 at 9:03:59 PM UTC-4, Michael Soulier wrote:

Hmm. Maybe I misunderstand how runsv connects the two. A simple shell test seems to behave more as expected. I'll need to dig.

I think I found it. On read I'm getting "resource temporarily unavailable", so I suspect runsv has set stdin to be non-blocking.

select() is the only way I know of to work with a non-blocking file descriptor. Now if I can only get select to work.

Mike 

[Ian Lance Taylor]

The golang.org/x/sys/unix package support Poll on GNU/Linux.

Ian

[Michael P. Soulier]

On 2016-06-27 11:26 AM, Ian Lance Taylor wrote:

> The golang.org/x/sys/unix package support Poll on GNU/Linux.

For now I'm using select so it works on darwin too. Seems to work so far.

Thanks,
Mike

[Konstantin Khomoutov]

On Mon, 27 Jun 2016 06:26:59 -0700 (PDT)
Michael Soulier <msou...@gmail.com> wrote:

> > Hmm. Maybe I misunderstand how runsv connects the two. A simple
> > shell test seems to behave more as expected. I'll need to dig.
> >
> I think I found it. On read I'm getting "resource temporarily
> unavailable", so I suspect runsv has set stdin to be non-blocking.
>
> select() is the only way I know of to work with a non-blocking file
> descriptor. Now if I can only get select to work.

A pipe has two ends. Is it possible for you to just make os.Stdin
blocking again before attempting to read from it?

Something like

import (
"os"
"syscall"
)
...
err = syscall.SetNonblock(os.Stdin.Fd(), false)