Out-of-band callbacks with cgo
Gustavo Niemeyer
I'd like to wrap the Zookeeper [1] C library for Go, and I believe
this is touching on an open problem related to C callbacks.
The thread-safe and recommended way that Zookeeper can be wrapped
spawns a thread in the background to handle incoming events such as
watches on nodes. Ian was telling me that there's no way for that
thread to callback into Go code, though. The potential workaround I
can see is to busy loop in the Go side, and dequeue callbacks when
they are available, but that's of course sub-optimal.
Is there a better way to do this at the moment, and is the solution to
the underlying problem of being unable to callback from C in the
horizon?
Thanks,
[1] http://hadoop.apache.org/zookeeper/
--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter
Florian Weimer
> The thread-safe and recommended way that Zookeeper can be wrapped
> spawns a thread in the background to handle incoming events such as
> watches on nodes. Ian was telling me that there's no way for that
> thread to callback into Go code, though. The potential workaround I
> can see is to busy loop in the Go side, and dequeue callbacks when
> they are available, but that's of course sub-optimal.
You could implement a queue using pthreads and access that from Go
code. The dequeing would be synchronous, so the whole operation would
be synchronous from Go's point of view, even though the queue is
filled concurrently.
Gustavo Niemeyer
Yeah, that's precisely what I suggested above. It's not a good
solution on itself because it requires polling.
Florian Weimer
> Hi Florian,
>
>>> The thread-safe and recommended way that Zookeeper can be wrapped
>>> spawns a thread in the background to handle incoming events such as
>>> watches on nodes. Ian was telling me that there's no way for that
>>> thread to callback into Go code, though. The potential workaround I
>>> can see is to busy loop in the Go side, and dequeue callbacks when
>>> they are available, but that's of course sub-optimal.
>>
>> You could implement a queue using pthreads and access that from Go
>> code. The dequeing would be synchronous, so the whole operation would
>> be synchronous from Go's point of view, even though the queue is
>> filled concurrently.
>
> Yeah, that's precisely what I suggested above. It's not a good
> solution on itself because it requires polling.
If you use a condition value, no polling is required. The cost is a
context switch, which can hardly be avoided.
⚛
> Hi Florian,
>
> >> The thread-safe and recommended way that Zookeeper can be wrapped
> >> spawns a thread in the background to handle incoming events such as
> >> watches on nodes. Ian was telling me that there's no way for that
> >> thread to callback into Go code, though. The potential workaround I
> >> can see is to busy loop in the Go side, and dequeue callbacks when
> >> they are available, but that's of course sub-optimal.
>
> > You could implement a queue using pthreads and access that from Go
> > code. The dequeing would be synchronous, so the whole operation would
> > be synchronous from Go's point of view, even though the queue is
> > filled concurrently.
>
> Yeah, that's precisely what I suggested above. It's not a good
> solution on itself because it requires polling.
and a goroutine:
- a pipe
- pthread_cond_signal (an example can be found here:
http://github.com/0xe2-0x9a-0x9b/Go-SDL/tree/master/sdl/audio/)
Gustavo Niemeyer
> and a goroutine:
>
> - a pipe
>
> - pthread_cond_signal (an example can be found here:
> http://github.com/0xe2-0x9a-0x9b/Go-SDL/tree/master/sdl/audio/)
Ahh, interesting. You mean maintaining the condition variable in the
C side, and then having Go loop calling a C function and blocking a
thread waiting on the condition. Yeah, that sounds like a reasonable
solution I didn't think of. Will give that a try, thanks!
Gustavo Niemeyer
> context switch, which can hardly be avoided.
I see what you mean. I didn't think about blocking a thread on the C
side. The context switch shouldn't be a big deal in this specific
case. I'm giving this a try, thanks for the suggestion.