Question about Go concurrency example
hernan43
a C like language. I've been interested in Go since they announced it.
Recently I've gotten a little more serious about learning it.
I've been reading through the various examples and blogs on Go and
came across "Go Concurrency Patterns: Timing out, moving on":
http://blog.golang.org/2010/09/go-concurrency-patterns-timing-out-and.html
For the most part, I understand what is going on in the code examples.
In the second code example there is a block of code that uses
goroutines to query a series of database connections in a non-blocking
fashion:
func Query(conns []Conn, query string) Result {
ch := make(chan Result, 1)
for _, conn := range conns {
go func(c Conn) {
select {
case ch <- c.DoQuery(query):
default:
}
}(conn)
}
return <- ch
}
The idea is that if a connection doesn't "return immediately" the
switch statement will fall through to the default condition so the
goroutine isn't hanging around. I understand that much. The question I
am trying to wrap my head around is what happens here if *none* of the
"c.DoQuery(query)" calls return immediately?
Is the idea that nothing gets returned? How immediate is "immediately"
in this case? It seems like database connections would very plausibly
not return immediately(maybe even most of the time if your environment
is loaded).
Am I overthinking this and missing the point totally? I apologize for
my ignorance.
--Ray
roger peppe
> func Query(conns []Conn, query string) Result {
> ch := make(chan Result, 1)
> for _, conn := range conns {
> go func(c Conn) {
> select {
> case ch <- c.DoQuery(query):
> default:
> }
> }(conn)
> }
> return <- ch
> }
>
> The idea is that if a connection doesn't "return immediately" the
> switch statement will fall through to the default condition so the
> goroutine isn't hanging around. I understand that much. The question I
> am trying to wrap my head around is what happens here if *none* of the
> "c.DoQuery(query)" calls return immediately?
All the DoQuery calls might take a long time to complete.
The channel send to ch only happens when a query has
completed. The first query that finishes will succeed in sending
its result to the channel (we know that for certain because the
channel has a buffer size of 1). That's the result that will
be returned from Query.
The second query that finishes will also succeed in sending
its result to the channel, if it does so after the first result has been
sent. however that result is never used. All the other result sends
will then block and fall through to the default clause accordingly.
Does that make things clearer?
Ramon Hernandez
Peter Kleiweg
> The second query that finishes will also succeed in sending
> its result to the channel, if it does so after the first result has been
> sent. however that result is never used. All the other result sends
> will then block and fall through to the default clause accordingly.
query is executed, right? This line:
case ch <- c.DoQuery(query):
My guess would be that first c.DoQuery(query) is evaluated (executing
the query), and then an atempt will be made to send the result to ch
(which fails). So the query is executed on all database connections,
but only one result is used.
Why would you want to do that?
Gustavo Niemeyer
It's a simple resource usage vs. performance trade off. If all the
machines are under-used, and it's critical to return an answer as fast
as possible, you could use that technique.
--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/plus
http://niemeyer.net/twitter
http://niemeyer.net/blog
-- I never filed a patent.
John Meinel
I think his original point was that if the "default:" section is taken for each conn, because it is slow, then the final return will hang and block.
I think the model is confusing, because it certainly is a rather special case.
I'm also a bit surprised that this works, but I guess if DoQuery puts data in its return channel before it hands the channel back, it could work.
I think the more obvious model would be to pass the go routine an "exit" channel. And then have them select from the DoQuery or exit. The outer routine waits for the first response, then signals the goroutines to exit. Im guessing it should do something about draining the return channel, since otherwise we have a bunch of blocked goroutines. And IIRC, blocked routines aren't gc'd.
John
=:->
roger peppe
> I think his original point was that if the "default:" section is taken for each conn, because it is slow, then the final return will hang and block.
this cannot happen.
>
> I think the model is confusing, because it certainly is a rather special case.
> I'm also a bit surprised that this works, but I guess if DoQuery puts data in its return channel before it hands the channel back, it could work.
I'm not sure what you are getting at here. DoQuery doesn't have a
return channel - it returns a
result that is then sent on a channel.
perhaps the example might have been clearer if it had
been written like this, which is functionally identical:
func Query(conns []Conn, query string) Result {
ch := make(chan Result, 1)
for _, conn := range conns {
go func(c Conn) {
r := c.DoQuery(query)
select {
case ch <- r:
default:
}
}(conn)
}
return <- ch
}
> I think the more obvious model would be to pass the go routine an "exit" channel. And then have them select from the DoQuery or exit.
the goroutines are no selecting from the DoQuery - they're selecting
on the send to ch, which is quite different.
one potentially legitimate gripe about the above code is that
if one DoQuery succeeds quickly then for as long as the
others hang around they are garbage which can't be collected.
if you're doing hundreds of requests a second, this might become
an issue. (it is solvable too, if the db connection allows some way
of cancelling outstanding requests, but that's outside the scope
of the example).