select eating up my CPU profile

[Dustin Sallings]

An application using a library I wrote is finding 70% of its time spent
unrolling selects (dequeueg).

You can see the profile report here:

http://bleu.west.spy.net/public/sgw-prof.svg

[English description of pool]:

The select is around my connection pool design which uses two buffered
channels:

connections // currently active connections
createsem // a buffered channel whose capacity specifies max connections

Any request for a connection from the pool first starts with an outer
select statement that waits up to 1ms for an existing connection to
become available (return to the pool places healthy channels back in the
connection buffer).

If it can't pull a connection from the first channel within that
millisecond, it waits for either a connection to become available, or
space to be available in the createsem channel (popped every time a
connection is actually closed).

There's also an overall timeout that's not relevant here. It's 30 days
in this case, so it only fires when there's a catastrophic issue bug.



[useful description of pool]:

func (cp *connectionPool) GetWithTimeout(d time.Duration) (*memcached.Client, error) {
if cp == nil {
return nil, errors.New("no pool")
}

t := time.NewTimer(time.Millisecond)
defer t.Stop()

select {
case rv, isopen := <-cp.connections:
if !isopen {
return nil, closedPool
}
return rv, nil
case <-t.C:
t.Reset(d) // Reuse the timer for the full timeout.
select {
case rv, isopen := <-cp.connections:
if !isopen {
return nil, closedPool
}
return rv, nil
case cp.createsem <- true:
// Build a connection if we can't get a real one.
// This can potentially be an overflow connection, or
// a pooled connection.
rv, err := cp.mkConn(cp.host, cp.auth)
if err != nil {
// On error, release our create hold
<-cp.createsem
}
return rv, err
case <-t.C:
return nil, TimeoutError
}
}
}


--
dustin

[Dmitry Vyukov]

Hi,
I can imagine that contended select is slow.
This looks like a good benchmark for optimization. Can you provide more details? Number of goroutines? Buffer sizes? DB request time?
If there is blocking on select, you may consider increasing number of connections. Otherwise you can add nonblocking recv before outer select (recv case and empty default case).

sent from phone

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.