Just a quick public acknowledgement for your recent efforts in the concurrency aspects of the runtime. I've just run a real world benchmark provided by someone using mgo with the r59 release, and it took about 5 seconds out of 20, without any changes in the code.
This note caught my eye because I just had another experience where the Go language just made my life a whole lot easier. So I'd like to follow up Gustavo's note by sending a general thank-you to the Go developers for making such a nice language, period.
I have a program I'm working on that ends up using HTTP to POST or PUT resources onto another server. The general flow of the first draft of the program was along the lines of:
for id := range idSet { entry := generateEntry(id) if doUpdate(entry) { if u.Update(entry) { success[entry.URL] = true } } }
I found that this was far too slow, but I also found that if I faked u.Update, to just write its data to /dev/null, the program was 25x faster than when I actually sent data to the remote server. That seemed to indicate I was probably spending a lot of time blocking, waiting on the remote server.
So with a few simple changes, thanks to Go's design, I added code to spawn a few goroutines to handle the updates:
func (u *Updater) Updater(entries chan Representation, status chan map[string]bool) { m := make(map[string]bool) for entry := range entries { m[entry.URL()] = u.Update(entry) } status <- m
}
and code to submit entries to those goroutines:
.... updateEntries := make(chan Representation) updateStatus := make(chan map[string]bool) for i := 0; i < u.Config.GoRoutines; i ++ { go u.Updater(updateEntries, updateStatus) } .... for id := range idSet { entry := generateEntry(id) if doUpdate(entry) { updateEntries <- entry } } ... close(updateEntries) ...
the code was able to execute many more operations per second. In this case, with 3 goroutines, I see a 3x to 4x speed-up:
My sentiments as well. I came from the "hardware is cheap" camp but it seems a terrible waste what we do with it at times. Thanks for this language, your hard work and giving us such a great tool that lets us build stuff quickly, that runs quickly, and is easy to develop. :) Brandon
On Wed, Aug 3, 2011 at 12:13 PM, jimr <jim.robin...@gmail.com> wrote: > This note caught my eye because I just had another experience where > the Go language just made my life a whole lot easier. So I'd like to > follow up Gustavo's note by sending a general thank-you to the Go > developers for making such a nice language, period.
> I have a program I'm working on that ends up using HTTP to POST or PUT > resources onto another server. The general flow of the first draft of > the program was along the lines of:
> for id := range idSet { > entry := generateEntry(id) > if doUpdate(entry) { > if u.Update(entry) { > success[entry.URL] = true > } > } > }
> I found that this was far too slow, but I also found that if I faked > u.Update, to just write its data to /dev/null, the program was 25x > faster than when I actually sent data to the remote server. That > seemed to indicate I was probably spending a lot of time blocking, > waiting on the remote server.
> So with a few simple changes, thanks to Go's design, I added code to > spawn a few goroutines to handle the updates:
> func (u *Updater) Updater(entries chan Representation, status chan > map[string]bool) { > m := make(map[string]bool) > for entry := range entries { > m[entry.URL()] = u.Update(entry) > } > status <- m > }
> and code to submit entries to those goroutines:
> .... > updateEntries := make(chan Representation) > updateStatus := make(chan map[string]bool) > for i := 0; i < u.Config.GoRoutines; i ++ { > go u.Updater(updateEntries, updateStatus) > } > .... > for id := range idSet { > entry := generateEntry(id) > if doUpdate(entry) { > updateEntries <- entry > } > } > ... > close(updateEntries) > ...
> the code was able to execute many more operations per second. In > this case, with 3 goroutines, I see a 3x to 4x speed-up:
I'd also like to give a note of public thanks to Dmitry for his ongoing efforts improving the runtime. Your work is very much appreciated and will benefit countless current and future Go users.
On Thu, Aug 4, 2011 at 4:48 AM, Andrew Gerrand <a...@golang.org> wrote: > Indeed. Thanks for your excellent work, Dmitry. > Nice to see it paying off so quickly. > Andrew
On Wednesday, August 3, 2011 9:35:28 AM UTC-3, Gustavo Niemeyer wrote:
> Just a quick public acknowledgement for your recent efforts in the > concurrency aspects of the runtime. I've just run a real world > benchmark provided by someone using mgo with the r59 release, and it > took about 5 seconds out of 20, without any changes in the code.
I second that and would also congratulate Gustavo. It is so much more common to see people complaining than recognizing good work. Kudos to both.
On 3 Aug., 14:35, Gustavo Niemeyer <gust...@niemeyer.net> wrote:
> Hey Dmitry,
> Just a quick public acknowledgement for your recent efforts in the
> concurrency aspects of the runtime. I've just run a real world
> benchmark provided by someone using mgo with the r59 release, and it
> took about 5 seconds out of 20, without any changes in the code.
> Please keep up the great work.
5 seconds out of 20 is huge, I second the above acknowledgement!
The primary reason I took an interest in the Go language was because
of its concurrency and scalability perspectives...
Imagine if you are running thousands of Servers (or perhaps 900000
Servers) and you save just 10% or 20% of your processor time and
energy only by using more efficient algorithms on the concurrency
side. Calculate that into Dollars saved or even just emissions saved.
Its a compelling business case :-) .
On Wednesday, August 3, 2011 4:35:28 PM UTC+4, Gustavo Niemeyer wrote:
> Hey Dmitry,
> Just a quick public acknowledgement for your recent efforts in the > concurrency aspects of the runtime. I've just run a real world > benchmark provided by someone using mgo with the r59 release, and it > took about 5 seconds out of 20, without any changes in the code.
> Please keep up the great work.
> Thank you, guys! You are welcome! It's really nice to hear such responses.
More is coming, stay tuned ;)
Btw, while you are here, I've started work on the concurrent "co" package, and would like to hear your thoughts on it, I will start a separate thread for it.
On Monday, August 8, 2011 9:01:57 PM UTC+2, Dmitry Vyukov wrote:
> On Wednesday, August 3, 2011 4:35:28 PM UTC+4, Gustavo Niemeyer wrote:
>> Hey Dmitry,
>> Just a quick public acknowledgement for your recent efforts in the >> concurrency aspects of the runtime. I've just run a real world >> benchmark provided by someone using mgo with the r59 release, and it >> took about 5 seconds out of 20, without any changes in the code.
>> Please keep up the great work.
>> Thank you, guys! You are welcome! It's really nice to hear such responses. > More is coming, stay tuned ;)
> Btw, while you are here, I've started work on the concurrent "co" package, > and would like to hear your thoughts on it, I will start a separate thread > for it.
