C10k@Go

283 views
Skip to first unread message

Fabian Moser

unread,
May 22, 2011, 7:34:49 AM5/22/11
to google-appengine-go
In order to pass the time until we get Channels API support, I have put
together a quick&dirty test application to investigate what performance and
resource usage I would have to expect from a Go Comet server. You can find the
full story on my blog at http://fabianmoser.at/blog/2011/05/22/go-c10k/ but
for those in a hurry, the source is pasted below. This seems to need around
32KB per connection. Is there room for improvement while sticking with pure
Go? Looking forward to your comments!

package main

import (
"fmt"
"http"
"log"
"time"
)

func main() {
http.HandleFunc("/test/", feed)
if err := http.ListenAndServe("127.0.0.1:8080", nil); err != nil {
log.Print(err)
}
}

func feed(w http.ResponseWriter, r *http.Request) {
ch := time.Tick(10000000000)
chunkedw := http.NewChunkedWriter(w)
fmt.Fprintf(chunkedw, "Goconntest welcomes you! Your Id: %s\n",
r.URL.String())
for N := 1; ; N++ {
<-ch
n, err := fmt.Fprintf(chunkedw, "Chunk %d for id %s\n", N,
r.URL.String())
if err != nil {
log.Print(err)
break
} else if n == 0 {
log.Print("Nothing written")
break
}
}
chunkedw.Close()
}

--
Fabian

Brad Fitzpatrick

unread,
May 22, 2011, 6:33:58 PM5/22/11
to Fabian Moser, google-appengine-go
Go's HTTP server does chunking by default if you don't set a Content-Length.  If you do your chunking you'll get chunking-in-chunking (wrong) or chunking-with-a-content-length (also wrong).  I should update the docs on NewChunkedWriter to warn about that.

Also, App Engine doesn't permit streaming a response, so this won't work there anyway.

As for 32kb-per-goroutine, that's probably the HTTP stack and various buffers.  Go's native goroutine cost is closer to 4Kb, plus whatever state you have per-goroutine.

John Davis

unread,
May 24, 2011, 6:29:30 AM5/24/11
to google-ap...@googlegroups.com, Fabian Moser
How far off is the comet support?

Andrew Gerrand

unread,
May 24, 2011, 6:55:00 PM5/24/11
to google-appengine-go, Fabian Moser
On 24 May 2011 20:29, John Davis <unic...@gmail.com> wrote:
> How far off is the comet support?

What do you mean? It's quite easy to implement comet-style http
handlers in Go already.

Or are you referring to the Channels API?

Andrew

murz

unread,
Jun 2, 2011, 3:38:44 PM6/2/11
to google-appengine-go
> What do you mean? It's quite easy to implement comet-style http
> handlers in Go already.

Can you elaborate on that? Are there any examples of implementing
comet-style http in Go? What are the limitations of running an app
like that on appengine-go?

I'm interesting in building a web app using Go and the Channels API,
but in the meantime comet-style http should work.

Brad Fitzpatrick

unread,
Jun 2, 2011, 4:00:00 PM6/2/11
to murz, google-appengine-go
On Thu, Jun 2, 2011 at 12:38 PM, murz <mmurr...@gmail.com> wrote:
> What do you mean? It's quite easy to implement comet-style http
> handlers in Go already.

Can you elaborate on that? Are there any examples of implementing
comet-style http in Go?

See TestStreamingGet here for the client and server:
 
What are the limitations of running an app
like that on appengine-go?

App Engine doesn't permit streaming requests or responses.  Your request and response are delivered in one chunk (up to 32 MB).

Andrew Gerrand

unread,
Jun 2, 2011, 6:21:26 PM6/2/11
to murz, google-appengine-go
On 3 June 2011 05:38, murz <mmurr...@gmail.com> wrote:
>> What do you mean? It's quite easy to implement comet-style http
>> handlers in Go already.
>
> Can you elaborate on that? Are there any examples of implementing
> comet-style http in Go?

Here's a complete framework for doing comet-style stuff in Go:
https://github.com/madari/go-socket.io

I could put together a quick example just using the Go standard
libraries, if you're really interested.

> What are the limitations of running an app
> like that on appengine-go?
>
> I'm interesting in building a web app using Go and the Channels API,
> but in the meantime comet-style http should work.

As Brad said, App Engine doesn't support streaming responses so the
comet technique won't work.

Don't worry, the Channel API for Go will be available soon.

Andrew

Fabian Moser

unread,
Jun 3, 2011, 2:16:57 AM6/3/11
to google-ap...@googlegroups.com
Am Freitag, 3. Juni 2011, 00:21:26 schrieb Andrew Gerrand:
> Here's a complete framework for doing comet-style stuff in Go:
> https://github.com/madari/go-socket.io

Thanks Andrew for that link! Due to probably very unfortunate choices, none of
my search patterns found that package for me.

--
Fabian

murz

unread,
Jun 21, 2011, 5:40:22 PM6/21/11
to google-appengine-go
> Don't worry, the Channel API for Go will be available soon.

Any clue as to when it will be available?

David Symonds

unread,
Jun 21, 2011, 6:04:36 PM6/21/11
to murz, google-appengine-go

On Jun 22, 2011 7:40 AM, "murz" <mmurr...@gmail.com> wrote:

> > Don't worry, the Channel API for Go will be available soon.
>
> Any clue as to when it will be available?

It's available now.

Dave.

murz

unread,
Jun 21, 2011, 7:27:26 PM6/21/11
to google-appengine-go
Wow, that is "soon". You guys are awesome :)

On Jun 21, 3:04 pm, David Symonds <dsymo...@golang.org> wrote:
Reply all
Reply to author
Forward
0 new messages