C10k@Go

[Fabian Moser]

In order to pass the time until we get Channels API support, I have put
together a quick&dirty test application to investigate what performance and
resource usage I would have to expect from a Go Comet server. You can find the
full story on my blog at http://fabianmoser.at/blog/2011/05/22/go-c10k/ but
for those in a hurry, the source is pasted below. This seems to need around
32KB per connection. Is there room for improvement while sticking with pure
Go? Looking forward to your comments!

package main

import (
"fmt"
"http"
"log"
"time"
)

func main() {
http.HandleFunc("/test/", feed)
if err := http.ListenAndServe("127.0.0.1:8080", nil); err != nil {
log.Print(err)
}
}

func feed(w http.ResponseWriter, r *http.Request) {
ch := time.Tick(10000000000)
chunkedw := http.NewChunkedWriter(w)
fmt.Fprintf(chunkedw, "Goconntest welcomes you! Your Id: %s\n",
r.URL.String())
for N := 1; ; N++ {
<-ch
n, err := fmt.Fprintf(chunkedw, "Chunk %d for id %s\n", N,
r.URL.String())
if err != nil {
log.Print(err)
break
} else if n == 0 {
log.Print("Nothing written")
break
}
}
chunkedw.Close()
}

--
Fabian

[Brad Fitzpatrick]

Go's HTTP server does chunking by default if you don't set a Content-Length.  If you do your chunking you'll get chunking-in-chunking (wrong) or chunking-with-a-content-length (also wrong).  I should update the docs on NewChunkedWriter to warn about that.

Also, App Engine doesn't permit streaming a response, so this won't work there anyway.

As for 32kb-per-goroutine, that's probably the HTTP stack and various buffers.  Go's native goroutine cost is closer to 4Kb, plus whatever state you have per-goroutine.

[John Davis]

How far off is the comet support?

[Andrew Gerrand]

On 24 May 2011 20:29, John Davis <unic...@gmail.com> wrote:
> How far off is the comet support?

What do you mean? It's quite easy to implement comet-style http
handlers in Go already.

Or are you referring to the Channels API?

Andrew

[murz]

> What do you mean? It's quite easy to implement comet-style http
> handlers in Go already.

Can you elaborate on that? Are there any examples of implementing
comet-style http in Go? What are the limitations of running an app
like that on appengine-go?

I'm interesting in building a web app using Go and the Channels API,
but in the meantime comet-style http should work.

[Brad Fitzpatrick]

On Thu, Jun 2, 2011 at 12:38 PM, murz <mmurr...@gmail.com> wrote:

> What do you mean? It's quite easy to implement comet-style http
> handlers in Go already.

Can you elaborate on that? Are there any examples of implementing
comet-style http in Go?

See TestStreamingGet here for the client and server:

http://code.google.com/p/go/source/browse/src/pkg/http/client_test.go#199

 

What are the limitations of running an app
like that on appengine-go?

App Engine doesn't permit streaming requests or responses.  Your request and response are delivered in one chunk (up to 32 MB).

[Andrew Gerrand]

On 3 June 2011 05:38, murz <mmurr...@gmail.com> wrote:
>> What do you mean? It's quite easy to implement comet-style http
>> handlers in Go already.
>
> Can you elaborate on that? Are there any examples of implementing
> comet-style http in Go?

Here's a complete framework for doing comet-style stuff in Go:
https://github.com/madari/go-socket.io

I could put together a quick example just using the Go standard
libraries, if you're really interested.

> What are the limitations of running an app
> like that on appengine-go?
>
> I'm interesting in building a web app using Go and the Channels API,
> but in the meantime comet-style http should work.

As Brad said, App Engine doesn't support streaming responses so the
comet technique won't work.

Don't worry, the Channel API for Go will be available soon.

Andrew

[Fabian Moser]

Am Freitag, 3. Juni 2011, 00:21:26 schrieb Andrew Gerrand:
> Here's a complete framework for doing comet-style stuff in Go:
> https://github.com/madari/go-socket.io

Thanks Andrew for that link! Due to probably very unfortunate choices, none of
my search patterns found that package for me.

--
Fabian

[murz]

> Don't worry, the Channel API for Go will be available soon.

Any clue as to when it will be available?

[David Symonds]

On Jun 22, 2011 7:40 AM, "murz" <mmurr...@gmail.com> wrote:

> > Don't worry, the Channel API for Go will be available soon.
>
> Any clue as to when it will be available?

It's available now.

Dave.

[murz]

Wow, that is "soon". You guys are awesome :)

On Jun 21, 3:04 pm, David Symonds <dsymo...@golang.org> wrote: