net/http: configurable write buffer size for responses

[Albert Strasheim]

Howdy

I've been doing some tests here with a net/http server where the handler makes many relatively small writes of binary data to the ResponseWriter. I am ranging over some key-values with a LevelDB iterator, and writing a bunch of the values.

The underlying bufio.Writer's size is hard-coded at 4k:

http://golang.org/src/pkg/net/http/server.go#L433

With this size, my throughput is about 65 MB/sec. Increasing to 64k gets me to 90MB/sec and a 1M buffer gets me to 140 MB/sec.

I have a relatively small number of clients, so I can afford to increase this buffer size.

You can't wrap the ResponseWriter in another bufio.Writer to override the hard-coded buffer size.

The code in the handler could be changed to "stage" its multiple small writes to a bigger buffer which will then bypass the underlying bufio.Writer's buffer when writing, but that seems rather cumbersome.

It looks like a simple 2-line change to add a WriteBufferSize (better name?) to the Server struct to make this configurable.

Thoughts?

Regards

Albert

[Brad Fitzpatrick]

Too late for Go 1.3. File a bug though so we don't forget for Go 1.4.

--

---
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Brad Fitzpatrick]

For the record: https://code.google.com/p/go/issues/detail?id=7467 

[elit...@gmail.com]

I'd also like to see this implemented, particularly when writing slices out to templates - there seems to be a fair bit of time spent increasing the size of the underlying writer. Being able to increase it to cover (i.e.) the 98% percentile of an application's response sizes would be a useful performance gain.