Disable buffering on http request?

778 views
Skip to first unread message

patrick...@gmail.com

unread,
Apr 7, 2015, 11:56:58 PM4/7/15
to golan...@googlegroups.com
I'm trying to make a streaming chunked-transfer http request, but the net/http package is buffering my request. This is a problem as the data source is slow, and I need the data to be sent as soon as it is available. Is there any way to disable the buffer and send as fast the req.Body reader provides the bytes?

For example, in the below code, the reader pauses for a millisecond before providing data. If you run this, you'll notice the output comes in bursts. I've thrown a tcpdump on the wire, and have found it's the client side doing the buffering. Digging through source I see where the buffer is being added (request.go#L384), but don't see a way of getting around it.

package main

import (
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"time"
)

type reader int

func (r *reader) Read(p []byte) (int, error) {
time.Sleep(time.Millisecond)
*r++
return copy(p, fmt.Sprintf("hello %d\n", *r)), nil
}

func handler(w http.ResponseWriter, req *http.Request) {
io.Copy(os.Stdout, req.Body)
}

func main() {
s := httptest.NewServer(http.HandlerFunc(handler))
r := new(reader)
req, _ := http.NewRequest("POST", s.URL, r)
req.ContentLength = -1
}


Any suggestions or solutions?

-Patrick

Brad Fitzpatrick

unread,
Apr 8, 2015, 4:34:29 AM4/8/15
to patrick...@gmail.com, golang-nuts

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages