Broken pipe while executing template with >2 concurrent connections

[Antoine Grondin]

Hello everybody,

I'm starting with go, and with web development in general. I'm building a basic blog engine for a course (intro to database). The website seems to work fine when I browse it, and when I run Apache Bench on the index page with 1 or 2 concurrent connections, it works without an itch.

However, as soon as I increase the number of concurrent connections, I start to get errors of this kind:

2013/04/01 11:06:27 IndexController, execute:  template: sidebar:21: write tcp 127.0.0.1:57047: broken pipe
2013/04/01 11:06:27 IndexController, execute:  template: sidebar:21: write tcp 127.0.0.1:57050: broken pipe

The template reported changes from call to call. The error is returned here:

https://github.com/aybabtme/goblog/blob/master/ctlr/index_controller.go#L55

I've put a panic(err) call into the error handling to get a stack trace, it gives me this :

/usr/local/Cellar/go/1.0.3/src/pkg/net/http/server.go:589 (0x889e5)
    _func_004: buf.Write(debug.Stack())
/usr/local/Cellar/go/1.0.3/src/pkg/runtime/proc.c:1443 (0x12ea7)
    panic: reflect·call(d->fn, d->args, d->siz);
/Users/antoine/gocode/src/github.com/aybabtme/goblog/ctlr/index_controller.go:57 (0x65ef7)
    com/aybabtme/goblog/ctlr._func_002: panic(err)
/usr/local/Cellar/go/1.0.3/src/pkg/net/http/server.go:703 (0x7c42b)
    HandlerFunc.ServeHTTP: f(w, r)
/Users/antoine/gocode/src/github.com/gorilla/mux/mux.go:86 (0x67815)
    com/gorilla/mux.(*Router).ServeHTTP: handler.ServeHTTP(w, req)
/usr/local/Cellar/go/1.0.3/src/pkg/net/http/server.go:941 (0x7d332)
    (*ServeMux).ServeHTTP: mux.handler(r).ServeHTTP(w, r)
/usr/local/Cellar/go/1.0.3/src/pkg/net/http/server.go:669 (0x7c23e)
    (*conn).serve: handler.ServeHTTP(w, w.req)
/usr/local/Cellar/go/1.0.3/src/pkg/runtime/proc.c:271 (0x10fad)
    goexit: runtime·goexit(void)

Any idea on what I'm doing wrong? I guess I must be doing a lot wrong.

[Nigel Tao]

Sorry for the late reply...

On Tue, Apr 2, 2013 at 2:25 AM, Antoine Grondin
<antoine...@gmail.com> wrote:
> Any idea on what I'm doing wrong? I guess I must be doing a lot wrong.

You're not obviously doing anything wrong. Are you still seeing this?
What version of Go are you running (release or tip)? What OS version
are you running? Do you know if you're running the 32-bit or 64-bit
flavor of Go?

[Antoine Grondin]

No problem! To be honest I forgot about this after submitting my project. Although the course was about SQL, the prof was interested to see a submission in Go - a language he didn't know. I think he liked it.

I'm using Go 1.0.3 on OS X 10.8.3, 64bits.

I thought about the problem a bit after submitting the project. I think it might be because of a race condition in the templates. The way my routing is done, I think that all the connections share the same controllers (which they get from this function) and thus the same template:

When two connections come to render the template, I assume they might from time to time meet one another in the process and result in corrupted data. I thought trying to put numCpu templates in a channel inside the controllers to make sure no two routines use the same template at the same time. I did not try my idea, however. I'm overloaded with exams these days.

A thing I did try was to change the pointer to a value, which I expect would give copies of the templates to the routines (and thus solve the possible race condition). It didn't change anything, the error still comes up.

However, I do not get the error consistently. I would say that the error appears 30% of the time right when I start the benchmark. The rest of the time, the benchmark goes through without a glitch. I find that in contradiction with my hypothesis of a race condition, in which case I would expect errors to happen at any time during the benchmark and not systematically at the beginning.

However, in contradiction with my impression