http server request queue

[Keith Brown]

Here is a scenario: a high traffic site which executes costly operations (database / filesystem / compute heavy operations). The web server is running standard go/http/server.

I would like to handle at most 100 requests and the remainder to queue -- all while the client (curl) is waiting. Are there tools to throttle the web server? And is there a way to view the queue of work? 

  

[m...@anmol.io]

Here is a simple solution. Write some middleware that will wrap your http.Handlers.

In order to handle at most 100 requests the middleware needs to have access to a channel with a buffer size of 100 (I'll call it x) and do the following:

1. Send to x

2. Defer a receive from x.

3. Run the original http.Handler

If at any point x's buffer comes full (at 100 concurrently running handlers), step 1 will block and wait until a handler finishes and receives from x.

If you want to view the queue of work you'd need a different solution. I'm not aware of an existing open source solution.

[Jonathan Yu]

chi includes a middleware for this, see: https://github.com/go-chi/chi/blob/master/middleware/throttle.go and some examples here: https://github.com/go-chi/chi/blob/master/_examples/limits/main.go

I haven't used it myself, but it seems to support having a fixed backlog as well, which is important for backpressure, see: https://www.somethingsimilar.com/2013/01/14/notes-on-distributed-systems-for-young-bloods/#backpressure

On Sat, Oct 28, 2017 at 6:23 AM, Keith Brown <keit...@gmail.com> wrote:

Here is a scenario: a high traffic site which executes costly operations (database / filesystem / compute heavy operations). The web server is running standard go/http/server.

I would like to handle at most 100 requests and the remainder to queue -- all while the client (curl) is waiting. Are there tools to throttle the web server? And is there a way to view the queue of work? 

  

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Jonathan Yu / @jawnsy on LinkedIn, Twitter, GitHub, Facebook

“Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.” — Samuel Beckett, Worstward Ho (1983) 

“In an adaptive environment, winning comes from adapting to change by continuously experimenting and identifying new options more quickly and economically than others. The classical strategist's mantra of sustainable competitive advantage becomes one of serial temporary advantage.” — Navigating the Dozens of Different Strategy Options (HBR)