anybody able to serve 100,000 or more http requests per second in go from a single machine?

[Mike Andrews]

and if so, how, which linux kernel, distro, & parameters, etc.? any code examples would be greatly appreciated.

i'm jealous of the claimed 1/2 million per second with nginx --- http://lowlatencyweb.wordpress.com/2012/03/20/500000-requestssec-modern-http-servers-are-fast/

[mjste...@gmail.com]

I always wonder what fraction of servers need to do anything near 500k req/sec. Anyway Go doing better than 100k req/sec on non static data...

http://www.techempower.com/benchmarks/

Matt

[Andy Bonventre]

It entirely depends on what you’re trying to serve in response to those requests.

One could probably write a server that is able to serve an absurd number of requests per second with a “hello!” plaintext response payload, but... why? Aside from pure curiosity, of course.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Mike Andrews]

thanks for that reference, matt!

i'm also wondering about what guarantees or expectations i could have about latency; e.g., could i expect those 100k/s to each be served in 10ms or under (for small responses, maybe <100 bytes)? i mean, since go-lang is not just raw linux with user-code-and-o/s-calls (like in c-language), but a whole additional layer of go's synchronization and scheduling? so for instance, in the go-lang net/http server architecture, there are two points in the code i'm concerned or curious about ---

1. http://golang.org/src/pkg/net/http/server.go?s=45668:45714#L1542 --- the "rw, e := l.Accept()" line where go-lang is waiting for a new tcp/ip socket connection.

2. http://golang.org/src/pkg/net/http/server.go?s=45668:45714#L1564 --- the "go c.serve()" line where go-lang is being asked to schedule a go-routine to handle the i/o on that new tcp/ip connection.

questions or concerns i have are:

1. in the midst of potentially thousands (or 10's of thousands) of other go-routines competing for cpu (in a 100k/s http server), might the "rw, e := l.Accept()" line receiving new tcp/ip connections not get called for extended periods of time? i mean, linux might notify the process in a microsecond, but go-lang's runtime might decide to wait 100ms before calling "l.Accept()?"

2. after accepting a new connection, might the "go c.serve()" line not cause a go-routine to start running immediately? again, i guess my question or concern is that the go-lang scheduler might be free to assign priorities in a way i'd disagree with.

best regards,

mike

[Michael Jones]

Did you notice the latency-related columns here?

http://www.techempower.com/benchmarks/

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Michael T. Jones | Chief Technology Advocate  | m...@google.com |  +1 650-335-5765

[Mike Andrews]

yes i did, but thanks for reminding me --- go-lang certainly does look excellent there, both throughput- and latency-wise, so i will try to reproduce those results. best regards, mike.

[RickyS]

I'm a little suspicious of those benchmarks because one popular combination, ruby-sinatra, is shown as having so many errors it only has a 1% success rate.  If that were realistic nobody would be using it.

I also notice that go was not run on the EC2 system.  I have to wonder if the competition between Amazon and Google has anything to do with that...

[Rodrigo Kochenburger]

I think those benchmarks are great, and it had been iterated over w/ help of communities, including the Ruby community.

I'm a long time Ruby developer and even though Ruby is widely adopted and sinatra as well, everyone knows it does not perform well. Most people that ran ruby in production and tried to scale had intermittent timeouts and db deadlocks because of long running queries, which seems to align w/ the "Multiple queries" test which is the only one that had ruby throwing so many errors.

Benchmarks are tricky but from the ones out there, that one is by far the one that makes most sense IMHO. 

- RK

On Mon, Nov 18, 2013 at 1:05 PM, RickyS <rickys...@gmail.com> wrote:

I'm a little suspicious of those benchmarks because one popular combination, ruby-sinatra, is shown as having so many errors it only has a 1% success rate.  If that were realistic nobody would be using it.

I also notice that go was not run on the EC2 system.  I have to wonder if the competition between Amazon and Google has anything to do with that...

[Ian Lance Taylor]

On Mon, Nov 18, 2013 at 3:32 AM, Mike Andrews <m...@xoba.com> wrote:
>

> i'm also wondering about what guarantees or expectations i could have about
> latency; e.g., could i expect those 100k/s to each be served in 10ms or
> under (for small responses, maybe <100 bytes)? i mean, since go-lang is not
> just raw linux with user-code-and-o/s-calls (like in c-language), but a
> whole additional layer of go's synchronization and scheduling? so for
> instance, in the go-lang net/http server architecture, there are two points
> in the code i'm concerned or curious about ---
>
> 1. http://golang.org/src/pkg/net/http/server.go?s=45668:45714#L1542 --- the
> "rw, e := l.Accept()" line where go-lang is waiting for a new tcp/ip socket
> connection.
>
> 2. http://golang.org/src/pkg/net/http/server.go?s=45668:45714#L1564 --- the
> "go c.serve()" line where go-lang is being asked to schedule a go-routine to
> handle the i/o on that new tcp/ip connection.
>
> questions or concerns i have are:
>
> 1. in the midst of potentially thousands (or 10's of thousands) of other
> go-routines competing for cpu (in a 100k/s http server), might the "rw, e :=
> l.Accept()" line receiving new tcp/ip connections not get called for
> extended periods of time? i mean, linux might notify the process in a
> microsecond, but go-lang's runtime might decide to wait 100ms before calling
> "l.Accept()?"

First I'll say that obviously the only thing that counts is measurement.

That said, your question is not quite correctly stated. The Go code
will have already called l.Accept. The descriptor will have been
added to the runtime poller. The question is the length of time it
will take from when the OS notifies the poller that the descriptor is
ready (via returning from epoll_wait) to when the goroutine will be
scheduled to run. This is obviously a key element of the Go runtime,
and it has been heavily optimized. If the program is otherwise idle
letting the goroutine continue will be quite fast, comparable to a
similar C program. But it is certainly true that in a loaded system
for the Go code to continue running past the point of Accept will mean
waiting for the Go runtime to allocate a thread, and the length of
time that will take depends entirely on the load.

> 2. after accepting a new connection, might the "go c.serve()" line not cause
> a go-routine to start running immediately? again, i guess my question or
> concern is that the go-lang scheduler might be free to assign priorities in
> a way i'd disagree with.

Yes, again, in a loaded system, a new goroutine will start soon but it
will not start right away. It will wait until some thread is
available.

Ian