Millions of requests per second in a realistic application

[Jens Theisen]

Hi,

I'm developing a logging ingestion application that has the special twist of concentrating more on "how many" rather than "what".

As such, I'm curious if I can make it scale to millions of requests per second, as the plaintext benchmarks indicate I should be able to.

The plaintext benchmarks, however, send multiple requests at once if I understand that correctly (that's what the pipeline.lua script is for), and also there's the question if one wrk instance generating all the load is even a realistic - I reckon TCP connections will be reused, for example.

I'm fairly confident I can optimize my own app enough to handle the load (ASP.NET Core and not even doing json parsing), but I have no idea what stress OS connection bookkeeping and whatever else there may be is an issue.

And I don't know how I would create more realistic load and whether it's worthwhile pursuing.

Existing load services have pitifully low rps and are more geared towards saturating a PHP script.

Any advice from the experts would be appreciated.

Thanks, Jens

[Jens Theisen]

Just to be more clear: If my app can handle a "wrk -s pipeline.lua -- 20" with millions of requests, will that mean it can handle the same amount if those requests all come from different machines? What about if they each come from a new connection?

[Matthieu Garrigues]

If you want performances without pipelining, check the performance of the json test. If you want the performances of a test with no keep alive, the tfb berchmark does not test this today so you have to test it yourself with ab without the -k option.

[zloster]

On 17.08.2020 22:09, Matthieu Garrigues wrote:
> If you want performances without pipelining, check the performance of the json test. If you want the performances of a test with no keep alive, the tfb berchmark does not test this today so you have to test it yourself with ab without the -k option.
>

Please note that in the later case (without "keep-alive") you will hit
very early the TCP/IP connections/socket limits. The defaults in Linux
kernels are around 20 000 - 30 000 used sockets per minute. It will
require tweaking the client (the load generator) and the server (and the
DB server).