vert.x webclient and golang fasthttp client(plow)

[Himanshu Gupta]

I am trying to write a performance test application using vert.x webclient which gives me 20k rps performance(near to what wrk is giving) while hitting the endpoint which is a simple quarkus application but same endpoint when hit with plow (which uses fasthttp) gives 40K rps.

This is how I am creating the webclient for each connection:

WebClient webClient = WebClient.create(vertx, new WebClientOptions()
.setSsl(false)
//.setPoolEventLoopSize(CpuCoreSensor.availableProcessors() * 2)
.setTcpFastOpen(true)
.setTcpQuickAck(true)
.setTcpKeepAlive(true)
.setTcpNoDelay(true)
//.setTcpCork(true)
.setReusePort(true)
//.setReuseAddress(true)
.setAlpnVersions(List.of(HttpVersion.HTTP_1_1))
.setTracingPolicy(TracingPolicy.IGNORE)
//.setSendBufferSize(1024 * 1024)
//.setReceiveBufferSize(1024 * 1024)
.setMaxWaitQueueSize(50)
//.setPipelining(true)
//.setPipeliningLimit(5)
.setDefaultHost(pertaasTestBean.getPerformanceTestRequest().hostname())
.setDefaultPort(pertaasTestBean.getPerformanceTestRequest().port())
.setHttp2ClearTextUpgrade(false)
.setProtocolVersion(HttpVersion.HTTP_1_1)
.setPoolCleanerPeriod(-1)
.setMaxPoolSize(1));

This is how I am running the constant load:

while (System.currentTimeMillis() <= pertaasTestBean.endTime()) {
webClient
.get("/couchbase/get")
.as(BodyCodec.none())
.putHeader("pertaas-id", pertaasTestBean.uuid())
.putHeader("pertaas-path", "/couchbase/get")
.send()
.await()
.indefinitely();
}

I tried with okhttp, apache httpclient and others but the results were more or less the same in the JVM based http clients.

What are the performance improvement being done in plow/fasthttp which is making it so performant? Can we look at bringing those into vert.x and eventually get into quarkus?

[Bruno F]

I don't know.

Benchmarking is hard.

In the techempowerbenchmark fiber-prefork (built on top of fasthttp) is only 5% faster than netty (but I guess this benchmark is about serving requests and not calling a server):

https://www.techempower.com/benchmarks/#section=data-r22&hw=ph&test=plaintextite

I would add that maybe you're comparing apples and oranges: vertx-web is probably a user friendly addition to netty and should rather be compared to fiber than to fasthttp.

But it is true that fiber seems to be very fast according to the same benchmark.

 

A few years ago I benchmarked some HTTP clients to call a high RPM microservice and I realized they all had their strengths and weaknesses depending on the size of the payload (I was benchmarking POST requests).

With small payloads (a few bytes) one version of okhttp was really brilliant (but more recent versions lost that capability) and Netty was the worse (that really came as a big surprise to me). 

But Netty was the fastest with a bit larger payloads (a few kilobytes).

Maybe plow is putting a big effort in tweaking parameters depending on the nature of the request?

[Bruno F]

I'm curious so I tried to test it by myself :-).

I'm using this HTTP server written in Rust:

https://gist.github.com/bfreuden/27b681aacccfbcb0a8be8dea9b8678b0

Then this WebClient benchmark program (Vert.x 4.3.8):

https://gist.github.com/bfreuden/342898907b19074f71307a45d9b957e1

And I also tested the lower-level Vert.x HttpClient (Vert.x 4.3.8):

https://gist.github.com/bfreuden/2655785718026ecdf8c479f78ad7c33f

Those benchmark verticles are an attempt to prevent thread context switching.

This is the output of plow:

plow http://127.0.0.1:3000/ -c 1 -n 100000

Summary:
  Elapsed        3.5s
  Count        100000
    2xx        100000
  RPS       27835.114

And this is the output of the WebClient benchmark (ran with a JDK11):

Requests: 100000
Successes: 100000
Failures: 0
Elapsed (ms): 4168
Request/sec: 23992
done

And this is the output of the HttpClient benchmark (ran with a JDK11):

Requests: 100000
Successes: 100000
Failures: 0
Elapsed (ms): 3894
Request/sec: 25680

done

So, in this particular use case and if my benchmark programs are correct, there is definitely not a x2 factor between plow and the Vert.x WebClient.
Plow is only 16% faster than the Vert.x WebClient.

And it is only 8% faster than the Vert.x HttpClient.

[Himanshu Gupta]

Hi,

I will try to share the similar benchmark with plow and share results. I do see you are using http/2 in your Webclient whereas plow only does http 1.1.

Thanks & Regards,
Himanshu Gupta

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/a43fe921-3d24-480c-901a-57a215184a78n%40googlegroups.com.

[Himanshu Gupta]

I think every time I go for reproducer, I end up finding the issue with my application. Thanks for the insights and here is the code repo that runs the same test with hostname, port, connection count and number of requests. It has the quarkus server running on virtual threads and the vertx test app.

https://github.com/himanshumps/performance-test

[Bruno F]

Good! Is your Webclient performance closer to plow now?

[Himanshu Gupta]

Almost identical .. I checked with wrk as well and all of them are on similar lines

Thanks & Regards,
Himanshu Gupta

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/f67138f6-5f0b-4a5e-932b-7ba5e9c29d99n%40googlegroups.com.

[Bruno F]

That's great! I'm glad it's working :-)

Bruno.