Looks like vertx web randomly returns 404 error under high load

[Sachin Mittal]

Hi,

I was testing vertx web version 4.5.7 against a high load real data with roughly 1000 requests per second.

My vertx web application is deployed as a docker container on AWS ECS and behind an Application Load Balancer.

I have added a LoggerHandler to log requests.

My server code is something like this:

Router router = Router.router(vertx);

// add CORS and other handlers

router.route().handler(LoggerHandler.create());
router.route().handler(getCorsHandler());
router.route().handler(ResponseContentTypeHandler.create());
router.route().method(HttpMethod.POST).handler(BodyHandler.create());

// coupe of get handlers ...

// handle post
router.post().produces("text/plain").handler(new PostHandler(...));

// error handler
router.route().failureHandler(getFailureHandler());

// Serve the static resources
router.route().handler(StaticHandler.create("webroot"));

vertx.createHttpServer().requestHandler(router).listen(8080);

I see something pretty strange in the logs:

2024-05-15 12:18:36,962 [INFO ] [vert.x-eventloop-thread-1] [?:?] - 172.30.x.xxx - - [Wed, 15 May 2024 12:18:36 GMT] "POST /StreamProducer HTTP/1.1" 200 2 "-" "Mozilla/5.0 ..." 2024-05-15 12:18:36,980 [WARN ] [vert.x-eventloop-thread-1] [?:?] - 172.30.x.xxx - - [Wed, 15 May 2024 12:18:36 GMT] "POST /StreamProducer HTTP/1.1" 404 53 "-" "Roku/DVP-12.5 ..."

As you can see from logs is that for same path one request produces correct 200 response, but for smilar request other produces a 404.

Why is this ?

How can I debug the cause of this and how to prevent such random 404s.

Thanks

Sachin