Luminus in Techempower benchmarks

[gvim]

In the latest round of Techempower benchmarks:

https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=json

... I was surprised to find Luminus performing no better than Hapi
(Node) and significantly worse than Java frameworks. Figures are
requests/second:

FORTUNES
- Hapi: 1.9
- Luminus: 0.9
- Gemini: 55.5

JSON SERIALISATION
- Hapi: 0.3 (Raw db)
- Luminus: 0.8
- Rapidoid: 78.4


SINGLE QUERY
- Hapi: 2.9
- Luminus: 8.7
- Gemini: 75.8

MULTI-QUERY
- Hapi: 33.0
- Luminus: 20.4
- Dropwizard: 65.8

DATA UPDATES
- Hapi: 20.9
- Luminus: 20.0
- Ninja: 54.7

PLAINTEXT
- Hapi: 0.7
- Luminus: 0.0
- Rapidoid: 100.0



Any ideas?

gvim

[Jim Crossley]

Hi,

Luminus uses Immutant, which uses Undertow, so it should be possible
to tune the Luminus app to approach the performance of the TechEmpower
Undertow app. The relevant options to immutant.web/run [1] are
:dispatch?, :io-threads, and :worker-threads.

The Undertow app sets IO threads here [2] and worker threads here [3],
so you can easily set those same values in the Luminus app.

But the real performance bump will come from the :dispatch? option. By
default, :dispatch? is true, i.e. requests handled by threads in the
IO pool are dispatched to a thread in the worker pool. For
compute-bound tasks like the JSON serialization benchmark, that
context switch is far more expensive than just having the IO thread
return the response. So you'd want to set :dispatch? to false in that
case.

The tricky bit here is that the Luminus app defines all its routes in
a single handler [4]. But for the more io-bound tasks, e.g.
DbSqlHandler [5], you'll want the default true value of :dispatch?. So
in order to get the best results for all the benchmarks, you'll need
to re-organize that app to run two handlers: one that dispatches and
one that doesn't. In Immutant, the simplest way to do this is to
distinguish each handler with a unique :path option.

I'm happy to assist whoever is maintaining that app with the tuning.

Thanks,
Jim

[1] http://immutant.org/documentation/current/apidoc/immutant.web.html#var-run
[2] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Java/undertow/src/main/java/hello/HelloWebServer.java#L122
[3] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Java/undertow/src/main/java/hello/HelloWebServer.java#L160
[4] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Clojure/luminus/hello/src/hello/routes/home.clj#L44
[5] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Java/undertow/src/main/java/hello/DbSqlHandler.java#L38

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Jim Crossley]

I just tried a few experiments and realized the :dispatch? option is broken in the latest Immutant release. :(

This is a result of some changes we made to better support WebSockets.

We were already hoping to get a release out this week, so we'll add that to the list of fixes.

Sorry about that,

Jim

[Leif]

The Pedestal app is also strangely underperforming in some cases.  A Pedestal expert may want to take a look at its configuration.

[Dmitri]

Hi Jim,

I'm the author of Luminus, and I'd love tow work with you to tune up the performance. Feel free to ping me via email or on GitHub.

[Jim Crossley]

Sounds good, Dmitri. I'll work up something on a fork in the next few
days and ping you. We can go from there.

Jim

[Dmitri]

Sounds fantastic!