[Review] Slow server, not maxing out CPU

[Вук Мировић]

I'm fooling around with Clojure and I implemented simple server that will randomly query/insert/update/delete random data from DB on every request.

When I benchmark (wrk) server is slow (compared to similar solution) and I noticed that CPU is not maxed out, it's about ~30-40%.

I used http-kit for HTTP server, Hakiri-CP as connection pool. Added some core.async hoping to improve the speed but results are the same.

There is clearly something wrong with my code but I can't figure it out. I'm hoping if someone could throw a glimpse on my code to give a sugestion.

Here is the code: https://github.com/wooque/bench/blob/master/clojure/src/bench/core.clj

[James Reeves]

If you're going to be working with blocking I/O, then you're probably going to get constrained by the default threadpool size of HTTP Kit, which is only 4 threads. Up it to around 1000 and see if that improves matters.

You can get around this by writing asynchronous code, but your async-handler isn't asynchronous.

- James

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ashish Negi]

with core async `go` you can not do blocking IO or any time consuming work.
`go` uses fixed threadpool (no of cpus + 2 or something).
and in your async-handler you are using `<!! c` which is blocking . https://clojure.github.io/core.async/

Try with `future`. https://clojuredocs.org/clojure.core/future

Simplest would be to pass your `channel` in with-channel to the future which would put the data in it after it gets from `work`.

[James Reeves]

On 21 July 2016 at 05:05, Ashish Negi <thisismy...@gmail.com> wrote:

with core async `go` you can not do blocking IO or any time consuming work.
`go` uses fixed threadpool (no of cpus + 2 or something).
and in your async-handler you are using `<!! c` which is blocking . https://clojure.github.io/core.async/

Try with `future`. https://clojuredocs.org/clojure.core/future

While futures will work, it's worth pointing out you'd get the same result just by increasing the thread count on the adapter. Either way you're using up a thread per request.

Since the database functions are not asynchronous, core.async isn't really useful in this context. My suggestion would be to drop it altogether.

- James

[Вук Мировић]

http-kit threadpool was definitely bottleneck, and probably hakiri-cp connection pool.

I reverted to initial version without core.async and bumped both pools to 128 and now CPU is fully saturated, and performance is much better. Comparable to Go version I wrote, maybe even better.

Thanks you both for your time :)