Re: kilim web server

[seth/nqzero]

finally got around to using async tomcat and did a comparison with
kws. as promised, here are some numbers ...

the bottleneck in my backend processing is a single-threaded-portion
and disk. i haven't tried the nonblocking network io, but my payloads
are small, so hoping that that's not a big factor. for testing i ended
up using apache bench. i added a command to my webapp that will
perform a "random" read-only action

linux limits the number of open file descriptors, which limits the
concurrent connections that you can test. i work around this for
testing by running kws, tomcat and ab in a bash instance that has been
run with sudo, eg

sudo bash -c "ulimit -n 70000; su username"
ab -r -T application/x-www-form-urlencoded -p doc/POST-read.txt -c
20000 -n 100000 localhost:8080/diary

ab is limited to 20000 concurrent connections. i have thin shims for
kws and tomcat that feed the request to a common processing engine and
i've mimicked the kilim portions of kws in my tomcat shim (standard
scheduler with 4 threads). i believe it's apples to apples, but of
course, the devil is in the details so buyer beware

tomcat 8.0.8, with <async-supported>, 1000000 total requests

throughput 95% latency
concurrency tomcat kws tomcat kws
(1000s) (req/s) (req/s) (ms) (ms)
01 8942.88 13813.25 1016 52
03 9952.78 14465.96 1051 1074
04 10261.91 14280.45 1233 1150
06 9836.01 14641.07 3028 1234
07 8849.40 14716.42 3044 1291
09 10442.45 14612.94 3074 1412
10 9998.81 14477.38 3103 1481
11 8940.53 14599.88 3242 1530
12 8608.96 14407.21 3282 1626
13 8642.78 14453.07 3557 1689
14 8614.16 14594.65 4304 1731
15 8707.69 14489.58 5079 1831
17 8577.84 14361.63 7051 1917
18 8548.55 14595.37 7060 2002
19 8467.92 14616.16 7081 2108
20 8294.48 14671.26 7121 2151

at high concurrency levels and when the total number of requests is
high, tomcat (or ab) appears to drop a few requests:
apr_socket_recv: Connection reset by peer (104)

my understanding is that these are more a symptom of how ab manages
the requests than a real problem, but i haven't investigated. with
kws, these drops only happens before the cache gets warmed up, ie when
disk is severely limiting

throughput doesn't seem sensitive to concurrency on either server from
1000 to 20000, with kws being 50% faster. as concurrency is increased,
kilim wins on latency. i haven't tried to tune tomcat, so perhaps
these numbers could be improved (eg by limiting the number of tomcat
threads or using a read/write listener). but at first glance, it looks
like tomcat async is decent when faced with 20k concurrent
connections, making it sufficient to use as a front end for a
kilim-based backend





On Fri, Aug 30, 2013 at 1:49 AM, seth/nqzero <bl...@nqzero.com> wrote:
> thanks for the reply sriram
>
>> The MOB tester is quite bitrotted, and not documented at all, so it'll
>> take a fair bit of effort to package up. There are a number of http testers
>> around anyway
>
>
> the blog posts comparing tornado and node.js all seem to use multiple
> httperf processes - a kilim based client might be lighter and allow for
> better simulated loads (and maybe comet ?). so if the code is unencumbered
> and it's just a matter of polishing the rot i'd be willing to try
>
>> As for the KWS, all the essential bits are already in kilim.http. If all
>> you're using Tomcat for is request and response handling (and not say, https
>> or cookies), then kilim http is more than sufficient and really really fast
>
>
> unfortunately i do need cookies to store a session id, but maybe i can parse
> that myself. i've mocked something up, but need to remove some vestigial
> framework stuff. i'll report back if i get some tomcat vs kws numbers
>
>> That said, there's no reason to fear threads
>
>
> my understanding is that a java thread uses about 1M for stack. i'm
> targeting lowendbox/vps setups so hoping to keep that footprint down. since
> i'm already using kilim for the backend, KWS seems like the perfect solution
>

[Sriram Srinivasan]

On Jun 28, 2014, at 12:24 PM, seth/nqzero <bl...@nqzero.com> wrote:

finally got around to using async tomcat and did a comparison with
kws. as promised, here are some numbers ...

Thanks much for the numbers. 

the bottleneck in my backend processing is a single-threaded-portion
and disk. i haven't tried the nonblocking network io, but my payloads
are small, so hoping that that's not a big factor.

That's probably not a good assumption -- you have 14000 requests per second converging on disk. I'm sure there's plenty of improvement possible in those numbers.  Given that the numbers sort of go up and down as the concurrency goes up,  it'd be good to see the jitter.

Incidentally, Gil Tene's talk on "How not to measure latency" has really opened my eyes. I now see the errors of my ways.

http://www.infoq.com/presentations/latency-pitfalls

for testing i ended
up using apache bench. i added a command to my webapp that will
perform a "random" read-only action

Are ab and the servers on the same machine?  How many processors in the machine(s)?

throughput doesn't seem sensitive to concurrency on either server from
1000 to 20000, with kws being 50% faster. as concurrency is increased,
kilim wins on latency. 

If the throughput is more or less the same, then I suspect the single backend is the limiting factor. And if it is a oneshot page (every request is independent of the next), then the decreased latency is coming from the way kilim/nio handles the NIO selector. 

--sriram.

[seth/nqzero]

>> i haven't tried the nonblocking network io, but my payloads are small,
>> so hoping that that's not a big factor

> That's probably not a good assumption -- you have 14000 requests per second converging on disk

for both tomcat and kws, all disk access is thru my kilim-based backend
tomcat 8 introduces nonblocking network io

https://weblogs.java.net/blog/swchan2/archive/2013/04/16/non-blocking-io-servlet-31-example

my understanding is that the only case in which this helps is if a
request or reply is larger than 1 packet, and then only if the network
device blocks, eventually resulting in thread exhaustion. my payloads
are small, i'm disk+cpu bound, and in one configuration my network is
just the loopback device, so i think it's safe to ignore this factor.
am i missing something ?

> Are ab and the servers on the same machine?
> How many processors in the machine(s)?

for the most part ab, the web server (tomcat or kws) and the backend
(kilim-based) all ran on the same machine, but i also tested with ab
on a separate machine on the LAN and didn't see a significant
difference. both machines i used are core i3 at approximately 3GHz

for both tomcat and kws, all access to disk (for these tests) was thru
my kilim-based backed. the backend is a single thread processing kilim
tasks, and additional threads for accessing disk. for the 14000
requests, most are handled by cache, ie they never hit disk (all the
numbers i gave were after being warmed up). the scheduler for the
backend is distinct from the kws scheduler

i really like kws, esp the easy deployment and imperative style.
however, the lack of features limits it's utility, eg ssl, websockets,
and cookies. my primary purpose in this benchmark was just to see if
tomcat was a viable frontend for a kilim-based backend. i haven't
looked at the code yet, but bayou.io looks interesting and i might try
it next

as for the lies that latency statistic tell, i think it's more a
question of semantics than actual lies. you can take the percentiles
and normalize them (similar to the monte-carlo method) to approximate
the numbers that gil tene seems to want to use. i would like a more
sophisticated tester. ab is a good start, but it can't be scripted and
the 20000 limit on concurrency isn't enough for real tests. an async
java based http client that worked like kws would be a boon, but i'm
not aware of anything

> --
> You received this message because you are subscribed to the Google Groups
> "kilimthreads" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kilimthreads...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.