[erlang-questions] performance of websocket

[envelopes envelopes]

from:  https://github.com/ericmoritz/wsdemo/blob/master/results.md

It seems Erlang's connection time is a little bit high. 

Summary

Implementation	Connection Time (mean)	Latency (mean)	Messages	Timeouts
Erlang	00.865ms	00.017ms	2849294	0
Java (Webbit)	00.567ms	00.835ms	1028390	157
Go	00.284ms	18.503ms	2398180	225
Node.js	00.768ms	42.580ms	1170847	4299
Python (ws4py)	01.561ms	34.889ms	1052996	5208

Both the Python gevent/ws4py implementation and the Node.js websocket implementation failed hard with around half of the connections hitting the 2 second TCP connection timeout.

I expected Go to kick Erlang's ass in the performance department but the message latency was much higher than Erlang's latency and we had 225 unhappy customers. Go only reached C9.775k; close but no cigar.

I did not know what to expect with the Java Webbit implementation. I expected it to perform somewhere close to the Go implementation. The Webbit implementation did do much better than the Go implementation but it still had 157 connection timeouts which is unacceptable.

I was very surprised that node.js fell down after 5001 connections and that gevent fell down at 4792 connections. Both platforms were specifically built to for the C10k problem and both platforms could barely handle C5k.

[Eric Moritz]

I made a dumb mistake at 3am on this table.  Those ms times are actually seconds.  I've updated the table at this link.  Disregard the table in this email.

https://github.com/ericmoritz/wsdemo/blob/master/results.md

If you look at the raw data you will also notice that the handshake times for 2,500 of the 10,000 connections are greater than 742ms which is pretty lousy as well. While none of the TCP connections timed out, 25% of the connections had to wait at least 743ms to start sending messages.

Once the handshake eventually finished, the per-message latency was really good compared to the others with a mean of 17ms.

Eric.

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Max Lapshin]

My main question is: why Java, Node and python are dropping connections?

[Eric Moritz]

I was wondering the same thing. I haven't had time to investigate the cause.

I stood up a naked m1.medium AWS instance with ubuntu-12.04 and ran this script:

https://raw.github.com/ericmoritz/wsdemo/master/configure_ubuntu.sh

The contents of the sysctl.conf file that I stole from Richard Jones
<http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1>

Is here:

https://github.com/ericmoritz/wsdemo/blob/master/etc/sysctl.conf

I ran the servers and the ./runtest script as root on seperate servers
instances with a ulimit of 999999.

I tried my best to document everything that was done to the servers in
order to trace any faults I had in the process.

[André Graf]

Thank you very much for sharing this!

Cheers

André