Eventbus Performance

[Oliver Rolle]

Setup:
* Cluster setup: 1 PingPong Verticle + 1 Monitor Verticle per Vertx Environment; 2 Vertx Environments on 1 Laptop
* PingPong Verticle: listens on a eventbus address, replies on a received message and than sends a newly constructed message to eventbus (with same eb-address and a reply handler); Every received message and received reply counts as 1 message.
* Monitor Verticle: receives duration time till each PingPong Verticle receives 1.000.000 messages.
* 8 messages in the system at a time.
* Measured on a laptop: Core 2 Duo 2.4 GHz (2 Cores), 3GB DDR2 Ram (@1066 MHz); Network IO over loopback device

Non-Cluster (reference values):
* 2 PingPong Verticle + 1 Monitor instance in 1 Vertx Environment
Min:  132187  msgs/sec
Avg:  150773 msgs/sec
Max: 161890 msgs/sec
Network IO: n/a
No reply to messages:
  +70% more msgs/sec (avg: ~260000 msgs/sec)
  Network IO: n/a

Cluster (on same machine):
* 2 PingPong Verticle + 2 Monitor instance in 2 Vertx Environment
Min:  33627 msgs/sec
Avg:  35222  msgs/sec
Max: 41187 msgs/sec
Network IO: 900 - 1000 KB/sec (in both directions)
No reply to messages:
  +4% more msgs/sec (avg: ~36500 msgs/sec)
  Network IO: 600 KB/sec (in both directions)

My conclusion:
Serialization and deserialization of JsonObjects is a very expensive operation and reduces the performance at an avg of 4 times. I would recommend to use Eventbus.registerLocalHandler() as much as possible, to send messages only to local handler if one is available as a general routing strategy and to have a more efficient serialization and deserialization mechanism.

[Daryl Teo]

Hi, thanks very much for your efforts =)

Could you try a similar setup but instead of using JSONObject, use POJO serialization?

In terms of performance, I suspect there isn't much one can do if one wants to communicate with Javascript clients or verticles.

Daryl

[Oliver Rolle]

Hi,

where can I find/configure the POJO serialization? Someone had implemented a BSON Eventbus, maybe I give a try.

I like the idea of your mod-promises! I appended my Parallel implementation to this mail, maybe you could add it to your impl?
How to use it:
parallel = Parallel.Completed(new Handler(){ ... For completed parallel processing ... ])
someAyncFct1(parameters, parallel.getHandler());
someAsyncFct2(parameters, parallel.getHandler());

[Pid]

On 08/01/2013 03:43, Oliver Rolle wrote:
> Hi,
>
> where can I find/configure the POJO serialization?

We don't have a built-in mechanism for this, but you can send a byte[]
which means you can implement serialization however you want (there are
a number of alternative strategies).


p

> Someone had
> implemented a BSON Eventbus, maybe I give a try.
>
> I like the idea of your mod-promises! I appended my Parallel
> implementation to this mail, maybe you could add it to your impl?
> How to use it:
> parallel = Parallel.Completed(new Handler(){ ... For completed parallel
> processing ... ])
> someAyncFct1(parameters, parallel.getHandler());
> someAsyncFct2(parameters, parallel.getHandler());
>
>
> Am Dienstag, 8. Januar 2013 02:04:08 UTC+1 schrieb Daryl Teo:
>
> Hi, thanks very much for your efforts =)
>
> Could you try a similar setup but instead of using JSONObject, use
> POJO serialization?
>
> In terms of performance, I suspect there isn't much one can do if
> one wants to communicate with Javascript clients or verticles.
>
> Daryl
>
> On Tuesday, 8 January 2013 07:09:01 UTC+8, Oliver Rolle wrote:
>

> *Setup:*

> * Cluster setup: 1 PingPong Verticle + 1 Monitor Verticle per
> Vertx Environment; 2 Vertx Environments on 1 Laptop
> * PingPong Verticle: listens on a eventbus address, replies on a
> received message and than sends a newly constructed message to
> eventbus (with same eb-address and a reply handler); Every
> received message and received reply counts as 1 message.
> * Monitor Verticle: receives duration time till each PingPong
> Verticle receives 1.000.000 messages.
> * 8 messages in the system at a time.
> * Measured on a laptop: Core 2 Duo 2.4 GHz (2 Cores), 3GB DDR2
> Ram (@1066 MHz); Network IO over loopback device
>

> *Non-Cluster (reference values):*

> * 2 PingPong Verticle + 1 Monitor instance in 1 Vertx Environment
> Min: 132187 msgs/sec
> Avg: 150773 msgs/sec
> Max: 161890 msgs/sec
> Network IO: n/a
> No reply to messages:
> +70% more msgs/sec (avg: ~260000 msgs/sec)
> Network IO: n/a
>

> *Cluster (on same machine):*

> * 2 PingPong Verticle + 2 Monitor instance in 2 Vertx Environment
> Min: 33627 msgs/sec
> Avg: 35222 msgs/sec
> Max: 41187 msgs/sec
> Network IO: 900 - 1000 KB/sec (in both directions)
> No reply to messages:
> +4% more msgs/sec (avg: ~36500 msgs/sec)
> Network IO: 600 KB/sec (in both directions)
>

> *My conclusion:*

> Serialization and deserialization of JsonObjects is a very
> expensive operation and reduces the performance at an avg of 4
> times. I would recommend to use Eventbus.registerLocalHandler()
> as much as possible, to send messages only to local handler if
> one is available as a general routing strategy and to have a
> more efficient serialization and deserialization mechanism.
>

> --
> You received this message because you are subscribed to the Google
> Groups "vert.x" group.
> To view this discussion on the web, visit
> https://groups.google.com/d/msg/vertx/-/yoKqxFitLhUJ.
> To post to this group, send an email to ve...@googlegroups.com.
> To unsubscribe from this group, send email to
> vertx+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/vertx?hl=en-GB.


--

[key:62590808]

[Daryl Teo]

I was looking into writing an async library that will allow people to do something similar, but I think someone mentioned that it would be difficult to do something like that in a type-safe manner. From the short glance, it would seem that the parallel tasks must all return the same type (E)? 

Daryl

[Daryl Teo]

Yup this was what I was referring to.

[Paulo Lopes]

On Tuesday, January 8, 2013 4:43:33 AM UTC+1, Oliver Rolle wrote:

Hi,

where can I find/configure the POJO serialization? Someone had implemented a BSON Eventbus, maybe I give a try.

Hi there, i implemented a basic BSON EventBus wrapper, you can get it here:
https://github.com/pmlopes/bson.vertx.eventbus

(it is still waiting to be merged in the "official" vertx module repo)

I did not test it for performance, so if you could share your testing code, it would be nice for me to profile and see which parts of the code need optimization.

Cheers,
Paulo

[Sergio Campos Valdés]

Oliver, I have a doubt about your report.
When you said "*No reply to messages*" , are you refering to the *lost messages*?

PS: Thanks for your benchmarks.

2013/1/7 Oliver Rolle <oliver...@googlemail.com>

--

You received this message because you are subscribed to the Google Groups "vert.x" group.

To view this discussion on the web, visit https://groups.google.com/d/msg/vertx/-/HsXltEtKa00J.

To post to this group, send an email to ve...@googlegroups.com.
To unsubscribe from this group, send email to vertx+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/vertx?hl=en-GB.

--
Deseando que Dios le bendiga se despide:

Sergio Andrés Campos Valdés
Ingeniero Informático.
Perfil LinkedIn: http://cl.linkedin.com/in/scamposv
Twitter: chech0x

[Oliver Rolle]

Hi Segio,

The eventbus does NOT loose messages. I tested with a total of 100 Million Messages per test, and not a single message was lost.

"*No reply to messages*" means, that on receive of a message my second test implementation will not call message.reply(JsonObject). I intended to test a different mode of message delivering. As you see it does have an impact on performance.

best regards
Oli

[Oliver Rolle]

Hi Paulo,

I will test it. maybe today or weekend.

best regards
oli

[Sergio Campos Valdés]

Oliver, i now understand, thanks. 

Another question, which vert.x version

did you use?

2013/1/9 Oliver Rolle <oliver...@googlemail.com>

--
 
 

[Oliver Rolle]

Setup:
* vertx 1.3.0 final (download from www.vertx.io)

* Cluster setup: 1 PingPong Verticle + 1 Monitor Verticle per Vertx Environment; 2 Vertx Environments on 1 Laptop
* PingPong Verticle: listens on a eventbus address, replies on a received message and than sends a newly constructed message to eventbus (with same eb-address and a reply handler); Every received message and received reply counts as 1 message.
* Monitor Verticle: receives duration time till each PingPong Verticle receives 1.000.000 messages.
* 8 messages in the system at a time.
* Measured on a laptop: Core 2 Duo 2.4 GHz (2 Cores), 3GB DDR2 Ram (@1066 MHz); Network IO over loopback device

Test: a message send via Eventbus.send() and then use Message.reply() to respond to that message (counts as 2 messages). Do this till 15 or 500 million messages send.

Non-Cluster (reference values; Total 500.000.000 Messages delivered):

* 2 PingPong Verticle + 1 Monitor instance in 1 Vertx Environment

             Orginal        BSON
MIN       111179       146939
AVG      178168       223871 (+25%)
MAX      239348       340425

Non-Cluster (?serialzation performance?; Total 500.000.000 Messages delivered):
* 1 PingPong Verticle + 1 Monitor instance in 1 Vertx Environment
             Orginal        BSON
MIN       152045       265886
AVG      249471       418801 (+67%)
MAX      333555       545553


Cluster (cluster performance; on same machine: Total 15.000.000 Messages delivered):

* 2 PingPong Verticle + 2 Monitor instance in 2 Vertx Environment

             Orginal        BSON
MIN       18798       32199
AVG      36819       49115 (+33%)
MAX      31253       54356
I/O        2 MB/s      2 MB/s

best regards
oli

[Paulo Lopes]

Thanks for your tests, i see that i have work to do :)

Can you share the test app? It would be nice to have it in order to run the tests while profiling and experimenting any potential optimisations :)

Thanks!

Paulo

[Oliver Rolle]

Hi,
I am not happy with my current test implementation bc its messy to switch between non-cluster and cluster mode and is not ready to adapt for the number of cores of the CPU.

https://github.com/orolle/vertx-eventbus-performance

[Oliver Rolle]

I am positively surprised that your impl is 33% faster without much optimization.

I am playing around with the idea to create a fault tolerant distributed complex event processing event bus bc it seems to have much possible use cases and technological opportunities. Interested?

[Paulo Lopes]

On Thursday, January 10, 2013 12:23:38 PM UTC+1, Oliver Rolle wrote:

I am positively surprised that your impl is 33% faster without much optimization.

Wow, i was reading your results wrong, i was under the impression my implementation was 33% slower :)

I do know that i can optimise the case when objects with several child objects are serialized because i do instantiate a new Buffer per object while i could just do it by keeping a reference list or something like that.

Also BSON spec requires a full document length as the first data in the binary format which is kind of useless and forces jumping back and forward in the buffer when it could be possible to decode without that info anyway.

 

I am playing around with the idea to create a fault tolerant distributed complex event processing event bus bc it seems to have much possible use cases and technological opportunities. Interested?

That sounds interesting what kind of serialization are you thinking? does it need to be language agnostic? (like BSON, Thrift, Protocol Buffers) or are you aiming to JVM messaging?

Please share more about it, maybe in a new Thread so we don't go off topic :)

Cheers,

Paulo

[Bruno Bonacci]

I am playing around with the idea to create a fault tolerant distributed complex event processing event bus bc it seems to have much possible use cases and technological opportunities. Interested?

Isn't what Storm does very well? (https://github.com/nathanmarz/storm/wiki)

Bruno 

[Oliver Rolle]

yes, I looked a little bit into storm and it is pretty interesting software!

[Sharat Koya]

Hi has anyone ran an eventbus throughput test like this since on vertx3, and does anyone know how to trap for dropped messages on the eb?

Thanks