Message System without persistence and security?

[Matteo Cusmai]

Hi all,
i am learning something about ZeroMQ that is the message system on which Storm is based.
Now, i have understood that 0MQ is powerful, very fast and simple to use, but it seems to have some advantages:

1. no message persistence;
2. no security and encryption layer;
3. furthermore, since JZMQ is a JNI wrapper on ZMQ c lib, there is a additional throughput in marshal and un-marshal data from java 2 c and vice versa.

My questions are:

1. Which are the pros to use 0MQ?
2. Do the above cons have negligible importance?
   
3. finally, is it pluggable into Storm architecture a different Message System, such as AMQP or MQTT.

Cheers,

Matteo.

[Yu Dongmin]

My understanding on the storm is the messages between the spouts and bolts doesn't need persistence.  

Because storm guarantees message processing by itself and  guaranteeing message processing is nothing to do with the persistence. 

So the persistence will not be an issue of the messaging in the storm.

Also a broker having system can cause a problem at the reliability and the scalability.

I don't have any experience on MQTT but AMQP must be slow for the real time processing in the storm. 

I think any alternative messaging will be great (or pluggable) if it has no broker and is faster than 0MQ.

But the security might have to be considered at anyhow.

Thanks

Min

[Matteo Cusmai]

Ok, the persistence could be handled at app side, but security in communications should be a service provided by message system.
Regarding performance, it should be interesting to compare 0MQ plus marshal and un marshal at jni side, with rabbitmq.
However, I think to have pluggable message service will allow us to choice the right system for specific context.
Bye,
Matteo.

[Nathan Marz]

We're planning to replace 0MQ with a pure Java solution: https://github.com/nathanmarz/storm/issues/372

There are other tickets open to allow you to hook into the serializer in order to add encryption.

--
Twitter: @nathanmarz
http://nathanmarz.com