Re: Transactional queues expected performance

[Talip Ozturk]

I would recommend comparing hazelcast 3 (which is not yet released)
with other solutions. Hazelcast 3 queues are great for fault tolerant
queues, it will scale much better if you have many queues. Hazelcast 3
is in alpha but we should have the beta in 10 days.

-talip

On Tue, May 21, 2013 at 5:53 PM, <jason....@gmail.com> wrote:
> If you want high performance queues I would recommend using something like HornetQ or RabbitMQ. A queue makes for a poor partitioned datastructure in general because of its strong ordering characteristics. I don't mean to imply that Hazelcast is bad in any way, only that you will see consistently better performance from a "real" MQ.
>
> What are you looking to accomplish?
>
> --
> You received this message because you are subscribed to the Google Groups "Hazelcast" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
> To post to this group, send email to haze...@googlegroups.com.
> Visit this group at http://groups.google.com/group/hazelcast?hl=en-US.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Jason Clawson]

In hazelcast 2.5 the expected performance is highly dependent on your environment.  Since hazelcast is an embedded technology it is very much affected by other things the JVM is doing.  I can't speak to how the queue implementation is done in Hazelcast 3.0 (I assume its no longer partitioned, or at least each entry is not partitioned randomly (it could be segmented into contiguous segments)?) but Hazelcast 2.5 partitions the queue across the whole cluster on the entry level using a map.  AFAIK (in 2.5): an enqueue operation has to make at least 2 network calls:  1) talk to the master to establish queue entry order and 2) to store the entry in a hazelcast map + a network call for each backup(s).  I imagine that in hazelcast 3.0 it has gone to more of a mirrored queue approach which is how things like RabbitMQ achieve HA.

We have used hazelcast queues in the past and had issues with them.  I would expect it to be able to handle 100-200 messages a second without issue.  What kind of numbers are you seeing?  If you need to do more than a few hundred a second, use an external MQ like RabbitMQ, HornetQ, or ActiveMQ or try out Hazelcast 3.0.  

Just remember that you always need to keep your hazelcast nodes that store data (non-lite members) healthy and minimize GC pausing because they will affect the entire cluster.

--

Jason Clawson

On Wednesday, May 29, 2013 3:45:21 AM UTC-6, ghep...@gmail.com wrote:

Hello,

thanks a lot for answering.

I am using HZ queues to have a lightweight, fast, reliable, messaging infrastructure, and it is being used in a real-time event-driven system.

Our internal performance tests are showing "strange" results, and our investigations at the moment puts the HZ queues as one of the possible causes of the issues we are observing, thus my question about which are the expected performances of those queues, both in the case of transactional one or non transactional in a cluster environment (say 10 nodes).

Do you have any numbers you can share ?

Thanks in advance.