Hazelcast on HPC with RDMA

[Jonas P]

Hi all,

I use Hazelcast for my HPC applications. Supports Hazelcast SPD with RDMA? Or other techniques for HPC optimizations?

I only found this:

https://hazelcast.com/resources/live-webinar-extreme-network-performance-hazelcast-torusware/

However, with my applications (and official Hazelcast examples), speedus is slower. Is torusware still recommanded for Hazelcast? With which Hazelcast-application speedus is faster?

Thanks and cheers,

Jonas

[Noctarius]

Hey Jonas,

We didn’t do any performance testing with torusware anymore but we massively improved our internal layers responsible for operation and socket handling. Between 3.5 and 3.8 there’s an x-speedup just by updating Hazelcast itself. That said it is very well possible your observation is correct and Hazelcast + torusware is not faster anymore than Hazelcast plain.

One thing to consider for highest throughput though is to use for example Solarflare cards to bypass the Linux TCP kernel stack and have everything handled in hardware. Latency and throughput will show improvements but the tradeoff of more money to spend on the hardware.

In general I would always start with plain Hazelcast (less components) and see how far it’ll get me and then, when there are bottlenecks, to solve them one by one.

Chris

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/686a113f-4015-4cf1-959c-1384a33ed68c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Peter Veentjer]

We don't have support for SDP.

What is your application doing? Have you figured out where the time is spend? What kind of hardware are you running on? 

I'm using perf in combination with flamegraphs to get very actionable information.

http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html

On Wed, Feb 22, 2017 at 6:35 PM, 'Noctarius' via Hazelcast <haze...@googlegroups.com> wrote:

Hey Jonas,

We didn’t do any performance testing with torusware anymore but we massively improved our internal layers responsible for operation and socket handling. Between 3.5 and 3.8 there’s an x-speedup just by updating Hazelcast itself. That said it is very well possible your observation is correct and Hazelcast + torusware is not faster anymore than Hazelcast plain.

One thing to consider for highest throughput though is to use for example Solarflare cards to bypass the Linux TCP kernel stack and have everything handled in hardware. Latency and throughput will show improvements but the tradeoff of more money to spend on the hardware.

In general I would always start with plain Hazelcast (less components) and see how far it’ll get me and then, when there are bottlenecks, to solve them one by one.

Chris

On 22 Feb 2017, at 10:23, Jonas P <jop...@gmail.com> wrote:

Hi all,

I use Hazelcast for my HPC applications. Supports Hazelcast SPD with RDMA? Or other techniques for HPC optimizations?

I only found this:

https://hazelcast.com/resources/live-webinar-extreme-network-performance-hazelcast-torusware/

However, with my applications (and official Hazelcast examples), speedus is slower. Is torusware still recommanded for Hazelcast? With which Hazelcast-application speedus is faster?

Thanks and cheers,

Jonas

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+unsubscribe@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/686a113f-4015-4cf1-959c-1384a33ed68c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+unsubscribe@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/1FB659FE-2955-4419-BE7E-55E22412675A%40googlemail.com.

[Jonas P]

Thanks for the answers.

I am running the applications on a cluster, each node has a Intel XeonProcessor E5-2680 and 64GB RAM. The nodes are connected with FDR-14 InfiniBand. The hardware is fix and I am not able to modifiy it.

I have a task pool system with a work stealing algorithm, a bottleneck is rejected steal requests. They could be reduced when checking before sending a request whether work is avaible with RDMA. 

With Solarfale cards you mean http://www.openonload.org/ ?

cheers,

Jonas

Am Mittwoch, 22. Februar 2017 17:41:32 UTC+1 schrieb peter veentjer:

We don't have support for SDP.

What is your application doing? Have you figured out where the time is spend? What kind of hardware are you running on? 

I'm using perf in combination with flamegraphs to get very actionable information.

http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html

On Wed, Feb 22, 2017 at 6:35 PM, 'Noctarius' via Hazelcast <haze...@googlegroups.com> wrote:

Hey Jonas,

We didn’t do any performance testing with torusware anymore but we massively improved our internal layers responsible for operation and socket handling. Between 3.5 and 3.8 there’s an x-speedup just by updating Hazelcast itself. That said it is very well possible your observation is correct and Hazelcast + torusware is not faster anymore than Hazelcast plain.

One thing to consider for highest throughput though is to use for example Solarflare cards to bypass the Linux TCP kernel stack and have everything handled in hardware. Latency and throughput will show improvements but the tradeoff of more money to spend on the hardware.

In general I would always start with plain Hazelcast (less components) and see how far it’ll get me and then, when there are bottlenecks, to solve them one by one.

Chris

On 22 Feb 2017, at 10:23, Jonas P <jop...@gmail.com> wrote:

Hi all,

I use Hazelcast for my HPC applications. Supports Hazelcast SPD with RDMA? Or other techniques for HPC optimizations?

I only found this:

https://hazelcast.com/resources/live-webinar-extreme-network-performance-hazelcast-torusware/

However, with my applications (and official Hazelcast examples), speedus is slower. Is torusware still recommanded for Hazelcast? With which Hazelcast-application speedus is faster?

Thanks and cheers,

Jonas

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/686a113f-4015-4cf1-959c-1384a33ed68c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.