Question: vert.x instances across multiple datacenters

[Amit Kumar]

1. I have two vert.x instances running on host-1 and host-2 in datacenter-1. These are clustered vert.x instances within datacenter-1.

2. I have another vert.x instance running on a host-2 in datacenter-2 (different datacenter). Is it possible to cluster this instance with the instances running in datacenter-1 ?

Need some guidelines for being able to cluster vert.x instances across multiple datacenter. Is it possible ? Any restrictions ? I believe Hazelcast supports clustering across multiple datacenter. Is there any special configuration I need to consider while doing it under vert.x ?

Your response much appreciated.

[Amit Kumar]

Experts, any input on this please ?

[Ryan Chazen]

vert.x uses hazelcast to find and manage which nodes are connected, and then regular tcp to connect the eventbus across the cluster. So there is no problem with multiple data centers that I can think of other than increased latency when processing event bus messages sent between data centers (should still be quick though, communication between data centers is reliable because they sit on such big pipes). But I think you could have variable time answering requests then - eg, request one sends a message on the eventbus to ask for data and gets routed to a local verticle and completes in 5ms. Request two gets routed to the next data center and takes 50ms. I could be wrong on this though, I'm not 100% sure on how the routing is chosen.

So the only thing you'd need to do is configure hazelcast to pick up all your servers across the different data centers. More info here: http://vertx.io/manual.html#configuring-clustering
In particular you'd want to set multicast enabled="false", tcp-ip enabled="true", and manually add the ips of some of your servers in different data centers.

[Morten Jensen]

Hey,

A follow-up question on this spurred on by the original clustering question. If the separate data centres aren't layer-2/VLAN stretched but are routed instead (so separate VLANs & networks/IP ranges - e.g nodes in separate Amazon AWS regions) how does hazelcast do node discovery? Are there multicast requirements and/or can you add known node IP addresses and/or ranges to configuration?

How does dynamic node discovery (as in spin up new instance - join existing cluster) work?

Not sure if this is one for vert.x - and if not, where would be the best source of this info?

Thanks.

Best regards,

Morten Jensen

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/3k6m1uGxbUc/unsubscribe.
To unsubscribe from this group and all of its topics, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Tim Fox]

For this kind of setup I wouldn't really recommend using Hazelcast to try and form a single cluster over separate LANs (not even sure if it's possible), especially if those LANs are linked by potentially unstable connections (e.g. WAN).

A Vert.x cluster is only really intended for use on a single reliable LAN. For connecting between separate clusters, I'd recommend writing a module which bridges the two, i.e. something that listens on the event bus and forwards traffic to the other cluster over a TCP connection(s). You can then use the inbuilt reconnect logic in the net client to handle the cases where the connection is down.

I don't imagine it would be too hard to write, it's something that would be really nice as a module that can be reused by others.

You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

[Morten Jensen]

Sorry, got my 2nd response posted to the wrong thread (see bottom). It indeed appears that hazelcast supports clustering across data centres.

From my perspective I was hoping to achieve a cluster of vert.x instances in AWS in a particular region (not inter-region) across availability zones. This mandates different networks with routing in between (as opposed to a single sub-net). Network latencies would still be in the low milliseconds and bandwidth in the 10's of MB(yte)/sec.

Tim, certainly your suggestion of proxying requests between clusters is an option but it makes for added complexity and a different handling of messages across the event bus. Do you know of anyone successfully clustering vert.x/hazelcast in AWS between availability zones (not regions)?

===========

Ah, I believe I have just potentially answered my own question - multicast per default but "full TCP/IP cluster" possible: https://code.google.com/p/hazelcast/wiki/ConfigFullTcpIp

So really my only question is: are there any vert.x constraints to clustering that do not exist in hazelcast?

Thanks.

Morten

[Amit Kumar]

Thank you all for your answers.

Tim, I can understand your suggestion on "not considering communication across unstable WAN links". However, Hazelcast seems to claim working across datacenter boundaries.

Again, thank you all for your guidance here.

[javadevmtl]

vertx doesn't use hazelcast for actual data transfer. It only uses hazelcast for node discovery. Once vertx has a list of ips it uses its own tcp/ip to send msgs. So doesnt matter what hazelcast can and canot do because its not used ;)