Rebalancing zones and nodes

[Jeronimo Rodriguez]

Hi all,

I am currently doing a  maintenance on Voldemort and I have several questions that do not quite understand (I've only been two months using the tool).

The maintenance's requirements is in an initial scenario where we have 2 nodes in one zone and we have to arrive to a final scenario where we have 2 zones and 8 nodes equally balance between this nodes without stopping service. 

When I expand the cluster from 2 to 8 nodes, I don´t have any problem. Rebalance goes without problem and Key samples recovered from rebalance are equals. In the next step from the rebalance plan, when I split the cluster in 2 zones, I used the script ./rebalance-zone-expansion.sh and I´m receiving a VoldemortException:

"Nodes do not have same state[ subset node state (Node mynode.localhost Id:1 in zone 0 with admin port 6689, socket port 6688, and http port 38188) not equal to superset node state (Node mynode.localhost Id:1 in zone 1 with admin port 6689, socket port 6688, and http port 38188) ]"

/**

* Confirms that any nodes from supersetCluster that are in subsetCluster

* have the same state (i.e., node id, host name, and ports). Specific

* partitions hosted are not compared.

* 

* @param subsetCluster

* @param supersetCluster

*/

See on: voldemort / src / java / voldemort / utils / RebalanceUtils#validateClusterNodeState

 

Reading source code, I could appreciate that this is because Voldemort try to move a Node from zone 0 to zone 1 without stopping service. 

My question is:  Is there any form to rebalance nodes from a zone to another zone without stopping service?

thanks in advance.