Hot Alignment still not recommended?

[S. Monkey]

The documentation for distributed architecture (http://orientdb.com/docs/last/Distributed-Architecture.html) states that hotAlignment should be set to false as true can lead to inconsistency.

Is this still the case in version 2.0.X (2.0.8 specifically)?

If so, what is the correct way to configure a distributed database? If autoDeploy is set to true then it appears that whenever starting a node that joins a distributed cluster, the database is automatically deployed, even if the node was previously a member of the cluster. For a large database this could be problematic, but it seems this would be necessary without a mechanism like hotAlignment.

[Luca Garulli]

Hi,

Unfortunately hotAlignment is still not working, in the meaning that cannot guarantee 100% consistency of database.

We are working with Hazelcast team for a more efficient solution (more reliable and much faster). This will be available in 2.2 (June 2015) or lately 3.0 (summer 2015).

Lvc@

Best Regards,

Luca Garulli

CEO at Orient Technologies LTD

the Company behind OrientDB

http://about.me/luca.garulli

--

---
You received this message because you are subscribed to the Google Groups "OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eric24]

Is hotAlignment working now in 2.2?

[Luca Garulli]

In v2.2 is not supported anymore in the meaning that hotAlignment is ALWAYS on, look at: https://github.com/orientechnologies/orientdb-docs/blob/master/Release-2.2.0.md#distributed.

Best Regards,

Luca Garulli

Founder & CEO

OrientDB

[Eric24]

Ah, very good. Thanks for that link (I don't think I had seen that page before). A couple of questions about some of those 2.2 details:

1) Is Hazelcast no longer used at all or are Hazelcast queues just not used for intra-node communications?

2) Is client-side load balancing supported on the node.js driver (orientjs)?

3) In a distributed application (say, 3 ODB nodes) where multiple end-client requests need to access and modify the same data (something like a session object in a multi-server web application), I'm concerned about consistency (i.e. end-client A updates a record when connected to node 1, then end-client B queries that record when connected to node 2, but gets stale data from before end-client A's update). I realize this is a common problem for any distributed system, but I don't fully understand ODB's consistency "guarantees". Assuming that writeQuorum is "majority", if node 1 and 2 both successfully commit the update, it is my understanding that end-client B will get the updated data (just as if it had connected to node 1), but what if node 2 fails the update (but node 3 successfully commits, thereby satisfying the writeQuorum)? What other scenarios should I watch out for (I'm keying on the comment about consistency mentioned in relation to per-request round-robin, but not per-connection round-robin, and I'm not sure why this would be particularly different)? Also, if the update is to just one record, is an explicit transaction of any kind needed here?

[Luca Garulli]

On 8 May 2016 at 14:49, Eric24 <er...@24x8.com> wrote:

Ah, very good. Thanks for that link (I don't think I had seen that page before). A couple of questions about some of those 2.2 details:

1) Is Hazelcast no longer used at all or are Hazelcast queues just not used for intra-node communications?

We still use HZ for:

• discovery of nodes
• configuration (HZ maps)
• distributed locks

But for intra-communications we drop them in favour of something custom: our binary protocol. Performance with 3 nodes are about 10x in v2.2 in comparison to v2.1!

2) Is client-side load balancing supported on the node.js driver (orientjs)?

OrientJs driver supports HA, so in case of crash it's able to automatically switch to another available node, but load-balancing is not yet implemented.

 

3) In a distributed application (say, 3 ODB nodes) where multiple end-client requests need to access and modify the same data (something like a session object in a multi-server web application), I'm concerned about consistency (i.e. end-client A updates a record when connected to node 1, then end-client B queries that record when connected to node 2, but gets stale data from before end-client A's update). I realize this is a common problem for any distributed system, but I don't fully understand ODB's consistency "guarantees". Assuming that writeQuorum is "majority", if node 1 and 2 both successfully commit the update, it is my understanding that end-client B will get the updated data (just as if it had connected to node 1), but what if node 2 fails the update (but node 3 successfully commits, thereby satisfying the writeQuorum)? What other scenarios should I watch out for (I'm keying on the comment about consistency mentioned in relation to per-request round-robin, but not per-connection round-robin, and I'm not sure why this would be particularly different)? Also, if the update is to just one record, is an explicit transaction of any kind needed here?

You're free to use or not transactions, so single create/update/delete can be executed distributed without transactions.

 

Now in the scenario you pictured, a client could read a stale data during distributed transaction commit/rollback. We decided to leave it like this to avoid slowing down too much reads, but it's straightforward implementing the reads that wait for locked resources in order to guarantee a stale record is never read.

[Eric Lenington]

Thanks for the details Luca. Just out of curiosity, are you planning to phase out HZ over time or are you generally satisfied with this solution?

Of course the challenge with load-balancing in a distributed environment (and probably moreso with a writeQuroum < totalNodes) is the trade-off between speed and "overhead" between nodes. In my experience, this is something that almost has to be left to the application (i.e. settable on a per-connection or maybe per-request basis), because without understanding what the application is trying to do with the underlying data, there really isn't any way for the database to make the decision. Is this your position as well, or do you feel that there is a better approach that can come closer to "automating" this?

You say "it's straightforward implementing the reads that wait for locked resources in order to guarantee a stale record is never read". Is this something I can do now in my code or something that could be done within ODB or the orientjs driver?

--Eric

--

---
You received this message because you are subscribed to a topic in the Google Groups "OrientDB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/orient-database/5NCen4Zxfjs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to orient-databa...@googlegroups.com.

[Luca Garulli]

Hi Eric,

HZ will be there for a while. For what we use it now, it's a perfect fit, so no reason to completely remove it.

About last question it's something we can do in OrientDB. I've just opened an issue for it: https://github.com/orientechnologies/orientdb/issues/6102

Best Regards,

Luca Garulli

Founder & CEO

OrientDB