OrientDB and replication, scalability and fault-tolerance: comments please!
724 views
Skip to first unread message
Luca Garulli
Sep 16, 2010, 6:49:01 AM9/16/10
to orient-database
Seems that the most missed feature in OrientDB is the support for clustering, and therefore high scalability, high availability and high volume of transactions that a single node can't handle. In the last months I studied the whole different architectures of other NoSQL solutions for clustering and I can say that the preferred until now is something similar to Mongo-DB approach with Master/Slaves architecture.
The current work-in-progress release 0.9.23 provides the first version of Replication in OrientDB. The features are:
- Master-Slaves type, where it can be only one Master and N Slaves. If the Master crashes a Slave is elected to be the new Master
- IP multicast to discover cluster nodes
- Configuration of nodes using TCP/IP, useful for Clouds that don't allow the IP multicast
- Two sync modes: full where all the database is compressed and sent over the network, and partial by sending only the changes happened since the last sync
- New database handled by the Master OrientDB Server instance to store all the pending records until a configurable threshold. Up this threshold the logs are deleted and the node need a full-sync on startup
- New console commands to display nodes, listen clustering messages and elect the master manually
What do you think? I'd like to receive your comments, thoughts, ideas and, why not, critiques.
Ciao,
Luca Garulli
David Marko
Sep 16, 2010, 10:56:03 AM9/16/10
to OrientDB
I have an experience with MongoDB and master/slave replication is very
nice feature and what is important that it is very easy to setup. One
can easily setup slave that works as on-line backup on some spare
machine.
Its really must-have feature!!!
David
On 16 zář, 12:49, Luca Garulli <l.garu...@orientechnologies.com>
wrote:
> - Configuration of nodes using TCP/IP, useful for Clouds that don't allow
> the IP multicast
> - Two sync modes: full where all the database is compressed and sent over
nice feature and what is important that it is very easy to setup. One
can easily setup slave that works as on-line backup on some spare
machine.
Its really must-have feature!!!
David
On 16 zář, 12:49, Luca Garulli <l.garu...@orientechnologies.com>
wrote:
> Seems that the most missed feature in OrientDB is the support for
> clustering, and therefore high scalability, high availability and high
> volume of transactions that a single node can't handle. In the last months I
> studied the whole different architectures of other NoSQL solutions for
> clustering and I can say that the preferred until now is something similar
> to Mongo-DB approach with Master/Slaves architecture.
>
> The current work-in-progress release 0.9.23 provides the first version of
> Replication in OrientDB. The features are:
>
> - Master-Slaves type, where it can be only one Master and N Slaves. If
> clustering, and therefore high scalability, high availability and high
> volume of transactions that a single node can't handle. In the last months I
> studied the whole different architectures of other NoSQL solutions for
> clustering and I can say that the preferred until now is something similar
> to Mongo-DB approach with Master/Slaves architecture.
>
> The current work-in-progress release 0.9.23 provides the first version of
> Replication in OrientDB. The features are:
>
> the Master crashes a Slave is elected to be the new Master
> - IP multicast to discover cluster nodes
> - Configuration of nodes using TCP/IP, useful for Clouds that don't allow
> the IP multicast
> - Two sync modes: full where all the database is compressed and sent over
> the network, and partial by sending only the changes happened since the last
> sync
> - New database handled by the Master OrientDB Server instance to store
> sync
> all the pending records until a configurable threshold. Up this threshold
> the logs are deleted and the node need a full-sync on startup
> - New console commands to display nodes, listen clustering messages
> the logs are deleted and the node need a full-sync on startup
Lvc@
Sep 17, 2010, 6:58:46 AM9/17/10
to OrientDB
Yes,
after a look to all the different solutions I found the MongoDB way
the best as cost/complexity/performance/scalability. Obviously only
the main Replication concepts will be taken from MongoDB
while the OrientDB implementation will have special features as
always.
bye,
Luca Garulli
after a look to all the different solutions I found the MongoDB way
the best as cost/complexity/performance/scalability. Obviously only
the main Replication concepts will be taken from MongoDB
while the OrientDB implementation will have special features as
always.
bye,
Luca Garulli
secmask
Sep 18, 2010, 4:46:45 AM9/18/10
to OrientDB
Sound great, that's the most feature I'm waiting for :)
Jani Laakso
Sep 18, 2010, 5:04:49 AM9/18/10
to OrientDB
The critical issue for me is this: http://code.google.com/p/orient/issues/detail?id=115
I guess it falls somewhat under the category of fault-tolerance
mentioned in this thread.
BR / Jani
I guess it falls somewhat under the category of fault-tolerance
mentioned in this thread.
BR / Jani
Luca Garulli
Sep 19, 2010, 5:57:50 PM9/19/10
to orient-database
Hi,
no fault-tolerance, it's a known bug on schema change. Follow the issue thread to know more and how to bypass it.
bye,
Luca Garulli
Guizmo
Oct 1, 2010, 3:27:46 AM10/1/10
to OrientDB
I were thinking OrientDB already have ha capabilities :) as the main
page delivers "The transactional engine can run in distributed systems
supporting up to 9.223.372.036 Billions of records for the maximum
capacity of 19.807.040.628.566.084 Terabytes of data distributed on
multiple disks in multiple nodes."
When thinking about mongo and if you follow its achitecture (http://
www.mongodb.org/display/DOCS/Sharding+Introduction) based concepts,
would it be possible to merge what they call "mongos" and "config
servers". I'm not sure to well understand why they split the two as :
- it seems more complex
- it surely demultiplies the numbers of needed servers, it's really a
farm of vps for sure :), i don't imagine with dedicated servers :)
- it adds network traffic and delay latency
- so in apparence 3 reasons which bring another one : it's clearly
more expensive, particulary in a hosting context whatever cloud,
hybrid cloud or dedicated servers.
These config servers make me think of the meta-data servers of several
distributed parallel fault-tolerant file systems. I mean it's great in
some ways but also have quite some requirements the list of
simplicity. I prefer the approach of the more and more popular
GlusterFS competitor with no metadata servers (maybe some ideas to
retrieve here in the field of distributed file systems which can be
good teacher and glusterfs a good inspiration as it's quite unique).
Regarding the "one Master and N Slaves", maybe an idea for OrientDB
(and finally i may missunderstand something and Mongo already do
that) : the possibility to run several servers per machine (vm or
dedicated) and avoiding a salve being on the same machine as the
master of course. That would calm down the number of needed machine.
They call this "partitionned servers" in Lotus notes environments.
Is auto-sharding in the works ?
Will OrientDB keep its ACID qualities in a clustered context (whatever
it is document or graph oriented) ?
Does an Orient database must be compacted periodically due to MVCC ?
If yes, can it be done even when the db is in use ?
One other thing to keep from Mongo or Sedna is the language binding
idea with a binary protocol as the rest approach is perfectly cool for
slow performance :)
so php client support would be welcomed :)
On 16 sep, 12:49, Luca Garulli <l.garu...@orientechnologies.com>
wrote:
> - Configuration of nodes using TCP/IP, useful for Clouds that don't allow
> the IP multicast
> - Two sync modes: full where all the database is compressed and sent over
page delivers "The transactional engine can run in distributed systems
supporting up to 9.223.372.036 Billions of records for the maximum
capacity of 19.807.040.628.566.084 Terabytes of data distributed on
multiple disks in multiple nodes."
When thinking about mongo and if you follow its achitecture (http://
www.mongodb.org/display/DOCS/Sharding+Introduction) based concepts,
would it be possible to merge what they call "mongos" and "config
servers". I'm not sure to well understand why they split the two as :
- it seems more complex
- it surely demultiplies the numbers of needed servers, it's really a
farm of vps for sure :), i don't imagine with dedicated servers :)
- it adds network traffic and delay latency
- so in apparence 3 reasons which bring another one : it's clearly
more expensive, particulary in a hosting context whatever cloud,
hybrid cloud or dedicated servers.
These config servers make me think of the meta-data servers of several
distributed parallel fault-tolerant file systems. I mean it's great in
some ways but also have quite some requirements the list of
simplicity. I prefer the approach of the more and more popular
GlusterFS competitor with no metadata servers (maybe some ideas to
retrieve here in the field of distributed file systems which can be
good teacher and glusterfs a good inspiration as it's quite unique).
Regarding the "one Master and N Slaves", maybe an idea for OrientDB
(and finally i may missunderstand something and Mongo already do
that) : the possibility to run several servers per machine (vm or
dedicated) and avoiding a salve being on the same machine as the
master of course. That would calm down the number of needed machine.
They call this "partitionned servers" in Lotus notes environments.
Is auto-sharding in the works ?
Will OrientDB keep its ACID qualities in a clustered context (whatever
it is document or graph oriented) ?
Does an Orient database must be compacted periodically due to MVCC ?
If yes, can it be done even when the db is in use ?
One other thing to keep from Mongo or Sedna is the language binding
idea with a binary protocol as the rest approach is perfectly cool for
slow performance :)
so php client support would be welcomed :)
On 16 sep, 12:49, Luca Garulli <l.garu...@orientechnologies.com>
wrote:
> Seems that the most missed feature in OrientDB is the support for
> clustering, and therefore high scalability, high availability and high
> volume of transactions that a single node can't handle. In the last months I
> studied the whole different architectures of other NoSQL solutions for
> clustering and I can say that the preferred until now is something similar
> to Mongo-DB approach with Master/Slaves architecture.
>
> The current work-in-progress release 0.9.23 provides the first version of
> Replication in OrientDB. The features are:
>
> - Master-Slaves type, where it can be only one Master and N Slaves. If
> clustering, and therefore high scalability, high availability and high
> volume of transactions that a single node can't handle. In the last months I
> studied the whole different architectures of other NoSQL solutions for
> clustering and I can say that the preferred until now is something similar
> to Mongo-DB approach with Master/Slaves architecture.
>
> The current work-in-progress release 0.9.23 provides the first version of
> Replication in OrientDB. The features are:
>
> the Master crashes a Slave is elected to be the new Master
> - IP multicast to discover cluster nodes
> - Configuration of nodes using TCP/IP, useful for Clouds that don't allow
> the IP multicast
> - Two sync modes: full where all the database is compressed and sent over
> the network, and partial by sending only the changes happened since the last
> sync
> - New database handled by the Master OrientDB Server instance to store
> sync
> all the pending records until a configurable threshold. Up this threshold
> the logs are deleted and the node need a full-sync on startup
> - New console commands to display nodes, listen clustering messages
> the logs are deleted and the node need a full-sync on startup
Luca Garulli
Oct 1, 2010, 10:59:10 AM10/1/10
to orient-database
Hi,
the first release of the distributed version of OrientDB doesn't provide auto-sharding, but it's planned for the future.
ACID transactions are guaranteed in the cluster version since everything happen only into the master node.
For the compacting OrientDB already recycles automatically holes in segments. However during the time there could be need a defragmentation. Now is in aplha status but it works while the database is opened, so it not requires stops.
About the binary protocol, OrientDB already has its own. Before to start the implementation I've tried Google Buffers with Apache Mina and JBoss Netty but at the end micro-benchmarks drove to me to implement from ground on top of raw TCP/IP sockets. It's really fast.
bye,
Lvc@
Guizmo
Oct 3, 2010, 7:02:30 PM10/3/10
to OrientDB
Thanks for the explanations.
Interesting for the auto-sharding, i'm not in a hurry for it but to
have even a vague idea, will it be
in a 2.x or in a 1.x release ? that helps to know if it's planned in
one or two years :)
For ACID, i were thinking of multi-master node, so my question is
asked too soon :)
Good news for the compact operations and also interesting you
considered
Google Buffers with Apache Mina and JBoss Netty, good references.
On the other hand, Thrift would have great added value for introducing
a range of language clients for OrientDB, is it a thing you can
consider ?
Best,
Alex
On 1 oct, 16:59, Luca Garulli <l.garu...@orientechnologies.com> wrote:
> Hi,
> the first release of the distributed version of OrientDB doesn't provide
> auto-sharding, but it's planned for the future.
>
> ACID transactions are guaranteed in the cluster version since everything
> happen only into the master node.
>
> For the compacting OrientDB already recycles automatically holes in
> segments. However during the time there could be need a defragmentation. Now
> is in aplha status but it works while the database is opened, so it not
> requires stops.
>
> About the binary protocol, OrientDB already has its own. Before to start the
> implementation I've tried Google Buffers with Apache Mina and JBoss Netty
> but at the end micro-benchmarks drove to me to implement from ground on top
> of raw TCP/IP sockets. It's really fast.
>
> bye,
> Lvc@
>
Interesting for the auto-sharding, i'm not in a hurry for it but to
have even a vague idea, will it be
in a 2.x or in a 1.x release ? that helps to know if it's planned in
one or two years :)
For ACID, i were thinking of multi-master node, so my question is
asked too soon :)
Good news for the compact operations and also interesting you
considered
Google Buffers with Apache Mina and JBoss Netty, good references.
On the other hand, Thrift would have great added value for introducing
a range of language clients for OrientDB, is it a thing you can
consider ?
Best,
Alex
On 1 oct, 16:59, Luca Garulli <l.garu...@orientechnologies.com> wrote:
> Hi,
> the first release of the distributed version of OrientDB doesn't provide
> auto-sharding, but it's planned for the future.
>
> ACID transactions are guaranteed in the cluster version since everything
> happen only into the master node.
>
> For the compacting OrientDB already recycles automatically holes in
> segments. However during the time there could be need a defragmentation. Now
> is in aplha status but it works while the database is opened, so it not
> requires stops.
>
> About the binary protocol, OrientDB already has its own. Before to start the
> implementation I've tried Google Buffers with Apache Mina and JBoss Netty
> but at the end micro-benchmarks drove to me to implement from ground on top
> of raw TCP/IP sockets. It's really fast.
>
> bye,
> Lvc@
>
Reply all
Reply to author
Forward
0 new messages