Difference between DRBD and XtreemFS

[mani.sh...@gmail.com]

Hi,

I have been using DRBD in many projects I have worked on. I do see below issues in DRBD:

1. The initial synch up during initialization takes a lot of time

2. The secondary nodes are not accessible for read/write

3. Sometimes it gets difficult to recover from split brain

I do see below advantages using DRBD:

1. Block replication is very fast

2. On failover, mounting on different node takes hardly any time

3. Synch mode is configurable. Can be async or sync

4. Allows failover handlers

Just wanted to know if XtreemFs can address the DRBD issues. Is there any other major difference in Xtreem and DRBD which I am missing. I am looking at synching multiple files of approx 10 MB size. Can XtreemFS be the replacement fro DRBD?

[Robert Schmidtke]

Hi,

let me try and address your questions one by one to give you a feeling of whether XtreemFS might be right for you. Please bear in mind that I have not worked with DRBD in practice so far.

Please also refer to the user guide if anything is not clear: http://xtreemfs.org/xtfs-guide-1.5.1/index.html

Issues:

1. Starting a new Object Storage Device (OSD, the service running on each node that stores the actual data) and having it working takes practically no time, as there is no synchronization required. New data is added to it only for files that are newly created, or if you re-replicate files to it that are already there. This is done manually.

2. When opening a file, OSDs decide which one becomes the primary. The amount of time that one OSD is a primary can be configured through the expiration of leases.

3. Depending on the replication factor and consistency requirements (e.g. quorum during read or write), certain operations are not possible in that scenario of course. We have not found issues when recovering from that, however.

Advantages:

1. The object size in XtreemFS can be configured and is reasonably fast in my opinion. Again I don't know about DRBD.

2. The Directory Service (DIR) and Metadata Catalog (MRC) do have a replication setting, however there are some issues during failover which may result in an unusable state. This would require manual intervention to restart the nodes. This configuration is therefore considered experimental, and should not be relied upon in production environments. Note that this may pose as a single point of failure in case of the DIR and MRC. There are currently no plans to address this issue, as XtreemFS is a research project and not exactly maintained by specifically hired individuals.

3. XtreemFS supports both synchronous and asynchronous I/O.

4. OSD failover works transparently, for DIR and MRC see 2.

You could set the object size to around 10 MB and do a benchmark. Having no experience with DRBD I cannot really advise for or against a switch to XtreemFS. I hope this helps you a bit.

Cheers

Robert