XtreemFS sustainability (XtreemFS as a project)

[Thomas Debesse]

Hi, I'm looking for a distributed file-system with bidirectional geo-replication abilities. I had a look on glusterfs but I discovered the geo-replication service is not bidirectional.
From what I read (haven't tried yet so I can be wrong) XtreemFS offers bidirectional geo-replication. So, my first question is this one: is it true ?

• Can I have one file-system with two or more geo-replicated mount point in different places of the world allowing everyone to write stuff on the nearest geographic server?

If yes, a second questions comes, the question of XtreemFS sustainability as a project.

I just discovered there is no blog news since 2015 and Quobytes announced former developers of XtreemFS now work on Quobytes instead. I'm not very happy with Quobytes because I want to avoid vendor lock-in. They say on their website
that “users no longer need to worry about vendor lock-in” but it looks like they have'nt noticed yet they became that vendor who locks-in users.

I've seen the GitHub's XtreemFS repository is still pretty active which is very reassuring, and there is recent stuff like debian stretch packages which looks pretty good. By the way, it raises the question of XtreemFS sustainability as a project.

• The footer of xtreemfs.org talks about organizations funding XtreemFS, do these organizations still funding XtreemFS development today?
• Even if Quobytes focus on their Quobytes solution now, do some of their employees still works on XtreemFS maintenance?
  I'm not expecting them to improve XtreemFS but do they at least do by themselves the minimal changes to keep it working on newer systems?
• Is there paid XtreemFS developers today?
  I don't mind who paids them but is there paid people working on XtreemFS today?
• If not, is there some organisations/companies still using XtreemFS today and able to at least do the needed maintenance tasks for themselves?

No need to worry, I just need to know what I get and what I not get, what I can expect and what I can't expect if I go the XtreemFS way.

Since there is already Debian Stretch packages available I know that if I go the XtreemFS way today the setup would at least live for the next 3 years without worrying, which is very OK and tells me choosing XtreemFS today still not look like a risk since it gives me 3 years to migrate to something else if something bad unexpectedly happens to XtreemFS as a project in the future. So, there is no need to worry but I just need to know what I can expect and what I can't expect, just to be fair.

Thanks in advance.

[Robert Schmidtke]

Hi Thomas,

please find answers/comments embedded below. Here is a link to the user guide, which I will be referring to: http://www.xtreemfs.org/xtfs-guide-1.5.1/index.html

Cheers

Robert

On Monday, December 11, 2017 at 4:56:34 PM UTC+1, Thomas Debesse wrote:

Hi, I'm looking for a distributed file-system with bidirectional geo-replication abilities. I had a look on glusterfs but I discovered the geo-replication service is not bidirectional.
From what I read (haven't tried yet so I can be wrong) XtreemFS offers bidirectional geo-replication. So, my first question is this one: is it true ?

• Can I have one file-system with two or more geo-replicated mount point in different places of the world allowing everyone to write stuff on the nearest geographic server?

You can create a single volume, and mount that from anywere on earth. When creating a file, you can choose the data center map based sorting policy (check the guide section 7.3), and have clients write to the nearest OSD. You will need to specify a data center map manually though (as explained in section 7.3.2).

 

If yes, a second questions comes, the question of XtreemFS sustainability as a project.

I just discovered there is no blog news since 2015 and Quobytes announced former developers of XtreemFS now work on Quobytes instead. I'm not very happy with Quobytes because I want to avoid vendor lock-in. They say on their website
that “users no longer need to worry about vendor lock-in” but it looks like they have'nt noticed yet they became that vendor who locks-in users.

As XtreemFS started out as a research project, and the former developers have left, there are currently 1-2 people (depending on their time), using XtreemFS in current research project. However, there are no new features being developed at the moment, so the main additions to future releases will be bug fixes. Please also note that DIR and MRC replication are experimental features, and we are aware of issues during failover. This is why we do not currently recommend using this feature in production.

 

I've seen the GitHub's XtreemFS repository is still pretty active which is very reassuring, and there is recent stuff like debian stretch packages which looks pretty good. By the way, it raises the question of XtreemFS sustainability as a project.

• The footer of xtreemfs.org talks about organizations funding XtreemFS, do these organizations still funding XtreemFS development today?

Funding is secured until fall next year, after which it may be extended. 

• Even if Quobytes focus on their Quobytes solution now, do some of their employees still works on XtreemFS maintenance?

No. 

• I'm not expecting them to improve XtreemFS but do they at least do by themselves the minimal changes to keep it working on newer systems?

They have a separate implementation, which to my understanding is more appropriate for production usage, both because of some major implementation changes, as well as support. Their implementation surely is "future-proof". We, on the other hand, will be doing our best to support new operating systems as they appear (to be honest, usually when a user requests it, we add it to our build service and see how it goes). 

• Is there paid XtreemFS developers today?
  I don't mind who paids them but is there paid people working on XtreemFS today?

Well, I'm being paid for research, and part of my research involves working with XtreemFS. Not sure what you're hinting at with this question? 

• If not, is there some organisations/companies still using XtreemFS today and able to at least do the needed maintenance tasks for themselves?

Since XtreemFS has a liberal license, we're not aware of all installations out there. There are some major sites using XtreemFS as far as we know, but in case of problems they contact this mailing list (as I'm sure you've seen). You're always welcome to fork XtreemFS for yourself and improve it. However I'm not aware of any development activities outside my research institute.