Interested in contributing omid

[Jian Fang]

Hi everyone,

I’m a master student from Carnegie Mellon. I found yahoo omid is quite interesting. So I want to make some contribution to this project. 

I read the MegaOmid page: https://github.com/yahoo/omid/wiki/MegaOmid . I think this idea is interesting. In that wiki page, it says MegaOmid is lacking two features now:

1. The recovery procedure for the global transactions is not integrated into recovery of local transactions.

2. The dynamic partitioning and dynamic status oracle membership is not implemented.

I’m considering to make some contributions on these two features. So I wrote this email to ask you guys whether or not these two features are worthy implementing. If so, could you give me some suggestions about how can I start and what materials I should read first (related papers, discussion or projects). I’ll appreciate your assistance.

BTW: I want to take this work as a course project (15-648 Big Data Systems Studio, taught by Garth Gibson http://www.cs.cmu.edu/~15648/)

Best Regards

________________________________

Jian Fang
Master Student at Carnegie Mellon University
ji...@cs.cmu.edu
+1(412)-608-2733

[Matthieu Morel]

Hello,

this would certainly be a great contribution, and it deals with various issues and tradeoffs for scaling out a system.

I'm not sure the megaomid branch is fully working nor in line with the master branch. You might need to rebase on top of the master branch and update a few things. Also note that you may develop/test in isolation from HBase.

As for reading material, the latest and most exhaustive paper about omid has been accepted at ICDE'14 but is not yet published (174 in http://ieee-icde2014.eecs.northwestern.edu/accepted.html). A related work would be Google's Percolator, though this uses locks written to the database whereas omid uses a lock-free approach.

You might also want to watch Daniel's presentation at Hadoop summit '13 here http://www.youtube.com/watch?v=Rhdmo9pVGgU

Regards,

Matthieu