ZooKeeper distributed lock in hbase

[Pari]

Hi Friends, I am trying to find how to use zookeeper lock in hbase. Hbase has rowkey lock functionality. Similarly I am trying with zookeeper. I would request somebody to give input on this. It would be helpful for me.

Pari.

[Ashwanth Kumar]

Have a look at Recipes under "Locks" section on http://curator.incubator.apache.org/curator-recipes/index.html

On Wed, Jun 26, 2013 at 9:12 PM, Pari <pari.ma...@gmail.com> wrote:

Hi Friends, I am trying to find how to use zookeeper lock in hbase. Hbase has rowkey lock functionality. Similarly I am trying with zookeeper. I would request somebody to give input on this.  It would be helpful for me.

Pari.

--
You received this message because you are subscribed to the Google Groups "Hadoop Users Group (HUG) Chennai" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chennaihug+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Ashwanth Kumar / ashwanthkumar.in

[Prasad S]

Hi Pari,

Can you explain a bit more on what scenario you want this zookeeper locking for HBase? I have used a similar approach for Cassandra since it does not have explicit row locking.

Regards,

Prasad

[Pari]

Hi Ashwanth,

                           Thank you for your reply and input. I will check that link and let you know.

Hi Prasad,
                          Thank for your reply. What I am trying to achieve is:

If you want update a row in hbase table in multi-user environment, one user has to explicitly make rowlock and update the row and release the lock. Like mentioned in this example: https://github.com/larsgeorge/hbase-book/blob/master/ch03/src/main/java/client/RowLockExample.java

Even hbase implicitly hold the lock when we try to update any row in hbase table. But i want to achieve the explicit lock on hbase table using zookeeper distributed lock. Please let me know if you have any reference link or any inputs. Thanks.

Pari.M

[Prasad S]

Hi Pari,

For this scenario, the first thing to understand is about the zookeeper distributed locking mechanism. 

Further in this case, I would try to do something like this.

1. Any user who wants to update a row in hbase will check for the znode with corresponding row key whose row he needs to update within the zookeeper.

2. If there is no znode available, then he would create a znode and then obtain the lock on the node.

3. Then he would go with the update of the row and finally release the lock at exit.

4. Incase, he sees a znode which is available already he would try to get a lock. If the lock is already acquired then he would register himself for the release of the lock and continues to wait.

5. Once the lock is released, he will acquire znode and continue the process.

6. Care should be taken to use ephemeral type to create the znode to ensure the node is deleted incase the client disconnects halfway without proper release of the lock.

All the best. Anyways I dont have a pseudo code for this. Happy coding!! :)

Regards,

Prasad

[Pari]

Hi Prasad,

                       Thank you for the quick reply and your inputs. I will check the same.

Pari.

[Pari]

Hi Prasad,

                        I just gone through the basics of zookeeper and followed this link for distributed lock for simple implement: http://architects.dzone.com/articles/distributed-lock-using

I hope, I have moved one step forward, I have written simple program to update row in a hbase table, so as you mentioned above I have created znode(used rowkey for znode path) in zookeeper and locked using distributed lock before Put operation. I was able to block the other concurrent update for same rowkey before I release the lock. I didn't solve this "Herd Effect" issue, I am just learning about it.

Thank you for your suggestions.

Pari.M

[Prasad]

Hi Pari,

Thats in turn a good effort put for the learning. :) Let me know if you are able to solve the issue. Else I have some other solution in my mind which I will share if you need.

Regards,

Prasad

[Pari]

Hi Prasad,

                       Please share your views, I am just going through some links & blogs to solve this issue.

Thanks for your inputs.

Pari.

[Prasad]

Hi Pari,

An approach which I can think of is

1. All clients those wanting to update a specific row will have to create a znode under the parent (rowkey) znode.

2. You need to ensure that all znodes created are sequential ephimeral type so that there will be an increased numbering with the znode being created.

3. After which the client has to list for children in parent node. 

4. If there are no children available, its obvious that it is the first to propose for row updation and so it can update the row.

5. But if there are children available, then this client will put a watch on the last created znode present under the parent node. 

6. So when the first client goes down after updating row, it deletes the node created and this will trigger the watch on the next registered client and the process will go on.

7. This ensures that we handle herd effect with not making the watch activated same time to all the registered clients on the rowkey node.

Let me know if you have further thoughts on this approach.

[Pari]

Hi Prasad,

                     Thank you. I will check this and I will let you know.

Many thanks.

Pari.

[Prasad]

Any progress on this Pari?

[Pari]

Hi Prasad,

                      Sorry for late update. I was engaged with other work. Still I didn't work on "herd effect" issue. I will let you once I work on the same.

Thanks & Regards,

Pari.M