RowLog vs HBase CoProcessors

[svarma]

This mail is to understand your future roadmap in terms of RowLog.

With the new HBase CoProcessors feature, and with its ability to
create RegionObservers and such, do you still see a need for the
RowLog implementation going forward? Or are you planning on re-
implementing RowLog using Co Processors?

We wanted a mechanism to trigger Solr index updates in response to
HBase table updates and were evaluating using RowLog; we found Co
Processors to be another implementation that might be a potential
solve.

So - just wondering if you too see similarities / plan to migrate to
use Co Processors for RowLog as part of your roadmap. Alternatively,
if you could point out features in RowLog that are still not available
with CoProcessors ... that's something I'd like to understand as well
and help us with the decision.

Thanks,
--Suraj

[Evert Arckens]

Hi,

I haven't had time yet to digg deep into HBase co-processors. So, correct me I make wrong assumptions.
Some of the features the rowlog provides which are not immediately solved by co-processors include:
* guaranteed message delivery : if a row gets updated a messages needs to be delivered even in case of node failures. If you would implement the sending of a message in a co-processor postPut call, what happens if a node fails before the co-processor gets executed. I don't think its execution is picked up by the new regions-server where the region moves to.
* atomicity : in the same area as the above feature, we want to make sure that a message is created when a row is updated, but also only when the row is updated. The co-processors are executed before or after the actual put.
* asynchronous execution : for some use cases we want to make sure a message is created when a row is updated, but that the message is delivered and acted upon in an asynchornous fashion, allowing to return the put call to the caller as fast as possible.

To conclude: I think some parts of the rowlog implementation could be handled by co-processors (like for instance preparing the row-local message information), but it will be hard to put the overall behaviour of the rowlog into just one or two co-processor calls.

Regards,
Evert Arckens.

--
Evert Arckens

http://outerthought.org/
Open Source Content Applications
Makers of Kauri, Daisy CMS and Lily

[svarma]

Hi Evert:
Thank you for the below details - this was exactly what I was looking
for.

Thanks,
--Suraj

On Dec 22 2010, 12:08 am, Evert Arckens <ev...@outerthought.org>
wrote:

> Evert Arckenshttp://outerthought.org/