Re: HBaseWD with ThriftAPI

[Alex Baranau]

Hi Syed,

Couple of Qs to understand the whole idea

1. Are you looking at implementing only the read path or you also going to write data thru thrift API? Looks like you also look at writing..

2.

> The Distributed Rowkey genration will be done at Server side where the Thrift is started.

Please, correct me if I'm wrong, but as far as I understand, thrift server is started on each RS and acts as a wrapper of RS. If you look at writing path in HBase, when certain Put (or Delete, etc.) comes to specific RS, there's no way to change the row key, as you already "stuck" with this specific RS (I believe I already explained that on HBase user ML). So to me it looks like the row key should be generated on client side.

Thoughts?

Alex

P.S. Please include hba...@googlegroups.com in "To" to share this info.

On Thu, Aug 9, 2012 at 2:52 PM, syed kather <in.a...@gmail.com> wrote:

Hi Alex ,

     The Distributed Rowkey genration will be done at Server side where the Thrift is started. The data will be Distributed evenly even If there are  'n' number of Thrift clients. 

     Now what's happening at the client side(thrift) 

     Wet will call our custom put and avoid touching then existing put(as user may need for some other purpose). I am planning to achieve that using increment column.

  I am planning to implement in on .Net environment initially, and later we can implement for other clients, once it is working fine

 Note : Custom Function which will emit the distributed key is as per HBaseWD. If the client is interested in getting the original key we have a   

 function available for that with thift clients.

  Distributed Row Key Bucket Number will be avaliable on hbase-site.xml and will be fetched from there when the thrift server is started. 

  Please correct me if you find the approach inappropriate and pardon my negligence.

 

            Thanks and Regards,
        S SYED ABDUL KATHER 
               

[syed kather]

Sorry alex for the delay ..

Thanks for ur reply ...
Yes i am trying for both read and write ..

I am agree with your comments and i had not thing in angle of starting thrift in different region servers .. generating distributed rowkey in client side is perfect idea and i will start implementing as per ur suggestion ... l

Thanks and regards ,
Syed Abdul Kather

[syed kather]

Hi Team ,
Let me know is there any suggestion. Please let me know . As planned to start devolping . I need your suggestion and design changes if required .

Thanks and regards
Syed abdul kather

[Alex Baranau]

Hi Syed,

I think the only thing to do w.r.t. thrift client is:

1) on reading end: port (re-implement) distributed result scanner in your scripting language

2) on writing end: similarly port (re-implement) row key generation

I guess that's the only way to do it... Yes, that would mean porting several classes and there's a question why want to preserve format of the key compliant with HBaseWD. I guess the reasons is to be able to later use java client API and MR integration (which HBaseWD provides). So it may be well worth it.

Thoughts?

Alex Baranau

------

Sematext :: http://sematext.com/ :: Hadoop - HBase - ElasticSearch - Solr