HBaseWD - Distribute Sequential HBase Writes Google Group

Calculation of HBASE Size after test run

llnaraya...@gmail.com (Narayan) — Wed, 10 Oct 2012 08:03:02 UT

Hi,
I was using HDFS+HBASE. I have created a database and ran a test for 2
hours. I want to check the increase in size of the database after test run.
I have tried to take readings using:
hadoop dfsadmin -report and
hadoop fs -dus /hbase.
Also tried to check the disk size using du -sk /HADOOP.
Observation is after the test is run, the size shows decreased instead of

Re: HBaseWD with ThriftAPI

alex.barano...@gmail.com (Alex Baranau) — Thu, 16 Aug 2012 20:07:30 UT

Hi Syed,

I think the only thing to do w.r.t. thrift client is:
1) on reading end: port (re-implement) distributed result scanner in your
scripting language
2) on writing end: similarly port (re-implement) row key generation

I guess that's the only way to do it... Yes, that would mean porting

Re: HBaseWD with ThriftAPI

in.ab...@gmail.com (syed kather) — Thu, 16 Aug 2012 19:18:19 UT

Hi Team ,
Let me know is there any suggestion. Please let me know . As planned to
start devolping . I need your suggestion and design changes if required .

Thanks and regards
Syed abdul kather

Re: HBaseWD with ThriftAPI

in.ab...@gmail.com (syed kather) — Wed, 15 Aug 2012 18:20:42 UT

Sorry alex for the delay ..

Thanks for ur reply ...
Yes i am trying for both read and write ..

I am agree with your comments and i had not thing in angle of starting
thrift in different region servers .. generating distributed rowkey in
client side is perfect idea and i will start implementing as per ur

Re: HBaseWD with ThriftAPI

alex.barano...@gmail.com (Alex Baranau) — Fri, 10 Aug 2012 23:01:15 UT

Hi Syed,

Couple of Qs to understand the whole idea

1. Are you looking at implementing only the read path or you also going to
write data thru thrift API? Looks like you also look at writing..

Thrift is started.

Please, correct me if I'm wrong, but as far as I understand, thrift server

Re: Pre-splitting table when writing to it with HBaseWD

alex.barano...@gmail.com (Alex Baranau) — Tue, 07 Aug 2012 01:06:52 UT

This is weird.

Well, I can explain that the first region doesn't get attention, since it
has literally nothing (the start key of the second region is the smallest
possible key). To fix that, just don't provide first key from distributor
as a split [1] - HBase will create first one anyways by default.

Re: Using HBaseWD with Thrift

alex.barano...@gmail.com (Alex Baranau) — Mon, 06 Aug 2012 19:57:47 UT

That would be really great. There's no formal process: just fork the
sources, add needed code and when ready, create a pool request thru github.

But first I think we should may be discuss what is the plan (in the
comments at: [link]). I.e. how do
you see it's going to be implemented. Just added small comment there.

Re: Pre-splitting table when writing to it with HBaseWD

in.ab...@gmail.com (syed kather) — Mon, 06 Aug 2012 18:14:38 UT

Alex

After implementing that snippet of code which you gave i have found there
still some problem in distribution

You can see the Red color high lighted request count is 288727 and green
color count is 6.

HBaseAdmin admin=new HBaseAdmin(HBaseConfiguration. create());

RowKeyDistributorByOneBytePref ix distributor = new

Re: Using HBaseWD with Thrift

in.ab...@gmail.com (syed kather) — Sun, 05 Aug 2012 17:20:30 UT

As i had disucussed in this earlier mail . I am very much interested to
contribute .. i will further update you once it started working .. this is
my first contribution too. Let me know is there any process i need to
follow

Syed Abdul kather
send from Samsung S3

Re: Using HBaseWD with Thrift

in.ab...@gmail.com (syed kather) — Sat, 04 Aug 2012 18:37:51 UT

Hi Alex,
Thanks for your response ..
Yes I would like to work on this .Let me know the process .

Thanks and regards
Syed Abdul kather
send from Samsung S3

Using HBaseWD with Thrift

alex.barano...@gmail.com (Alex Baranau) — Sat, 04 Aug 2012 01:52:24 UT

RE using HBaseWD with Thrift (changed subject for this part of the thread
to be about Thrift).

There's currently no ports to any of the scripting languages (like Python
and such). I've added an issue to HBaseWD [1], but I can't promise it will
get attention from me, as I'm mostly using Java in the development of the

Pre-splitting table when writing to it with HBaseWD

alex.barano...@gmail.com (Alex Baranau) — Sat, 04 Aug 2012 01:38:02 UT

RE pre-splitting the table (changed subject for this part of the thread to
"Pre-splitting table when writing to it with HBaseWD"):

There's no helper for that at the moment in HBaseWD lib (but going to be:
[1])
// creating pre-splits
RowKeyDistributorByOneBytePref ix distributor = new
RowKeyDistributorByOneBytePref ix((byte) 32);

Re: Atomic increment Distribute Sequential HBase

in.ab...@gmail.com (syed kather) — Fri, 03 Aug 2012 16:44:05 UT

Team,
How to do Pre splitting for HBase table . As i mention in the earlier
mail that my rowkey is followed by incremental sequence

when i had created a table like give below . I had notice the startkey and
endkey is looking like that

bin/hbase org.apache.hadoop.hbase.util.R egionSplitter ObjectSequence4 -c 3

Re: Atomic increment Distribute Sequential HBase

in.ab...@gmail.com (Syed Abdul Kather) — Fri, 03 Aug 2012 15:54:48 UT

Alex , Sorry that my mistake i had understood the problem the constructor
is keep on getting called . i am trying to fix now .

Re: Atomic increment Distribute Sequential HBase

in.ab...@gmail.com (Syed Abdul Kather) — Fri, 03 Aug 2012 15:54:28 UT