Re: Scan Hbase Table rows

[Klaas Bosteels]

You definitely have to use the deprecated version, since Hadoop Streaming doesn't support the new API yet. I had a quick look at the code of that version and there doesn't seem to be any place where you read in the hbase.mapred.tablecolumns value and I'm also not finding where you're opening the table itself? Guess you might want a configure method that reads in config values and uses that info to do the necessary initializations.

-K

On Fri, Apr 20, 2012 at 4:10 AM, Syamlal L S <itssya...@gmail.com> wrote:

More details:


Goal : Read a hbase table from a mapreduce process



Using un-deprecated version
-----------------------------------------

dumbo dbtest.py -hadoop /usr/local/hadoop -input test -inputformat sls.MyTableInputFormat -output /wc4/wc4_op.txt -outputformat org.apache.hadoop.mapreduce.TextOutputFormat -hadoopconf hbase.mapreduce.scan.columns="col:age" -hadoopconf hbase.mapreduce.inputtable="test" -libjar sls.jar -libjar /usr/local/hbase/hbase-0.92.1.jar

Output
---------
Exception in thread "main" java.lang.RuntimeException: class sls.MyTableInputFormat not org.apache.hadoop.mapred.InputFormat



Using deprecated version
------------------------------------

dumbo dbtest.py -hadoop /usr/local/hadoop -input test -inputformat sls.MyTableInputFormat -output /wc4/wc4_op.txt -outputformat org.apache.hadoop.mapred.TextOutputFormat -hadoopconf hbase.mapred.tablecolumns="col:age" -libjar sls.jar

Output
----------

2/04/20 07:32:51 ERROR security.UserGroupInformation: PriviledgedActionException as:hadoop cause:java.io.IOException: No table was provided
12/04/20 07:32:51 ERROR streaming.StreamJob: Error Launching job : No table was provided
Streaming Command Failed!

Code
-------

I am attching source code of custom InputFormat with this mail. Both deprecated and un-deprecated versions are attached.


Any suggestions?

On Thu, Apr 19, 2012 at 9:18 PM, Syamlal L S <itssya...@gmail.com> wrote:

Hi,

I have installed Hadoop 1.0.2, Hbase 0.92.1 and Dumbo. I would like to read from a Hbase table in the mapper processes and write to another Hbase table in reducer process.

I tried to use lasthbase ( link: http://dumbotics.com/2009/07/31/dumbo-over-hbase/ ). Unfortunately it does not work for recent versions of hbase and hadoop. Then I tried to create my own InputFormat java class ( by modifying InputFormat.java in hadoop and hbase src). I got the following error,

ExcepException in thread "main" java.lang.RuntimeException: class org.apache.hadoop.hbase.mapreduce.TableInpution in thread "main" java.lang.RuntimeException: class   org.apache.hadoop.hbase.mapreduce.TableInputFormat not org.apache.hadoop.mapred.InputFormat

Can I fix this? Or is there any other ways to read from Hbase tables (I want to read the entire table) in MapReduce stages?

--
You received this message because you are subscribed to the Google Groups "dumbo-user" group.
To view this discussion on the web visit https://groups.google.com/d/msg/dumbo-user/-/THaWanVxjw0J.
To post to this group, send email to dumbo...@googlegroups.com.
To unsubscribe from this group, send email to dumbo-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/dumbo-user?hl=en.

--
with regards,
Syamlal L S

--
You received this message because you are subscribed to the Google Groups "dumbo-user" group.

To post to this group, send email to dumbo...@googlegroups.com.
To unsubscribe from this group, send email to dumbo-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/dumbo-user?hl=en.