Error with using HBaseTap on HBase 0.20.0

[dekumar]

I recently performed an upgrade of HBase from 0.19.1 to 0.20.0. I had
a working piece of code that used HBaseTap for the earlier version of
HBase. After the upgrade, I got the HBaseTap for HBase 0.20.0 from -

http://github.com/cwensel/cascading.hbase/network

However, on using this code, I get the following error -

Testcase: testFlow took 8.399 sec
Caused an ERROR
flow failed: AP
cascading.cascade.CascadeException: flow failed: AP
at cascading.cascade.Cascade$CascadeJob.call(Cascade.java:428)
at cascading.cascade.Cascade$CascadeJob.call(Cascade.java:369)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask
(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: cascading.flow.FlowException: unhandled exception
at cascading.flow.Flow.complete(Flow.java:607)
at cascading.cascade.Cascade$CascadeJob.call(Cascade.java:419)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
class org.apache.hadoop.hbase.mapreduce.TableOutputFormat not
org.apache.hadoop.mapred.OutputFormat
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:
840)
at org.apache.hadoop.mapred.JobConf.getOutputFormat(JobConf.java:488)
at org.apache.hadoop.mapred.JobClient.submitJobInternal
(JobClient.java:772)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
at cascading.flow.FlowStep$FlowStepJob.call(FlowStep.java:470)
at cascading.flow.FlowStep$FlowStepJob.call(FlowStep.java:409)
Caused by: java.lang.RuntimeException: class
org.apache.hadoop.hbase.mapreduce.TableOutputFormat not
org.apache.hadoop.mapred.OutputFormat
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:
834)

----------------------------------------------------------------------------------------------------------------

Any ideas on what might be causing this ??

[Chris K Wensel]

You might try this branch:
http://github.com/cwensel/cascading.hbase/tree/hbase-0.20

No guarantees though. A lot has changed with HBase .20. this branch
uses the deprecated apis.

It will be some time before I can get a stable build of Cascading
against the hadoop .20 api changes, until then, hbase taps will need
to use the deprecated api calls (the branch above).

ckw

--
Chris K Wensel
ch...@concurrentinc.com
http://www.concurrentinc.com

[dekumar]

Thanks a lot for that Chris !

I used the HBaseScheme file from the location you specified and my
code ran without a problem !!

Thanks again !!

[mlimotte]

Hi Chris,

This HBaseTap source worked fine for us, but we did have to make one
change to the source. Just wanted to let you know what it was as an
FYI.

The code has constructors where you can specify the ZooKeeper quorum
servers explicitly. If you don't use that, it defaults to
"localhost". We wanted the list of quorum servers to be pulled from
the hbase-site.xml file (which needs to be in the classpath).

This is actually the default behavior for hbase, so we simply had to
remove the lines from your source where you were explicitly setting
that property:

conf.set( "hbase.zookeeper.quorum", quorumNames );

Thanks for sharing the tap code with us.

Marc

[Chris K Wensel]

Awesome, thanks!