using HBaseWD with Put

[Marcin Cylke]

Hi

I've been trying to use HBaseWD with my MapReduce Job, but failed. My code works by simply putting some data from HDFS to HBase - using PUTs. 

I've created a GIST: https://gist.github.com/3039334 with rough idea how my code is working.

With HBaseWD I'm not getting any PUTs to HBase. Am I doing something incorrectly here?

Regards

Marcin

[Alex Baranau]

So the only thing you've changed is:

  String recalculatedKey = keyDistributor.getDistributedKey(key + (Long.MAX_VALUE - timestamp))

return new Put(recalculatedKey);

instead of

return new Put(key + (Long.MAX_VALUE - timestamp))

right? Btw, didn't know there's a Put(String) ctor :)

And the problem is that you cannot read this data later? If yes, how do you read it? If not, could you provide more details, where you see the problem?

Thanx,

Alex

[Marcin Cylke]

On Tuesday, 3 July 2012 16:47:38 UTC+2, Alex Baranau wrote:

So the only thing you've changed is:

  String recalculatedKey = keyDistributor.getDistributedKey(key + (Long.MAX_VALUE - timestamp))

return new Put(recalculatedKey)

I've also added a line initializing keyDistributor into Mapper's setup() method.

 

instead of

return new Put(key + (Long.MAX_VALUE - timestamp))

right? Btw, didn't know there's a Put(String) ctor :)

There isn't :) It is a mistake. Should be Put(Bytes.toBytes(key)) - but let's treat it just like a pseudo-code.

 

 

And the problem is that you cannot read this data later? If yes, how do you read it? If not, could you provide more details, where you see the problem?

The problem is that those PUTs don't show up in destination table. The HBase table is empty. There are no errors also. 

Marcin

[Alex Baranau]

Hm. How do you check that? In shell? Could you try with shell (sorry for obvious).

This looks suspicious:

context.write(new ImmutableBytesWritable(Bytes.toBytes(OUTPUT_TABLE_NAME)), putForClass(key, ts, elems));

If I'm not mistaken, the key should be not the table name, but the "row key" (i.e. Put's key). Can the problem be in this?

Alex

[Marcin Cylke]

On Thursday, 5 July 2012 21:41:12 UTC+2, Alex Baranau wrote:

Hm. How do you check that? In shell? Could you try with shell (sorry for obvious).

I've checked that with Shell. I've executed just "scan 'tableName'"

 

This looks suspicious:

context.write(new ImmutableBytesWritable(Bytes.toBytes(OUTPUT_TABLE_NAME)), putForClass(key, ts, elems));

If I'm not mistaken, the key should be not the table name, but the "row key" (i.e. Put's key). Can the problem be in this?

This line works when not using HBaseWD - However maybe it works by accident? :) I'll try passing a key there.

Marcin