Pig Support - Load/Store Functions

[sloan]

Has anybody written or know of a Pig Load/Store Function for use with Hypertable? Looking to roll out hypertable into a production environment and wanted to checking tooling support and or efforts to implement tooling.

Thanks

[Doug Judd]

Hypertable does have a Pig connector.  We run a Pig test, which tests Load/Store, prior to each release.  Attached is the test, which includes the Pig scripts.

- Doug

On Fri, Feb 26, 2016 at 10:53 AM, sloan <sl...@ironbridgeintegration.com> wrote:

Has anybody written or know of a Pig Load/Store Function for use with Hypertable? Looking to roll out hypertable into a production environment and wanted to checking tooling support and or efforts to implement tooling.

Thanks

--
You received this message because you are subscribed to the Google Groups "Hypertable User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hypertable-us...@googlegroups.com.
To post to this group, send email to hyperta...@googlegroups.com.
Visit this group at https://groups.google.com/group/hypertable-user.
For more options, visit https://groups.google.com/d/optout.

--

Doug Judd

CEO, Hypertable Inc.

[sloan]

Doug, 

Thank you very much for bringing these to my attention. I will look into this and provided any helpful feedback.

Thanks again.

[sloan]

Doug,

The Loader is working great for reading however I'm having issues using STORE. Do you happen to have an example of what it is expecting as the input (e.g. {(row,column_family,column_qualifier,value),...})?

Example:

STORE results INTO 'hypertable://{namespace}/table'

USING org.hypertable.hadoop.pig.HypertableStorage(); // Using default constructor for STORE should I specific columns that will be stored?

I've tried several tried several solutions but continue to get an ArrayIndexOutOfBoundsException: 1 which is indicating to me it isn't finding the tuples as I would have expected.

Thanks,

Sloan

On Friday, February 26, 2016 at 1:04:07 PM UTC-6, Doug Judd wrote:

[Doug Judd]

Did you try this:

STORE results INTO 'hypertable://namespace/table'

USING org.hypertable.hadoop.pig.HypertableStorage();

(without the curly braces)

Also, did the ArrayIndexOutOfBoundsException come with a stack trace?

[sloan]

Sorry I didn't post an update. I was able to get everything working by writing a custom UDF to ensure the format of the outer bag was proper. After doing this everything is working just fine. Thanks again.

[Doug Judd]

No problem.  Thanks for the update!