Paruqet File on Hive

[M]

Has anyone worked with parquet files on hive using cascading? I've been trying to work with the cascading-hive and cascading-parquet extensions together but I've been having trouble figuring out how it should work. Does anyone have any advice or know of any projects that would help me figure it out? Right now I'm just trying to set up a dataflow with a source tap from a parquet table on hive which just writes the data to another parquet table on hive.

[Andre Kelpe]

I believe that CommBank uses cascading-hive with parquet: https://github.com/CommBank/ebenezer/

- André

On Thu, Jul 23, 2015 at 10:30 PM, M <matthe...@gmail.com> wrote:

Has anyone worked with parquet files on hive using cascading? I've been trying to work with the cascading-hive and cascading-parquet extensions together but I've been having trouble figuring out how it should work. Does anyone have any advice or know of any projects that would help me figure it out? Right now I'm just trying to set up a dataflow with a source tap from a parquet table on hive which just writes the data to another parquet table on hive.

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/62cf0659-dc91-46e0-af81-ff731728e289%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

André Kelpe
an...@concurrentinc.com
http://concurrentinc.com

[M]

Thanks, I'll give that a try. This may have an obvious answer that I'm just missing but how do I include a project like that into my own project? Is it just a dependency or do I have to copy the whole ParquetTupleScheme.java file into my own project? Thanks!

[jlief...@gmail.com]

Has anyone been able to use CommBank in their own project? I want to use it in my project but don't know how to include it. I'm using gradle.