Reading a hive partitioned table with Cascading-Hive

[Felix Sanchez]

Hi,

I'm trying to use Cascading-Hive to read a table that is partitioned. I understood that HivePartitionTap is meant to be able to do that because:

1- I don't seem to be able to figure out how to indicate which partitions I want to read.  

2- The HivePartitionTab code complains that PartitionCollector has private access in PartitionTab.

Any ideas?

Thanks!

Felix

Cascading-Hive Cas

[Andre Kelpe]

Hi,

which version of cascading-hive and which version of Cascading are you using? PartitionCollector was made public in Cascading 2.5.4: https://github.com/Cascading/cascading/commit/3642c145588b73612abe4fbff7cc5ecb5fec0ae6 You should really use Cascading 2.6. It is backwards compatible and has all the latest bug-fixes.

HiveTap is a sub-class of PartititionTap. You can find more usage examples in the tests:

https://github.com/Cascading/cascading/blob/2.6/cascading-platform/src/test/java/cascading/tap/TapPlatformTest.java#L286

- André

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/2bf418b1-f2a4-4936-9a5b-09c03485ee15%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

André Kelpe
an...@concurrentinc.com
http://concurrentinc.com

[Felix Sanchez]

Andre,

thanks for you quick response. You were indeed right about the version, I was using Cascading 2.5.3. I'm still confused about the example, though. 

How do you specify which partition do you want to read from in a HivePartitionTap? 

Cheers,

Felix

[Elliot West]

I don't think the PartitionTap supports this currently.

To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/ea619ab1-b3f4-406d-9409-800d45953a0a%40googlegroups.com.

[shree]

May i know any updates on Parition Tap for Partitioned Hive table . As  i am looking for solution to create a pipe on partitioned hive table with parquet format

It would be really helpful, thank you.