Cascalog Avro

[Gerrard McNulty]

Hi,

Has anybody used cascalog with avro? What are your thoughts/
experiences?

[nathanmarz]

I haven't, but you may want to check out this project that integrates
Cascading with Avro:

https://github.com/bixolabs/cascading.avro

It's fully compatible with Cascalog since Cascading taps are first-
class.

-Nathan

[Andrew Xue]

I just started messing with avro and cascalog. Was able to create the
a tap with AvroScheme no problem, but encountered
cascading.flow.FlowException: unhandled exception (NO_SOURCE_FILE:0)
when actually trying to query data.

seems like it could be really sweet if i got it working. one thing
though: one of the features of avro is that the schema is specified in
the data itself. but with cascading.avro theres no way to recover it,
so you have to specify it yourself somewhere.


On Sep 16, 11:49 am, Gerrard McNulty <gerrard.mcnu...@gmail.com>
wrote:

[nathanmarz]

Hey Andrew,

Whenever you get a FlowException like that, you should be able to find
the actual cause of the error in the task logs. That will help with
diagnosing why it's not working.

-Nathan

[Andrew Xue]

Hi Gerrard,

Any luck with this? I got through some of initial issues but now stuck
on the fact that cascalog vars/fields have a prefix that avro doesn't
seem to like

Caused by: org.apache.avro.SchemaParseException: Illegal initial
character: !id
at org.apache.avro.Schema.validateName(Schema.java:1042)
at org.apache.avro.Schema.access$200(Schema.java:77)
at org.apache.avro.Schema$Field.<init>(Schema.java:383)
at org.apache.avro.Schema$Field.<init>(Schema.java:379)

same for ? as well

On Sep 16, 11:49 am, Gerrard McNulty <gerrard.mcnu...@gmail.com>
wrote: