gzipped input into Scalding job

[George Blazer]

Hello there,

Is there any way to feed a gzipped Tsv or Textline file into a Scalding job.

If so, can you please point me in the right direction.

Thanks.

[Oscar Boykin]

We don't use gzipped sources, but I think I suggested to someone else on to just try it and it works since Hadoop supports it. Did you try it and have it fail? If so, send your complete code and exception/failure reports.

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/15f268c9-f181-4ddc-bf1d-15bae8df347a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Oscar Boykin :: @posco :: http://twitter.com/posco

[Sanchit Grover]

It works as it is, you don't need to make any changes in your code. Just give the .gzip input to Textline function as you give a normal tsv to it. 

[Jeff T]

we have all gzipped sources and this is what we use

hadoop jar ./your.jar com.twitter.scalding.Tool -Dio.compression.codecs=org.apache.hadoop.io.compress.GzipCodec yourClass