[scalding] Some questions regarding scalding steps

13 views
Skip to first unread message

Hagai Attias

unread,
Jun 20, 2016, 1:52:10 PM6/20/16
to cascading-user
Hey guys, I have some questions regarding scalding steps:

1. Does scalding writes the intermediate data (that is persisted to hdfs between steps) steps compressed? Can I change it?
2. I want to control the number of input splits (and thus control the number of mappers) when reading the output of the reducers in a previous step. How should I go about that?

Thanks,
Hagai.

Reply all
Reply to author
Forward
0 new messages