Changes to config in 2.0?

30 views
Skip to first unread message

jd

unread,
Mar 22, 2012, 7:33:30 PM3/22/12
to cascading-user
Also I notice while migrating from 1.2 to 2.0 that setting the number
of reducers (and I assume other properties) no longer works.

for example:
myFlow.getConfig().setNumReduceTasks(1);

But when running on the cluster, the number of reducers is unchanged.

Chris K Wensel

unread,
Mar 23, 2012, 9:46:12 AM3/23/12
to cascadi...@googlegroups.com
you cannot change properties after the Flow has been instantiated unless you write a FlowStepStrategy that lets you change the properties of each MR job immediately before its submitted to the cluster.

Pipes also now have a FlowDef allowing you to set local properties during assembly.

ckw

> --
> You received this message because you are subscribed to the Google Groups "cascading-user" group.
> To post to this group, send email to cascadi...@googlegroups.com.
> To unsubscribe from this group, send email to cascading-use...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.
>

--
Chris K Wensel
ch...@concurrentinc.com
http://concurrentinc.com

Reply all
Reply to author
Forward
0 new messages