How can i identified Best number of Mappers and Reducers in my cluster

20 views
Skip to first unread message

Sivaji

unread,
Oct 6, 2014, 10:17:38 AM10/6/14
to chenn...@googlegroups.com
Hi Every one,


In my organisation we are using 6 node cluster, i am running some pig scripts on more then 10 m of records, i want set no.of mappers and reducers, 

HOW can i know how many reducers i can set for my pig script ? what is the best performance of mappers size ? i am using below code to set mapps and reducers,

SET default_parallel 200;
SET mapred.min.split.size 256000;
SET mapred.max.split.size 256000;
SET pig.noSplitCombination true;
SET mapred.max.jobs.per.node 1; 

Thanks
Reply all
Reply to author
Forward
0 new messages