MR counter - Num_Keys_Flushed

[Ashish]

Hi Team, 

I would like to understand the significance of Cascading MR counter - Num_Keys_Flushed.

While analysing one of production job I cam across this counter so would like to understand what this counter indicates so we can look possible optimisations.

cascading.​flow.​SliceCounters	Name Map Reduce Total Process_Begin_Time 1,474,798,437,657,894 0 1,474,798,437,657,894 Process_End_Time 812,615,484,602,069 0 812,615,484,602,069 Read_Duration 590,689,444 0 590,689,444 Tuples_Read 460,525,182 0 460,525,182 Tuples_Written 326,774,887 0 326,774,887 Write_Duration 179,431,191 0 179,431,191
cascading.​flow.​StepCounters	Name Map Reduce Total Tuples_Read 460,525,183 0 460,525,183
cascading.​pipe.​assembly.​AggregateBy​$Flush	Name Map Reduce Total Num_Keys_Flushed 160,436,704 0 160,436,704

Thanks,

Ashish

[Chris K Wensel]

cache evictions from the LRU. tuning discussed on the javadoc of AggregateBy

ckw

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at https://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/d52b81b9-ef38-4c59-9001-67f107179d71%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

—

Chris K Wensel

415-203-5022

ch...@wensel.net

https://www.linkedin.com/in/cwensel