Google's Map Reduce as Groovy Closures

[Jesse O'Neill-Oine]

http://kenbarclay.blogspot.com/2008/01/googles-mapreduce-programming-model.html

Much of this goes over my head, but I figured there might be a few of you who would find this interesting.

Jesse

--
----------------------------------------------------------
Jesse O'Neill-Oine // je...@refactr.com
Refactr LLC // http://refactr.com
mobile // 612-670-5037
----------------------------------------------------------

[Hamlet D'Arcy]

I am not an expert, bu I think the problem is a lot simpler than this
post makes it blog makes it look.

Simply put:

mapReduce is a function with 3 parameters
1 - a map (like java.util.Map)
2 - a function to perform on each element of the map, which returns
a new map
3 - a function to perform on the result of step 2

This groovy code is /almost/ correct:

def mapReduce = { data, mapFunction, reduceFunction ->
def mappedData = data.collect(mapFunction)
return reduceFunction(mappedData)
}

Whether you toss each iteration of data.collect onto a separate thread
or a separate process is your own business.

I say /almost/ correct because Groovy's Collection.collect method
returns a java.util.List and not a java.util.Map...

... that's my take on it at any rate.

On Jan 11, 10:15 am, "Jesse O'Neill-Oine" <je...@refactr.com> wrote:
> http://kenbarclay.blogspot.com/2008/01/googles-mapreduce-programming-...

>
> Much of this goes over my head, but I figured there might be a few of you
> who would find this interesting.
>
> Jesse
>
> --
> ----------------------------------------------------------
> Jesse O'Neill-Oine // je...@refactr.com

> Refactr LLC //http://refactr.com
> mobile // 612-670-5037
> ----------------------------------------------------------

[Hamlet D'Arcy]

you can make the collect method generate a map by doing this:

def startingCharacter = {
it.value = it.key[0]
return it
}

"it" happens to be an Entry object. I didn't realize that.

http://hamletdarcy.blogspot.com/2008/01/mapreduce-for-mere-mortals.html