We were talking about stepper cooling in another thread and I figured it merited its own discussion.
I've taken a lot of temp measurements of my R2x. The Y stepper moves the most mass and consistently runs about +25C above build chamber ambient temp during long prints (mostly 90mm/s). The X stepper consistently runs about +20C above ambient. An active extruder stepper touching the cooling block runs ~10C above ambient (0.2mm layer with ABS). This is measured in the middle of the black laminated section where the surface is hottest. (The internal windings will obviously be a lot hotter than that.) My contact thermocouple and IR thermometer have good agreement on these numbers.
Stepper motors are designed to run hot, and are usually rated up to 50C ambient. No R2 will ever reach this temperature. My stock R2x barely gets up to 50C with both extruders + HBP running and the enclosure sealed up pretty tight. Running my Y stepper at 75C surface temp has been completely fine so far. R2 owners don't need to worry about X+Y stepper cooling.
Extruder steppers are different because they can contribute to heat creep, particularly with aluminum extruders -- I think putting a heat sink on the extruder stepper when printing PLA might be a good idea, particularly with hot end setups that separate the stepper from the cooling bar.
But I seriously question the value of simple heat sinks -- if you really look at spec sheets for plain 40mm heatsinks, the "natural" heat dissipation is around 1 watt per 15C temperature difference. (With forced air circulation, you roughly triple the heat removal rate.) So a naked heat sink is adding mass to the gantry for what is probably less than one watt of cooling. Kind of questionable to my mind. Maybe worthwhile if you already have forced air circulation throughout the chamber? But then your steppers are already getting some air cooling. I don't know.
I'm actually working on running my chamber higher than 50C, so I'm getting "advanced" with stepper cooling. Putting a fan and heatsink on the Y stepper dropped the temp by 5C. I made a clip to easily attach a stock 55mm computer heatsink to the stepper.
http://www.thingiverse.com/thing:249804
And I am currently playing with a 50W peltier cooler module strapped to my Y stepper. The nice thing about this arrangement is that the extra heat generated by the peltier (because it's inefficient) adds 7-8C to the build chamber temp, and makes it warm up a lot faster at the start of a print. My Y stepper now runs at ambient chamber temp because heat is being actively pumped out of it. I modified my fan clip design above to provide some plastic insulation and it's been working great for the last 30-40 hours or so of printing.

I'm printing at 56C with the chamber door slightly ajar right now. Great for ABS. But now my X stepper is reaching 75C, so I need to figure out a way to cool it next. There isn't enough open surface area to attach anything substantial. But I think Carl's Alu X-Ends will help.
If I do add active X stepper cooling, the chamber temp will come up another 5-10C and all the plastic parts in the printer are at risk of serious warping/sagging. My Z-arms, carriage, and X-ends are all going aluminum. But I'm not sure what to do about the plastic pillow blocks. At a certain point, you have to start sucking outside air into the machine to keep things cool rather than ramping the chamber temp up with thermoelectric cooling.