I've been doing some more research on optical flow sensors.
I've found two interesting products, one is the adns3080
sensor sold by diydrones (which they don't seem to sell anymore), and the other is the PX4Flow -
https://pixhawk.ethz.ch/px4/modules/px4flow - also sold by diydrones. I suspect that an optical flow sensor could work really well on a rover, as the sensor could be kept parallel to the ground (except when the rover tilts over or becomes airborne, etc). I always thought that the main problem would be distinguishing lateral x,y movement from rotation, but then I realized that a gyro would work great for compensating for rotation. And actually the PX4Flow looks like it does this, and does a really great job (there is a picture of a ground trace using only the optical flow sensor and it at least looks really good:
https://pixhawk.ethz.ch/px4/_detail/modules/px4flow_trajectory.png?id=modules%3Apx4flow). But, there is a problem I think, in order for a pixel to be found in sequential frames, the rover can not be moving too fast (or the sensor has to high up, or it needs a faster frame rate). The PX4Flow claims a 2 to 1 altitude to max speed (5 m above ground gives 10 m/s). I couldn't find a specification for the ADNS3080. At any rate, I had something more on the order of a few cm above ground, which is clearly unworkable.