I've also found that a good way to do these things is to figure out
the actual size of the final piece, then make sure to set the font
size to the font size you want in place.
eg. If your paper/site is in 9 pt type, setup the theme with:
theme_set(theme_gray(base_size=9))
Then save the plot to the actual height and width you want (in inches
or pixels) using ggsave(). That helps avoid situations of poor
fontsize balance due to scaling down (eg by includegraphics in latex).
Tangental, I know :)
--J