Sorry, you cannot style a client-side GChart's gridlines.
You can use the setGridColor method, which may help do part of what
you are looking for:
http://gchart.googlecode.com/svn/trunk/doc/com/googlecode/gchart/client/GChart.html#setGridColor%28java.lang.String%29
If you must have dotted gridlines, and don't mind a bit of extra work,
you can add a curve with a SymbolType of YGRIDLINE (or XGRIDLINE) add
points at the y-coordinates where you want to draw the gridlines, and
use setBorderStyle("dotted") on that curve, and appropriately change
its width, height, etc (basically, you will be drawing your own
gridlines via a special curve; since curves have more options, you can
get more varied gridlines).
Thanks for your question. If you think more gridline options are
needed, please consider submitting an issue to the GChart issue
tracker.
John C. Gunther
http://gchart.googlecode.com