I have no affiliation with google. Just a fellow user like you. In my experience there are 2 cases where the refresh doesn't work:
1. When you exchange their code with non-empty scopes as a parameter. Make sure to leave this blank (not sure which client library you're using) and only include non-empty scopes when you create the initial redirect url.
2. When the redirect url parm of the exchange code request is not the same as the redirect url you provided the user (must be identical)
I also can't find this in the documentation or anywhere else in the forum; I just had to figure it out painfully.