Don't worry about what you can't control, only worry about what you can, and even then sometimes what PageSpeed suggests isn't relevant (e.g. telling me to minimize html - to save less than 1k)
Downloading libs locally so you can set a cache header may give you a better score but may slow your site down because even without a cache header, browsers will often still cache JavaScript - and with 3rd party hosts (such as
code.google.com) there's a good chance visitors to your site already have it cached.
Worry about what makes sense to fix, not the stuff you have no control over.