This is a very old thread and is probably too little help too late, but I think I know what is going on here. You are passing in a PKCS12 key when a PEM file is required. See the following comment from the jwt.Token struct (
http://godoc.org/code.google.com/p/goauth2/oauth/jwt#Token):
"When obtaining a key from the Google API console it will be downloaded in a PKCS12 encoding. To use this key you will need to convert it to a PEM file. This can be achieved with openssl."
I hope this helps, and let me know if you encounter any further errors.