Upgrade from ClientLogin to Oauth2

[Sean Busa]

Our company is currently using OAuth2 for adWords authentication. We are using OAuth2Mode = Application and have our clients accounts linked to our mcc account so that we can manage their accounts using our mcc.

Since this is a project set up in the Google APIs Console and we have Google Analytics turned on and are using the client libraries (dotnet),

Can we link our clients GA accounts to the same MCC account and use the same authentication clientId, secret and refresh token?

Thanks,

Sean

[Sean Busa]

Anyone from Google Analytics have a reply or update here before the April 21 deprecation happens?

[mschenkel]

It appears there is not much discussion within this group these days.  I too had a recent question about the 04/21 deprecation:

https://groups.google.com/forum/#!topic/google-analytics-data-export-api/MedhDNFdmVk

I recall seeing some mentions even within some google documentation pages that said stackoverflow is now being monitored by Google personnel.  You may want to try there (stackoverflow.com).

What do you mean by MCC account? Based on my knowledge, I think as long as your Master Account supports OAuth, then any accounted shared to it via Google Adword or Google Analytics should still be accessible.

[Mike Sullivan]

If you are using Oauth2 for AdWords and have GA selected in the APIs Console, you should be able to use the same clientid and secret. You will need to broaden the scope to include the GA scope endpoint.

If I remember correctly, you simply append the second scope to the first with a space.

[Pete]

Mike is on the right track but your questions is a bit ambiguous as to what you're trying to do.

A request to the Analytics API requires that the user (or application) has (1) permission to take that action on the Analytics entity and (2) that the correct Analytics scopes have been granted by the user (or application) which means the user has to authorize for the correct Analytics auth scopes for the client (or clientId as you refer to it). If you look at the API reference for AdWords Link in the Analytics Management API, this is how you would make the request and it shows the required scopes to do so.

I'm not sure that's what you're trying to do. Since you know the implementation well can't you just test this out and see if it works based on what's required to insert an AdWords link using the Management API? Have you received some error message?

[Sean Busa]

Hi Pete,

So let me explain our situation a little further. I work for a very large search marketing company which manages 40+ large clients with at least 50M in spend a month. Currently in our implementation, we manage all of reporting needs for AdWords through linking all of our clients accounts to our MCC. This makes it easy on the API side where we use our oAuth token and secret embedded in our config file and we pass the client account id and have access to all of their data.

Now for Google Analytics.. We currently manage our clients accounts and have login and password to use the v3 analytics client library to pull reports  for the client. Some of these clients don't necessarily want their GA account linked to their adWords account. This is going away with clientLogin deprecating on the 21st of this month.

I'm not really understanding the idea around setting up the permissions side from the client end, as I don't typically have much to do with this process.

I guess what I'm asking is if there is a way for our Google APIs console project (where our agency mcc account is managing the adwords accounts) to manage all of our analytics accounts in a similar way that we're managing the AdWords accounts? Is there a way to link our clients accounts to our API console project or do we have to create a project for each client (which is a huge pain). I'm looking for the easiest way to manage many accounts and to ensure that there is no lapse in reporting data to our clients before the 4/21 deadline. Any help that you can provide would be greatly appreciated.

Thanks,

Sean

[Sean Busa]

So when I log in with the account which manages our api console project and add a GA account id to the API's explorer I get back a 400 error, which I believe says that we don't have permission to administer this

400 Bad Request

- Show headers -

Cache-Control:  private, max-age=0
Content-Type:  application/json; charset=UTF-8
Date:  Thu, 09 Apr 2015 14:07:37 GMT
Expires:  Thu, 09 Apr 2015 14:07:37 GMT
Server:  GSE
Transfer-Encoding:  chunked
Vary:  X-Origin

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "badRequest",
    "message": "invalid accountId: UA-7974431-1."
    ...
   }
   ...
  ],
  "code": 400,
  "message": "invalid accountId: UA-7974431-1."
  ...
 }
 ...
}

[Pete]

ok, so the general answer to your question is No but the problem is that you're conflating a few things.

some details for you:

1) There is no equivalent of an MCC in Analytics

2) You're currently accessing Google Analytics accounts using clientlogin (user/pass) so this means you have access already and permissions. You just need to use OAuth 2.0 for authorization instead. Again, since you know the implementation details you're going to have to decide which auth approach to use but it sounds like a Service Account is what you want since that effectively does what you're describing, link your API console project.

If you don't know how OAuth 2.0 works then you should review the docs:

https://developers.google.com/identity/protocols/OAuth2#serviceaccount

Finally, the error message from the APIs Explorer you've shared says invalid account id...because you've entered an invalid Account ID. If it was a permissions issue, it would say you don't have sufficient permissions.

UA-XXXX-YY is a property ID. There is an account id, property id, and profile(view) id. Maybe you should use the Account Explorer to see what these IDs look like so that you are working with the correct IDs. 

[Sean Busa]

okay, this is an answer. Not the answer I was hoping for, but an answer. I know now to stop going down the path I was headed.

I tried again with the inverse account_id, profile and got the permission error, so I know I'm going to be using a service account to manage the accounts then as our systems are all automated scheduled server calls to the service. We are using p12 keys for clients using Google Cloud Storage. In that case our service account id is added to the client's cloud storage project with the appropriate permissions. I'm guessing that we'll have to set up a cloud storage project for each client then?

Thanks,

Sean

[Tim Bongers]

Hi Sean,

what you need is one service account, for which you then grant access to your clients' Analytics account (at least) on the profile/view level. The service account has its own e-mail address which can be added as a user via the UI in the same way as you'd add a normal Google account. The service account can then be used to access the Analytics API via OAuth2 using a refresh token (preferrably by using one of the client libraries to make things easier, i.e. Java, PHP etc.).

For 40+ accounts this should be working fine. In our experience you might run into account limits when handling more than 100 accounts with one service account, in which case I'd try to contact a Google representative.

Sadly, there is no MCC solution equivalent to AdWords (as stated above). I still hope for Google to realize the need for this and an implementation in the future ;)

Regards,

Tim

[Sean Busa]

Thanks Tim,

I'm going to try this today. Will let you know how it works out.

Regards

[Sean Busa]

Hi Tim or Mike,

Okay, I added the service account id (email) to a client at the account level which is associated with a p12 key and I'm still getting a 403 error. Is there something else that needs to be enabled on the client GA account?

Google.Apis.Requests.RequestError
User does not have sufficient permissions for this profile. [403]
Errors [
 Message[User does not have sufficient permissions for this profile.] Location[ - ] Reason[insufficientPermissions] Domain[global]
]

On Friday, April 10, 2015 at 10:06:40 AM UTC-4, Tim Bongers wrote:

[Tim Bongers]

Hi Sean,

are you sure that you are using the correct profile id? You need the ID of the profile/view to request data, not the account ID. Could you probably provide some example code?

Regards,

Tim

[Sean Busa]

Hi Tim,

I actually got it working for most of our clients who've added our serviceAccountId to their GA accounts. For the request to the

service.Management.Profiles.List(uaAccount(1), "~all")

You have to list the Account Id.

I'm also calling the Goal method to get a list of goals

service.Management.Goals.List

And finally the data get method

service.Data.Ga.Get

I'm having one more issue where it seems that clients who are using GA premium are possibly not able to get data via adding the service account to their account.

I'm getting the following error :

Error:"invalid_grant", Description:"", Uri:""

But the methods are currently pulling data for other clients.

Is this a known issue?

thanks,

Sean

[Pete]

can you confirm the behavior and provide more details on when you get the invalid grant error? I.e. is it only for premium, any method? any other details to help troubleshoot.

Most invalid grant issues are because of server times being out of sync with NTP.

see https://developers.google.com/analytics/devguides/reporting/core/v2/gdataAuthentication#helpme

[Sean Busa]

Hey Pete,

I get the error on the actual request. It doesn't make sense that it would be the server time as we get some data for some clients but not for others and it's all the same process.

I also can call the profile service and get a list of profiles for all clients (or at least I could prior to 9am). I'm thinking that this has something to do with quota. I don't see how it could be the second item on the list either (ie. The refresh token limit has been exceeded) as we're not using tokens but the service account with a p1

[Pete]

Unless you can reliably reproduce the issue and provide some useful details it's going to be hard to troubleshoot the problem. Why do you think it's quota related, did you receive some error message indicating this?

Consider the following:

When you make a request there is Auth (used by all Google APIs) and then Analytics, these are 2 separate services. Once you successfully complete the auth part then the actual request is going to the Analytics service. 

The Auth service is returning the invalid grant error so the question is, if it's not related to NTP or token limits, or your implementation (which I have no visibility into) then is it specific to certain GA accounts and reproducible?

[Sean Busa]

Hey Pete,

That seems to be one of the big problems. I cannot consistently reproduce the issue.

I think it's quota related because, sometimes I can pull data for a client and then other times it get's this invalid_grant. I cannot fathom why an account would authenticate on one call then not 10 minutes later.

This issue is now crossing all of our clients and seems to be worse the later you get in the day.

We are using the same Google APIs Console project for all of our AdWords pulls and the same project for Analytics.

Here's an example, I just tried calling the profile service (to get a list of profile ids) for one of our clients 10 minutes ago, it worked. Now for the same client 10 minutes later I'm getting the invalid_grant error.

This doesn't sound like it's a problem in our code or in our authentication or any thing else I can possibly think of.

Sean

[Mike Sullivan]

Sean, are you getting a new oauth access token with each query sequence or re-using the existing one until it expires? I have found in the past that if you ask for a new access token too often, the authentication server rejects it every now and then. Reusing the access token until it expires solved my problems back then, and I haven't seen that type of problem since.

Mike

[Sean Busa]

Thanks Mike,

I'm not quite sure how I would go about doing that, I'm currently using the p12 key for 2 legged oAuth call using a service account. We do not store the token on our side.

We're using the dotNet client library, so I'm not sure what's going on behind the scenes in the call. My project was working for some clients, now isn't working for any, I'm getting the invalid_grant on every request.

My solution is set up the exact way as I'm using for Google Cloud Storage calls.

Here's my code in case anyone notices something that I'm overlooking.

Public Function getData(ByVal site_id As Integer, ByVal ua As String, ByVal profile_id As String, ByVal con As SqlConnection, ByVal startDate As Date, ByVal endDate As Date, ByVal dimensions As String, ByVal metrics As String, ByVal filters As String, ByVal startIndex As Integer, ByVal prefix As String) As GaData

Dim certificate As New X509Certificate2(keyPath, "notasecret", X509KeyStorageFlags.Exportable)

Dim credential = New ServiceAccountCredential(New ServiceAccountCredential.Initializer(serviceAccountId) With {.Scopes = {AnalyticsService.Scope.AnalyticsReadonly}}.FromCertificate(certificate))

Dim service = New AnalyticsService(New BaseClientService.Initializer() With {.HttpClientInitializer = credential, .ApplicationName = "ImpaqtPPC"})

Dim sDate As String

Dim eDate As String

sDate = startDate.ToString("yyyy-MM-dd")

eDate = endDate.ToString("yyyy-MM-dd")

Dim acct() = ua.Split("-")

Dim request = service.Data.Ga.Get("ga:" & profile_id, sDate, eDate, metrics)

request.Dimensions = dimensions

request.Sort = "ga:date"

request.StartIndex = startIndex

request.PrettyPrint = False

request.Output = 1

request.SamplingLevel = DataResource.GaResource.GetRequest.SamplingLevelEnum.DEFAULT

request.MaxResults = 10000

If Not filters = "" Then

request.Filters = filters

End If

Dim data As GaData = request.Execute()

Return data

　

End Function

[Pete]

You're going to have to try to isolate the problem because you have the most information about what you've done and how your code works.

Invalid grant

#1) in most invalid grant cases it's because system clock is out of sync (possibly it is drifting over the day).

#2) The other is "too many tokens".

It looks like you run through auth each time getData is called. So are you looping through N profiles and calling getData for each one, in other words going through the authenticating process for each data request? Considering #2 above and Mike's comments don't you think that maybe this is the problem? I.e. why do you need to authenticate on each request to get data, why not do that once and pass in the already authenticated/authorized service object? 

If you fix that problem and it isn't working then try new credentials (p12).

[Sean Busa]

I downloaded a new key, and am only making one call right now to the service for one profile and still get the invalid_grant. My time actually matches NTP perfectly (within 1 sec due to network on a utility that I ran).

This also isn't a problem with the Google Cloud storage implementation that we have (running a different service account from another Google Apis Console project), so I think that eliminates the time variable here.

Sean

[Sean Busa]

Okay, I finally got it working. Ready for this.... I went back in tfs and compared the last working state. I had ended up moving the p12 key file to the production server and passing in the key path to that new location.

To get it working I had to change back the keypath to the original server that it was installed on. It must be tied to the original download (server installation) location?

Thanks again for everyone's comments.

Sean