RGoogleAnalytics Scripts Generating : 401 Unauthorized

[Matt]

Last year, I wrote several scripts using R and r-google-analytics ( http://code.google.com/p/r-google-analytics/) to pull data from Google Analytics.  The scripts run in batch mode (Windows) every week and generate reports.

The scripts stopped running around July 7th, 2012 and now generate, "Error in GetDataFeed(query.builder$to.uri()) : 401 Unauthorized"

I'm assuming that this issue is tied to the deprecation of the v2 of the Google Analytics Data Export API Data Feed and the Oauth 2 requirement, but I'm not really sure.

Has anyone updated r-google-analytics to work with the newer APIs? 

Does anyone know of a work around that would enable me to authenticate and then run the R scripts without modifying them?

Here's a simple sample of the script that I'm using:

ga <- RGoogleAnalytics()

ga$SetCredentials("MY EMAIL,"MY PASSWORD")

profiles <- ga$GetProfileData()

#Build the Data Export API query

query <- QueryBuilder()

query$Init(start.date = format(Sys.time()-(24*60*60*7), "%Y-%m-%d"),end.date = format(Sys.time()-(24*60*60), "%Y-%m-%d"),dimensions = "ga:pagePath",metrics = "ga:pageviews", sort = "-ga:pageviews", max.results = 10000,table.id = "ga:MYID")

ga.data <- ga$GetReportData(query)

Thanks,

Matt

[Jeetendra S.]

We are continuing to resend v2.3 requests to the new API, which means all your v2.3 requests will start seeing the new v2.4 response. That said, client login is still supported (although not recommended) on v2.4 of the API. So I am not sure why you are seeing 401 errors. I suspect it's an issue with the scripts that you're using. May be you can check with the script authors to see if they can look into it.

Otherwise, we recommend that you migrate to the newer versions of the API and start using OAuth 2.0.

Jeetendra.

[Nick]

Hi yes,

With migration to the new version of the API, you need to register in the google apis console to get an API key. Then append that key to all of your API requests.

When we get a chance we'll update the R library.

-Nick

On Tuesday, July 17, 2012 9:11:05 AM UTC-7, Matt wrote:

[Matt]

Nick,

I appended the API key to my API requests, but I still get the 401 Unathorized authError.  (I tried both the Browser key with "any referer allowed" and the Server key with my IP address.)

It feels like I am missing something really basic.

This is the legacy authentication that I'm using:

    auth.resp <- postForm("https://www.google.com/accounts/ClientLogin",

                          Email = username,

                          Passwd = pass,

                          accountType = "GOOGLE",

                          source = "r-google-analytics",

                          service = "analytics")

After I authenticate, I can access my GA profiles and IDs:

    google.auth <- paste("GoogleLogin auth=", auth.token)

    response <-

      getURL("https://www.google.com/analytics/feeds/accounts/default",

             .encoding = 'UTF-8',

             httpheader = c("Authorization" = google.auth,

               "GData-Version" = 2),

             headerfunction=header$update)

But, I can't get the data:

    google.auth <- paste("GoogleLogin auth=", auth.token)

    response <- getURL(query.uri,

                       .encoding = 'UTF-8',

                       httpheader = c("Authorization" = google.auth,

                         "GData-Version" = 2),

                       headerfunction=header$update)

The resulting URL is:

https://www.google.com/analytics/feeds/data?start-date=2012%2D07%2D12&end-date=2012%2D07%2D18&dimensions=ga%3ApagePath&metrics=ga%3Apageviews&max-results=10000&ids=ga%3A<<MyID>>&key=<<MyKey>>

Do you see anything that looks incorrect?

Thanks,

Matt

[Neil Charles]

We've got exactly the same problem with this. It's bizarre - some Table IDs will run and others return a 401 error on the same Google account.

A fix for RGoogleAnalytics would be a huge help as it's knocked over some regular reporting and at the moment, I'm going to have to code a new interface in order to get them back up again.

[Theodore Van Rooy]

Has anyone solved this problem yet?  I'm setting off down this path to try and get mine fixed as well.. any updates would be appreciated.

Theo

[Jenna Lee-Dussud]

An update: http://code.google.com/p/r-google-analytics/issues/detail?id=4

[Ajay Ohri]

apparently this can be done by simply adding an additional paste in the google query part of the package (please correct me if I am wrong here)

from
https://developers.google.com/analytics/resources/articles/gdata-migration-guide

https://www.google.com/analytics/feeds/data?ids=ga:XXXX&dimensions=ga:date&metrics=ga:visits&start-date=2012-04-01&end-date=2012-04-02

After: API Key associated with Request
https://www.google.com/analytics/feeds/data?ids=ga:XXXX&dimensions=ga:date&metrics=ga:visits&start-date=2012-04-01&end-date=2012-04-02&key=API_key

Switch to v2.4 by replacing the base url with: https://www.googleapis.com/analytics/v2.4/data
regards

Ajay


Websites-
Technology
http://decisionstats.com
 

--
 
 

[Ajay Ohri]

I meant the Query Builder part of the package

[Jenna Lee-Dussud]

There is one more issue beyond adding the API key to the query and changing the base url on QueryBuilder.R.

In RGoogleAnalytics.R, there is a reference to the account feed, "https://www.google.com/analytics/feeds/accounts/default".  This feed is no longer available now that the Data Export API v2.3 had been shut down. I'm not quite sure how to work around this.

[Ky]

Changing line 327 of RGoogleAnalytics.R from

google.auth <- paste("GoogleLogin auth=", auth.token)

to

google.auth <- paste("GoogleLogin auth=", auth.token, sep="")

fixed this for me.

[Jenna Lee-Dussud]

Thank you so much Ky!  This worked for me and my day has been that much better for it.  I really appreciate your help.  :)

[Ajay Ohri]

Cool. Now we have a solution Can someone let us know when the R package has been updated on the code.google website

Regards

Ajay Ohri

http://decisionstats.com

[fegatochirurgia]

BUMP

I wondered if anyone has moved forward on this issue, since I myself am interested in carrying out some analysis on R of ga data.

I was hacking the code around and made some marginal progress, and would appreciate to know if someone resolved any of the issues.

I managed thus far to read the webproperties profiles, but it comes out in an non-XML format, and thus the getURL/ParseXML couple fail in the current RGoogleAnalytics script.

I also used googleanalytics4r (a defunct R package) which basically hardcodes all what is required for data retrieval from ga and, correcting some minor points to make it compatible with v2.4 or v3 API, I managed to get the required data and do some plotting. So, it is possible, and I would just love to have a working RGoogleAnalytics package.

So, bump, anyone?

[fegatochirurgia]

PROGRESS REPORT:

Google Analytics API v3

Queries come as JSON, not XML (at least, as far as I can tell).

I managed to retrieve the profiles data, and to get it into an R list. However, downstream functions are coded for an XML object.

IMHO the simpler approach would be to recode the XML parsing functions, rather than to hammer the JSON result into an XML format to be reparsed.

I used the rjson library to transform the ga "feed" into a list, as follows:

    response <- fromJSON(getURL("https://www.googleapis.com/analytics/v3/management/accounts/~all/webproperties/~all/profiles?key=<MY KEY>", .encoding = 'UTF-8', httpheader = c("Authorization" = google.auth, "GData-Version" = 2), headerfunction=header$update ))

I couldn't get the getURL result directly into the variable, so I transformed it previously.

It must also be noted that for the time being the OAuth 1 Token (key) is hard-coded. For either OAuth 2 or 1, these tokens should be saved into the working(?) directory, and called upon by the script, preferably as an function option (to allow for direct key input or change of location).

The output of the GetProfileData() function looks like this:

> ga$GetProfileData()
$kind
[1] "analytics#profiles"

$username
[1] "your.log...@here.net"

$totalResults
[1] 1

$startIndex
[1] 1

$itemsPerPage
[1] 1000

$items
$items[[1]]
$items[[1]]$id
[1] "12345678"

$items[[1]]$kind
[1] "analytics#profile"

$items[[1]]$selfLink
[1] "https://www.googleapis.com/analytics/v3/management/accounts/246810/webproperties/XX-000000-00/profiles/12345678"

$items[[1]]$accountId
[1] "1098765"

$items[[1]]$webPropertyId
[1] "XX-000000-00"

$items[[1]]$internalWebPropertyId
[1] "12341234"

$items[[1]]$name
[1] "www.yourwebsite.org"

$items[[1]]$currency
[1] "USD"

$items[[1]]$timezone
[1] "America/Phoenix"

$items[[1]]$websiteUrl
[1] "http://www.yourwebsite.org"

$items[[1]]$created
[1] "2001-01-01T00:00:00.000Z"

$items[[1]]$updated
[1] "2012-12-31T00:00:00.000Z"

$items[[1]]$parentLink
$items[[1]]$parentLink$type
[1] "analytics#webproperty"

$items[[1]]$parentLink$href
[1] "https://www.googleapis.com/analytics/v3/management/accounts/246810/webproperties/XX-000000-00/"


$items[[1]]$childLink
$items[[1]]$childLink$type
[1] "analytics#goals"

$items[[1]]$childLink$href
[1] "https://www.googleapis.com/analytics/v3/management/accounts/246810/webproperties/XX-000000-00/profiles/12345678/goals"

I do not manage multiple profiles with my account, so I do not know what exactly the output would look like. So, anyone? Ideas?

My 2¢ worth.

[Jerry Who]

2012/9/6 fegatochirurgia <in...@fegatochirurgia.com>:

> BUMP
>
> I wondered if anyone has moved forward on this issue, since I myself am
> interested in carrying out some analysis on R of ga data.

I've just pushed my fork of the original package to github:
https://github.com/JerryWho/rgoogleanalytics
I've adjusted to work with v2.4 of the API

I works for me - but since I'm new to R and especially to R packages -
there may be some points to be done better.

But maybe someone wants to help ;-)

JW

[fegatochirurgia]

Thank you.

I downloaded your fork and will soon check if it works for me. I did notice that the RGoogleAnalytics.R script still uses the legacy url for pre 2.3 api:
https://www.google.com/analytics/feeds/accounts/default

The problem for me was precisely in that function (GetAccountFeedXML) where problems arose.

I was wondering, would it be better to keep using api v2.4 or to go for v3?

On an unrelated subject: can I contribute code to your git repository, if ever I decide to push my code out into the wild?

BTW, on another thread  greentheo (Theodore Van Rooy) posted a script to get the OAuth 2 token validation to work:
https://github.com/greentheo/ROAuthSamples
See thread:
http://code.google.com/p/r-google-analytics/issues/detail?id=4

Theo too was wondering where to contribute new code. I think your git could be the place, if you agree.

What do you say we get this package in CRANtastic shape?

[Theodore Van Rooy]

We're working on getting those Oauth exchanges and validations built into the RGoogleAnalytics package at:  http://code.google.com/p/r-google-analytics/  

The XML/JSON is another beast altogether...

Theo

--
 
 

--
Theodore Van Rooy
http://greentheo.scroggles.com