GA Data Export API data download limits and customization

[saulat78]

Hello

We got a website, according to GA we have 500,000 Visits per day, is
this possible for GA Data Export API to download that 500,000 data on
daily basis. We need to store that data in our data warehouse and
present it to our end users on weekly or daily basis.

I know there is limit of 10,000 queries per 24 hours and every query
can return max 10,000 rows. I am thinking to use pagination to
download all the relevant data.

Is there any documentation or dimensional model available for the Data
warehouse that google uses to store our data. I have seen valid
combination of dimensions and metrics that we can query through GA
Api, but it would be rather good to have their dimensional model.

Further, do GA have any option to use our custom variables for e,g, I
want to know weather the existing visitor is my registered member or
not.

Thanks
Syed Saulat Rizvi

[Nick]

Hi,

No there is no public dimensional model. Some of the data is pre-
aggregated (ie you can make 1 request an get 1 row back saying for a
particular day you got 500,000 visits) while other requests like for a
particular page, what were all the entrance source, are dynamically
calculated (ie GA goes back and re calculates that data on the fly)

So the answer to your general question, is yes, you can create a
script to automate retrieving daily visit numbers. All our
documentation is on google code. There is an interactive query
explorer that can help you build queries.

And currently it is against the GA Terms of Service to combine any
Google Analytics data with personally identifiable information, even
if it's already in your data warehouse.
-Nick

[GreenStreet]

I would like to import analytics data into my warehouse to run queries
about site usage of my more loyal customers.
Would this violate the terms and conditions even if these reports do
not contain explicit personal information?
I would like to use setCustomVar to include a unique identifier for
each logged in visitor to correlate them back to this other data.

(Note, we can't use the ECommerce api because we have many sales
channels.)

Also, I would like to pull all this data down nightly into the data
warehouse. Is something like this possible? It seems the api is
limited in that you can only grab certain metrics at a time as opposed
to pulling all the data into our warehouse.

On Oct 13, 9:37 am, Nick wrote:
> Hi,
>
> No there is no public dimensional model. Some of the data is pre-
> aggregated (ie you can make 1 request an get 1 row back saying for a
> particular day you got 500,000 visits) while other requests like for a
> particular page, what were all the entrance source, are dynamically
> calculated (ie GA goes back and re calculates that data on the fly)
>
> So the answer to your general question, is yes, you can create a
> script to automate retrieving daily visit numbers. All our
> documentation is on google code. There is an interactive query
> explorer that can help you build queries.
>
> And currently it is against the GA Terms of Service to combine any
> Google Analytics data with personally identifiable information, even

> if it's already in yourdata warehouse.

> -Nick
>
> On Oct 12, 10:28 pm, saulat78 wrote:
>
> > Hello
>
> > We got a website, according to GA we have 500,000 Visits per day, is
> > this possible for GA Data Export API to download that 500,000 data on
> > daily basis. We need to store that data in ourdata warehouseand
> > present it to our end users on weekly or daily basis.
>
> > I know there is limit of 10,000 queries per 24 hours and every query
> > can return max 10,000 rows. I am thinking to use pagination to
> > download all the relevant data.
>

> > Is there any documentation or dimensional model available for theData> warehousethat google uses to store our data. I have seen valid

[Nick]

Hi There,

Currently it is against the GA TOS to store personally identifiable
information in Google Analytics as well as correlating Google
Analytics data with personally identifiable information outside of
Google Analytics.

So currently setting a unique id in a custom variable and using it to
do joins within your data warehouse is against the terms of service.

For data requests that are OK to do, yes you can create a cron job to
pull data every night to get the previous days work of data. Many
developers have been doing this and it should work fine. The way we
store our data is a bit different than a standard relational data base
and as such, not all combinations of data are available. Nor do we
have any standard schema for the data. Your best bet will be to define
which keys you'll want from GA and see how many other dimensions and
metrics you can combine with them.

-Nick