Best practice to process Big Data?

[Qaiser Javed]

Hi,

Please suggest the quick way to process Google Ads Big Data.

Case Study:

Let's suppose there is an account having 1000 campaigns and 1 million keywords.

I want to get account level insight that how many impressions are there for keyword tail length like 1 word keyword, 2 word keyword up to 5 word keyword. For example:

1 Word Keywords   =   xxxxx impressions

2 Words Keywords = xxxxx impressions

.

.

5+ Words Keywords = xxxxx impressions

For this type of analysis I have to get 1 million keywords in my local database then process locally and make required result.

But this is very expensive call to get 1 million keywords.

What you suggest what is best practice?

OR

Can I process this huge dataset on Google Ads platform and get only required row?

Regards,

Qaiser

[googleadsapi...@google.com]

Hello Qaiser,

Could you please confirm if you are testing using the AdWords API or the Google Ads API? If you are using AdWords API, you could run the report with just the keyword and impression metrics by avoiding the segment fields. This will help in reducing the size of the downloaded file. There is no explicit limit to the rows the API can extract in one go. But if the report is too big, the API may timeout. In such cases, you may need to group your campaigns to limit the number of keywords in one report call. Additionally you can try to use the gzip compressed report data formats as well. One option maybe to use the labels when you create the keyword to group them as 1 word keyword, 2 word keyword etc. In your report, you could filter based on the labels instead of campaigns.

If you are using Google Ads API, you should ideally be able to extract bigger data-set without issues. It would still be recommended to use only the required attributes and metrics in your call.

Thanks,
Sreelakshmi, Google Ads API Team

=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and discussion group:
    http://googleadsdeveloper.blogspot.com/search/label/adwords_api
    https://developers.google.com/adwords/api/community/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~

Was your question answered? Please rate your experience with us by taking a short survey.
If not -- reply to this email and tell us what else we can do to help.

Take Survey

Also find us on our blog and discussion group:
http://googleadsdeveloper.blogspot.com/search/label/adwords_api
https://developers.google.com/adwords/api/community/

--
--
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog:
https://googleadsdeveloper.blogspot.com/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
 
You received this message because you are subscribed to the Google
Groups "AdWords API and Google Ads API Forum" group.
To post to this group, send email to adwor...@googlegroups.com
To unsubscribe from this group, send email to
adwords-api+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/adwords-api?hl=en
---
You received this message because you are subscribed to the Google Groups "AdWords API and Google Ads API Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to adwords-api+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/adwords-api.
To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/8258cf54-b28c-487f-87f2-5e3a55d760e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Qaiser Javed]

Hello Sreelakshmi,

I am using AdWords API.

gzip compression data format is a good option I will try this.

How this is possible to apply Label on keywords as you suggest in some one else campaign programmatically. To do this we still need to apply filter.

Actually the solution comes in my mind there should be an option to apply regular expressions to filer data and get only total row (one row).

If we get bulk rows then we have to filter rows by sending keyword IDs in where clause, there is also a limit of 10000 keywords in where clause in operator. Some metrics possible to total manually like impressions, clicks etc. But %age metrics like Impression Share, EMIS etc not match correctly with Google Ads GUI. So to match these values we have to get Total column from Google Ads.

I hope can elaborate my problem well. Please suggest me.

Regards,

Qaiser

[googleadsapi...@google.com]

Hi Qaiser,

You are right, applying label will work only if you are managing the campaigns and keywords as well. The AdWords API does give the summary row for the fields selected if you set the skipReportSummary to false. But, that is just to aggregate the data in the downloaded report. It is not possible to get just the summary and not the rows using the API. Perhaps, you may need to group the keywords at your end using a separate logic and filter the report using the keyword ids of the same type. If the number of keywords in the predicate is too high, you may have to split the report calls and do the aggregation at your end later. 

If you are worried about hitting the daily report download limit with the basic access, you could request to upgrade that to standard access which will enable you to perform unlimited report downloads in a day. 

Thanks,

Sreelakshmi, Google Ads API Team

=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and discussion group:
    http://googleadsdeveloper.blogspot.com/search/label/adwords_api
    https://developers.google.com/adwords/api/community/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~

[Zweitze]

You will need one report. I am pretty sure one report should be enough to get all records.

Did you try this?

I looked in my logs this morning, and saw that the software downloaded a keyword report with 750,000+ records. Its format was XML_GZIP, 26 MB. Downloading (the time between the request and the last byte received, storing in a memory buffer) took 24 seconds.

Note: that particular report was a keyword structure report - it had much more columns then you need, but impressions were not among them. Anyway, just try.

If you get timeouts, you may want to look at your internet connection. Or consider moving your software in the cloud.

If you tried but didn't get timeouts, what is the exact problem?

[googleadsapi...@google.com]

Hi,

Thank you Zweitze for sharing your experience!

@Qaiser,
I will share your feedback with the team. However, since Google Ads API V1 is released, there is possibility that new features may only be added to the Google Ads API and not in the AdWords API anymore. You could checkout the reporting functionality in Google Ads API. Please check here for more details for migration. 

Thanks,
Sreelakshmi, Google Ads API Team

=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and discussion group:
    http://googleadsdeveloper.blogspot.com/search/label/adwords_api
    https://developers.google.com/adwords/api/community/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~