How to identify low search volume keywords in TargetingIdeaService?

[Benjamin Morel]

I'm trying to use TargetingIdeaService to filter out low search volume keywords (actually, EAN numbers) before adding them to an ad group, as I was previously advised.

I've been running this service on 1.3 million keywords, and it turned out that only 72,000 of them had the SEARCH_VOLUME attribute set.

If on the other hand, I add all these keywords to a few ad groups (which I did), I can see that 767,000 of them are eligible (systemServingStatus == ELIGIBLE).

I was hoping that TargetingIdeaService would help me identify which keywords are Low Search Volume, and which ones are Eligible, but as far as I can see, the presence of a search volume number in TargetingIdeaService, and the systemServiceStatus of this keyword in an ad group (given similar geo and language targets) seem to be very different matters.

Did I miss something? Can I accurately use TargetingIdeaService to identify low search volume keywords before adding them to my ad groups?

-----

For information, here are the parameters I sent to TargetingIdeaService:

IdeaType=IdeaType::KEYWORD

requestType=RequestType::STATS

requestedAttributeTypes=[KEYWORD_TEXT, SEARCH_VOLUME]

searchParameters=[RelatedToQuerySearchParameter([keyword1, keyword2, ...])]

I tried with and without additional searchParameters, such as LanguageSearchParameter, LocationSearchParameter, NetworkSearchParameter, and the numbers did not change much, to my surprise (just ~10% more keywords with search volumes when removing all parameters).

Thanks in advance for any insight!

[Shwetha Vastrad (AdWords API Team)]

Hi Benjamin, 

Have you tried using the Keyword Optimizer to identify high-quality keywords? You may see a difference in the search volume statistics and the actual traffic your keywords receive for various reasons. Since web traffic is influenced by seasonality, current events, and a number of other factors, the number of searches on your keywords is constantly fluctuating. Your estimated impressions take your bid, budget, and historical ad quality into account, but your search volume statistics don’t. The search volume is determined only for exact keyword matches, while your estimated impressions are based on your selected match types. Please try using the Keyword Optimizer and let me know if the results match your requirements. If you still see a considerable difference in the results from TargetingIdeaService and the actual serving status of your keywords, please send over a few samples along with your clientCustomerId and I'll take a look. Please use Reply privately to author when responding.

Regards,

Shwetha, AdWords API Team.

[Benjamin Morel]

Hi Shwetha,

No I haven't, but I'm not sure how much the Keyword Optimizer would be useful for my use case.

Basically, I just need a way, given a list of keywords, to identify which ones are low search volume (the ones that will yield systemServingStatus==RARELY_SERVED); I don't think (correct me if I'm wrong) that this is related to my account history in any way, as new keywords (unknown to my account) are instantly categorized as ELIGIBLE or RARELY_SERVED once I add them to an ad group.

Because I potentially have more product references than the maximum number of keywords allowed in a single AdWords account, the only way I can think of, at the moment, is to use an ad group dedicated to categorizing keywords, and to:

- add a batch of 20,000 keywords to the ad group

- get the systemServingStatus for each keyword

- remove the keywords from the ad group

- rinse and repeat with the next keyword batch

This would work, but seems highly inefficient (and will leave a huge number of deleted keywords in the account).

Is there another AdWords API service that would allow me to know the systemServingStatus without adding the keywords to an ad group?

Thanks in advance for any help,

Benjamin

[Shwetha Vastrad (AdWords API Team)]

Hi Benjamin,

You need to use Keyword Planner to get keyword ideas and traffic estimates. TargetingIdeaService and TrafficEstimatorService in AdWords API provide functionality similar to the Keyword Planner in the UI. Could you provide a sample of keywords which have systemServingStatus as ELIGIBLE but did not have a searchVolume attribute in the response form TargetingIdeaService?

Thanks,
Shwetha, AdWords API Team.

[Benjamin Morel]

Hi Shwetha,

Sure, there are so many! Here are a few:

9780691034713

9781594603242

9781556509063

9780823013210

9781401323523

It would be nice if you could have a look and let me know why these are eligible but do not have a searchVolume attribute in the response from TargetingIdeaService.

As far as I can see, there is no better solution at the moment than the one I mentioned above (adding keywords, checking their systemServiceStatus, and deleting them).

As I said though, it will leave dozens of millions of deleted keywords in the ad group. May this become a problem, given that deleted keywords are never physically deleted?

Thanks in advance,

Benjamin

[Shwetha Vastrad (AdWords API Team)]

Hi Benjamin, 

Could you enable logging and provide the complete SOAP request and response logs so I can take a look? Since AdGroupCriterionIds are not globally unique, it'll be difficult for me to troubleshoot. Please use Reply privately to author when responding. 

[Benjamin Morel]

SOAP logs sent as private message!

Thank you,

Benjamin

[Thanet Knack Praneenararat (AdWords API Team)]

Hello Benjamin,

Let me confirm with engineering why the search volumes are not returned.

Will get back to you once I get more information.

Cheers,

Thanet, AdWords API Team

[Thanet Knack Praneenararat (AdWords API Team)]

Hello Benjamin,

It turns out that you've not got any search volumes because there are no search volumes at all for those keywords.

That's why the API servers don't return a value.

Hope this help.

Cheers,

Thanet, AdWords API Team

[Benjamin Morel]

Hi Thanet,

Thanks for your reply. I've run a few tests, and it looks like the search volumes are more likely to be returned if the keywords have been sitting there in my account for some time, so unless I missed something, it would look as if the search volumes are tied to the account, not global. This seems to be very different from the systemServingStatus, which according to my tests is not dependent on account history, and not dependent on campaign country/language targeting either.

So I guess what I'm really interested is the systemServingStatus of a keyword; as far as I understand, there is no API to get this value without actually adding the keyword to an ad group?

Is it a problem if I batch add/reload/delete millions of keywords in a single ad group to get all the serving statuses? I would do this as part of a batch job before syncinc the ELIGIBLE ones to a bunch of other, permanent ad groups.

Thanks in advance!

Benjamin

[Thanet Knack Praneenararat (AdWords API Team)]

Hi Benjamin,

Let me confirm with engineering about this too.

If possible, I think we should avoid adding millions of keywords just to get systemServingStatus.

That put unnecessary burdens on the API servers and also waste your rate limits. :)

Best,

Thanet, AdWords API Team

[Benjamin Morel]

That's what I'm afraid of, but the fact is, there doesn't seem to be any alternative at all at the moment!

[Thanet Knack Praneenararat (AdWords API Team)]

Hi Benjamin,

I've been confirmed that the SEARCH_VOLUME returned by TargetingIdeaService, is not affected by your account settings--the result is globally generated.

The systemServingStatus is generated by a different service, so the result could be different.

I can investigate more about it if you still would like to know, but it may take some time. :)

Best,

Thanet, AdWords API Team

[Benjamin Morel]

Hi Thanet,

I'll stick with my solution for now (adding/checking/deleting keywords to get their systemServingStatus) and see if I encounter any issue.

I'm staying within my rate limits and handle RateExceededError appropriately.

Also, I noticed that deleted keywords that haven't had any history seem to be deleted permanently, and therefore do not pollute the ad group needlessly.

So all in all it's less bad than I thought, and seems to be working so far.

Anyway if you know of any other way to get the systemServingStatus without adding a keyword to an ad group, please let me know :)

Otherwise it would be a nice feature request to file.

Thanks for your help anyway!

Benjamin

[Thanet Knack Praneenararat (AdWords API Team)]

Hi Benjamin,

Thanks for your valuable opinion. 

I'll pass this as a feature request to engineering.

Have a nice day.

Thanet, AdWords API Team