Targeting idea for RelatedToUrlSearchParameter returns the same keywords for different pages on a domain

[KG]

Last year, I used RelatedToUrlSearchParameter with idea type KEYWORD when doing keyword research. It worked great.

It used to return results similar to what Display Planner returns, sorted by relevance and for that specific URL (not for domain).

These days, when I try it, it seems to return the same set of keywords for different URLs on a domain.

It returns different sets of keywords for different pages of large sites, but it no longer works for smaller sites. (Not tiny, but smaller than, say, NYtimes or Reddit or sites of that size.)

For example, I would supply URLs like these:

example.com/
example.com/some-page
example.com/another-page

(at different runs not at the same time)

But I would get the same set of keyword ideas for all those URLs. And the content on those pages would be different.

Trying, right now, Display Planner in Adwords provides the expected results (different sets of keyword ideas for different pages on a site), but Adwords API doesn't (the same keywords).

Does anyone else see the same thing or is it something I'm doing wrong on my end?

[Danial Klimkin]

Hello KG,

To make sure we are looking at the same data, could you please provide a few API requests as an examples and corresponding UI screenshots showing different data?

-Danial, AdWords API Team.

[KG]

Thanks for getting back to me.
Here is an example.

===***===
URL #1:
http://www.uglyhedgehog.com/
(a home page of a photography forum)

Top 10 (or first 10) keywords returned by the API:
a canon 7d
buy canon 60d
canon 7d pricing
photography editing programs
buy a canon 60d
canon 60d buy
canon 7d prices
price canon 7d
canon 7d cost
photo editing software programs

Keywords returned by Display Planner:
photography photo gallery
photography wedding photographer
photography
photograph photography
photos photo
royalty free photo images
photographers family photography
photography contest
art photography prints
wildlife photography contest

screenshot of the Display Planner page:
http://www.goudkov.com/temp/aw-api-kw-ideas-01.png

===***===
URL #2:
http://www.uglyhedgehog.com/t-147398-1.html
(a thread discussing UV filters)

keywords returned by the API:
a canon 7d
buy canon 60d
canon 7d pricing
photography editing programs
buy a canon 60d
canon 60d buy
canon 7d prices
price canon 7d
canon 7d cost
photo editing software programs

Keywords returned by Display Planner:
hoya uv filter review
uv filter review
camera uv filters
nikon uv filters
hoya 55mm uv filter
hoya uv filter 62mm
hoya 62mm uv filter
tiffen uv filter
uv filter 55 mm
uv filter 72 mm

screenshot of the Display Planner page:
http://www.goudkov.com/temp/aw-api-kw-ideas-02.png

===***===
URL #3:
http://www.uglyhedgehog.com/t-148205-1.html
(a thread discussing graphics editors, gimp in particular)

keywords returned by the API:
a canon 7d
buy canon 60d
canon 7d pricing
photography editing programs
buy a canon 60d
canon 60d buy
canon 7d prices
price canon 7d
canon 7d cost
photo editing software programs

Keywords returned by Display Planner:
how to make gimp
how to take good photos with a digital camera
gimp how to
how to do gimp
how to use gimp
photo editing gimp
gimp photo editor
how to take great pictures with a digital camera
gimp crop image
gimp

screenshot of the Display Planner page:
http://www.goudkov.com/temp/aw-api-kw-ideas-03.png


=====
As you can see, Display Planner actually gets them right.
The home page is vague, which is to be expected.
But individual thread pages are right on spot.
I just included the home page as well to show that the API always returns the same set of keywords.
In the past, the API always returned the keywords sorted according to relevance.
I haven't seen anything in the docs suggesting that it changed. So I'm assuming, the API is still supposed to return the results based on relevance, which is the default sort order for Display Planner too.
Yet, we can see a discrepancy.
===
Here is the relevant bits of code I use with the client library:

String url="THE USR IS SUPPLIED HERE, ELSEWHER IN THE CODE";
int maxResults=10; //just as a test

TargetingIdeaSelector selector = new TargetingIdeaSelector();
selector.setRequestType(RequestType.IDEAS);
selector.setIdeaType(IdeaType.KEYWORD);
selector.setRequestedAttributeTypes(new AttributeType[] { AttributeType.KEYWORD_TEXT });

RelatedToUrlSearchParameter relatedToUrlSearchParameter = new RelatedToUrlSearchParameter();
relatedToUrlSearchParameter.setUrls(new String[] { url });
relatedToUrlSearchParameter.setIncludeSubUrls(false);

//I'm assuming 2840 is the US
LocationSearchParameter ctsp = new LocationSearchParameter();
Location usLoc = new Location();
usLoc.setId(new Long(2840));
ctsp.setLocations(new Location[] { usLoc });

//I'm assuming 1000 is English
LanguageSearchParameter ltsp = new LanguageSearchParameter();
Language englishLang = new Language();
englishLang.setId(new Long(1000));
ltsp.setLanguages(new Language[] { englishLang });

selector.setSearchParameters(new SearchParameter[] { relatedToUrlSearchParameter, ctsp, ltsp });

Paging paging = new Paging();
paging.setStartIndex(0);
paging.setNumberResults(maxResults);
selector.setPaging(paging);

//tried either of the two methods, the same result
TargetingIdeaPage page = targetingIdeaService.get(selector);
//TargetingIdeaPage page = targetingIdeaService.getBulkKeywordIdeas(selector);

//outside code deals with the list
List kwdList = new LinkedList();
if (page.getEntries() != null && page.getEntries().length > 0) {
    for (TargetingIdea targetingIdea : page.getEntries()) {
        kwdList.add(((StringAttribute) targetingIdea.getData()[0].getValue()).getValue());
    }
}
return kwdList;

[Danial Klimkin]

Thanks for the detailed report. It looks like an issue, we'll look into it internally.

-Danial, AdWords API Team.

[Danial Klimkin]

Hello Konstantin,

I've checked with the team and we found this is expected behavior. TIS currently is the API version of Keyword Planner available in the UI. Comparing TIS and KP results, they fully match.

We currently don't have Display Planner functionality in the API. I'll check if we can introduce it in the future versions.

-Danial, AdWords API Team.

On Monday, September 16, 2013 4:12:50 PM UTC+4, KG wrote:

[KG]

I see. That explains it. I just tried Keyword Planner and see the same results.

Could you please tell me if it's expected for different URLs on the name domain to return the same set of keywords?

Basically, with the Display Planner, the resulting keywords differ based on the theme of the page (as shown in my example).
But with the Keyword Planner, I get the same set of keywords for different URLs on a domain.

It differs for pages/sections of very large websites, but for smaller sites I get the same keywords.

It most definitely used to be different in the past.I remember because I used to rely on this functionality during my keyword research, and it was extremely helpful.

Could you please confirm that this (the same keywords for different pages) is the expected behavior and is not likely to change in the (near) future?

Don't get me wrong, I'm not complaining. I just want to try and plan out how to use this part of the API going forward. Given the current information.

And of course, I'll wait for the possibility of the addition of the Display Planner to the API.

Thanks for the info. I appreciate it.

[Danial Klimkin]

Hello KG,

Yes, this is the expected behavior. I've logged Display Planner support as a feature request and it will be considered for the future API versions.

-Danial, AdWords API Team.

[Evgeni]

Hi,

I'm facing the same issue with v201409 version. Am I missing something or it wasn't changed yet?

Thanks in advance!