Missing keywords in KEYWORDS_PERFORMANCE_REPORT

[Syrtsov Evgeny]

Dear forum,

it's seems that the KEYWORDS_PERFORMANCE_REPORT returns not all
existing keywords.

We have a multi customer system and we download the
KEYWORDS_PERFORMANCE_REPORT for each of them.

Basically we do the following steps before we aggregate later:

1) We parse the downloaded xml from the KEYWORDS_PERFORMANCE_REPORT
and insert all data into our database
2) We are parsing the AD_PERFORMANCE_REPORT and trying to find the
corresponding keywords in our keywords table, but some of keywords are
not exist.

We try to select the keyword by campaign_id, adgroup_id and
keywords_id.
After analyzing the the data we found out that 3.5% of the keywords
are missing. Our total amount is 1.2 Million keywords and 44619
keywords are missing.

At the moment we use the cross-client report form v13 where everything
works fine and we try to switch to the lastest v201109.

We found that problem while trying to obtain the keyword data for the
keywords ids from the AD_PERFORMANCE_REPORT.

What we already tested:
Filesize checks after multiple downloads of the same report
Including zero impression keywords
Internal checks of database problems
Switched between diffrent days (today, last day, -6 and -31 days)
Updated to current PHP Adwords API scripts form googlecode
Diffrent operating systems

Hopefully you got an idea.

[Evgeniy Bogdanov]

Hi Evgeny.

Just to clarify - you don't able to get data for this 44619 keywords
for any parameters set?

Regards,
Evgeniy.

[Syrtsov Evgeny]

Hi Evgeniy,

we download all the data without any parameters. Zero Impressions flag
is set to true.

[Steve B]

We're running into the very same problem for the Keywords Performance Report - a small number of keywords are missing in the report, which causes us problems when we try to cross reference the keyword ID from the Ad Performance Report.

We've tried a variety of predicates to see if it makes any difference, and tried experimenting with requesting and not requesting various fields, and it doesn't make any difference - we get the same set of keywords back, missing several keywords.

This is a kind of critical problem here, so we're hoping some solution has been presented by now.

[Steve B]

Let me clarify that the reason this is critical is that with the keyword related data no longer available in the Ad Performance Report, we *must* cross reference the keyword ID with the Keyword Performance report in order to fill in fields that used to be in the v13 Ad Performance Report.   There is NO other way to get the keyword text, status, quality score, etc.

I'll also mention that the v13 keyword report *does* seem to include the missing keywords, so it's a major discrepancy here.

[Mark Gormley]

In your Keywords Performance Report, try including all the metrics (impressions, clicks, cost, conversions, etc) that you have in your Ad Performance Report.

I ran into a similar problem where I was running an Ad Performance Report that returned a row with impressions = 0 , clicks = 1, and cost = 0.42. When I ran a Keywords Performance Report to get the corresponding keyword data, I wasn't getting that row. I originally ran the Keywords Performance Report with just the impression metric, but with I included clicks or cost I started getting that missing row.

It seems that the reports will only return rows if at least one of the metrics included in it is greater than zero. I also found that predicates didn't really work with values of zero.

It took me a lot of messing around to figure out why I wasn't getting this one keyword. This doesn't really seem like desirable functionality...especially with how many people are merging Ad Performance and Keyword/Criteria Performance Reports.

[Steve B]

Wow.

The mind boggles that this could possibly be necessary to work, but given all the other truly crazy things which we've run into in the process of this migration, I'm sure it does, and am giving it a shot ASAP.

It's especially confusing why this should be the case, when they have fields which *explicitly* state that if you include this field, zero impressions rows won't be returned.   Which therefore implies that zero impressions rows are returned by default.   Which is what we want here.

I have to say I'm *far* from impressed with this switchover -- seems to me Google needs to start using their own APIs so they can get the craziness out of the reports -- *before* inflicting them on the paying customers.

(Oh, and may I add that the "new and improved" Groups/Discussion board UI is the worst thing ever?   I desperately want to go back to the old Groups interface…)

[Steve B]

Alas, that didn't work for me, at least in most cases.   I've did a lot of analysis, cross checking IDs against the v13 report data, and have discovered they all fall into the following categories.

1) Many of the missing keywords are reported in the v13 keyword report as domain names, with a kwType of "Website".

2) Other keywords which are reported in the v13 keyword report with a kwType of "BoomUserList", where the keyword itself is "boomuserlist >> nnnnnn" where nnnnnn is a number.

3) Two of the entries are reported as having a keywordid of 3000000 - and aren't reported in the v13 keyword list either.   In the Ad Performance Reports, this keyword was reported with a kwType of Content and a keyword of Total - content targeting

It's really unclear to me whether the necessary data exists in the Ad Performance Report to deal with these.   I can handle the third case by explicitly checking for that keyword ID, but I don't have any clue how to fill in the blanks on the first two cases.   They don't seem to be covered in the docs or any other discussion on the web, as yet.   Can anyone from Google elaborate on this, please?

[Kevin Winter]

Hi,

1) These "keywords" are part of a feature that has not yet been released in the API - logic was never added to filter it out of v13, hence v13 incorrectly returned this data.  The reporting data returned in the UI also will not include this data.

2) Similar to #1, the UI won't return these values and therefore v201109 reports won't either.

3) This is the special "Content" keyword which represents the clicks/impressions/etc associated with the "sum" of all keywords in an adgroup that caused it to trigger on Content networks (i.e. display) when you added keywords to an adgroup that displays on websites (placements).

V13 uses a completely different reporting backend from both the UI and v201109 reports.  When troubleshooting discrepancies, please compare API reports to the UI, as there is no guarantee that v13 behavior is still possible. (i.e. the product moved in a different direction and v13 didn't keep up).

- Kevin Winter

AdWords API Team

[Steve B]

Yes, they may be getting filtered out of the Keyword Performance Report now, but...  We're getting these rows in the v201109 Ad Performance Report, despite your comment in #2, and the fact the Keyword ID now doesn't map to anything in the Keyword Performance Report is a significant issue.   Either the Ad Performance Report shouldn't be returning those rows, OR there should be a field in the Ad Performance Report which lets us identify those rows and their usage and treat them accordingly, OR the Keyword Performance Report should be returning something which matches those Keyword IDs.   Having those rows dangle with unspecified purpose is problematic -- just leaving them in there with a dangling keyword ID isn't enough for us to work with.

-->Steve  

[Kevin Winter]

Hi Steve,

  Could you please contact me off-list with the ReportDefinition XML you are using and the Customer ID of the account that are seeing these results in the Ad Performance Report?  I'd like to investigate this further.

- Kevin Winter

AdWords API Team

[Kevin Winter]

Hi Steve,

  I ran some tests and spoke with my colleagues.  It turns out I was 50% correct (and 50% wrong).  BoomUserList will not return in v201109 Keywords report as this report only returns keywords.  However, the BoomUserList is the same as our remarketing list criteria in the new API.  These create ARE returning via the CRITERIA_PERFORMANCE_REPORT.  I wrote a quick python script that worked with the results of the AdPerformanceReport and the CriteriaPerformanceReport and cross referenced on AdGroupId and KeywordId (still not named well) and found that for your account the CriteriaPerformanceReport did return values for each keywordId seen in the AdPerformance report.  You should be able to use this as a workaround.  (The only caveat is that KeywordMatchType is not available via the Criteria report (for technical reasons), you'll have to use the Keyword report if you need match type).

- Kevin Winter

AdWords API Team

[Steve B]

Does the Criteria Performance Report also return the Website and Content info?   The vast majority of the missing "Keywords" are website entries.

-->Steve

[Kevin Winter]

Hi Steve,

  The Criteria field is a string representation of the criteria (http://code.google.com/apis/adwords/docs/appendix/reports.html#criteria).  It's format differs depending on the type of the criteria.  For placements, the Criteria field will return a value like this: "bookfactory.de".

- Kevin Winter

AdWords API Team