[v201209] Discrepancy between API results and web UI search volume results

[Sebastiaan Moeys]

I'm migrating my systems From V201109 to V201209, and I'm struggling with a number of things.

This issue is related to retrieving search volume for keywords I already know.

Googler David Torres said:

The new SEARCH_VOLUME is equivalent to GLOBAL_MONTHLY_SEARCHES when no LocationSearchParameter and LanguageSearchParameter are specified, on the flip side SEARCH_VOLUME is equivalent to AVERAGE_TARGETED_MONTHLY_SEARCHES if a language and/or a location is required.

So I did a test with the keyword 'games':

• API v201209 | No Targeting: SEARCH_VOLUME 20,400,000
  
• API v201109 | EXACT: GLOBAL_MONTHLY_SEARCHES 16,600,000
  
• Keyword Tool Web UI | EXACT: Global Monthly Searches 16,600,000
  

That's about 20% of non-equivalence.

Then I did a local/targeted comparison with the Dutch word 'spelletjes', meaning games:

• API v201209 | Dutch | The Netherlands: SEARCH_VOLUME 1,000,000
  
• API v201109 | EXACT | Dutch | The Netherlands: AVERAGE_TARGETED_MONTHLY_SEARCHES 823,000
  
• Keyword Tool Web UI | EXACT | Dutch | The Netherlands: Global Monthly Searches 823,000

Right now there will be a split between results from the web interface and the API. This split is causing trouble in the team I work in.

Some questions:

1. What is the exact difference in behaviour between the two API versions? Why isn't this difference documented? Why isn't there a migration guide?
   
2. Are you planning on permanently removing match type targeting (i.e. broad/exact/phrase)? If so, could you tell why? Something else than 'back-end changes' would be appreciated. Those back-end changes are made for a reason. Could be a good reason, but we'd like to know.
3. Do you plan on keeping the pre-v2012 features in the UI? That's my greatest concern, since different people will be working with different data.

Otherwise very happy to be working with the API you provided, but would like to hear from you about this.

[Martin]

Hello,

i have the same problem.

The values are equal with the Column "google-search-ad-network" (sorry for that translation, i hope you know what i mean) in the External-Keyword-Tool not with the monthly-local-search.


Here is my java-code:

        AdWordsUser user = new AdWordsUser();

        TargetingIdeaServiceInterface targetingIdeaService = user
                .getService(AdWordsService.V201209.TARGETING_IDEA_SERVICE);

        // Create selector.
        TargetingIdeaSelector selector = new TargetingIdeaSelector();
       
       
        selector.setRequestType(RequestType.STATS);
        selector.setIdeaType(IdeaType.KEYWORD);

        selector.setRequestedAttributeTypes(new AttributeType[] {
                AttributeType.KEYWORD_TEXT,
                AttributeType.SEARCH_VOLUME,
        });

        Language language = new Language();
        language.setId(1001L);

        // Countrycodes
        // http://code.google.com/apis/adwords/docs/appendix/countrycodes.html
        Location location = new Location();
        location.setId(2276L);

        RelatedToQuerySearchParameter relatedToQuerySearchParameter = new RelatedToQuerySearchParameter();
        relatedToQuerySearchParameter.setQueries(keywords
                .toArray(new String[0]));
       

        LocationSearchParameter locationSearchParameter = new LocationSearchParameter();
        locationSearchParameter.setLocations(new Location[]{location});
       
        LanguageSearchParameter languageSearchParameter = new LanguageSearchParameter();
        languageSearchParameter.setLanguages(new Language[]{language});
       
        selector.setSearchParameters(new SearchParameter[] { relatedToQuerySearchParameter, locationSearchParameter, languageSearchParameter });

        selector.setLocaleCode("DE");

        Paging paging = new Paging();
        paging.setStartIndex(0);
        paging.setNumberResults(keywords.size());
        selector.setPaging(paging);

        TargetingIdeaPage page = targetingIdeaService.get(selector);
        TargetingIdea[] ideaArray = page.getEntries();

[Martin]

Any Idea???

[veseo]

I'm having the same problem, described here: https://developers.google.com/adwords/api/community/?place=topic%2Fadwords-api%2F_Jx6stWJ7zk%2Fdiscussion

[Sébastien de Salvador]

Same problem here with the .NET API, here's the code:

            TargetingIdeaPage page;

            var user = new AdWordsUser();

            var searchVolumes = new Dictionary< string, long >();

            var targetingIdeaService = ( TargetingIdeaService )user.GetService( AdWordsService.v201209.TargetingIdeaService );

            var paging = new Paging { startIndex = 0, numberResults = 3 };

            var keywordSelector = new TargetingIdeaSelector

                {

                    requestType = RequestType.STATS,

                    ideaType = IdeaType.KEYWORD,

                    requestedAttributeTypes = new[]

                        {

                            AttributeType.KEYWORD_TEXT,

                            AttributeType.SEARCH_VOLUME

                        },

                    paging = paging

                };

            var relatedToQuerySearchParameter = new RelatedToQuerySearchParameter { queries = new[] { "jobbörse", "jobsuche", "stellenangebote" } };

            var locationSearchParameter = new LocationSearchParameter { locations = new[] { new Location { id = 2276, CriterionType = "Location" } } };

            var deviceTypeSearchParameter = new DeviceTypeSearchParameter { deviceType = DeviceType.DESKTOPS_AND_LAPTOPS };

            keywordSelector.searchParameters = new SearchParameter[] { relatedToQuerySearchParameter, locationSearchParameter, deviceTypeSearchParameter };

            page = targetingIdeaService.get( keywordSelector );

            if ( page.entries == null || page.entries.Length <= 0 ) return;

            foreach ( var targetingIdea in page.entries )

            {

                var keyword = ( targetingIdea.data.Single( d => d.key == AttributeType.KEYWORD_TEXT ).value as StringAttribute ).value;

                var searchVolume = ( targetingIdea.data.Single( d => d.key == AttributeType.SEARCH_VOLUME ).value as LongAttribute ).value;

                searchVolumes.Add( keyword, searchVolume );

            }

            foreach ( var searchVolume in searchVolumes )

                Console.WriteLine( "{0}, {1}", searchVolume.Key, searchVolume.Value );

Output:

jobbörse, 1830000

jobsuche, 110000

stellenangebote, 368000

Where the UI reports the following:

jobbörse, 1,830,000

jobsuche, 90,000

stellenangebote, 246,000

The only thing I wasn't able to specify in code is the match type ("exact") but I think I read somewhere that it actually defaults to that when not specified...

[Sébastien de Salvador]

Actually I've just noticed that using the attribute SEARCH_VOLUME returns the value from the "Google Search Network" column, how can I get the value from the "Local Monthly Searches" column?

[Sebastiaan Moeys]

Sebastien:

1) Match type is no longer supported in newer API versions. One does wonder why it is still present in the Web UI, still waiting for a response from Google there.

2) SEARCH_VOLUME will return local results if you specify location targeting criteria. Those results seem to be slightly different, but strongly correlated, with local results in the Web UI.

[Sébastien de Salvador]

Ah well, I guess we'll stick to v201109 for now since it returns the right numbers...

[Sébastien de Salvador]

So my code is actually right, thanks for your answer, but the business wants those exact figures, I'm just the dev in the story so I'll stick to the older API :)

[Kevin Winter]

We recently published a blog post talking about these numbers: http://googleadsdeveloper.blogspot.com/2012/11/understanding-adwords-api.html

Please note that v201109 has sunset and should not be used.

- Kevin Winter

AdWords API Team

[Sébastien de Salvador]

As far as I'm concerned I think this is a poor decision, the company I work for relies on those numbers and explicitly asked for the local monthly volumes without partners data (or google network as you call it).

Don't get me wrong I'm not being judgmental, I'm just the dev in the story, but if for some reason one day I decide to change the behaviour of my applications/code without notice I don't think the business will appreciate it...

How long will the 201109 version be supported? The code still works for the moment... 

[Kevin Winter]

v201109 officially sunset on 10/26: https://developers.google.com/adwords/api/docs/sunset-dates

You should migrate off this version immediately.

- Kevin Winter

AdWords API Team

[Adrian Perez]

Same results everywhere from everyone.

I'm not judge of anything, but I don't really like this move. 

I mean, what's in for users?

The takeaway here is, and correct me if I'm wrong, that you can't simply get global monthly searches without the google partners results in new api versions, coupled to the fact that old API versions are no longer usable, leaves the clients with little to none alternatives.

I'm using the global network, as a last resort, I have to make two separate requests (let me rephrase, the upgrade has forced me to make two separate requests) in order to do a crazy nested hash merge of the global results I want.

Is we API users who suffer, at least I'm suffering ;)

If they were thinking to include a way to prevent that from happening, such as a filter, IMHO this should have been carried over first, or at least not deprecate the old functionality leaving us without options.

Have you found other alternatives to getting those results? Feel free to post if you do. I just did with what I've done.

[Adwords user]

On Friday, November 9, 2012 12:48:39 AM UTC+8, Kevin Winter wrote:

We recently published a blog post talking about these numbers: http://googleadsdeveloper.blogspot.com/2012/11/understanding-adwords-api.html

 
Any update on the "plan" mentioned in that blog post? Now that v201109 is officially offline (it still worked until Nov. 19), we don't have any way to retrieve the correct local search volume of keywords.

Just like what Adrian Perez said, the API behavior shouldn't have been changed in the first place. And if it's really needed, you should've at least left your clients a way to retrieve the same old values - either by leaving the deprecated APIs online or by immediately releasing a new version that supports those values

[Kevin Winter]

The earliest we'd be able to add such support is in the next major release.  See this blog post about our published release schedule.

- Kevin Winter

AdWords API Team

[Sébastien de Salvador]

Haha! That's a joke right?
You guys really take the piss, and you really get too much money and credits for the poor service you provide...
Not once did I see a customer oriented answer, and not once did you guys apologise or at least pretend to be sorry...

Amazing...

[Adwords user]

In case anyone stumbled on this thread, this has been addressed in v201302. Even though the migration notes didn't mention anything about it, a new search parameter has been added to support custom network targeting. In PHP, the required code is as follows:

<?php

$networkSetting = new NetworkSetting();

$networkSetting->targetContentContextual = false;

$networkSetting->targetContentNetwork = false;

$networkSetting->targetGoogleSearch = true;

$networkSetting->targetSearchNetwork = false;

$networkSetting->targetPartnerSearchNetwork = false;

$networkSearchParameter = new NetworkSearchParameter();

$networkSearchParameter->networkSetting = $networkSetting;

// And don't forget to add the parameter in your selector object

$selector->searchParameters[] = $networkSearchParameter;

?>

[Afrooz Shaikh]

I just came Accross this issue, initialy i taught every thing was working fine because the UI and API wer Giving same data, bt as the number of keywords kept on increasing i saw difference of about 100 times in the search volume of 30% of the keywords i used. 

package uber.keyword.bussiness;


import java.io.IOException;
import java.sql.CallableStatement;
import java.sql.SQLException;
import java.util.HashMap;
import java.util.Map;


import com.google.api.ads.adwords.axis.factory.AdWordsServices;
import com.google.api.ads.adwords.axis.v201406.cm.ApiException;
import com.google.api.ads.adwords.axis.v201406.cm.Language;
import com.google.api.ads.adwords.axis.v201406.cm.Location;
import com.google.api.ads.adwords.axis.v201406.cm.Money;
import com.google.api.ads.adwords.axis.v201406.cm.Paging;
import com.google.api.ads.adwords.axis.v201406.o.Attribute;
import com.google.api.ads.adwords.axis.v201406.o.AttributeType;
import com.google.api.ads.adwords.axis.v201406.o.IdeaType;
import com.google.api.ads.adwords.axis.v201406.o.LanguageSearchParameter;
import com.google.api.ads.adwords.axis.v201406.o.LocationSearchParameter;
import com.google.api.ads.adwords.axis.v201406.o.LongAttribute;
import com.google.api.ads.adwords.axis.v201406.o.MoneyAttribute;
import com.google.api.ads.adwords.axis.v201406.o.RelatedToQuerySearchParameter;
import com.google.api.ads.adwords.axis.v201406.o.RequestType;
import com.google.api.ads.adwords.axis.v201406.o.SearchParameter;
import com.google.api.ads.adwords.axis.v201406.o.StringAttribute;
import com.google.api.ads.adwords.axis.v201406.o.TargetingIdea;
import com.google.api.ads.adwords.axis.v201406.o.TargetingIdeaPage;
import com.google.api.ads.adwords.axis.v201406.o.TargetingIdeaSelector;
import com.google.api.ads.adwords.axis.v201406.o.TargetingIdeaServiceInterface;
import com.google.api.ads.adwords.axis.v201406.o.Type_AttributeMapEntry;
import com.google.api.ads.adwords.lib.client.AdWordsSession;
import com.google.api.ads.common.lib.auth.OfflineCredentials;
import com.google.api.ads.common.lib.auth.OfflineCredentials.Api;
import com.google.api.ads.common.lib.conf.ConfigurationLoadException;
import com.google.api.ads.common.lib.exception.OAuthException;
import com.google.api.ads.common.lib.exception.ValidationException;
import com.google.api.client.auth.oauth2.Credential;
import com.google.api.client.googleapis.auth.clientlogin.ClientLoginResponseException;


import java.sql.Connection;


import uber.connection.Dbcon;
import uber.keywordbeans.AdwordBean;
import uber.keywordbeans.GoogleAPIBean;


public class Calculate_CPC_MCC {


 private static final String clientId =
  private static final String clientSecret =
  private static final String refreshToken =
  private static final String developerToken =
  private static final String userAgent =
 
 public AdwordBean getCPC_MCC(String keyword) throws ClientLoginResponseException, IOException, ValidationException,
 
    ConfigurationLoadException, OAuthException, ApiException, ClassNotFoundException, InstantiationException, IllegalAccessException, SQLException
 {
 String[] keywords=new String[]{keyword};
 String finalresult="";
 String[] locationNames = new String[] { "Paris", "Quebec", "Spain", "Deutschland" };
        String kwd=keyword;
        Long monthlySearches=null;
        String monthltSearches_string="--";
        Money avgCpc=null;
        int k_id=0;
        String k="--";
        AdwordBean ad=new AdwordBean();
        Connection con=new Dbcon().getConnection();
      try {
 
      Credential oAuth2Credential = new OfflineCredentials.Builder()
          .forApi(Api.ADWORDS)
          .withClientSecrets(clientId, clientSecret)
          .withRefreshToken(refreshToken)
          .build()
          .generateCredential();


          AdWordsSession session = new AdWordsSession.Builder()
          .withDeveloperToken(developerToken)
          .withUserAgent(userAgent)
          .withOAuth2Credential(oAuth2Credential)
          .build();
   


      AdWordsServices adWordsServices = new AdWordsServices();
     
      TargetingIdeaServiceInterface targetingIdeaService = adWordsServices.get(session,TargetingIdeaServiceInterface.class);

      TargetingIdeaSelector selector = new TargetingIdeaSelector();


      selector.setRequestType(RequestType.STATS);
      selector.setIdeaType(IdeaType.KEYWORD);

      selector.setRequestedAttributeTypes(new AttributeType[] { AttributeType.KEYWORD_TEXT, AttributeType.SEARCH_VOLUME, AttributeType.AVERAGE_CPC });

      Language language = new Language();

      language.setId(1000L);

 
      Location location = new Location();

      location.setId(2356L);

      RelatedToQuerySearchParameter relatedToQuerySearchParameter = new RelatedToQuerySearchParameter();

      relatedToQuerySearchParameter.setQueries(keywords);

      LocationSearchParameter locationSearchParameter = new LocationSearchParameter();
      locationSearchParameter.setLocations(new Location[] { location });


      LanguageSearchParameter languageSearchParameter = new LanguageSearchParameter();
      languageSearchParameter.setLanguages(new Language[] { language });


      selector.setSearchParameters(new SearchParameter[] { relatedToQuerySearchParameter, locationSearchParameter,

              languageSearchParameter // if not provided locationSearchParameter, languageSearchParameter then result
                                      // is global
      });


      selector.setLocaleCode("IN");
     // System.out.println(selector.getLocaleCode());

      Paging paging = new Paging();
      paging.setStartIndex(0);

      paging.setNumberResults(keywords.length);

      selector.setPaging(paging);


      TargetingIdeaPage page = targetingIdeaService.get(selector);

      if (page.getEntries() != null && page.getEntries().length > 0) {
          for (TargetingIdea targetingIdea : page.getEntries()) {
              Map<AttributeType, Attribute> data = toMap(targetingIdea.getData());
              kwd = ((StringAttribute) data.get(AttributeType.KEYWORD_TEXT)).getValue();
               monthlySearches = ((LongAttribute) data.get(AttributeType.SEARCH_VOLUME)).getValue();
               avgCpc = ((MoneyAttribute) data.get(AttributeType.AVERAGE_CPC)).getValue();
             System.out.println(monthlySearches);
          System.out.println( kwd);
              if ((avgCpc==null)|| avgCpc.equals("")){
            k="--";
           
              //finalresult =  kwd + "@@NULL@@" + monthlySearches;
             // System.out.println(kwd + ", " + k + ", " + monthlySearches );
              }
              if((monthlySearches==null|| monthlySearches.equals(""))){
             monthltSearches_string="--";
              }
              if(avgCpc!= null && monthlySearches!= null) {
             monthltSearches_string=monthlySearches.toString();
            k=String.format("%.1f",(avgCpc.getMicroAmount().floatValue()/1000000.0)) +"";
            //finalresult =  kwd + "@@" + avgCpc.getMicroAmount() / 1000000.0 + "@@" + monthlySearches ;
              //System.out.println(kwd + ", " + avgCpc.getMicroAmount() / 1000000.0 + ", " + monthlySearches);
              }
           
          }
      }
     
//      CallableStatement cs1=con.prepareCall("{call sp_RetriveKwId(?,?)}");
// cs1.setString(1,kwd );
// cs1.registerOutParameter(2,java.sql.Types.INTEGER);
// cs1.executeUpdate();
// k_id=cs1.getInt(2);
//      
//      
//      ad.setCpc(k);
//      ad.setKw_id(k_id+"");
//      ad.setKeyword(kwd);
//      ad.setMCC(monthltSearches_string);
//      ad.setFlag(true);
     
      }
      catch (Exception e){
      e.printStackTrace();
      }
     
 
 return ad;
 
 
 }
 
   
    public static Map<AttributeType, Attribute> toMap(Type_AttributeMapEntry[] data) {
      Map<AttributeType, Attribute> result = new HashMap<AttributeType, Attribute>();
      for (Type_AttributeMapEntry entry: data) {
          result.put(entry.getKey(), entry.getValue());
      }
      return result;        
  }
    public static void main(String args[]) throws ApiException, ClientLoginResponseException, OAuthException, ClassNotFoundException, InstantiationException, IllegalAccessException, IOException, ValidationException, ConfigurationLoadException, SQLException
    {
    new Calculate_CPC_MCC().getCPC_MCC("Ballerina");
    }
}

I hav used the location and the language setings and it gives the output for balerina as "Ballerina" for Location India Language English it shows 33100, where as in the UI the it shows 1,900. Y is there such a vast differnce?? it is the same case for many keywords, like Boots . Plz Chck this. @kevin

[Danial Klimkin]

This thread is a few years old. Please create a new one instead of reopening such an old thread.

-Danial.

...