Random RateExceededError with rateName=RATE_LIMIT using v201806

[Andreas Toresäter]

Hello,

I am having problems getting the GetKeywordIdeas to work reliably. It will randomly throw this error. 

1. I am not making excessive requests, I manually make them from the IDE.

2. When I got the errors I always waited for at least 30 seconds

3. I have tried 3 different accounts, both have spent 10 of thousands for ads and it happens on all of them.

4. The very two first queries I ever did got this error, how can I have exceeded a limit if I never used the API before?

Is there a solution to this issue, or a workaround? It is a great source of information for my customers but I need to be able to guarantee I can deliver it to them. Below is complete SOAP log for last time this happened:

[Thread-1] WARN com.google.api.ads.adwords.lib.client.AdWordsServiceClient.requestInfoLogger - Request made: Service: TargetingIdeaService Method: get clientCustomerId: 723-498-3557 URL: https://adwords.google.com/api/adwords/o/v201806/TargetingIdeaService Request ID: 00058345c16d3c6a0ac1260cc0031364 ResponseTime(ms): 137 OperationsCount: 1 IsFault: true FaultMessage: ApiException{applicationExceptionType=ApiException, errors=[RateExceededError{apiErrorType=RateExceededError, errorString=RateExceededError.RATE_EXCEEDED, fieldPath=, rateName=RATE_LIMIT, rateScope=ACCOUNT, reason=RATE_EXCEEDED, retryAfterSeconds=30, trigger=}]}

[Thread-1] INFO com.google.api.ads.adwords.lib.client.AdWordsServiceClient.soapXmlLogger - SOAP request:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

    <soapenv:Header>

        <ns1:RequestHeader xmlns:ns1="https://adwords.google.com/api/adwords/o/v201806" soapenv:mustUnderstand="0">

            <ns2:clientCustomerId xmlns:ns2="https://adwords.google.com/api/adwords/cm/v201806">723-498-3557</ns2:clientCustomerId>

            <ns3:developerToken xmlns:ns3="https://adwords.google.com/api/adwords/cm/v201806">REDACTED</ns3:developerToken>

            <ns4:userAgent xmlns:ns4="https://adwords.google.com/api/adwords/cm/v201806">Aperico (AwApi-Java, AdWords-Axis/4.4.0, Common-Java/4.4.0, Axis/1.4, Java/1.8.0_191, maven)</ns4:userAgent>

            <ns5:validateOnly xmlns:ns5="https://adwords.google.com/api/adwords/cm/v201806">false</ns5:validateOnly>

            <ns6:partialFailure xmlns:ns6="https://adwords.google.com/api/adwords/cm/v201806">false</ns6:partialFailure>

        </ns1:RequestHeader>

    </soapenv:Header>

    <soapenv:Body>

        <get xmlns="https://adwords.google.com/api/adwords/o/v201806">

            <selector>

                <searchParameters xmlns:ns7="https://adwords.google.com/api/adwords/o/v201806" xsi:type="ns7:RelatedToQuerySearchParameter">

                    <ns7:queries>portable solar charger</ns7:queries>

                    <ns7:queries>best mppt charge controllers</ns7:queries>

                </searchParameters>

                <searchParameters xmlns:ns8="https://adwords.google.com/api/adwords/o/v201806" xsi:type="ns8:LanguageSearchParameter">

                    <ns8:languages>

                        <ns9:id xmlns:ns9="https://adwords.google.com/api/adwords/cm/v201806">1000</ns9:id>

                    </ns8:languages>

                </searchParameters>

                <searchParameters xmlns:ns10="https://adwords.google.com/api/adwords/o/v201806" xsi:type="ns10:NetworkSearchParameter">

                    <ns10:networkSetting>

                        <ns11:targetGoogleSearch xmlns:ns11="https://adwords.google.com/api/adwords/cm/v201806">true</ns11:targetGoogleSearch>

                        <ns12:targetSearchNetwork xmlns:ns12="https://adwords.google.com/api/adwords/cm/v201806">false</ns12:targetSearchNetwork>

                        <ns13:targetContentNetwork xmlns:ns13="https://adwords.google.com/api/adwords/cm/v201806">false</ns13:targetContentNetwork>

                        <ns14:targetPartnerSearchNetwork xmlns:ns14="https://adwords.google.com/api/adwords/cm/v201806">false</ns14:targetPartnerSearchNetwork>

                    </ns10:networkSetting>

                </searchParameters>

                <ideaType>KEYWORD</ideaType>

                <requestType>IDEAS</requestType>

                <requestedAttributeTypes>KEYWORD_TEXT</requestedAttributeTypes>

                <requestedAttributeTypes>SEARCH_VOLUME</requestedAttributeTypes>

                <requestedAttributeTypes>AVERAGE_CPC</requestedAttributeTypes>

                <requestedAttributeTypes>COMPETITION</requestedAttributeTypes>

                <paging>

                    <ns15:startIndex xmlns:ns15="https://adwords.google.com/api/adwords/cm/v201806">0</ns15:startIndex>

                    <ns16:numberResults xmlns:ns16="https://adwords.google.com/api/adwords/cm/v201806">100</ns16:numberResults>

                </paging>

            </selector>

        </get>

    </soapenv:Body>

</soapenv:Envelope>

[Thread-1] INFO com.google.api.ads.adwords.lib.client.AdWordsServiceClient.soapXmlLogger - SOAP response:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">

    <soap:Header>

        <ResponseHeader xmlns="https://adwords.google.com/api/adwords/o/v201806" xmlns:ns2="https://adwords.google.com/api/adwords/cm/v201806">

            <ns2:requestId>00058345c16d3c6a0ac1260cc0031364</ns2:requestId>

            <ns2:serviceName>TargetingIdeaService</ns2:serviceName>

            <ns2:methodName>get</ns2:methodName>

            <ns2:operations>1</ns2:operations>

            <ns2:responseTime>137</ns2:responseTime>

        </ResponseHeader>

    </soap:Header>

    <soap:Body>

        <soap:Fault>

            <faultcode>soap:Server</faultcode>

            <faultstring>[RateExceededError &lt;rateName=RATE_LIMIT, rateKey=null, rateScope=ACCOUNT, retryAfterSeconds=30&gt;]</faultstring>

            <detail>

                <ApiExceptionFault xmlns="https://adwords.google.com/api/adwords/o/v201806" xmlns:ns2="https://adwords.google.com/api/adwords/cm/v201806">

                    <ns2:message>[RateExceededError &lt;rateName=RATE_LIMIT, rateKey=null, rateScope=ACCOUNT, retryAfterSeconds=30&gt;]</ns2:message>

                    <ns2:ApplicationException.Type>ApiException</ns2:ApplicationException.Type>

                    <ns2:errors xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:RateExceededError">

                        <ns2:fieldPath/>

                        <ns2:trigger/>

                        <ns2:errorString>RateExceededError.RATE_EXCEEDED</ns2:errorString>

                        <ns2:ApiError.Type>RateExceededError</ns2:ApiError.Type>

                        <ns2:reason>RATE_EXCEEDED</ns2:reason>

                        <ns2:rateName>RATE_LIMIT</ns2:rateName>

                        <ns2:rateScope>ACCOUNT</ns2:rateScope>

                        <ns2:retryAfterSeconds>30</ns2:retryAfterSeconds>

                    </ns2:errors>

                </ApiExceptionFault>

            </detail>

        </soap:Fault>

    </soap:Body>

</soap:Envelope>

Request failed due to ApiException. Underlying ApiErrors:

  Error 0: RateExceededError{apiErrorType=RateExceededError, errorString=RateExceededError.RATE_EXCEEDED, fieldPath=, rateName=RATE_LIMIT, rateScope=ACCOUNT, reason=RATE_EXCEEDED, retryAfterSeconds=30, trigger=}

Thankful for any advice!

[googleadsapi...@google.com]

Hello Andreas, 

Based on the details shared, it looks like you're receiving an error with the rate scope as Account. This indicates that application is making a high number of requests per second on a single AdWords account. You could wait at least the amount of time specified in the retryAfterSeconds and retry the request again. You will need to consider all requests made against that account irrespective of whether the call was made via API or web interface. Please note that the API call might still fail as the rate limits are not based on a constant QPS and are subjected to change. 

For the Rate limits, when you retry the requests, your application should use an exponential backoff policy. For example, if you first pause 5 seconds before the first retry, you could pause 10 seconds after the second and 20 seconds after the third retry. Exponential backoff helps ensure you are not calling the API too aggressively. This guide has best practices that can be implemented to handle the rate limit errors. Please take a look at the system limits guide for the TargetingIdea specific limits. You can also take a look at Batching Targeting Ideas to make requests more efficiently. Could you please give this a try and let me know if you're still having an issue?

Thanks,
Bharani, AdWords API Team

=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and discussion group:
    http://googleadsdeveloper.blogspot.com/search/label/adwords_api
    https://developers.google.com/adwords/api/community/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~

Was your question answered? Please rate your experience with us by taking a short survey.
If not -- reply to this email and tell us what else we can do to help.

Take Survey

Also find us on our blog and discussion group:
http://googleadsdeveloper.blogspot.com/search/label/adwords_api
https://developers.google.com/adwords/api/community/

--
--
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog:
https://googleadsdeveloper.blogspot.com/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
 
You received this message because you are subscribed to the Google
Groups "AdWords API and Google Ads API Forum" group.
To post to this group, send email to adwor...@googlegroups.com
To unsubscribe from this group, send email to
adwords-api+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/adwords-api?hl=en
---
You received this message because you are subscribed to the Google Groups "AdWords API and Google Ads API Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to adwords-api+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/adwords-api.
To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/b2c6a5e0-14ad-4841-8ccc-9af039154b60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andreas Toresäter]

Hello and thank you for the answer.

From the details I can read on all this there must then be a specific problem with my dev token and a copy paste standard answer will not do?

Based on the details shared, it looks like you're receiving an error with the rate scope as Account. This indicates that application is making a high number of requests per second on a single AdWords account.

The details I shared was that there are minutes apart of the requests.

You could wait at least the amount of time specified in the retryAfterSeconds and retry the request again.

Yes, as I said in my question I do that.

 You will need to consider all requests made against that account irrespective of whether the call was made via API or web interface.

I am the only one making calls for these 3 accounts, and I do it by manually doing one query with one operation at the time from my IDE.

Please note that the API call might still fail as the rate limits are not based on a constant QPS and are subjected to change.

What exactly does this mean? Do you say I can get this error even if I do let's say 1 request with one operation every hour?

For the Rate limits, when you retry the requests, your application should use an exponential backoff policy.

How will this help if I still get this error when I just tried a query after waiting for the whole night? 

Please take a look at the system limits guide for the TargetingIdea specific limits.

I am not getting any of those errors and I am well below the limits for seed keywords, at the moment I am testing with 2-3. 

You can also take a look at Batching Targeting Ideas to make requests more efficiently.

I will look into this but I do not see how it will help in my case since I get the error when I am using only a fraction of the rate limits you publish.

Please look into what is wrong with my dev token, thank you!

[Andreas Toresäter]

*edit*

I just saw that I am already using batching, in fact, your SDK example is by default. I have not got further than to use your own SDK example: GetKeywordIdeas.java, since I can not reliably use it without getting random rate exceeded errors.

// Create related to query search parameter.
RelatedToQuerySearchParameter relatedToQuerySearchParameter =
    new RelatedToQuerySearchParameter();

relatedToQuerySearchParameter.setQueries(keywords.toArray(new String[0]));
searchParameters.add(relatedToQuerySearchParameter);

[googleadsapi...@google.com]

Hi Andreas,

I looked into your account from your SOAP XML and noticed the account does not have any active ads. Please read this section which is relevant to your issue. My recommendation would be choosing an account that has an active campaign with recent ad spend, or create a test account to test the code sample. 

Thanks,
Poki, AdWords API Team

=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and discussion group:
    http://googleadsdeveloper.blogspot.com/search/label/adwords_api
    https://developers.google.com/adwords/api/community/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~

[Andreas Toresäter]

Thank you for your reply,

I had read that section a couple of times when searching for the problem. I do not interpret the text the way that you need active ads? It only says that new accounts can have this problem?

When you implement queuing or rate limiter strategies, it's important to keep in mind that a new Google Ads account may have a significantly more restrictive rate limit (i.e., lower QPS) than established accounts.

 

Typically, the more restrictive limits on new Google Ads accounts are relaxed once the account has published ads. 

Maybe I do not understand the definition of established account, two of the accounts I am testing on are started years ago and have spent 10s of thousands EUR on ads. Are they not considered established?  Would having an active ad campaign solve this even though it is never mentioned in that section? I don't want to be confronting, I just want to be sure if this is indeed a fix.

Thank you!

[googleadsapi...@google.com]

Hi Andreas,

I'm looking into the exact issue and explanation. I will keep you posted once I have an update. Thank you very much for your patience. 

Thanks,
Poki, AdWords API Team
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and discussion group:
    http://googleadsdeveloper.blogspot.com/search/label/adwords_api
    https://developers.google.com/adwords/api/community/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~

[googleadsapi...@google.com]

Hi Andreas,

Would you mind running these examples and see if you get rate limited errors?

- Get keyword traffic estimates

- Get campaigns

I'm trying to see if this is service specific issue. 

Thanks,

Poki, AdWords API Team
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and discussion group:
    http://googleadsdeveloper.blogspot.com/search/label/adwords_api
    https://developers.google.com/adwords/api/community/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~

On 03/07/19 01:38:33 aperi...@gmail.com wrote:

[Andreas Toresäter]

Hello,

thank you for getting back to me!

I ran "Get keyword traffic estimates" SDK example 30 times about 10 seconds apart and got no rate exceeded. Then I tried the GetKeywordIdeas SDK example and got rate exceeded RATE_LIMIT on the second try.

I also started running new ads on the 726-xxx-xxx account (which is the account I use for API queries now) three days ago to see if that would indeed help. It has displayed about 20k ads by now (I do not want to spend much money on ads before I get this sorted). It seems it did not help, are there certain limits to spendings or number of ads displayed etc that need to be met to unlock the API rate limit? If this is indeed the case that you need active ads running?

[googleadsapi...@google.com]

Hi Andreas,

Based on what you shared, it seems like you are only rate limited to TargetingIdeaService, which is not related to active ads. I'll look further into it. Sorry for the confusion.

Thanks,

Poki, AdWords API Team
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and discussion group:
    http://googleadsdeveloper.blogspot.com/search/label/adwords_api
    https://developers.google.com/adwords/api/community/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~

[googleadsapi...@google.com]

Hi Andreas,

This issue should be fixed. Can you please verify if you still experience the same issue?

Thanks,

Poki, Google Ads API Team

=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and discussion group:
    http://googleadsdeveloper.blogspot.com/search/label/adwords_api
    https://developers.google.com/adwords/api/community/
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~

On 03/13/19 02:57:22 aperi...@gmail.com wrote: