Ads API call takes an hour and fails with bad gateway

[Sophie Kühn]

Hello everyone,

since beginning of the week, a Google Ads API search call we do a lot without problems suddenly fails. It takes about an hour and stops then with status code 14, detail "bad gateway". No more details about what was wrong, no headers.

When i retry the query now, it executes just fine, so the reason can't be the query. Only the time of day this happens is always around 4:25 - 4:30 am (UTC+2), while the error itself is then thrown more or less exactly an hour later.

Anyone else an idea what wrong here?

Request
-------
Method Name: /google.ads.googleads.v10.services.GoogleAdsService/Search
Host: googleads.googleapis.com
Headers: {
    "x-goog-api-client": "gl-php\/8.1.5 gccl\/15.0.0 gapic\/15.0.0 gax\/1.12.2 grpc\/1.48.0 rest\/1.12.2",
    "x-goog-request-params": "customer_id=########",
    "developer-token": "REDACTED",
    "login-customer-id": "########"
}
Request:
{"customerId":"########","query":"SELECT campaign.name, campaign.id, campaign.status, metrics.impressions, metrics.clicks, metrics.cost_micros, metrics.conversions, metrics.conversions_value, metrics.search_impression_share, metrics.search_top_impression_share, metrics.search_absolute_top_impression_share, metrics.absolute_top_impression_percentage, metrics.top_impression_percentage FROM campaign WHERE segments.date = \"2022-08-09\" AND campaign.advertising_channel_type != \"HOTEL\""}

Response
-------
Headers: []

Fault
-------
Status code: 14
Details: 502:Bad Gateway
Failure: {}

[Google Ads API Forum Advisor]

Hi Sophie,

Thank you for reaching us out. I am Sherwin from Google Ads API support team. I hope that you are doing well today.

Moving forward to your concern, there could be many possible reasons why you are getting this kind of 502 gateway error. For us to further investigate, could you please provide us the complete request and response logs with request ID and request header generated on your end? You can provide it via Reply privately to author option. If this option is not available, then send it instead on this email address googleadsa...@google.com. If you haven't enabled the logging yet, logging can be enabled by navigating to the Client libraries > Your client library (ex. Java) > Logging documentation, which you can access from this link. Please include also the customer-id as this is important for us to investigate the issue.

Add to that, please check the network traffic when you are getting that error because the network fluctuation could also cause that error.

Kind regards,

Sherwin Vincent Google Ads API Team

ref:_00D1U1174p._5004Q2dl69N:ref

[Shipra]

Hey,
We are facing same issue, The calls are taking 1 hr. and after that are returning 502 Bad Gateway error. Is there any update?
One thing I have observed is this error is happening at same time across multiple calls. i.e at particular time all calls made end up returning Bad Gateway (502).

[Google Ads API and AdWords API Forum]

Re-posting the last inquiry (https://groups.google.com/g/adwords-api/c/paRyst75QKQ) from the forum as it wasn't routed to our support queue.

Kind regards,
Heidi

Google Ads API Team

[Google Ads API and AdWords API Forum]

Hi Shipra,

 

Thank you for reaching out to us.

 

Kindly note that the error '502 Bad Gateway' could be a transient issue which is caused when the servers are not reachable. However, for our team to further investigate and validate the error, could you please also provide us with the complete API logs(request and response with request-id) generated on your end? Note that this can be requested or provided to the developer handling the Google Ads API transactions when logging of the API requests has been enabled. You may check here and click your client library for the specific guideline to enable it.

 

For REST interface requests, you can enable logging via the curl command by using the -i flag. This will cause the tool to include the HTTP response headers in the output.

You may then send the requested logs via the Reply privately to author option. If this option is not available, you may send the details directly to our googleadsa...@google.com alias instead.

Kind regards,
Heidi

Google Ads API Team

On Tuesday, October 18, 2022 at 10:21:08 PM UTC+8 Shipra wrote:

[Shipra]

Hey we are also facing lot of internal error occurred and bad gateway instances, every hours there is one or the other occurrence of this error. One observation is: multiple calls return Internal error at that moment of time. I am sharing logs for few of the occurrences from a big list.

The below are request ids for Internal error occured.
The response we get from api is:

Error received from peer ipv4:142.251.16.95:443
{
  "StatusCode": 13,
  "Details": "Internal error encountered.",
  "RequestId": "dcZW9VUH6th01UsUvyenmQ",
  "Failure": {
    "errors": [
      {
        "errorCode": {
          "internalError": "INTERNAL_ERROR"
        },
        "message": "An internal error has occurred."
      }
    ],
    "requestId": "dcZW9VUH6th01UsUvyenmQ"
  }

Some other request ids, where we have received same error includes:
  1. SL9YC4jWewzUA5jV30A0_Q
  2. dGa0rhA9fxpZHdyl0m3G4g
  3. 0Z4U7tdhI3k5NW16BTHv4Q
  4. DkHq4R6-7Iex_O4DnRbDzg

For request id's of bad gateway errors: as it comes as grpc exception instead of google ads exception we are not able to record request ids, and as the occurrence is not reproable rather it occurs randomly we are not able to use logging method provided in doc.. and practically recording each and every call for request as well as response becomes costly. So we don't have request ids for same.

[Google Ads API Forum Advisor]

Hi Shipra,
 

Thank you for getting back to us. I hope you are doing well today.

Upon seeing the provided logs, it appears that you have encountered an internal error. This error occurs when something unexpected happened while processing the request and the API isn't functioning correctly due to a bug. However, this is transient in nature and the API request will work again when retying it after 30 seconds or some time. That said, please retry your request again to see if this error will still occur.
 

If the issue persists after trying the provided suggestion, then you may provide us with the complete API logs (request and response with request-id) generated on your end so our team can further check this.
 

You can provide it via Reply privately to the author option. If this option is not available, then send it instead on this email address googleadsa...@google.com.

Regards,

Darwin Google Ads API Team

ref:_00D1U1174p._5004Q2dl69N:ref