HTTP request fails with "502: Bad Gateway"

[clafollett]

We've been experiencing horrendous performance from the Google API
today. Upon reviewing our processing logs, I noticed a lot of .NET
exceptions being logged with the message of "The request failed with
HTTP status 502: Bad Gateway". I sure hope that this isn't an
indication of the unscheduled maintenance performed yesterday???

We also received a bunch of the following two errors as well:

"An internal error has occurred. Please retry your request"

"The Attempted modication failed due to change made concurrently by
another agent or user"

Can some one from Google please comment?

Thanks!

[Albert]

We've been getting the same 502 for 2-3 days now. Below is the .net
stack trace

System.Net.WebException: The request failed with HTTP status 502: Bad
Gateway. at
System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage
message, WebResponse response, Stream responseStream, Boolean
asyncCall) at
System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String
methodName, Object[] parameters) at
logic.refCriterion.CriterionServiceService.getAllCriteria(Int32
adGroupId) at logic.webLogic.myCriterions(String[] myDetails, DataTable
dtAdGroups) at ADvance2._5.AdWordsAPI_Creatives.splitLongBatch(DataSet
ds, Int32 maxRows, action myAction, String[] details) at
ADvance2._5.AdWordsAPI_Keywords.Keywords_Action(Int32 campaignID,
String[] myDetails, Boolean testRun, Boolean noDelta, Int32 bidTypeID,
Boolean importOnly)

[michael_zlatanov]

Google what about this problem?
I have it too.

[michael_zlatanov]

This happened on the "getOrders()" method.

[robot]

I have got it on several methods for a couple of weeks now, I try to
avoid it by recovering from the RemoteException thrown (Java) and
trying again after a while.

[clafollett]

My daily processing is taking about 6-8 hours as it is (full pull and
synchronization with my local copies as well as daily "keyword
performance" reports for the previous day) This is on over 100+
accounts with some accounts having over 100K keywords. If I waited to
only try later, my processes would never get done.

> > This happened on the "getOrders()" method.- Hide quoted text -- Show quoted text -

[robot]

Well we have exactly 551 accounts (processing takes about 2-3 hours
with extensive threading ;-)) at the moment, it doesn't matter if you
pause a 10 seconds or so before you retry, allot of times you can
recover after a couple of reties but sometimes there is no other way to
abort of course.

[clafollett]

I've got multiple threads kicked off and can't even come close to 2-3
hours. Are you doing full pulls of all AdGroups, Creatives and Keywords
everyday?

I currently have a max "retry" of 6 times. If it that fails, then the
exception is logged and no further attempts to retry are made. A lot of
times, the failures are happening during a multithreaded operation. For
instance, attempting to get the creatives for 1 adgroup out of several
fails while the other threads are gathering the other AdGroup's
Creatives. So now the processing is left with an incomplete set of
creatives.

[Pete Lavetsky]

Currently getting this every time while querying for criterion stats.

[AdWordsAPIAdvisor]

clafollett wrote:
> We've been experiencing horrendous performance from the Google API
> today. Upon reviewing our processing logs, I noticed a lot of .NET
> exceptions being logged with the message of "The request failed with
> HTTP status 502: Bad Gateway". I sure hope that this isn't an
> indication of the unscheduled maintenance performed yesterday???

It *is* the maintenance we're doing this week. You can expect these
502s for the next 12 hours. If you get one, best bet's to retry your
request.

> We also received a bunch of the following two errors as well:
>
> "An internal error has occurred. Please retry your request"

Are you attempting to add criteria you previously removed? We've got a
bug with that description.

> "The Attempted modication failed due to change made concurrently by
> another agent or user"

This's thrown when concurrent requests try to update the same ad group.
It's more of an issue now given the system slowness.

Sorry about all the problems!

-- Brian Kennish, AdWords API Team

[clafollett]

> > We've been experiencing horrendous performance from the Google API
> > today. Upon reviewing our processing logs, I noticed a lot of .NET
> > exceptions being logged with the message of "The request failed with
> > HTTP status 502: Bad Gateway". I sure hope that this isn't an
> > indication of the unscheduled maintenance performed yesterday???

> It *is* the maintenance we're doing this week. You can expect these
> 502s for the next 12 hours. If you get one, best bet's to retry your
> request.

Bummer.... Retrying is easier said than done.

> > We also received a bunch of the following two errors as well:
>
> > "An internal error has occurred. Please retry your request"

> Are you attempting to add criteria you previously removed? We've got a
> bug with that description.

Its quite possible that keywords that were once removed are being added
again. Our internal system allows them to move keywords around in
Categories (similar to AdGroups) or to change AdCopy. Either of which
could cause a "move" from one AdGroup to another. The only real way to
do this is to delete from one AdGroup and move it to another in your
system.

> > "The Attempted modication failed due to change made concurrently by
> > another agent or user"

> This's thrown when concurrent requests try to update the same ad group.
> It's more of an issue now given the system slowness.

Hmmm..... I'll double check my code but I'm not sure its possible for
the above situation to take place.

> Sorry about all the problems!
>
> -- Brian Kennish, AdWords API Team

Thanks Brain. I appreciate the enormity of a system like AdWords. I
wish I were there working on it rather than here trying to use it.

It wouldn't be so bad if the API didn't cost us an arm and leg to use.
It would be even better if you guys allowed "No Impressions" listings
in custom reports so I wouldn't have to make a bazzilion very costly
calls just to pull down my campaigns and their AdGroups and listings.
HINT HINT!!!! ;-)

[clafollett]

On Jan 11, 1:26 pm, "clafollett" <c...@sendtec.com> wrote:
> > > We also received a bunch of the following two errors as well:
> > > "An internal error has occurred. Please retry your request"

> > Are you attempting to add criteria you previously removed? We've got a
> > bug with that description.

> Its quite possible that keywords that were once removed are being added
> again. Our internal system allows them to move keywords around in
> Categories (similar to AdGroups) or to change AdCopy. Either of which
> could cause a "move" from one AdGroup to another. The only real way to
> do this is to delete from one AdGroup and move it to another in your
> system.

Brian,

After further monitoring, a majority of these errors are indeed from
keywords once deleted being "readded". BUT, on some occassions,
keywords are being added that were never deleted and that do not exist
but the error still occurs. Hopefully I can get some time soon for
further investigation.

[AdWordsAPIAdvisor]

clafollett wrote:
> After further monitoring, a majority of these errors are indeed from
> keywords once deleted being "readded". BUT, on some occassions,
> keywords are being added that were never deleted and that do not exist
> but the error still occurs.

Thanks for the update. Any chance the other faults are coming back with
error code 113? There's a bug filed against those, but we don't know
what's causing them yet.

-- Brian