Many CANCELED for Batch jobs today

[Ершов Александр]

Hello!

We have received a number of CANCELED errors today.

Batch job ID 707432799 has status 'CANCELED'.

Processing error [0]: errorType=BatchJobProcessingError, trigger=, errorString=BatchJobProcessingError.INTERNAL_ERROR,fieldPath=, reason=INTERNAL_ERROR

array(1) {
 [0]=>
 object(Google\AdsApi\AdWords\v201802\cm\BatchJobProcessingError)#49352 (7) {
   ["reason":protected]=>
   string(14) "INTERNAL_ERROR"
   ["fieldPath":protected]=>
   string(0) ""
   ["fieldPathElements":protected]=>
   NULL
   ["trigger":protected]=>
   string(0) ""
   ["errorString":protected]=>
   string(38) "BatchJobProcessingError.INTERNAL_ERROR"
   ["ApiErrorType":protected]=>
   string(23) "BatchJobProcessingError"
   ["parameterMap":"Google\AdsApi\AdWords\v201802\cm\ApiError":private]=>
   array(1) {
     ["ApiError.Type"]=>
     string(12) "ApiErrorType"
   }
 }
}

 [18:38:18 05.04.2018] - [ADWORDS PUSHER]run 2840 operations
[18:38:20 05.04.2018] - [ADWORDS PUSHER]Uploaded 2840. operations for batch job with ID: 707158643
[18:38:20 05.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...
[18:38:24 05.04.2018] - [ADWORDS PUSHER]Batch job ID 707158643 has status ACTIVE. percents: 0
[18:38:24 05.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...
[18:38:28 05.04.2018] - [ADWORDS PUSHER]Batch job ID 707158643 has status CANCELING. percents: 0
[18:38:28 05.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...
[18:38:33 05.04.2018] - [ADWORDS PUSHER]Batch job ID 707158643 has status CANCELED. percents: 0
[18:38:33 05.04.2018] - [ADWORDS PUSHER]report : {"SUCCESS":0,"FAILURE":35}

Etc.

About 20 total for today.

Could you please take a look what has happened

[Peter Oliquino]

Hi,

Could you also provide your clientCustomerId and if possible, the other batch job IDs as well so I can better look into this issue? You may reply using the Reply privately to author option.

Best regards,

Peter

AdWords API Team

[Ершов Александр]

Hello, Peter!

I appreciate you responding!

I sent to you privately some batch jobs privately.

Thank you in advance!

[hong anh]

me to :((

[peey rity]

me to, please, helping me to fix this

On Thursday, April 5, 2018 at 6:44:24 PM UTC-7, Ершов Александр wrote:

[David Fishman]

Noticed this happening randomly since at least April 6. Re-submissions were successful to batch service after waiting a few hours.

[Anash P. Oommen (AdWords API Team)]

Folks,

We had an issue at our end that might have triggered this error. For now, we rolled back the build, so errors should go away. If it continues, feel free to ping this thread with specific job IDs and I'll investigate.

Cheers

Anash P. Oommen,
AdWords API Advisor.

[Ершов Александр]

Hello, Anash P. Oommen!

Thank you very much for response!

For sure we will report, if this errors show up again!

I highly appreciate your efforts on fixing this issue!

Thanks!

[Ершов Александр]

Hello, Anash P. Oommen!

This error has came back today

 [11:43:09 12.04.2018] - [ADWORDS PUSHER]Running handler of the adwords push. account: TO queue: 2018-04-12 10:32:24

 [11:43:10 12.04.2018] - [ADWORDS PUSHER]run 7241 operations

 [11:43:14 12.04.2018] - [ADWORDS PUSHER]Uploaded 7241. operations for batch job with ID: 713387427

 [11:43:14 12.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...

 [11:43:18 12.04.2018] - [ADWORDS PUSHER]Batch job ID 713387427 has status ACTIVE. percents: 0

 [11:43:18 12.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...

 [11:43:22 12.04.2018] - [ADWORDS PUSHER]Batch job ID 713387427 has status CANCELING. percents: 0

 [11:43:22 12.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...

 [11:43:27 12.04.2018] - [ADWORDS PUSHER]Batch job ID 713387427 has status CANCELED. percents: 0

 [11:43:27 12.04.2018] - [ADWORDS PUSHER]report : {"SUCCESS":0,"FAILURE":29}

On Tuesday, April 10, 2018 at 7:00:54 PM UTC+3, Anash P. Oommen (AdWords API Team) wrote:

[Anash P. Oommen (AdWords API Team)]

Hi,

Thanks for confirming, I'll investigate.

Anash

[Ершов Александр]

Hello, Anash!

A lot of jobs failing today

 [14:00:22 18.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...

 [14:00:27 18.04.2018] - [ADWORDS PUSHER]Batch job ID 717691209 has status CANCELED. percents: 0

 [13:57:48 18.04.2018] - [ADWORDS PUSHER]Batch job ID 717644287 has status ACTIVE. percents: 9

 [13:57:48 18.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...

 [13:57:52 18.04.2018] - [ADWORDS PUSHER]Batch job ID 717644287 has status ACTIVE. percents: 9

 [13:57:52 18.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...

 [13:57:56 18.04.2018] - [ADWORDS PUSHER]Batch job ID 717644287 has status ACTIVE. percents: 9

 [13:57:56 18.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...

 [13:58:01 18.04.2018] - [ADWORDS PUSHER]Batch job ID 717644287 has status ACTIVE. percents: 9

 [13:58:01 18.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...

 [13:58:07 18.04.2018] - [ADWORDS PUSHER]Batch job ID 717644287 has status CANCELING. percents: 9

 [13:58:07 18.04.2018] - [ADWORDS PUSHER]Sleeping 4 seconds...

 [13:58:11 18.04.2018] - [ADWORDS PUSHER]Batch job ID 717644287 has status CANCELED. percents: 9

[Anash P. Oommen (AdWords API Team)]

Hi,

The issue should be fixed at our end now. Could you continue monitoring at your end to see if more tasks are failing?

Cheers

Anash P. Oommen,
AdWords API Advisor.

[Ершов Александр]

Hello, Anash!

Thanks! Yep, we will!