Batch job processing speed

[aleksand...@rocket-internet.com]

I am testing the upload of a batch job of:

975 campaigns;

64720 ad groups;

1242360 keywords;

242700 expanded ads;

1551730 items total;

to a test account 212-023-9393; batch job id 528294875. After uploading, the batch job is in "ACTIVE" status and is being processed by the adwords backend; I monitor the job's status.

After half an hour I look at the account and see all the campaigns, ad groups and 13142 keywords appeared. I thus estimate the entire job will be processed in 9 hours.

Questions:
1. Is it normal speed of processing batch jobs?

2. Can we get it faster?

3. Does processing speed differ between test and real accounts?

4. Does it depend on batch job size?

That's a test upload; in production I plan to split big uploads between multiple smaller batch jobs.

CONFIDENTIALITY NOTICE: This message (including any attachments) is confidential and may be privileged. It may be read, copied and used only by the intended recipient. If you have received it in error please contact the sender (by return e-mail) immediately and delete this message. Any unauthorized use or dissemination of this message in whole or in parts is strictly prohibited.

DISCLAIMER: The information contained in this email, attachment, document and/or presentation have been researched by us with due care and all and any evaluations or assessments stated herein represent our personal opinions. We advise you that some of the available information has been independently verified and may be based on statements by third persons, but no representation or warranty, expressed or implied, is made as to, and no reliance should be placed on, the fairness, accuracy, completeness or correctness of this information or opinions contained herein. Certain statements contained herein may be statements of future expectations and other forward-looking statements that are based on our current personal views and assumptions and involve known and unknown risks and uncertainties that cause actual results, performance or events to differ materially from those expressed or implied in such statements. None of our team shall assume any liability whatsoever (in negligence or otherwise) for any loss howsoever arising from any use of this email, attachment and/or presentation or the statements contained herein as to unverified third person statements, any statements of future expectations and other forward-looking statements, or the fairness, accuracy, completeness or correctness of statements contained herein, or otherwise arising in connection with this email, attachment, document and/or presentation. 

This email, attachment, document and/or presentation does not constitute an offer or invitation to purchase or subscribe for any investment and neither it nor any part of it shall form the basis of or be relied upon in connection with any commitment or contract whatsoever, and is solely intended to communicate a favourable impression of our team members and their business track-records. No one should base their investment decisions on this email, attachment, document and/or presentation, which principally constitutes an advertising message, as their main source of information or reliance but should form their personal opinions solely on the basis of further available information and evaluations and assumptions. The legal documentation held ready for any prospective investor shall be authorised as the sole source of relevant information and neither it nor any part of it shall constitute any representation, warranty or other undertaking, expressed or implied, as to the economic success of the offered investments.

[Sreelakshmi Sasidharan (AdWords API Team)]

Hi Aleksandr, 

Please find my answers inline:

1. Is it normal speed of processing batch jobs?

A: It is hard to comment on the actual time that a batch job will take to complete processing. As long you observe progress in the specific batch job, it should be good. 

2. Can we get it faster?

A: There are some best practices that you can follow to optimize the batch processing. Please check this section of the guide to understand more about that. 

3. Does processing speed differ between test and real accounts?

A: The speed between the test and production accounts should be comparable. Large variations among test and production accounts are not expected.

4. Does it depend on batch job size?

A: It is better to have fewer larger batch jobs over many smaller batch jobs considering the overall performance. As described in the best practices section, it also depends on other factors like grouping similar operations, avoiding operations on the same object etc. You may also want to keep in mind that there is an upload limit of 1 GB at a given time for an account. 

From what you have explained, the performance seems to be as expected. If your batch jobs are getting cancelled or if it fails due to some reason feel free to post again with the batch job Id.

Thanks,

Sreelakshmi, AdWords API Team

[aleksand...@rocket-internet.com]

Hi Sreelakshmi,

Obviously, batch processing has some advantages over mutate requests: you can use temporary IDs, it handles transient errors for you, overall setup is simpler, etc. But uploading the same generation (1.2M keywords) with mutate requests will take much less the 9 hours.

So could you advise in which cases batch processing should be preferred over "mutate" requests and vice versa? For which dataset sizes batch uploading is recommended - over 100K items, 1M, 10M?

Thanks,

Aleksandr

[Sreelakshmi Sasidharan (AdWords API Team)]

Hi Aleksandr, 

You are right, BatchjobService is extremely helpful in the cases where the operations are dependent (need to use temp Ids), has a bunch operations to be performed etc. BatchJobService also retries operations that fail due to transient errors like rate limit errors. The downside is that you will not have much control over the time in which the job is going to be completed. From the number of entities to be created in your case, you may have to make approximately the given number of API mutate calls (considering the maximum possible operations/request) : 

975 campaigns ~1 mutate call
64720 ad groups ~ 13 mutate calls
1242360 keywords ~ 249 mutate calls
242700 expanded ads ~ 49 mutate calls

In addition to that, you will also have to take care of spacing the requests in such a way to avoid rate limit errors, have proper mechanism to handle any errors while execution etc. There isn't a guideline to use/not use BatchJobService based on the upload size of the Job. Instead, BatchjobService should be looked at as an alternative to perform API operations asynchronously. It does seem like you are concerned about the execution time of the batch job. If so, using mutate requests might be a better option for you.

[ax...@turo.com]

Hi Aleksandr, 

You are mentioning time after uploading but how long did it take to upload all these operations?

 One thing we ended up doing is separating our operations in several batchjobs, once the first batchjob operations are uploaded and we are waiting for it to be "COMPLETED", we launch the upload of the second batchjob etc... etc.. 

In our case, it cut the operation time by at least 50%.

Axel