BulkMutateJob and extra identifiers

[Kyle LeNeau]

I have a process that maintains an Adwords campaign on behalf of a
customer. The process will either purchase new keywords, or update
keywords (destination URL only) nightly. The source of truth for what
keywords to maintain is in a database on our end and a user/customer
was the ability to insert new records into the table, thus triggering
a purchase of a new keyword.

When a new keyword is purchased the record that exists in my database
is updated with the criterion id, adgroup id, campaign id, url,
keywords and more. I then use the URL later to determine if the
keyword should be updated. In the previous version of the API (v13) I
was able to just update the keyword quickly and catch/handle any
errors. This happened very quickly; our small customer of 7000
keywords would run and usually update 80% of the keywords in a little
over an hour. With the new API (v200909) it is extremely slow, the
same 7000 keywords and 80% update is approaching the 6 hour mark and
has only updated 1/3 of the total.

This is a C# program and I am using the client library off of google
code. I have been debuging it all day and am getting a lot of
Database.CONCURRENT_MODIFICATION errors and am only running on one
thread. While debugging I have noticed that the call to update/mutate
a criterion is sometimes taking 30+ seconds to return, this was not
the case in the last version.

Which brings me to my main question about the Bulk Mutate Job. Is
there a way to pass along extra identifiers to the job/operation so
that when it returns and the job is complete I can loop over the
results and grab my database ID's (primary key) from the criterion to
update my system accordingly? If not, what are some alternatives to
keeping two systems connected like this? If this is not currently
supported can it be a feature request, sometime we want or need to
pass along our identifiers so we know what to do with it when it
returns.

Thanks for the help, Kyle.

[AdWords API Advisor]

Hi Kyle,

Thank you for the detailed description of your goals and the problems
you have encountered. Some points I'd like to mention:

- A simple way to improve the performance of keyword updates is to
only send the information you wish to change. So for instance if you
only want to change the bid on a keyword only send the adGroupId,
criteriaId, and bids fields. Sending other fields that you aren't
changing can often lead to increased request time since those other
fields must also be processed and can sometimes trigger other
workflows.

- CONCURRENT_MODIFICATION errors can happen when you attempt to update
the same keyword in quick succession. After you update a keyword it
may need to be re-approved, and the back end approval workflow can
sometimes try to modify a keyword at the same time you do. For this
reason it is best to leave a few seconds before attempting to to make
another update to the same keyword.

- It is possible to retrieve the resulting IDs with the
BulkMutateJobService, by setting the resultPartIndex on the
BulkMutateJobSelector. We have an example of this included with our
client libraries:

http://code.google.com/p/google-api-adwords-java/source/browse/trunk/examples/v200909/PerformBulkMutateJob.java#334

In general though the BMJS makes no guarantees that it will process
results any faster than the synchronous services, as that is not the
main goal of the service. Under certain circumstances you may see
some improvement, but the service is best used for changes that aren't
as time-sensitive.

Best,
- Eric Koleda, AdWords API Team

[Kyle LeNeau]

Hi Eric,

I appreciate the detailed response. However I am still unclear on why
performance is incredibly slow. I still (last 2 days) trying up
update 7,000 keywords and I am getting a TON of concurrent
modification errors.

Response to topic #1: I am only sending the destination url to be
updated with this code and it is still running SUPER slow to the point
of unusable, what gives:

Criterion criterion = new Criterion();
criterion.id = criterionId;
criterion.idSpecified = true;

BiddableAdGroupCriterion biddableAdGroupCriterion = new
BiddableAdGroupCriterion();
biddableAdGroupCriterion.adGroupId = adGroupId;
biddableAdGroupCriterion.adGroupIdSpecified = true;
biddableAdGroupCriterion.destinationUrl = destinationUrl;
biddableAdGroupCriterion.criterion = criterion;

AdGroupCriterionOperation operation = GetAdGroupCriterionOperation
(Operator.SET, biddableAdGroupCriterion);
m_adGroupCriterionService.mutate(new AdGroupCriterionOperation[]
{ operation });

Response to topic #2: I am not re-updating the same keyword after
failure, I moving on the next one.

Response to topic #3: To clarify is there a way to send additional
parameters what will just be sent back to with the criterion so that I
can parse it and match it to my DB, sort of like a known request ID
that is unique across all criterion. I also think of this as like and
extra public property on the criterion of type string that I can put
something into (like a JSON structure). Then I can de-serialize it on
the return. I can seem this being a well used feature in the API.

This way I can create a bulk update job, send it to the API and then
check on it every X minutes (asynchronously). This will reduce the
traffic to outside the building and reduce the SSL handshakes down
significantly. It is assumed that your API has written good logic to
process bulk jobs efficiently in house faster and with higher
priority.

Thanks Eric, I hope to hear from you again or someone else on your
team.

Thanks,
Kyle

On Jan 27, 10:13 am, AdWords API Advisor

<adwordsapiadvi...@google.com> wrote:
> Hi Kyle,
>
> Thank you for the detailed description of your goals and the problems
> you have encountered. Some points I'd like to mention:
>
> - A simple way to improve the performance of keyword updates is to
> only send the information you wish to change.  So for instance if you
> only want to change the bid on a keyword only send the adGroupId,
> criteriaId, and bids fields.  Sending other fields that you aren't
> changing can often lead to increased request time since those other
> fields must also be processed and can sometimes trigger other
> workflows.
>
> - CONCURRENT_MODIFICATION errors can happen when you attempt to update
> the same keyword in quick succession.  After you update a keyword it
> may need to be re-approved, and the back end approval workflow can
> sometimes try to modify a keyword at the same time you do.  For this
> reason it is best to leave a few seconds before attempting to to make
> another update to the same keyword.
>
> - It is possible to retrieve the resulting IDs with the
> BulkMutateJobService, by setting the resultPartIndex on the
> BulkMutateJobSelector.  We have an example of this included with our
> client libraries:
>

>  http://code.google.com/p/google-api-adwords-java/source/browse/trunk/...

[Rich Johns]

Just a couple of thoughts... when updating a keyword's destinationUrl,
there is edit review period. Do each of the bulk operations block
until the edit review period completes? Another thought (although this
seems unlikely and would suck if it were true) is that the AdGroup is
placed in some sort of edit review state such that subsequent
keyword updates are blocked?

> --
> You received this message because you are subscribed to the Google Groups "AdWords API Forum" group.
> To post to this group, send email to adwor...@googlegroups.com.
> To unsubscribe from this group, send email to adwords-api...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/adwords-api?hl=en.
>
>

--
rp johns

[AdWords API Advisor]

Hi Kyle,

Looking at your code I believe the main cause of your slow processing
time is that you are calling mutate() with only one operation at a
time. This is a very inefficient way to process operations, since
there is a round-trip across the network and a fixed cost on the
AdWords API servers for each request. The mutate method takes an
array of operations, and you can pass in a few thousand at a time.

I think Rich is on to something, that the CONCURRENT_MODIFICATION
errors are due to the internal keyword approval workflow. When the
system goes back in to update the approval status is may be putting a
temporary lock on the entire ad group, which is why subsequent keyword
additions to the same ad group are failing. Once again the solution
is to match together your operations and make fewer mutate calls.

There is no way to attach arbitrary information to an entity in the
BMJS, but I don't believe this is necessary. The order of the
operations in your BulkMutateRequest is the same as the operation
results in your BulkMutateResponse, so you can match up your inputs
and outputs.

As I mentioned before the BMJS is not meant to process operations
faster, and you should not make any assumptions that it will improve
the overall time it takes to add your keywords.

Best,
- Eric

[Kyle LeNeau]

Hi Eric,

I understand the multiple operations per call, that makes sense. I do
agree that making the request one by one is slowing me down a bit.

Is there anyway around the keyword workflow, I can see this as a big
problem when I need to edit anything more than one keyword in an
adgroup. Would doing a Bulk update or sending more operations on an
adgroup in one mutate request alleviate any of this problem?

I figured there wasn't any way to attach arbitrary information to a
keyword, I can always revert back to a query string param in the URL
and parse/look for it on the return of adding a new keyword, this
would work nicely I think (assuming I don't hit the length limit of
the destination URL field).

So the purpose of a BMJS is to reduce the cost then. Are there known
performance limits that people have seen as far as the number of
operations vs operation streams vs requests vs mutate jobs.

Thanks,
Kyle

On Jan 29, 8:50 am, AdWords API Advisor <adwordsapiadvi...@google.com>

[AdWords API Advisor]

Hi Kyle,

The keyword approval workflow is automatic and can't be skipped. To
avoid the concurrency error ensure that all changes to the same ad
group are made either in the same request or are made a few seconds
apart. Alternatively, you can catch the CONCURRENT_MODIFICATION
errors and automatically retry the request.

There are no specific differences in the performance between BMJS and
the synchronous services.

Best,
- Eric