Setting the CPC bid on product in shopping campaign.

[Chuck Reeves]

Currently we create keywords for each product and set a bid on that keyword.  Now that we need to update to a shopping campaign, and the process looks a little different.  It looks like the way that this needs to be done, is by creating a Product Group that is filtering by the Id/SKU we want.  Is there a limit to the number of product groups that an ad group can have? if so how many?

[Josh Radcliff (AdWords API Team)]

Hi,

AFAIK, the limits on the # of ProductPartitions in your accounts/adgroups/campaigns is dictated by the same limits AdWords uses for other types of criteria, as documented here:

https://support.google.com/adwords/answer/1704396 (expand the AdWords account limits section)

You may find the Shopping Campaigns Guide and PLA to Shopping Campaigns Migration Guide helpful if you're making this transition. In Shopping Campaigns, you can set up bidding by item/offer ID using ProductOfferId criteria on your AdGroupCriterion objects, but that would result in a fairly large number of criteria. If you don't actually need to bid separately on a per-item basis, Shopping Campaigns let you define your product partition tree using a variety of ProductDimensions so you could, for example, set up bidding by ProductCondition or ProductBiddingCategory. The guides I mentioned go into this in more detail.

Cheers,

Josh, AdWords API Team

[Christian]

Hi Josh,

we would like to set up shopping campaigns for quite large product feed (e.g. 100000 products). Here we are facing limitation issues (concerning the number product partitions per ad group) because we would like to set up our shopping campaigns with a 1-to-1 mapping, i.e. one partition for one product using the dimension ProductOfferId. We are aware of the fact that other dimensions are available too, but still the direct mapping is the best choice for us.

Now for the limitation, we plan to work around this problem by creating one ad group per product, each with two product partitions: a biddable one for the corresponding product ID and a negative one for all other products. From the theory that sounds like it should work. However, I'd be interested if you see any issues concerning the functionality of shopping campaigns.

I'm looking forward to hear your feedback on this issue.

Best,

Christian

[Josh Radcliff (AdWords API Team)]

Hi Christian,

I'll get back to you with a more complete answer, but one issue that immediately comes to mind with the solution below is that you'll quickly exceed the limits on the number of ad groups per campaign (currently 20,000).

https://support.google.com/adwords/answer/1704396

Best regards,

Josh, AdWords API Team

[Christian]

Hi Josh,

thanks for your feedback. We are also aware of the limitation of ad groups per campaign, but we already deal with that fact by creating multiple campaigns for a product feed if the number of products exceeds the limit.

According to the linked answer, I cannot see any problems concerning the display of product ads when using the one-adgroup-per-product approach for shopping campaigns. Please let me know if I am wrong here.

Best regards,

Christian

[Josh Radcliff (AdWords API Team)]

Hi Christian,

[NOTE (added 2014-07-11 15:00 EDT): The limit of product partitions per campaign below is incorrect. Please see later posts for the corrected limits.]

I'm a little concerned about the manageability of so many campaigns and the ad group-per product approach. I also found that there's a limit of 20,000 product partitions per campaign (noted here).

Have you considered using a CampaignCriterion with criterion type ProductScope to filter products at the campaign level (also described in our Shopping Campaigns Guide)? This could greatly reduce the number of ad groups per campaign and may make your campaigns more manageable. What I'm envisioning is something like this for your 100,000 products:

1. Create 5 (or more) campaigns to ensure you stay under the 20,000 ProductPartitions per campaign limit.

2. On each campaign, add a ProductScope (which can be based on up to 7 different ProductDimensions) CampaignCriterion to limit the number of products in each campaign to < 20,000.

3. Create one or more ad groups per campaign, each consisting of a set of ProductOfferId-based ProductPartitions. I don't see much benefit in multiple ad groups here unless you want to split up the products for other purposes besides managing the bids.

This approach assumes that your products have attributes that will allow you to perform the high level filtering in step 2.

Note that the advantages of the tree-based bidding structure of new shopping campaigns is that you don't have to set up such fine-grained bids, and the "other" category at each leaf node lets you set bids for anything that falls outside of the specific ProductPartitions you created (the ones with a value).

Best regards,

Josh, AdWords API Team

[GregT]

We have also in the past (with PLA campaigns) been creating a product target per product, each in its own ad group (splitting into multiple campaigns when necessary), and in our migration to shopping campaigns, plan on doing the same thing.  We have many clients, some with large product feeds like Christian is talking about.  Continuing to do multiple ad groups for each product is worthwhile so that we can set different mobile bid modifiers (which are at the ad group level) for different products (and presumably if/when shopping campaigns support remarketing, the bid modifiers will also be at the ad group level, as they are with display & search), as well as to support flexibility for different promotions for different products (because the ads/promotions are at the ad group level).

What looks to us is going to be the biggest issue is what is mentioned in this post: 

https://groups.google.com/forum/#!topic/adwords-api/fqQl076oA1o

Only being able to modify 2 ad groups at a time, with any reasonably sized merchant center feed, is not really going to work.  Whether it's just creating the original ad groups & product partitions, or updating bids, the amount of time it will take to do anything looks to be prohibitive.  With other kinds of campaigns, we send 1,000 criterion (keywords, placements, audience bid modifiers, etc.) at a time, and/or use the MutateJobService to send 5,000 at a time.  Unfortunately, the MutateJobService doesn't support shopping campaigns.  Also, unfortunately, using multiple threads will not help any either, as the thing that will most hinder doing anything with/for these campaigns is the rate limiting that will very quickly occur when we have to make 50,000 quick calls on the same account just to change bids on 100k products.

Josh and/or others, any way this limit of 2 ad groups can be increased and/or any good workarounds?  It's amazingly small compared to any other equivalent kind of limit in AdWords.  If the limit were instead a number of product partition nodes that can be updated in one call, it seems like it would be much more reasonable -- so if you want to add/update 1,000 product partition nodes in a single ad group or spread out across 1,000 ad groups, it still works?

Thanks,

Greg

On Friday, July 11, 2014 10:17:07 AM UTC-5, Josh Radcliff (AdWords API Team) wrote:

Hi Christian,

I'm a little concerned about the manageability of so many campaigns and the ad group-per product approach. I also found that there's a limit of 20,000 product partitions per campaign (noted here).

Have you considered using a CampaignCriterion with criterion type ProductScope to filter products at the campaign level (also described in our Shopping Campaigns Guide)? This could greatly reduce the number of ad groups per campaign and may make your campaigns more manageable. What I'm envisioning is something like this for your 100,000 products:

1. Create 5 (or more) campaigns to ensure you stay under the 20,000 ProductPartitions per campaign limit.

2. On each campaign, add a ProductScope (which can be based on up to 7 different ProductDimensions) CampaignCriterion to limit the number of products in each campaign to < 20,000.

3. Create one or more ad groups per campaign, each consisting of a set of ProductOfferId-based ProductPartitions. I don't see much benefit in multiple ad groups here unless you want to split up the products for other purposes besides managing the bids.

This approach assumes that your products have attributes that will allow you to perform the high level filtering in step 2.

Note that the advantages of the tree-based bidding structure of new shopping campaigns is that you don't have to set up such fine-grained bids, and the "other" category at each leaf node lets you set bids for anything that falls outside of the specific ProductPartitions you created (the ones with a value).

Best regards,

Josh, AdWords API Team

[Josh Radcliff (AdWords API Team)]

Hi Greg,

I'll look into the limit you mentioned below on ad group mutates.

Please note that the limit I quoted from here of 20,000 ProductPartitions per campaign is incorrect. The actual limit is the same as for other criteria, that is:

• 20,000 per ad group
• 5 million per campaign

Thanks,

Josh, AdWords API Team

[Christian]

Hi Josh and Greg,

thanks for your feedback.

Similar to Greg, in the past we also created PLA campaigns for big feeds by creating one ad group per product and by using more than one campaign if the products exceed the ad group limit. While the suggestion to use a product scope as an intermediate splitting of products sounds reasonable, for us it is more practical to use only one level of splitting, i.e. ad groups, for which we already have an implementation.

I also second Greg that the limit of 2 ad groups per mutate call is a serious issue for dealing with large product feeds.

It would be great if this situation is fixed in the next API version -- ideally together with the possibility to manage shopping campaigns using the mutate job service and partial failures.

Thanks again and best regards,

Christian

[GregT]

Just to put in my two cents, if it makes any difference... 

While it'd be great to have the mutate job service and partial failures in the next release, at the least, we'd really really like to see the limit increased sooner than the next API release -- seeing as Google is converting all PLA campaigns over to shopping campaigns and retiring PLA next month.  As it stands, this limit is a huge roadblock for us.  I think I understand why there's such a tiny limit (because the API requires you to send the entire product partition tree for an ad group in one call, and an ad group's partition tree could be up to 20,000 criterion), but maybe my suggestion of making the limit apply to the # of partitions and not the # of ad groups could work?

Thanks again,

Greg

[david]

On Wednesday, 16 July 2014 15:56:38 UTC+1, GregT wrote:

 (because the API requires you to send the entire product partition tree for an ad group in one call, and an ad group's partition tree could be up to 20,000 criterion), 

Slightly OT but just in case it helps, the above isn't quite the case. You just need to make sure you have a valid tree after each call.

So our first call creates a SUBDIVISION + single UNIT child for the "other" case. We take a note of the parentId.

Then we add UNITs in call batches of 2000 to that parent. You can also update UNIT (eg. change the maxCpc) with a mutate applying to a single child.

See Matt Scales post here:

https://groups.google.com/forum/#!msg/adwords-api/BbbsW1MSobw/7GnXo5wP2F8J

[Josh Radcliff (AdWords API Team)]

Hi,

David - thanks for clarifying. You are absolutely correct: your mutates simply need to keep the tree in a valid state, but you can send a mutate that only touches a subtree.

I have some good news - the limit of 2 ad groups per mutate request does not apply to SET operations anymore. This means that you can mutate more than 2 ad groups in a single call as long as you are updating existing BiddableAdGroupCriterion or NegativeAdGroupCriterion objects, e.g., changing their bids. Would that help with your one ad group per product approach, or will you have large, ongoing changes to the population of products as well?

Thanks,

Josh, AdWords API Team

[GregT]

Hi, Josh.

Thanks much for looking into it and for the good news -- that will be extremely helpful for us, and will solve a large part of the issue on an ongoing basis.  I do, unfortunately, think that the limitation on ADD operations is still going to be an issue.  On a shorter term basis, we have some clients with over 1 million products in their PLA campaigns in a single AdWords account.  Converting clients like them over to shopping campaigns right now would take 500,000 calls to the AdGroupCriterionService.  I'm sure you have a much better idea of the rate limiting effect on this, but I wouldn't be surprised if it took weeks to make those 500,000 calls [each with very, very little data in them] for clients like those.  On an ongoing basis, we don't really need to change the partitions after creation (except to change bids), but we would have new ad groups (& their partitions) created as new products are added to our clients' merchant center accounts, which happens pretty frequently.  Most of the time, there won't be a ton of new products at any given time, but there are occasions where we'll need to add multiple thousands of new products for a client, and this will probably be an issue then as well.

Given that an entire tree does not have to be created in a single ADD call like I had thought (but just needs to be valid), that seems to me to make it even more reasonable to change the limit from an ad group level one to a ProductPartition one?  I.e. instead of adding product partitions in 2 ad groups being the limit, it instead being something like 2,000 or 5,000 product partitions in a call, regardless of how many ad groups they happen to be in.  Just a suggestion. :)  Even a relatively low limit by AdWords API standards, like 500, would probably be enough to be able to work around this.

Thanks again, we appreciate it!

Greg

[Josh Radcliff (AdWords API Team)]

Hi Greg,

Thanks for all of the feedback - the details you've provided are extremely helpful in explaining why the mutate limit is problematic for you. I've passed this information on to the Shopping team and will let you know as soon as I have any news.

Thanks,

Josh, AdWords API Team

[Chuck Reeves]

I was trying to push the partition tree using MutateJobService.  I sent 2,500 partitions to be added and started polling for a response.  The stats for the job showed as status of PROCESSING and  at 60 percent complete.  After 20 min, no change in the status or progress.  I then decided to try again with another ad group.  The 2nd attempt I got a status of PROCESSING with no percentage gain.  Now when I try to view those ad groups in the UI, I keep getting a red banner stating: 

"There was an error with your operation. If you were trying to make a change, it may not have saved.

Please refresh this page to try again.

If the error continues, log out of your AdWords account, then log in again and return to this page." 

Aside from starting the campaign and groups over again, I cannot dismiss this error, or get the jobs I sent to complete. Is there anything that I can do to resolve this error?  

Is there another way for me to send the tree?

[david]

See this link in case it helps https://groups.google.com/d/msg/adwords-api/Ou-PSngYi7k/ZD-xuWexxA0J

Looks like the BulkMutateService does not support shopping campaigns. But usually the error CAMPAIGN_TYPE_NOT_COMPATIBLE_WITH_PARTIAL_FAILURE is given.

So we have to send our partition tree in batches directly to AdGroupCriterionService

We have encountered a similar error when we had an invalid tree that the GUI couldn't display.

(And incidentally still get that error but only when trying to list the products within our "Other" node.)

[Chuck Reeves]

I actually tried to use the AdGroupCriterionSerivce.  After I sent in the 1st ad group, I was rate limited.  Between the two calls I waited for 60 seconds (with other services, I only needed to wait for ~30 seconds between calls).  I never got any errors response from the BulkMutateService  (in fact the 1st ad group got up to 60% complete).  Those 2 ad groups in the UI still show the red banner.  We are not trying to set up on ad group per product.  Instead we are just create one ProductOfferId for each product.  In that way, we can set the bids individually on each product.  

You mentioned that you had success with the AdGroupCriterionSerivce.  What (if anything) did you have to do in order to prevent getting rate limited?

[david]

On Wednesday, 23 July 2014 14:57:13 UTC+1, Chuck Reeves wrote:

I actually tried to use the AdGroupCriterionSerivce.  After I sent in the 1st ad group, I was rate limited.  Between the two calls I waited for 60 seconds (with other services, I only needed to wait for ~30 seconds between calls).  I never got any errors response from the BulkMutateService  (in fact the 1st ad group got up to 60% complete).  Those 2 ad groups in the UI still show the red banner.  We are not trying to set up on ad group per product.  Instead we are just create one ProductOfferId for each product.  In that way, we can set the bids individually on each product.  

You mentioned that you had success with the AdGroupCriterionSerivce.  What (if anything) did you have to do in order to prevent getting rate limited?

We just synchronously send batches of 2000 product partitions (segmented on ProductOfferId also), all contained in the same ad group... it's quite slow anyway so rate limiting has never come up as an issue. 

[Chuck Reeves]

David,

Thank you for the help on this.  I went ahead and switched back the AdGroupCirterionService and tied setting up the partition table for 20,000 products.  I would send the service batches of 2000 for each call.  It worked well for the 1st 4 calls and then I got hit with a rate limit.  The API told me to wait for 86400 seconds.  I guess there is no away around this right now so we will just have to inform our customers that it will take some time to push the whole catalog out to Ad Words.

I only hope that in the future, Google will lessen this restriction on this service.  At the present, I do not see any other way around this.

[Josh Radcliff (AdWords API Team)]

Hi,

There is now an upgrade tool available -- please see the related blog post from earlier this week. Hopefully the tool will make it easier to upgrade some of those large campaigns without using the API (and thereby avoid rate limit issues).

Cheers,

Josh, AdWords API Team