cannot create shopping ProductGroup tree in one batch. Getting LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE error

[Inilien Inilien]

I am using Python google-ads module and Google Ads API v2 (Beta)  

In my test account's shopping campaign I am trying to create 3 ProductGroups in an empty AdGroup.

I can successfully create single root level AdGroupCriterion with listing_group type: UNIT but when I am trying to create 3 AdGroupCriterions: 1 root and 2 children I am getting errors.

Here are my AdGroupCriterionOperation protobuf messages:

[create {
   resource_name: "customers/5936312994/adGroupCriteria/82410873572~-25"
   status: PAUSED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   listing_group {
     type: SUBDIVISION
   }
 }, create {
   resource_name: "customers/5936312994/adGroupCriteria/82410873572~-27"
   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   negative {
     value: true
   }
   listing_group {
     type: UNIT
     case_value {
       listing_custom_attribute {
         index: INDEX0
       }
     }
     parent_ad_group_criterion {
       value: "customers/5936312994/adGroupCriteria/82410873572~-25"
     }
   }
 }, create {
   resource_name: "customers/5936312994/adGroupCriteria/82410873572~-26"
   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   cpc_bid_micros {
     value: 10000000
   }
   listing_group {
     type: UNIT
     case_value {
       listing_custom_attribute {
         value {
           value: "1000-2000"
         }
         index: INDEX0
       }
     }
     parent_ad_group_criterion {
       value: "customers/5936312994/adGroupCriteria/82410873572~-25"
     }
   }
 }]

Essentially I got them from my production account.

When I am trying to call mutate method of AdGroupCriterionService I am getting these errors (one per operation):

errors {
  error_code {
    ad_group_criterion_error: LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE
  }
  message: "Subdivided listing groups must have an \"others\" case."
  trigger {
    string_value: "ProductPartition{id=TempCriterionId{id=25}, bidSimulatorStatus=null, partitionType=SUBDIVISION, parentCriterionId=null, productGroup=null, productGroupObsoleteStatus=null, caseValue=null, decisionPath=[], salesCountry=null, merchantId=null, hasPromotedSuggestion=null}"
  }
}
errors {
  error_code {
    ad_group_criterion_error: LISTING_GROUP_DOES_NOT_EXIST
  }
  message: "Listing group referenced in the operation was not found in the ad group."
  trigger {
    string_value: "TempCriterionId{id=25}"
  }
  location {
    field_path_elements {
      field_name: "operations"
      index {
      }
    }
    field_path_elements {
      field_name: "create"
    }
    field_path_elements {
      field_name: "listing_group"
    }
    field_path_elements {
      field_name: "parent_ad_group_criterion"
    }
  }
}
errors {
  error_code {
    ad_group_criterion_error: LISTING_GROUP_DOES_NOT_EXIST
  }
  message: "Listing group referenced in the operation was not found in the ad group."
  trigger {
    string_value: "TempCriterionId{id=25}"
  }
  location {
    field_path_elements {
      field_name: "operations"
      index {
        value: 1
      }
    }
    field_path_elements {
      field_name: "create"
    }
    field_path_elements {
      field_name: "listing_group"
    }
    field_path_elements {
      field_name: "parent_ad_group_criterion"
    }
  }
}

I am assuming that if root level operation were to succeed then both children would succeed as well. But I cannot figure out how to solve the LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE error.

I would appreciate any help. Thank you in advance.

[Google Ads API Forum Advisor Prod]

Hello, 

The error LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE indicates that the Others case is missing in the tree. Each subdivision must always be completely partitioned, so it must contain a node representing Other which has an "empty" case_value of the correct type. This child is usually referred to as Other because it represents "all other values" for that ListingDimensionInfo. Please refer to this guide for more information. I have attached the sample logs from my testing as a reference. Could you please take a look and let me know if you encounter any issues?

Thanks,
Bharani, Google Ads API Team

ref:_00D1U1174p._5001UEJA9R:ref

[Inilien Inilien]

Thanks for your reply.

Sadly I cannot overcome the issue. 

I do not see any difference (related to the "Others" case problem) between my protobuf messages and the link to the guide you posted (in fact I used the same guide and the fuller version of that code as a reference). I have a single root with 2 children one of which doesn't have `value` field in its `case_value` field which should make it the "Other" node.

Just to be sure, I also have tried to make my operations more like in the Java example:

[create {
   resource_name: "customers/5936312994/adGroupCriteria/82410873572~-1"
   status: ENABLED
   listing_group {
     type: SUBDIVISION
   }
 }, create {

   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   cpc_bid_micros {
     value: 10000000
   }
   listing_group {
     type: UNIT
     case_value {
       listing_custom_attribute {

         index: INDEX0
       }
     }
     parent_ad_group_criterion {
       value: "customers/5936312994/adGroupCriteria/82410873572~-1"
     }
   }
 }, create {

   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   cpc_bid_micros {
     value: 10000000
   }
   listing_group {
     type: UNIT
     case_value {
       listing_custom_attribute {
         value {
           value: "1000-2000"
         }
         index: INDEX0
       }
     }
     parent_ad_group_criterion {

       value: "customers/5936312994/adGroupCriteria/82410873572~-1"
     }
   }
 }]

but it still doesn't work - the same 3 errors:

errors {
   error_code {
     ad_group_criterion_error: LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE
   }
   message: "Subdivided listing groups must have an \"others\" case."
   trigger {

     string_value: "ProductPartition{id=TempCriterionId{id=1}, bidSimulatorStatus=null, partitionType=SUBDIVISION, parentCriterionId=null, productGroup=null, productGroupObsoleteStatus=null, caseValue=null, decisionPath=[], salesCountry=null, merchantId=null, hasPromotedSuggestion=null}"

   }
 }
 errors {
   error_code {
     ad_group_criterion_error: LISTING_GROUP_DOES_NOT_EXIST
   }
   message: "Listing group referenced in the operation was not found in the ad group."
   trigger {

     string_value: "TempCriterionId{id=1}"

   }
   location {
     field_path_elements {
       field_name: "operations"
       index {
       }
     }
     field_path_elements {
       field_name: "create"
     }
     field_path_elements {
       field_name: "listing_group"
     }
     field_path_elements {
       field_name: "parent_ad_group_criterion"
     }
   }
 }
 errors {
   error_code {
     ad_group_criterion_error: LISTING_GROUP_DOES_NOT_EXIST
   }
   message: "Listing group referenced in the operation was not found in the ad group."
   trigger {

     string_value: "TempCriterionId{id=1}"

   }
   location {
     field_path_elements {
       field_name: "operations"
       index {
         value: 1
       }
     }
     field_path_elements {
       field_name: "create"
     }
     field_path_elements {
       field_name: "listing_group"
     }
     field_path_elements {
       field_name: "parent_ad_group_criterion"
     }
   }
 }

[Google Ads API Forum Advisor Prod]

Hello, 

Thank you for checking this. Let me explain this in detail. Based on your logs shared, you are creating a SUBDIVISION which will introduce the new level in tree. Here you are creating two units with index: INDEX0. This is still missing the Others node. Each subdivision must always be completely partitioned, so it must contain a node representing Other. Each subdivision must contain an "empty" case_value of the correct type. This child is usually referred to as Other because it represents "all other values" for that ListingDimensionInfo. Could you please take a look at the shared logs for on how the Others is implemented? 

Regards,

Bharani, Google Ads API Team

ref:_00D1U1174p._5001UEJA9R:ref

[Inilien Inilien]

Hmm..

If I understood you correctly, the main hypothesis is that the `case_field` of my create operation which should create the "Other" node is wrong.

I have tried several different variants of setting it:

attempt 1: remove `index` from `listing_custom_attribute`

[create {

   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   cpc_bid_micros {

     value: 14000000

   }
   listing_group {
     type: UNIT
     case_value {
       listing_custom_attribute {
         value {

           value: "3000-4000"

         }
         index: INDEX0
       }
     }
     parent_ad_group_criterion {
       value: "customers/5936312994/adGroupCriteria/82410873572~-1"
     }
   }
 }, create {
   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   cpc_bid_micros {
     value: 10000000
   }
   listing_group {
     type: UNIT
     case_value {
       listing_custom_attribute {
       }
     }
     parent_ad_group_criterion {
       value: "customers/5936312994/adGroupCriteria/82410873572~-1"
     }
   }
 }, create {

   resource_name: "customers/5936312994/adGroupCriteria/82410873572~-1"
   status: ENABLED
   listing_group {
     type: SUBDIVISION
   }

 }]

This results in the error:

errors {
 error_code {
   field_error: REQUIRED
 }
 message: "The required field was not present."
 trigger {
   string_value: ""

 }
 location {
   field_path_elements {
     field_name: "operations"
     index {
       value: 1
     }
   }
   field_path_elements {
     field_name: "create"
   }
   field_path_elements {
     field_name: "listing_group"
   }
   field_path_elements {

     field_name: "case_value"
   }
   field_path_elements {
     field_name: "listing_custom_attribute"
   }
   field_path_elements {
     field_name: "index"
   }
 }
}

Attempt 2: (just to be sure) Clear `case_value` field at all

[create {

   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   cpc_bid_micros {

     value: 14000000

   }
   listing_group {
     type: UNIT
     case_value {
       listing_custom_attribute {
         value {

           value: "3000-4000"

         }
         index: INDEX0
       }
     }
     parent_ad_group_criterion {
       value: "customers/5936312994/adGroupCriteria/82410873572~-1"
     }
   }
 }, create {
   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   cpc_bid_micros {
     value: 10000000
   }
   listing_group {
     type: UNIT
     case_value {
     }
     parent_ad_group_criterion {
       value: "customers/5936312994/adGroupCriteria/82410873572~-1"
     }
   }
 }, create {

   resource_name: "customers/5936312994/adGroupCriteria/82410873572~-1"
   status: ENABLED
   listing_group {
     type: SUBDIVISION
   }

 }]

Strangely enough this produces the same errors as in my first message in this thread:

Attempt 3: I have tried to use `listing_brand` instead of `listing_custom_attribute` since that field works in the Java example:

[create {

   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   cpc_bid_micros {

     value: 14000000
   }
   listing_group {
     type: UNIT
     case_value {
       listing_brand {
         value {
           value: "\321\215\320\272\320\276\321\207\320\265\320\273\320\276\320\262\320\265\320\272\320\270"

         }
       }
     }
     parent_ad_group_criterion {
       value: "customers/5936312994/adGroupCriteria/82410873572~-1"
     }
   }
 }, create {
   status: ENABLED
   ad_group {
     value: "customers/5936312994/adGroups/82410873572"
   }
   cpc_bid_micros {
     value: 10000000
   }
   listing_group {
     type: UNIT
     case_value {

       listing_brand {

       }
     }
     parent_ad_group_criterion {
       value: "customers/5936312994/adGroupCriteria/82410873572~-1"
     }
   }
 }, create {

   resource_name: "customers/5936312994/adGroupCriteria/82410873572~-1"
   status: ENABLED
   listing_group {
     type: SUBDIVISION
   }

 }]

p.s. the brand value here is "экочеловечки" - it's Cyrillic in UTF-8 encoding. I copied it from Google Ads web interface from another AdGroup so it should be correct and exist in the feed.

Again I receive the same errors as in the very beginning if this thread:

From what I understand, the last one (with listing_brand) is a correct input but still it results in errors. For the Custom labels, if I gathered it right, the index field is required one and should also be present in the Other node. And I would like to remind here that I copied the original protobuf messages from my working production account and just trying to recreate the same nodes in my test account, so the protobuf messages (the original ones) should be correct (I changed only CusteomerId, AdGroupId, and all AdGroupCriteriaIds to Temprorary ids)

If I didn't understand correctly what you meant, could you please provide protobuf messages of operations which you think should work with my feed values? (custom label or brand as in the examples above) or at least the part which contains error (before and after correction).

Thank you for your time.

[Google Ads API Forum Advisor Prod]

Hello, 

Thank you for trying out various scenarios. I have made a few changes to the code to add the tree with ListingCustomAttribute type, to create a UNIT and OTHERS node. Could you please take a look and let me know if you encounter any issues? I have attached the corresponding SOAP logs for your reference. 

[Inilien Inilien]

Sorry, just to be sure that we are on the same page, I would like to remind that I am using Python client library `google-ads`

I appreciate the java code you created very much but sadly it is of little use to me (((

I compared the protobuf messages for operations you created with that Java code though with the protobuf messages of operations I have created in my second post in this thread and they are basically the same (apart from different ids and string value of custom label).

The fact that you have no issues running that code leaves me with only one conclusion - something wrong with Python library (((

Just to be clear, have you also tried to create these Product Groups on your test Google Ads account? 

[Google Ads API Forum Advisor Prod]

Hi Inillien - 

Thanks for this question and for sending over all of this detailed information. I'm the maintainer of our Python client library so should be able to help with this issue. Unfortunately we don't have an example for adding a product listing group tree in Python, but that will hopefully change soon.

I believe the issue here is that one of the AdGroupCriterion in your request needs to have a case_value field that is not set. The problem is it's not very clear on how to do that in the Python interface for Protobuf message classes because it seems like the only way to add a field to a message is to give it a value.

If you look closely at the request Bharani shared you'll noticed that one of the criterion has a listing group with a case_value that looks like this:

case_value {
    product_condition {
    }
}

That represents an empty case_value field.

This set of documentation shows how to set an empty message field in Python. On one of your AdGroupCriterion objects you'll need to do something like this:

ad_group_criterion.listing_group.case_value.CopyFrom(client.get_type('ListingDimensionInfo'))

Hope this helps, please let me know if you have any further questions.

Best,
Ben Karl, Google Ads API Team

ref:_00D1U1174p._5001UEJA9R:ref

[Inilien Inilien]

Hi Ben.

the issue here is that one of the AdGroupCriterion in your request needs to have a case_value field that is not set

Sorry, but I thought I already tried that. Here are my protobuf messages and responses https://groups.google.com/d/msg/adwords-api/MKoKH8xVn-k/0q3JVm2nAgAJ (I don't have a lot of experiences with Google Groups so I hope that tрe link will be correct. In case it is not, this is the link to my 3rd message in this thread from August 20th)

First 2 attempts in this message are errors which I get if I try to use `listing_custom_attribute` without `index` field. In the 3rd attempt I tried to use `listing_brand` field to match the Java examples from Bharani's messages. But still I've got an error.

it's not very clear on how to do that in the Python 

In order to set empty protobuf message I just use these methods:

SetInParent, ClearField and the like. For example:

ad_group_criterion.listing_group.case_value.listing_brand.SetInParent()

This set of documentation shows how to set an empty message field in Python.

Just to be sure, if my string representations of protobuf messages (which I've been posting in previous messages) are correct, then the protobuf messages they were created from are also correct. Right? I mean, is it possible that I am somehow creating incorrect messages which are causing the error but still getting correct string representation? 

For example, I didn't use `CopyFrom` to create messages, I am creating them using this code:

ad_group_criterion_operation = client.get_service('AdGroupCriterionOperation',  version=GADS_API_V)
ad_group_criterion = ad_group_criterion_operation.create

and then just filling value of the `ad_group_criterion` using standard protobuf methods.

[Inilien Inilien]

---------- Forwarded message ---------
From: Google Ads API Forum Advisor Prod <adsapifor...@gmail.com>
Date: Mon, Aug 26, 2019 at 11:05 PM
Subject: Re: cannot create shopping ProductGroup tree in one batch. Getting LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE error
To: ini...@gmail.com <ini...@gmail.com>

Hi Inilien,

I'll respond to your questions inline below:

1. My advice was to resolve the error LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE, but it does seem that you were able to overcome that. Can you clarify exactly which error message you're currently receiving? There's a few in that chain and I want to make sure I'm addressing the correct error.
2. SetInParent appears to have the same effect as CopyFrom and may be a bit simpler. Have you tried calling SetInParent on case_value directly? For example: ad_group_criterion.listing_group.case_value.SetInParent()
3. That is the correct way to create an operation.

Thank you,
Ben, Google Ads API Team
 

ref:_00D1U1174p._5001UEJA9R:ref

[Inilien Inilien]

 Hi Ben.

You wrote:

 Have you tried calling SetInParent on case_value directly? For example: ad_group_criterion.listing_group.case_value.SetInParent()

Actually, not. I used ClearField - I was modifying my protobuf messages from my very first post in this thread. But it is my understanding that it doesn't matter how the protobuf message was created as long as it has the correct contents. You can see the result of an empty `case_value` in this message from me (which I posted earlier in my previous message) https://groups.google.com/forum/#!msg/adwords-api/MKoKH8xVn-k/0q3JVm2nAgAJ look for attempt 2. (there are 3 attempts. For each I provided input protobuf messages and errors which I am getting as a result. But some code blocks are hidden by default for some reason. Look for `- show quoted text -` to show them. 

 My advice was to resolve the error LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE, but it does seem that you were able to overcome that.

No, actually I wasn't. Again, pleas look at my post https://groups.google.com/forum/#!msg/adwords-api/MKoKH8xVn-k/0q3JVm2nAgAJ (same link as above).

To summarize them again here:

attempt 1: difference with the protobuf messages from my very first post in this thread is that I  removed `index` from `listing_custom_attribute` which resulted in different error, namely that for the case of `listing_custom_attribute` the `index` field is `REQUIRED`

attempt 2:I also tried to remove `listing_custom_attribute` at all (leave `case_value` empty) which resulted in the same error as in my very first post in this thread, namely: `LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE` for the root node and (I assume as a result of this error) `LISTING_GROUP_DOES_NOT_EXIST` for children nodes.

attempt 3: Here I tried to replace `listing_custom_attribute` with `listing_brand` since that is what Bharani successfully used in his Java code in previous posts and also was used in documentation. And as you can see for the `Other` node I used this construction

case_value {
  listing_brand {
  }
}

since it was used by Bharani and in Java example. But still I've got the same error `LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE`

P.s. I've also tried the protobuf messages with `listing_brand` when `case_value` for `Other` node is empty. In that case I am getting the same errors, namely `LISTING_GROUP_SUBDIVISION_REQUIRES_OTHERS_CASE`

Thank you for trying to help me, but could you please reread previous messages more accurately? It is the second time that I have to post link to my previous message and it takes very long time between responses. Essentially a week has passed since I mention for the second time to Bharani that I am using Python and he asked you to join the discussion. But sadly we haven't made any progress at all.

Oh, and could you please post responses in the Google Group thread? in case someone else will face the same issue it would be better if the hole discussion was in the same place (And I believe that at least gmail formatting of these messages is different from Google Groups' one. Not sure about other email providers)

[Inilien Inilien]

I found this pull request with usage of listing group tree in Python.

After fixing the code (in several cases wrong variables were used for some reason) I was able to successfully create listing tree.

The odd thing is, I compared generated protobuf messages with the ones I've been generating in my own code and they are identical (at least their string representations are) but nevertheless my code continues to throw the errors and the modified code from the pull request works just fine.

I am trying to figure out the difference now... Just wanted you to know that the protobuf messages seem to be correct (at least the ones with the `listing_brand` case_value)

[Google Ads API Forum Advisor Prod]

Hi Inilian,

Apologies for the delayed response. That PR you found is still under review and definitely not final, so that's why there are style issues and misnamed variables. However I'm glad that you were able to use it to make a successful request.

Are there any differences between the code in the example and your code that might indicate the cause of the problem?

Best,

[Inilien Inilien]

Hi Ben.

I've finally found time to compare differences in implementations.

Turns out that the reason for errors I was getting is the order of operations in array which I give as parameter to the `mutate_ad_group_criteria` method.

Originally, when I make the API call, the `root` node is on the last place and its 2 children (`Other` node and another one node) are before that. If I place the `root` node at the beginning of `operations` array then I receive no API errors.

I did not expect such behavior so much that I made the stupidest thing possible and changed the order of my protobuf operations in the arrays when I posted them in this thread (for "readability purposes" was my thinking - it would be easier to read if the root node would be the first one) :facepalm:. So I was hoping that someone would help me to fix the problem when I provided them with the changed data... I am very sorry about that.

I don't know if this is a feature or a bug but if it is the former then I would ask to tell API users about that as much as possible.

For example, on this documentation page about creating shopping listing groups I couldn't find any mentions of such a behavior nor have I been able to find any mentions about that on this page with example of creating the whole tree in Java.

[Inilien Inilien]

Turns out that for deletion there is the same problem: 

If I am trying to remove parent and child nodes in the same mutate request and if in my `operations` list I specify parent first, then I will get `LISTING_GROUP_DOES_NOT_EXIST` error for child remove operation. I assume this is happening because after removing parent, all its children are removed automatically as well and when the next operation is trying to be processed it results in error.

So I should either check and delete all remove operations for children nodes from my `operations` list (if they are children of a removing node) or reorder operations in a reverse breadth-first search order.

Once again I do not know if this is a bug or a feature but providing information about this in documentation would be very helpful.

[Inilien Inilien]

Also, if I have AdGroup with root node without children and want to add children to it, then I need to remove current root node and create new one (since `listing_group.type` is immutable field and update operation is prohibited). But the order of operations in this case is also crucial - the `remove` operation should be specified first and creation of root - second (with children after that).

It really would be nice if bulk operations did not require strict order.

[Google Ads API Forum Advisor Prod]

Hi Inilien,

Thanks for all the feedback! I'm really glad you were able to get past the issue you were having. I completely agree that we should make these details clear in our documentation. I'm pretty sure all of this functionality is working as expected, so I'll work on some updates to our docs so this is less confusing in the future.