GoogleAdsRow object not iterable?

[Deny Watanabe]

WHY?

WHY?

WHY????

[Google Ads API Forum Advisor Prod]

Hi Deny,

Thanks for the post, and sorry that you're having an issue here.

Can you share a little more detail about the problem you're having? I might be able to help you get around the issue, and if not can help post a feature request so that we can fix it in the future.

It seems like you might be having an issue with one of the client libraries. Can you let me know which language you're using?

Thanks!
Ben Karl, Google Ads API Team

ref:_00D1U1174p._5004Q25Wv5e:ref

[Watanabe, Deny (SPL-MEW)]

Hi Ben,

 

I’m using the python library and was very disappointed to find out that the results from the API queries are delivered as objects which are non-iterable, making the use of data dictionaries a lot more difficult and a lot less elegant.

 

Other Google APIs present results as dictionaries which can be inspected for content (fields and values) according to the same data dictionary that was used for the request. That makes configuring API extractions easy to build, maintain and improve, using code such as:

 

for element in my_api_dict:

  if element[“id”] in returned_values:

    results.append(returned_values[element[“id”]]

 

On the other hand, the new Google Ads API (which should supposedly be better than previous ones?) demands that the result set elements are named explicitly, like:

results = [returned_values.table1.column1,

  returned_values.table1.column2,

  returned_values.table1.column3, …

]

 

I feel very dirty to admit I came up with a fugly and shameful solution which is converting the results to string and then making some modifications to turn it into a yaml string and parse it, so I could transform it into a dictionary. Ugly solutions make me extremely uncomfortable and frustrated, and I don’t think I should have to do things like this in 2020, *especially* dealing with an API developed by the almighty Google.

 

I can live with the ugly solution if you can live with my deep disappointment in your company development teams.

 

Cheers,

Deny

This message contains information which may be confidential and privileged. Unless you are the intended recipient (or authorized to receive this message for the intended recipient), you may not use, copy, disseminate or disclose to anyone the message or any information contained in the message. If you have received the message in error, please advise the sender by reply e-mail, and delete the message. Thank you very much.

[Zweitze]

You should ask the client library developers in the respective GitHub project.

For instance, the .NET libraries have the same problem, but somewhere deeply hidden in the report results was another class with an attribute listing all column names that were returned. You could ask if the python library has a similar feature.

[Mat]

Hi Deny,

I feel with you. The Google Ads API documentation seems to be written by a mean bot.

Using the Python library, you can fetch reports like this:

###

VERSION = 'v5'

ga_service = google_ads_client.get_service('GoogleAdsService', version=VERSION)

query = ('SELECT '

          'campaign.id, '

          'campaign.name, '

          'ad_group.id, '

          'ad_group.name, '              

          'FROM ad_group '

          )    

response = ga_service.search_stream(str(customer_id), query=query)

for batch in response:

for row in batch.results:    

campaign = row.campaign

ad_group = row.ad_group

campaign_id = campaign.id.value

campaign_name = campaign.name.value

ad_group_id = ad_group.id.value

ad_group_name = ad_group.name.value      

###

If the value is an emum, you can handle it like this:

###

ad_group_criterion_type_enum = google_ads_client.get_type( 'CriterionTypeEnum', version=VERSION).CriterionType

criterion_type = ad_group_criterion_type_enum.Name(ad_group_criterion.type)

###

Regards

Mat

[Deny Watanabe]

Exactly Mat, that's what drove me nuts and made me have to come up with a (much) less than elegant solution because I really can't afford to write non-reusable code at this point in my life for a matter of principle. I'll share the snippet below, can't guarantee it will work without modifications for any given resultset but it has been working flawlessly for me so far:

import yaml

import codecs

# your loop goes here (for row in ...)

   tmp_row = str(row).replace(" {", ":").replace("}", "").replace('\\"', "")

   tmp_row = codecs.escape_decode(tmp_row)[0]

   tmp_row = tmp_row.decode("utf-8")

   new_row = yaml.load(tmp_row, Loader=yaml.FullLoader)

# Note: had to also use a codecs function to fix encoding which is messed up as well.

[Mat]

Hi Deny,

that's an interesting solution. I've never thought about handle the response that way but I'll definitely check it out, thx.

Regarding the encoding, I've had troubles too, but fixed them with this line below the #!/bin/bash:

#coding=utf-8

Don't know if that helps in your case, though.

Regards

Mat

[Google Ads API Forum Advisor Prod]

Hi Deny,

Just to clarify your issue, are you hoping to iterate over the fields of, for instance, a GoogleAdsRow object, which can have varied fields depending on the query you sent, (i.e. row.campaign, row.metrics, etc)? If so I can see why it's confusing, because those objects aren't like dicts, where you can simply use a for loop to easily access each field.

For background, these objects are protobuf message instances and behave quite differently from other traditional Python objects. I admit that the interface these objects present is definitely confusing and not Pythonic at all, and in fact we're working to make the surface more sensible and hope to produce a number of improvements in the future.

In the meantime, it's possible to know from the request object which fields are present on each GoogleAdsRow instance, here's an example using search_stream:
 

import functools

# Borrowed from: https://github.com/googleads/google-ads-python/blob/master/google/ads/google_ads/util.py#L51
def get_nested_attr(obj, attr, *args):
    def _getattr(obj, attr):
        return getattr(obj, attr, *args)

    return functools.reduce(_getattr, [obj] + attr.split("."))

for batch in response:
    fields = batch.field_mask.paths 
    for row in batch.results:
        for field in fields:
            # field will be a str path, i.e. "campaign.id" or "metrics.average_cost"
            value = get_nested_attr(row, field)
            # you can also define logic based on the field name
            if "campaign.id" in field:
               # do something

That's not particularly elegant, but it might be a bit faster than your solution that requires the entire object to be serialized.

Like I mentioned, making this interface simpler is a big priority for us. If you have any other issues/questions/feature requests related to this it may be better to post directly on our Github Issues Tracker. In Github you can also keep track of new releases/changes and hopefully we'll be able to release some improvements there soon.

Best,

[Deny Watanabe]

Hi Ben, thanks for replying.

This is what worked for me, and didn't break so far, after weeks importing data for several clients:

from google.protobuf.json_format import MessageToJson

# ...

response = service.search_stream(customer_id, query=query)

for batch in response:

for row in batch.results:

serialized_json = MessageToJson(row).replace('\n', '')

snakefied = re.sub(r'\"([a-zA-Z0-9_]+?)\"\:\s', self.camel2snake, serialized_json)

values = json.loads(snakefied)

# ...

Cheers,

Deny

[Google Ads API Forum Advisor Prod]

Hi Deny,

Thanks for sharing - I just realized there's also a MessageToDict method in the protobuf json_helper module that might be helpful so that you can skip the JSON serialization.

Hope that helps.

Best,
Ben, Google Ads API Team

ref:_00D1U1174p._5004Q25Wv5e:ref