AdWords API Python - Internal Error or Parsing Error

[te...@segment.com]

See code with commented errors here- https://gist.github.com/tejasmanohar/bff2fe3360673a823d31484a130d4bd8

We're seeing this error consistently (every single run) for one customer when fetching their Campaign Performance Reports. We stream 1 report per campaign so that we can filter from the campaign's start time onwards. I feel as though both errors are from the same source and the CSV parsing one is just occurring because some sort of non-CSV error is returned.

Could it be a load issue? How should we proceed? Given an email address of someone from the AdWords API team, I can send over additional information about the customer / request that's failing to narrow things down.

[Shwetha Vastrad (AdWords API Team)]

Hi Tejas,

Have you tried using the download_criteria_report_as_stream.py example to stream AdWords reports? Could you use the below code snippet to process stream_data?

 try:
   while True:
      chunk = stream_data.read(CHUNK_SIZE)
      if not chunk: break
      report_data.write(chunk.decode() if sys.version_info[0]
                              == 3 and getattr(report_data, 'mode', 'w') == 'w' else chunk)
   print report_data.getvalue()
finally:
   report_data.close()
   stream_data.close()

If this doesn't work, please provide the complete report definition used and any error message received along with your clientCustomerID. Please use Reply privately to author when responding. 

Regards,

Shwetha, AdWords API Team.

[Tejas Manohar]

Hm, that doesn't parse the CSV though.

--
--
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and Google+:
https://googleadsdeveloper.blogspot.com/
https://plus.google.com/+GoogleAdsDevelopers/posts
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
 
You received this message because you are subscribed to the Google
Groups "AdWords API Forum" group.
To post to this group, send email to adwor...@googlegroups.com
To unsubscribe from this group, send email to
adwords-api...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/adwords-api?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "AdWords API Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/adwords-api/Jmbqifyr_Ro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to adwords-api...@googlegroups.com.
Visit this group at https://groups.google.com/group/adwords-api.
To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/7dd4adad-b8bd-4825-8639-5bef60ccd42c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Best regards,

Tejas Manohar

[te...@segment.com]

Hm, that does not parse the CSV though. Other than that, it's practically the same, no?

[Shwetha Vastrad (AdWords API Team)]

Hi Tejas,

This issue is related to how Python handles data streams and not specific to AdWords API. I suggest that you post your query in the AdWords API Python library repository as they are better equipped to help you out.

[Tejas Manohar]

Hey Shwetha,

I've confirmed that this issue is resolved by retrying, but it happens very consistently. Since it's resolved by retrying and I'm using codecs#getreader which knows how to handle Python unicode in both v2 and v3, I really believe this issue is due to malformed data sent back from the API as a result of an error. Let me know what more info I could provide to help. Thanks!

--

--
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and Google+:
https://googleadsdeveloper.blogspot.com/
https://plus.google.com/+GoogleAdsDevelopers/posts
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
 
You received this message because you are subscribed to the Google
Groups "AdWords API Forum" group.
To post to this group, send email to adwor...@googlegroups.com
To unsubscribe from this group, send email to
adwords-api...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/adwords-api?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "AdWords API Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/adwords-api/Jmbqifyr_Ro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to adwords-api...@googlegroups.com.
Visit this group at https://groups.google.com/group/adwords-api.

To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/780b5d92-b91a-41d5-9467-246325408472%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Mark Saniscalchi]

Hello Tejas,

I'm the owner of the Python Client Library, and am taking a look at this issue. It's difficult to say for certain whether this is a client or server-based issue because the snippet and stacktrace are incomplete, but I'm leaning towards this being a server issue or a mixture of both. The AdWordsReportBadRequestError is raised when a reportDownloadError is detected in the response--this is usually the result of bad input.

The UnicodeDecodeError is a bit of a mystery at the moment, as I can't replicate it (also in Python 3) with a data source known to contain utf-8. The fact that it seems to be expecting ascii input is telling, and I'm wondering if something else in your program would be expecting ascii input? It might be the TextIOWrapper, which will default to the system encoding (which may be ascii in your case).

In short, I can't specifically point out what went wrong here because I don't have the full picture. I need to at least see what the report definition contains in order to attempt to reproduce theAdWordsReportBadRequestError; it'd also be helpful if I knew what locale settings are.

Regards,

Mark

[Tejas Manohar]

Hey Mark,

Thanks! I will get all this information to you on Monday.

To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/e19ae838-4b54-420d-9945-629c26205f5d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Tejas Manohar]

Hey Mark,

Sorry for the delay here. Here's report definition + error in logs- https://gist.github.com/tejasmanohar/fd539957f8653a6c8375cd3b5a2cd7f5. How can I determine the locale of customers? Btw, this error happens on many reports for this customer, not just one, so I can provide more examples, and retries are solving the problem for now, but it slows everything down. 

Let me know what else I can provide to help. Thanks!

[Mark Saniscalchi]

Hello Tejas,

The stacktrace you linked to actually seems indicative of a separate issue, I suspect you're seeing this issue instead. I can tell because the UnicodeDecodeError you're receiving indicates that you're likely receiving something that was gzipped, but your report definition indicates that you set the format to CSV.

That said, I can't verify whether the original issue is still in effect. If you're still experiencing issues with new reports, can you send another report definition / stacktrace?

Regards,

Mark

[Tejas Manohar]

Got it. We have not seen the error today. I'll follow-up if it comes back. Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/d996c1d9-22bd-4f61-a1a6-c036664e43ff%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Tejas Manohar]

Hey Mark,

The issue is back and is not being solved by retries for our customer. Here's code + logs- https://gist.github.com/tejasmanohar/24f15ddb830f8bc611e9e50fdf56f688. Let me know what other information I can provide to help you debug this. It's currently blocking syncs with our beta AdWords Source at Segment (https://segment.com/sources) for a subset of customers (launch date next week).

[Mark Saniscalchi]

Hello Tejas,

Based on the stacktrace, I can see that the root cause is that you're decoding with ASCII specified as the encoding. That will produce a UnicodeEncode error if non-ASCII text appears in the reporting data. It appears that one of the libraries you're using is probably the culprit here, but it's hard to tell which. I'm pretty confident this isn't caused by the googleads library though. I suggest taking a closer look at TextIOWrapper and the parse_csv method.

Regards,

Mark

[Tejas Manohar]

from csv import DictReader as parse_csv

from io import TextIOWrapper

Gotcha. Those are both standard library methods. Do you have an alternative way you'd suggest decoding in a Python 3 environment (given the Unicode changes)? I'd be happy to migrate. If really needed, I can move this service to Python 2 (fixing this error is top priority)... but that may require some widespread changes (though small ones). Thanks again!

(Sorry if I've asked a stupid question. Python frankly isn't my strong point. Used it because maintaining an AdWords SOAP wrapper in Go or Node is quite a burden.)

To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/aa2aa911-259f-4259-b599-b83eabfc4993%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Tejas Manohar]

Moreover, looks like TextIOWrapper accepts an encoding option... https://docs.python.org/2/library/io.html#io.TextIOWrapper - what would you suggest here (considering what sort of data the googleadwords lib / AdWords API is returning)?

[Mark Saniscalchi]

Hello Tejas,

The googleads library is compatible with Python 3, as indicated by the fact that the report data was successfully retrieved. However, if you're decoding the report output (containing non-ascii data) to ascii at some point, that will raise an Exception. Often times, the encoding will default to the system encoding, which could potentially be something other than utf-8. If an option to set the encoding is offered, try setting it to utf-8.

Regards,

Mark

[Tejas Manohar]

Ah yes, i tried utf8 yesterday and t worked. Thanks a lot Mark! :)

To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/841cef47-3021-4d56-b940-c4f8b576fd2c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.