AdWords API Python - Internal Error or Parsing Error

146 views
Skip to first unread message

te...@segment.com

unread,
Jul 18, 2016, 2:42:14 PM7/18/16
to AdWords API Forum

We're seeing this error consistently (every single run) for one customer when fetching their Campaign Performance Reports. We stream 1 report per campaign so that we can filter from the campaign's start time onwards. I feel as though both errors are from the same source and the CSV parsing one is just occurring because some sort of non-CSV error is returned.

Could it be a load issue? How should we proceed? Given an email address of someone from the AdWords API team, I can send over additional information about the customer / request that's failing to narrow things down.
Message has been deleted

Shwetha Vastrad (AdWords API Team)

unread,
Jul 18, 2016, 5:10:00 PM7/18/16
to AdWords API Forum
Hi Tejas,

Have you tried using the download_criteria_report_as_stream.py example to stream AdWords reports? Could you use the below code snippet to process stream_data?


 
try:
   
while True:
      chunk
= stream_data.read(CHUNK_SIZE)
     
if not chunk: break
     
report_data.write(chunk.decode() if sys.version_info[0]
                             
== 3 and getattr(report_data, 'mode', 'w') == 'w' else chunk)
   
print report_data.getvalue()
finally:
   report_data
.close()
   stream_data
.close()

If this doesn't work, please provide the complete report definition used and any error message received along with your clientCustomerID. Please use Reply privately to author when responding. 

Regards,
Shwetha, AdWords API Team.

Tejas Manohar

unread,
Jul 19, 2016, 10:21:29 PM7/19/16
to adwordsapiad...@google.com, AdWords API Forum
Hm, that doesn't parse the CSV though.

--
--
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and Google+:
https://googleadsdeveloper.blogspot.com/
https://plus.google.com/+GoogleAdsDevelopers/posts
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
 
You received this message because you are subscribed to the Google
Groups "AdWords API Forum" group.
To post to this group, send email to adwor...@googlegroups.com
To unsubscribe from this group, send email to
adwords-api...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/adwords-api?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "AdWords API Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/adwords-api/Jmbqifyr_Ro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to adwords-api...@googlegroups.com.
Visit this group at https://groups.google.com/group/adwords-api.
To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/7dd4adad-b8bd-4825-8639-5bef60ccd42c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Best regards,

Tejas Manohar

te...@segment.com

unread,
Jul 19, 2016, 10:22:15 PM7/19/16
to AdWords API Forum
Hm, that does not parse the CSV though. Other than that, it's practically the same, no?

Shwetha Vastrad (AdWords API Team)

unread,
Jul 20, 2016, 4:56:04 PM7/20/16
to AdWords API Forum
Hi Tejas,

This issue is related to how Python handles data streams and not specific to AdWords API. I suggest that you post your query in the AdWords API Python library repository as they are better equipped to help you out.

Tejas Manohar

unread,
Jul 26, 2016, 9:18:17 PM7/26/16
to adwordsapiad...@google.com, AdWords API Forum
Hey Shwetha,

I've confirmed that this issue is resolved by retrying, but it happens very consistently. Since it's resolved by retrying and I'm using codecs#getreader which knows how to handle Python unicode in both v2 and v3, I really believe this issue is due to malformed data sent back from the API as a result of an error. Let me know what more info I could provide to help. Thanks!

--
--
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and Google+:
https://googleadsdeveloper.blogspot.com/
https://plus.google.com/+GoogleAdsDevelopers/posts
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
 
You received this message because you are subscribed to the Google
Groups "AdWords API Forum" group.
To post to this group, send email to adwor...@googlegroups.com
To unsubscribe from this group, send email to
adwords-api...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/adwords-api?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "AdWords API Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/adwords-api/Jmbqifyr_Ro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to adwords-api...@googlegroups.com.
Visit this group at https://groups.google.com/group/adwords-api.

For more options, visit https://groups.google.com/d/optout.

Mark Saniscalchi

unread,
Jul 29, 2016, 6:05:14 PM7/29/16
to AdWords API Forum, adwordsapiad...@google.com
Hello Tejas,

I'm the owner of the Python Client Library, and am taking a look at this issue. It's difficult to say for certain whether this is a client or server-based issue because the snippet and stacktrace are incomplete, but I'm leaning towards this being a server issue or a mixture of both. The AdWordsReportBadRequestError is raised when a reportDownloadError is detected in the response--this is usually the result of bad input.

The UnicodeDecodeError is a bit of a mystery at the moment, as I can't replicate it (also in Python 3) with a data source known to contain utf-8. The fact that it seems to be expecting ascii input is telling, and I'm wondering if something else in your program would be expecting ascii input? It might be the TextIOWrapper, which will default to the system encoding (which may be ascii in your case).

In short, I can't specifically point out what went wrong here because I don't have the full picture. I need to at least see what the report definition contains in order to attempt to reproduce theAdWordsReportBadRequestError; it'd also be helpful if I knew what locale settings are.

Regards,
Mark

Tejas Manohar

unread,
Jul 29, 2016, 6:13:46 PM7/29/16
to msanis88.adxb...@gmail.com, AdWords API Forum, adwordsapiad...@google.com
Hey Mark,

Thanks! I will get all this information to you on Monday.


For more options, visit https://groups.google.com/d/optout.

Tejas Manohar

unread,
Aug 19, 2016, 10:12:39 PM8/19/16
to msanis88.adxb...@gmail.com, AdWords API Forum, adwordsapiad...@google.com
Hey Mark,

Sorry for the delay here. Here's report definition + error in logs- https://gist.github.com/tejasmanohar/fd539957f8653a6c8375cd3b5a2cd7f5. How can I determine the locale of customers? Btw, this error happens on many reports for this customer, not just one, so I can provide more examples, and retries are solving the problem for now, but it slows everything down. 

Let me know what else I can provide to help. Thanks!

Mark Saniscalchi

unread,
Aug 23, 2016, 7:20:35 PM8/23/16
to AdWords API Forum, msanis88.adxb...@gmail.com, adwordsapiad...@google.com
Hello Tejas,

The stacktrace you linked to actually seems indicative of a separate issue, I suspect you're seeing this issue instead. I can tell because the UnicodeDecodeError you're receiving indicates that you're likely receiving something that was gzipped, but your report definition indicates that you set the format to CSV.

That said, I can't verify whether the original issue is still in effect. If you're still experiencing issues with new reports, can you send another report definition / stacktrace?

Regards,
Mark

Tejas Manohar

unread,
Aug 23, 2016, 7:57:56 PM8/23/16
to msanis88.adxb...@gmail.com, AdWords API Forum, adwordsapiad...@google.com
Got it. We have not seen the error today. I'll follow-up if it comes back. Thanks!


For more options, visit https://groups.google.com/d/optout.

Tejas Manohar

unread,
Aug 25, 2016, 4:24:43 PM8/25/16
to msanis88.adxb...@gmail.com, AdWords API Forum, adwordsapiad...@google.com
Hey Mark,

The issue is back and is not being solved by retries for our customer. Here's code + logs- https://gist.github.com/tejasmanohar/24f15ddb830f8bc611e9e50fdf56f688. Let me know what other information I can provide to help you debug this. It's currently blocking syncs with our beta AdWords Source at Segment (https://segment.com/sources) for a subset of customers (launch date next week).

Mark Saniscalchi

unread,
Aug 25, 2016, 4:58:22 PM8/25/16
to AdWords API Forum, msanis88.adxb...@gmail.com, adwordsapiad...@google.com
Hello Tejas,

Based on the stacktrace, I can see that the root cause is that you're decoding with ASCII specified as the encoding. That will produce a UnicodeEncode error if non-ASCII text appears in the reporting data. It appears that one of the libraries you're using is probably the culprit here, but it's hard to tell which. I'm pretty confident this isn't caused by the googleads library though. I suggest taking a closer look at TextIOWrapper and the parse_csv method.

Regards,
Mark

Tejas Manohar

unread,
Aug 25, 2016, 5:05:34 PM8/25/16
to msanis88.adxb...@gmail.com, AdWords API Forum, adwordsapiad...@google.com
from csv import DictReader as parse_csv
from io import TextIOWrapper

Gotcha. Those are both standard library methods. Do you have an alternative way you'd suggest decoding in a Python 3 environment (given the Unicode changes)? I'd be happy to migrate. If really needed, I can move this service to Python 2 (fixing this error is top priority)... but that may require some widespread changes (though small ones). Thanks again!

(Sorry if I've asked a stupid question. Python frankly isn't my strong point. Used it because maintaining an AdWords SOAP wrapper in Go or Node is quite a burden.)


For more options, visit https://groups.google.com/d/optout.

Tejas Manohar

unread,
Aug 25, 2016, 5:07:01 PM8/25/16
to msanis88.adxb...@gmail.com, AdWords API Forum, adwordsapiad...@google.com
Moreover, looks like TextIOWrapper accepts an encoding option... https://docs.python.org/2/library/io.html#io.TextIOWrapper - what would you suggest here (considering what sort of data the googleadwords lib / AdWords API is returning)?

Mark Saniscalchi

unread,
Aug 26, 2016, 1:55:24 PM8/26/16
to AdWords API Forum, msanis88.adxb...@gmail.com, adwordsapiad...@google.com
Hello Tejas,

The googleads library is compatible with Python 3, as indicated by the fact that the report data was successfully retrieved. However, if you're decoding the report output (containing non-ascii data) to ascii at some point, that will raise an Exception. Often times, the encoding will default to the system encoding, which could potentially be something other than utf-8. If an option to set the encoding is offered, try setting it to utf-8.

Regards,
Mark

Tejas Manohar

unread,
Aug 26, 2016, 1:57:10 PM8/26/16
to msanis88.adxb...@gmail.com, AdWords API Forum, adwordsapiad...@google.com
Ah yes, i tried utf8 yesterday and t worked. Thanks a lot Mark! :)

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages