I process transaction data on PayPal by uploading from csv reports generated by PayPal. Running on PythonAnywhere.com using web2py 2.14.6 and Python 2.7.13.
The file is located using a form containing INPUT(_type='file',_name='bkfile').
This is then processed using
reader = csv.reader(request.vars.bkfile.file)
for row in reader:
....
The problem is that the first cell of the first row is not decoded properly because of the presence of BOM (byte order marker) for UTF-8. Thus where I expect to get the headers:
['Date', 'Time', 'TimeZone', 'Name', ...
I actually get:
['\xef\xbb\xbf"Date"', 'Time', 'TimeZone', 'Name', ....
A workaround is to load the downloaded file into Excel, then re-save it in .csv format. The resulting file works fine.
Editors like Notepad++ also seem to handle the BOM just fine.
Is there any way to get Python/web2py to handle the BOM correctly?
Thanks!