Problem with csv files downloaded from PayPal

17 views
Skip to first unread message

David Manns

unread,
Oct 5, 2017, 11:24:33 AM10/5/17
to web2py-users
I process transaction data on PayPal by uploading from csv reports generated by PayPal. Running on PythonAnywhere.com using web2py 2.14.6 and Python 2.7.13.

The file is located using a form containing INPUT(_type='file',_name='bkfile').

This is then processed using

            reader = csv.reader(request.vars.bkfile.file)
           
            for row in reader:
                 ....

The problem is that the first cell of the first row is not decoded properly because of the presence of BOM (byte order marker) for UTF-8. Thus where I expect to get the headers:
        
            ['Date', 'Time', 'TimeZone', 'Name', ...

I actually get:

            ['\xef\xbb\xbf"Date"', 'Time', 'TimeZone', 'Name', ....

A workaround is to load the downloaded file into Excel, then re-save it in .csv format. The resulting file works fine.

Editors like Notepad++ also seem to handle the BOM just fine.

Is there any way to get Python/web2py to handle the BOM correctly?

Thanks!

Reply all
Reply to author
Forward
0 new messages