appadmin CSV Import / Manual insertion ascii "ordinal out of range" error

[Jesse Hunt]

Good afternoon all,

I have been coming across errors while trying to work with the built-in appadmin to set up tables by manually inserting entries or importing CSVs (using MSSQL primarily). Put simply, any text with German umlauts (characters with the symbols above them, such as Ä, ö) would cause web2py to raise an error:

UnicodeDecodeError: 'ascii' codec can't decode byte ___ in position ___: ordinal not in range(128)

I have been searching in vain for months trying to find a solution as most queries lead to issues from 2015 with pg8000, etc.

My (dirty but) simple fix:

When you are encountering the DAL hanging up with special characters, it is most likely caused by the pyodbc package. Even if you remove your own pyodbc package, web2py still has it’s internal version to go off of.

 

To disable it:

Remove pyodbc from your local environment that web2py is running in.

Remove pypyodbc.py from web2py/gluon/contrib/pypyodbc.py

 

When the pyDAL loads up, it will not complain, as per line 63 from web2py/gluon/packages/dal/pydal/drivers.py:

 

 

try:

    import pyodbc

    DRIVERS['pyodbc'] = pyodbc

    #DRIVERS.append('DB2(pyodbc)')

    #DRIVERS.append('Teradata(pyodbc)')

    #DRIVERS.append('Ingres(pyodbc)')

except ImportError:

    try:

        import pypyodbc as pyodbc

        DRIVERS['pyodbc'] = pyodbc

    except ImportError:

        pyodbc = None

 

(If the driver is not found, it will just label pyodbc as none)

I hope this saves someone else the trouble in the future.