Error trying to go back to python 2 after testing python 3: TypeError: translate() takes exactly one argument (2 given)

[Lisandro]

I'm testing my web2py application to see what I need to fix in order to make it compatible with python3.

I'm using web2py Version 2.17.1-stable+timestamp.2018.08.06.01.02.56

My application has been running with python2 for a long time.

Yesterday I tried with python3 (setting up a virtual environment) and everything went ok.

But now I can't go back to use python2, any request to the application fails with this error:

TypeError: translate() takes exactly one argument (2 given)

Traceback (most recent call last):
  File "/home/gonguinguen/medios/gluon/main.py", line 435, in wsgibase
    session.connect(request, response)
  File "/home/gonguinguen/medios/gluon/globals.py", line 996, in connect
    response.cookies[response.session_id_name] = response.session_id
  File "/usr/lib64/python2.7/Cookie.py", line 592, in __setitem__
    self.__set(key, rval, cval)
  File "/usr/lib64/python2.7/Cookie.py", line 585, in __set
    M.set(key, real_value, coded_value)
  File "/usr/lib64/python2.7/Cookie.py", line 459, in set
    if "" != translate(key, idmap, LegalChars):
  File "/usr/lib64/python2.7/string.py", line 493, in translate
    return s.translate(table, deletions)
TypeError: translate() takes exactly one argument (2 given)

I've already flushed redis cache, restarted webserver, cleared browser data... 

Any other suggestion?

Thanks in advance

Regards,

Lisandro

[Dave S]

Delete all the .pyc files?

/dps

[Lisandro]

I've found the issue, it's not web2py related, sorry about that.

My web2py instance has several applications running, each one is attached to a domain.

I store a the map of domains:apps in a dictionary that I save to a .pkl file.

Then my routes.py reads that file and loads the map of domains:apps

I write the .pkl file like this:

with open('map.pkl', 'wb') as file:
    pickle.dump(dictionary_map, file, protocol=2)

Notice I use protocol=2 because I want to be able to read/write the file with python 2 and 3.

In my routes.py I read the file like this:

map = pickle.load(open('domains_apps.pkl', 'rb'))

routers = dict(
    BASE=dict(
        default_controller='default',
        default_function='index',
        domains=map,
        map_static=True,
        exclusive_domain=True,
    )
)

However, after writing .pkl the file with python 3 and returning to python 2, my applications fail with the error reported in my first message. The error goes away if I replace the .pkl file with an old backup I had made before using python 2.

I have noticed that once the .pkl file is written with python 3, then reading it with python 2 and 3 throws different results:

with python 3:

>>> r = pickle.load(open('domains_apps.pkl', 'rb'))

>>> print(r)

{'prod.com': 'prod', 'test.com': 'test'}

with python 2:

>>> r = pickle.load(open('domains_apps.pkl', 'rb'))

>>> print(r)

{u'prod.com': u'prod', u'test.com': u'test'}

Notice that in python 2 reading the .pkl file (that was written with python 3 using protocol=2) returns unicode strings. This doesn't happen in python 3. But i'm not sure what protocol to use. 

I'll do some more tests and I'll post here whatever solution I can find. 

Thanks for your time!

Regards,

Lisandro.

[Massimo Di Pierro]

I am puzzled by this too. The error is not in web2py code. The error is in the string.py module.

Also in your json example you are getting unicode both in py2 and py3 except py3 does not put the u'' in front of unicode strings because they are default.

[Lisandro]

> Also in your json example you are getting unicode both in py2 and py3 except py3 does not put the u'' in front of unicode strings because they are default.

I've used type() to inspect the keys and values of the dictionary in py2 and py3:

py2

>>> map = pickle.load(open(path, 'rb'))
>>> first_key = list(map.keys())[0]
>>> print(type(first_key))
<type 'unicode'>
>>> print(type(map[first_key]))
<type 'unicode'>

py3

>>> map = pickle.load(open(path, 'rb'))
>>> first_key = list(map.keys())[0]
>>> print(type(first_key))
<class 'str'>
>>> print(type(map[first_key]))
<class 'str'>

Notice in python3 it says "class" not "type" and it is str, while on python2 it says unicode :/

Anyway, I don't pretend to bother with this issue unrelated to web2py.

My goal is to be able to read/write the dicti to a .pkl file with py2 and py3, so I'll keep testing and I'll post the solution here if I find it :)

[Lisandro]

Well, in case someone else is facing the same issue, I'll post how I solved it.

First, let me add that I had the same issue using json library instead of pickle: once the file (json or pkl) was written using py3, then I was able to read it with both py3 and py2, but py3 read it as str while py2 reads it as unicode. 

And in the case of py2, unicode wasn't working, so I ended up doing a small fix in my routes.py file:

map = pickle.load(open('domains_apps.pkl', 'rb'))



# ---------- TEMP FIX ---------------
new_map = {}
for key in list(map):
    new_map[str(key)] = str(map[key])
map = new_map
# -----------------------------------

routers = dict(


    BASE=dict(
        default_controller='default',
        default_function='index',
        domains=map,
        map_static=True,
        exclusive_domain=True,
    )
)


Basically, what I do is to create a new dictionary where I explicitly set keys and values using str()

It's not an elegant solution, but it seems to work for py2 and py3.

Anyway, I'll only keep the fix until I'm sure I don't need to go back to py2.

Thanks for the help!

[Massimo Di Pierro]

In python 2 type str means bytes and type unicode means unicode

In python 3 class bytes means bytes and class str means unicode.

This is my biggest problem with py3. They changed the meaning of a keyword to mean the opposite of what it did in py2. It makes things really hard to debug.