New production install Ubuntu/nginx/uwsgi - connect to redis

[Jim S]

I'm getting this on every request

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19.

Traceback (most recent call last): File "/home/www-data/web2py/gluon/main.py", line 461, in wsgibase session._try_store_in_db(request, response) File "/home/www-data/web2py/gluon/globals.py", line 1239, in _try_store_in_db record_id = table.insert(**dd) File "/home/www-data/web2py/gluon/contrib/redis_session.py", line 150, in insert pipe.execute() File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 3443, in execute return execute(conn, stack, raise_on_error) File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 3309, in _execute_transaction if EMPTY_RESPONSE not in options]) File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 683, in pack_commands for chunk in self.pack_command(*cmd): File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 659, in pack_command for arg in imap(self.encoder.encode, args): File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 113, in encode raise DataError("Invalid input of type: 'bool'. Convert to a " DataError: Invalid input of type: 'bool'. Convert to a byte, string or number first.

I'm not sure what I'm missing.  This is a fresh install on an Ubuntu 18.04 box after running the setup-web2py-nginx-uwsgi-ubuntu.sh script (which by the way, needs some updating).

Anyone have any ideas off the top of their head or is this going to require some digging on my part?

-Jim

[Jim S]

I've narrowed this down a little bit:

Here is the code at the top of my db.py:

from gluon.contrib.redis_utils import RConn
from gluon.contrib.redis_session import RedisSession
rconn = RConn(ccfg.redis.server, ccfg.redis.port)
sessiondb = RedisSession(redis_conn=rconn, session_expiry=False)
session.connect(request, response, db=sessiondb)

If I comment out the last line everything works.  But then my sessions aren't handled by redis.

Here are my versions in use:

web2py™ Version 2.17.2-stable+timestamp.2018.10.06.11.34.06

Python Python 2.7.15rc1: /usr/bin/python (prefix: /usr)

Redis server v=4.0.9 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=1bc80a08306a3efd

Any ideas?

-Jim

[Jim S]

Also

I have another desktop linux machine here running Ubuntu 18.10 and this all works just fine.  The web2py version there is:

web2py™ Version 2.17.2-stable+timestamp.2018.10.06.18.54.02

so I tried that on the failing server and it too fails with the same traceback.  Same python version, same redis-server version.

-Jim

[Jim S]

And another thing...    This fails on the server with the rocket webserver as well as with the nginx/uwsgi server.

-Jim

[Leonel Câmara]

Does this still happen if you clear the cookies in the browser (or use a private window) before visiting the URL?

[Jim S]

Yes, same result.

I clear history/cookies and still got the message

I tried a private window and it happened.

Also got the same result testing from a different browser.

-Jim

[Jim S]

Ok, I appear to have it working:

Here's what I did.  I need someone much smarter than me to determine whether or not this is wise:

Original web2py/gluon/globals.py at line 1230

        dd = dict(locked=False,
                  client_ip=response.session_client,
                  modified_datetime=request.now,
                  session_data=session_pickled,
                  unique_key=unique_key)

I changed it to:

        dd = dict(locked='False',
                  client_ip=response.session_client,
                  modified_datetime=str(request.now),
                  session_data=session_pickled,
                  unique_key=unique_key)

Now it is working for me.

-Jim

[Leonel Câmara]

That's the same as locked=True as the string 'False' evaluates is a Truthy value. What you should try is to use:

RedisSession(redis_conn=rconn, session_expiry=False, with_lock=True)




Notice the added with_lock=True

[Jim Steil]

WooHoo! I think I finally figured it out.

Python redis package was at a different level on the other machine.

Python redis 2.10.6 works

Python redis 3.0.1 does not.  Seems to only want strings passed to it.

On my new server I ->   pip install redis==2.10.6    and now it works.

By default it is installing 3.0.1.

3.0.1 is breaking backward compatibility ->  https://pypi.org/project/redis/

What is the proper way to handle this differently for different versions of redis?  Line 1230 of globals.py is where the values are set, but redis_session.py is where the call is failing.  

-Jim

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/PdquGF_9a2E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Leonel Câmara]

Whoa nice find:

redis-py 2.X attempted to coerce any type of input into a string. While occasionally convenient, this caused all sorts of hidden errors when users passed boolean values (which were coerced to ‘True’ or ‘False’), a None value (which was coerced to ‘None’) or other values, such as user defined types.

[Jim Steil]

Leonel - what's the process to get this fixed in web2py?  I don't think I know enough about the redis library to offer a fix.  I also found a couple bugs in the ubuntu/nginx/uwsgi install script.

Would it make sense to make these install scripts version specific?

-Jim

On Fri, Dec 7, 2018 at 11:15 AM Leonel Câmara <leonel...@gmail.com> wrote:

Whoa nice find:

redis-py 2.X attempted to coerce any type of input into a string. While occasionally convenient, this caused all sorts of hidden errors when users passed boolean values (which were coerced to ‘True’ or ‘False’), a None value (which was coerced to ‘None’) or other values, such as user defined types.

--

[Leonel Câmara]

You can make a pull request with your fixes in github if you know how to fix it or you can open an issue in github.

https://github.com/web2py/web2py