Centos 7 uwsgi no driver for psycopg2

[urban....@gmail.com]

Hi,

Trying to setup web2py, python3 centos 7, nginx, uwsgi.

The application is already running on centos 7 with python 2.7.

With python 3.6 I get the following error:

class 'RuntimeError'> Failure to connect, tried 5 times:
Traceback (most recent call last):
  File "/opt/web2py/gluon/packages/dal/pydal/base.py", line 514, in __init__
    self._adapter = adapter(**kwargs)
  File "/opt/web2py/gluon/packages/dal/pydal/adapters/postgres.py", line 28, in __call__
    return AdapterMeta.__call__(cls, *args, **kwargs)
  File "/opt/web2py/gluon/packages/dal/pydal/adapters/__init__.py", line 41, in __call__
    obj = super(AdapterMeta, cls).__call__(*args, **kwargs)
  File "/opt/web2py/gluon/packages/dal/pydal/adapters/postgres.py", line 67, in __init__
    after_connection,
  File "/opt/web2py/gluon/packages/dal/pydal/adapters/base.py", line 424, in __init__
    super(SQLAdapter, self).__init__(*args, **kwargs)
  File "/opt/web2py/gluon/packages/dal/pydal/adapters/base.py", line 85, in __init__
    self.find_driver()
  File "/opt/web2py/gluon/packages/dal/pydal/adapters/base.py", line 134, in find_driver
    "No driver of supported ones %s is available" % str(self.drivers)
RuntimeError: No driver of supported ones ('psycopg2',) is available

This only occurs when running the application through uwsgi, but not when I run the web2py application directly through:

> python3 web2py.py

I also tried importing psycopg2 in the welcome app and got the following error:

  File "/opt/web2py/gluon/custom_import.py", line 78, in custom_importer
    result = sys.modules[modules_prefix]
KeyError: 'applications.welcome.modules.psycopg2'

Any clue as to what might be the problem? Apparently python is unable to locate the psycopg2 app when ran via uwsgi.

I'm not sure if this is enough information or what other details I should provide.

[Jim S]

When I've run into similar situations on Ubuntu the resolution was to install the package using sudo.  If you just pip installed the package under your user then I believe it is stored in .local/lib/python3.x/site-packages in the user home directory.

This would explain why it works when you run it interactively.  What user is running your uwsgi process? 

Also, if possible I'd move to a newer version of python3.  Python 3.6 is end-of-life as of 12/23/2021

-Jim

[urban....@gmail.com]

Thank you for the reply!

I believe we've already tried sudo pip install psycopg2. 

Psycopg2 seems to be located at /usr/local/lib64/python3.6/site-packages

Other packages that don't exhibit the same problem are also there. 

There's another installation of psycopg2 in

/home/ouruser/.local/lib/python3.6/site-packages/psycopg2

I'm not sure about the user running the process, but I think it isn't root. I'll have to check that with the colleague chiefly wrestling with this migration.  

Yes, we're going to move to a newer version. In fact we started on a newer version but ran into this problem. We're now on 3.6 because we were checking if the problem would persist since 3.6 was the default python3 on the machine. 

It's interesting that this is an issue in particular with psycopg2.

[Jim S]

You don't have different versions of psycopg2 in the ourusr vs system drivers do you?  Looks like newer versions of psycopg2 no longer support python 3.6

https://www.psycopg.org/psycopg3/docs/basic/install.html

[urban....@gmail.com]

Upon inspection it seems psycopg2 is installed at 3 locations. .

/home/ouruser/.local/lib/python3.6/site-packages/psycopg2

/usr/local/lib64/python3.6/site-packages

AND

/usr/lib64/python3.6/site-packages/psycopg2

Two first two are 2.9.3 and the last one is 2.8.6.

When I log the packages using this code

import pkg_resources
pkgs = str({p.project_name: p.version for p in pkg_resources.working_set})

I get 'psycopg2': '2.9.3' among others. However, importing it does not work. 

Perhaps we should try with a newer version of python again with sudo pip install. Though if it were a compatibility problem I don't see why it should work without wsgi. 

[Jim S]

I only mentioned that in case there were different versions of psycopg2 in the user home vs the system one.  That could have caused some issues but since all are pre-3.1 it seems like it shouldn't cause a problem.

-Jim

[urban....@gmail.com]

Am I correct in assuming this error is occurring because python is unable to import the module and so web2py is trying to import it from local modules:

  File "/opt/web2py/gluon/custom_import.py", line 78, in custom_importer
    result = sys.modules[modules_prefix]
KeyError: 'applications.welcome.modules.psycopg2'

[Dr. Clemens R. Odendahl]

I think, that Jims answer is right. Have a try sudo-pip psycopg2 (--> sudo -H)

Regards
Clemens

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/2fffeec0-8de9-40be-8e35-18011e458415n%40googlegroups.com.

[urban....@gmail.com]

I think we've somehow solved this issue. I will report back tomorrow or when I'm sure what exactly we did.