setting up pscycopg2

[Dave S]

When I check Last week I ended up making the move to Postgres on the busier of my production servers, and the changeover appears to be successful (imported existing data via CSV).  I was interested to see that the import preserved some gaps in the id field, and that Postgres was filling some of those gaps and holding off on using id's above the imported max(id).  But I'm also getting errors about 'ERROR 23505, duplicate key value value viotlates unique contraint "updreq_y_pkey".  Key(id)=(23681) already exists.'

Looking at this made me realize I was running pg8000 (still supplied in contrib in 2.15.4), and I wanted to switch to pscyopg2.  So I tried

pip install psycopg2

and that was progressing fine until it got to copying the built module into /usr/lib64/python2.7/dist-packages/psycogpg2 -- permission denied (not too surprising).  So I tired the obvious:

sudo pip install psycopg2

Which ends abruptly with

[traceback excerpt]
The 'pip==9.0.3' distribution was not found and required by the application

When I check pip's version

pip --version

gives

pip 18.0 for  /usr/.../pip (python2.7)

and

sudo pip --version

gives the 9.0.3 not found message.

What can I do now?
(AWS Linux, up to date, OS_VERSION says "2018.03")

/dps

[Dave S]

On Monday, May 13, 2019 at 2:31:33 PM UTC-7, Dave S wrote:

When I check Last week I ended up making the move to Postgres on the busier of my production servers, and the changeover appears to be successful (imported existing data via CSV).  I was interested to see that the import preserved some gaps in the id field, and that Postgres was filling some of those gaps and holding off on using id's above the imported max(id).  But I'm also getting errors about 'ERROR 23505, duplicate key value value viotlates unique contraint "updreq_y_pkey".  Key(id)=(23681) already exists.'

Looking at this made me realize I was running pg8000 (still supplied in contrib in 2.15.4), and I wanted to switch to pscyopg2.  So I tried

pip install psycopg2

and that was progressing fine until it got to copying the built module into /usr/lib64/python2.7/dist-packages/psycogpg2 -- permission denied (not too surprising).  So I tired the obvious:

sudo pip install psycopg2

Which ends abruptly with

[traceback excerpt]
The 'pip==9.0.3' distribution was not found and required by the application

[...]

 

What can I do now?
(AWS Linux, up to date, OS_VERSION says "2018.03")

 

 
Okay, I found the -t option, and from the web2py directory specified site-packages
Now, how do I get web2py.py to find the driver?

/dps

[Dave S]

Is site-packages the wrong location?

/dps

 

[Dave S]

That's web2py/applications/myapp/site-packages

/dps

 

[Dave S]

Help!

/dps

[Nico Zanferrari]

Hi Dave,

I've faced the same problem  trying to add psycopg2 on MacOs binaries with python 2.7. Following some advices, I've resolved it by installing unixodbc (with brew) before it.

I hope it helps,

Nico

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/b2aecb42-d28c-42a7-84fe-1de80100f735%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave S]

On Saturday, June 1, 2019 at 1:41:12 AM UTC-7, Nico Zanferrari wrote:

Hi Dave,

I've faced the same problem  trying to add psycopg2 on MacOs binaries with python 2.7. Following some advices, I've resolved it by installing unixodbc (with brew) before it.

I hope it helps,

Nico

 Sadly, pip tells me "could not find a version that satisfies te requirement unixodbc"

And to help with my memory later, -t is an option to "pip install", not to pip in general.

Oh, and I once said

>>> Is site-packages the wrong location?

>> That's web2py/applications/myapp/site-packages

No, it's not.  It's just web2py/site-packages.

And to expand on my comments about the original traceback ... sudo pip --version gives the same traceback, while pip --version works.  Probably a path issue, but I'm not sure I want to mess around with sudoing sudos.

/dps

To unsubscribe from this group and stop receiving emails from it, send an email to web...@googlegroups.com.

[Dave S]

On Tuesday, June 4, 2019 at 3:42:49 PM UTC-7, Dave S wrote:

 

>>> Is site-packages the wrong location?

>> That's web2py/applications/myapp/site-packages

No, it's not.  It's just web2py/site-packages.

I see gluon/packages/dal/pydal/drivers.py attempts to import psycopg2.  What's the search path for that import?

/dps

 

[Annet]

Hi Dave,

I work both on Mac OS X and Linux and have Postgres databases.

On Mac OS X I located the pg_config file and added its location to the .bash_profile file:

export PATH=/usr/local/bin:$PATH
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/9.3/bin

Then I got pip and installed psycopg2

lionserver:~ userx$ sudo python /Users/userx/Downloads/get-pip.py
lionserver:~ userx$ python -m pip install psycopg2

On Linux I ran the following code to install psycopg2

export PATH=/usr/pgsql-9.1/bin/:$PATH
mkdir -p ~/lib/python2.7
easy_install-2.7 psycopg2

As far as I know Web2py will always use psycopg2 instead of pg8000 if psycopg2 is installed.

I hope this helps you solve the issue.

Kind regards,

Annet

[Dave S]

On Wednesday, June 12, 2019 at 2:29:51 AM UTC-7, Annet wrote:

 

[...]

On Linux I ran the following code to install psycopg2

export PATH=/usr/pgsql-9.1/bin/:$PATH
mkdir -p ~/lib/python2.7
easy_install-2.7 psycopg2

This isn't done as sudo?
 

 

As far as I know Web2py will always use psycopg2 instead of pg8000 if psycopg2 is installed.

The trick seems to letting web2py know it is installed in its local environment.

/dps
 

I hope this helps you solve the issue.

Kind regards,

Annet

Thanks for your suggestions.

/dps
 

[Annet]

This isn't done as sudo?

With these lines I installed psycopg2 in my hosting environment, no sudo needed.

 

| The trick seems to letting web2py know it is installed in its local environment.

I installed psycopg2 in the site-packages folder of the python version I am using

same folder pip resides.

I am glad you solved the issue.

Kind regards,

Annet.

[Dave S]

On Thursday, June 13, 2019 at 1:47:34 AM UTC-7, Annet wrote:

This isn't done as sudo?

With these lines I installed psycopg2 in my hosting environment, no sudo needed.

In my environment, the easy_install fails because I don't have write-permission to /usr/lib/python2.7/dist-packages

The easy_install instructions for putting a package in a non-standard directory seem fussy, and they say "try the other ways first".

I'm not sure what the purpose of the mkdir is, since easy_install  doesn't know about that directory.

 

| The trick seems to letting web2py know it is installed in its local environment.

I installed psycopg2 in the site-packages folder of the python version I am using

same folder pip resides.

I am glad you solved the issue.

Unfortunately, I've only identified a task to be done, not how to do the task.

/dps
 

[Dave S]

I think have now solved the issue  of installing psycopg2.  Once again, pip install failed due to permissions, and sudo pip install failed due to wrong version of pip.  This time, a small grey cell said ..."use whereis", so I did, figured out which of the listed pips was correct, and put that full path into the sudo command.  Bingo!

it still seems to me I should have been able to put psycopg2 in the web2py tree, but I haven't quite figured out what gluon/packages/dal/pydal/adapters/base.py and gluon/packages/dal/pydal/adapters/postgres.py do when they are trying to locate drivers, and obviously I didn't find the right spot in the tree for psycopg2.

No matter, I've got things running now, and I'm off to keep an eye on the scheduler memory usage.

/dps