Psycopg2 version support

[Marc Tamlyn]

Hi all,

Related ticket - https://code.djangoproject.com/ticket/24335

Django 1.8 will necessarily be the first version with a true minimum requirement on psycopg2 version. Historically we have never documented a required version.

- The new UUIDField requires at least 2.0.9

- Some code which is currently contained in the backend, not in contrib.postgres, but I think only relates to arrays of (Generic)IPAddressFields requires 2.4.5. As yet I have not tried moving this compat code so it only runs if contrib.postgres installed.

- Range fields (in contrib.postgres) require 2.5

- There may be other things I'm not responsible for I don't know about, with different version requirements.

Release history for psycopg2:

- 2.0.9 is extremely old (date unknown)

- 2.4.5 was released in March 2012

- 2.5 was released in April 2013

- 2.6 was released this month (Feb 2015)

According to Claude, some distros have a python-psycopg2 package which is still on 2.4.5 (or maybe older?).

We have several options:

- Ensure as much as possible works with old versions of psycopg2 by shuffling code around and/or using conditional imports.

- Making sure everything *outside contrib* doesn't require newer psycopg2 that 2.0.9, and making contrib.postgres require 2.5. This may not be possible, but I think it should be.

- Change nothing but docs, so outside contrib requires 2.4.5, contrib.postgres requires 2.5.

- Just document that Django 1.8+ needs psycopg2 2.5 (pip install it...)

The latter would be my preferred option, but then I've never understood the argument for distro packages instead of pip. In any case, I feel that any distro not supporting a two year old version of psycopg2 is unlikely to be distributing a brand new version of Django.

As far as I know we have no official policy on dependencies like this (we don't have many). 

Marc

[Shai Berger]

Hi all,

On Saturday 14 February 2015 22:53:13 Marc Tamlyn wrote:
>
> Django 1.8 will necessarily be the first version with a true minimum
> requirement on psycopg2 version. Historically we have never documented a
> required version.
>

FWIW, we have long had minimum requirements for other backends -- all other
backends, if I'm not mistaken.

>
> Release history for psycopg2:
> - 2.0.9 is extremely old (date unknown)
> - 2.4.5 was released in March 2012
> - 2.5 was released in April 2013
> - 2.6 was released this month (Feb 2015)
>

During the run to 1.7, we defined a rule about the DBMSs themselves: We do not
support versions post their end-of-life. That is why Django 1.8 will no longer
support Postgres 8.4, for example. I suppose a similar rule should apply to
the drivers. So, the key question becomes, is PG 2.0.x still supported? is
2.4.x? Since you mention significant new features in 2.4.5, is it even valid to
talk about 2.4.x?

> According to Claude, some distros have a python-psycopg2 package which is
> still on 2.4.5 (or maybe older?).
>

Yep. Debian stable, for one.

> We have several options:
> - Ensure as much as possible works with old versions of psycopg2 by
> shuffling code around and/or using conditional imports.
> - Making sure everything *outside contrib* doesn't require newer psycopg2
> that 2.0.9, and making contrib.postgres require 2.5. This may not be
> possible, but I think it should be.
> - Change nothing but docs, so outside contrib requires 2.4.5,
> contrib.postgres requires 2.5.
> - Just document that Django 1.8+ needs psycopg2 2.5 (pip install it...)
>

1.8 is our next LTS. Whatever we commit to, binds us until 2018, more or less.
I think defining separate dependencies for contrib makes no sense as long as it
isn't distributed separately. Or rather, unless someone distributes a Django-
without-contrib. So I tend towards your latter option.

> The latter would be my preferred option, but then I've never understood the
> argument for distro packages instead of pip.

On reasonable distros, you can set things so you get critical updates (and
only critical updates) automatically. That is very attractive for low-
personnel production sites, and there's no reasonable way to achieve this with
pip.

My 2 cents,
Shai.

[Tim Graham]

Unless any counter arguments are presented, I suggest let's require psycopg 2.5 for now. Is there a scenario where you could pip install Django but not pip install psycopg2?

[Christophe Pettus]

On Feb 15, 2015, at 4:14 PM, Tim Graham <timog...@gmail.com> wrote:

> Is there a scenario where you could pip install Django but not pip install psycopg2?

Installing psycopg2 does require development tools, while Django does not. I'm not offering this as a compelling argument for anything, just an observation.

--
-- Christophe Pettus
x...@thebuild.com

[Collin Anderson]

Right, I think the situation would be if you don't have admin/sudo and the -dev / -devel header files are not present.

Seems to me 2.4.5 would be a safe minimum today, but supporting it until 2018 seems a bit excessive.

[Florian Apolloner]

Core could stay at 2.4.5 and contrib.postgresql could require a newer version :)

[Gert Van Gool]

For what it's worth, CentOS 6 (supported until November 30, 2020) ships with 2.0.14. CentOS 7 has 2.5.1.

-- Gert

Mobile: +32 498725202

Twitter: @gvangool
Web: http://gertvangool.be

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/dbde195e-39e8-438c-a30d-f95911e1b5ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Stephenson]

> Installing psycopg2 does require development tools, while Django does not.  I'm not offering this as a compelling argument for anything, just an observation. 

The postgres-*.*-dev libraries are a requirement for both 2.0.14 and 2.5, so if it's possible to install 2.0.14 via the package manager then it's possible to pip install 2.5.

[Thomas Stephenson]

> Installing psycopg2 does require development tools, while Django does not

Installing psycopg2 2.0.0 requires exactly the same `postgres-*-dev` libraries that 2.5 does as far as I know. If it is possible to install psycopg2 via the OS package manager then 2.5 can be installed via pip.

[Raphael Michel]

Am Mon, 16 Feb 2015 01:15:53 -0800 (PST)
schrieb Thomas Stephenson <ova...@gmail.com>:

> The postgres-*.*-dev libraries are a requirement for both 2.0.14 and
> 2.5, so if it's possible to install 2.0.14 via the package manager
> then it's possible to pip install 2.5.

No, I believe this is not correct. The system package manager might (or
in most cases, will) distribute a _compiled_ version of 2.0.14 where you
do not need the -dev libraries to execute them, while pip will download
_sources_ and compile them for you and therefore needs the -dev
libraries.

Raphael

[Florian Apolloner]

Not exactly, cause installing psycopg2 via the OS package manager should __not__ require dev packages at all.

[Thomas Stephenson]

My mistake, sorry.

So, to get the conversation back on track, is `2.4.5` acceptable as the minimum version, even if users who

- use CentOS 6
- install dependencies via the OS package manager
- don't have sudo/root
- can't request the sysadmin to install dev packages; and
- keep django at the latest version

don't have a valid upgrade path?

Thomas

[Gert Van Gool]

In CentOS 6, you can't run Django 1.8 anyway. It ships Python 2.6.

So if you want to run Django 1.8 on CentOS 6, you need to compile Python yourself. In that case I don't think they'll complain about having to build psycopg2 separately as well.

In the RHEL/CentOS universe, anyone who wants to run Django 1.8 needs either SCL's (software collections that are installed next to system binaries) or has access to psycopg2 2.5.

So my point for CentOS/RHEL 6 can be ignored.

-- Gert

Mobile: +32 498725202

Twitter: @gvangool
Web: http://gertvangool.be

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4efc59f0-0b97-4ed9-bbce-bd1b124ea978%40googlegroups.com.

[Tim Graham]

How about this plan:
* Document 2.4.3 as the minimum if you don't want to use contrib.postgres
* Recommend 2.5+ and document it as the minimum to use contrib.postgres
* Require 2.5+ for Django's test suite to avoid having to add conditional imports/test skipping.

[Wim Feijen]

Hi, 

I am definitely in favor of just one version requirement in order not to complicate things. So then the minimum version would be either 2.5; or 2.4.5 and write special cases in contrib.postgres to deal with that.

Regards, Wim

[Tim Graham]

Hi Wim,

Thanks for the feedback. My thinking is that requiring 2.5 will needlessly cut off anyone who really wants to use an older version (like Debian stable), but patching contrib.postgres to work with < 2.5 will add complexity which we want to avoid.

Here's the pull request that implements my proposal:

https://github.com/django/django/pull/4148

I think it's simple enough. How about you?

[Thomas Stephenson]

2.5 is relatively easy to work around in contrib.postgres and a consistent minimum everywhere is more straightforward. There is just a conditional import on the fields.Range and forms.Range modules to deal with. 

I'd also add in a module level check that the psycopg2 version is OK, so that the module loads with a clear error. Not everyone checks the documentation.

[Thomas Stephenson]

ps. 2.4.5 is the real minimum. The db.backends module loads without error at 2.4.3, but proper support for Inet arrays was not added until 2.4.5, so any code that uses GenericIPAddressField could fail unexpectedly.

And now I'll shut up, because I'm not entitled to an opinion here yet :D

Thomas

[Collin Anderson]

On Mon, Feb 16, 2015 at 1:41 PM, Thomas Stephenson <ova...@gmail.com> wrote:

And now I'll shut up, because I'm not entitled to an opinion here yet :D

By the way, Thomas, your opinion is welcome. :)

Collin