Overhaul extra_requires to include more optional dependencies

[jr...@leukeleu.nl]

Django has a number of optional features that require the installation of additional Python packages.

Right now it's possible to $ pip install Django[bcrypt,argon2] to install Django and the optional packages (at the correct minimum version) required for bcrypt/argon2 support.

However this is not possible for other optional features, like database/cache backend, gis, certain image related features etc.

I've created a pull request adding the optional dependencies found in the test requirements: https://github.com/django/django/pull/9440 / https://code.djangoproject.com/ticket/28905

So end users can do things like:

$ pip install Django[mysql,sqlparse,images,memcached]
$ pip install Django[postgresql,gis]
$ pip install Django[selenium,test-parallel]
 

Not only would this make it much easier to install a particular combination of Django and optional dependencies correctly, it also makes some of the optional features/dependencies a little bit more discoverable.

This might not the full set of optional dependencies, or the right grouping/naming, so a review is more than welcome (i.e. should sqlparse be added to the mysql/oracle extra requirements? Should a sqlite extra be added that only includes sqlparse?).
 
Besides adding this to setup.py, additional changes to the documentation, test instructions and scripts, code warnings etc. will probably be necessary.

If anyone objects to adding these extra requirements I'd also like to hear about it.

[Reinout van Rees]

Op 11-12-17 om 11:26 schreef jr...@leukeleu.nl:

> Django has a number of optional features that require the installation
> of additional Python packages.
>

> ... snip ...

>
> I've created a pull request adding the optional dependencies found in
> the test requirements: https://github.com/django/django/pull/9440 /
> https://code.djangoproject.com/ticket/28905
>
> So end users can do things like:
>
>
> $ pip install Django[mysql,sqlparse,images,memcached]
> $ pip install Django[postgresql,gis]
> $ pip install Django[selenium,test-parallel]

Good idea! I have two comments:


First, in the ticket, I see psycopg2 as requirement for the "postgresql"
extra. Fine. That's totally required.
For the "gis" extra it is numpy and geoip2. Numpy isn't strictly
required: according to the readme it is 'highly recommended'.

=> What is the meaning of such an 'extra'? Does it contain 'highly
recommended' and higher? Or also optional packages?

You *do* mention 'optional' in the ticket. So if I do 'Django[gis]' I
ought to get every package that's mentioned as "hey, that's handy" in
the documentation, right?



Second, I wonder whether the one-item extras are really needed. If you
want to use numpy, you'll install numpy yourself. You don't really need
'Django[numpy]' to tell you to install numpy.

admindocs->docutils: yes, that's handy
numpy->numpy: seems less useful
gis->numpy and geoip2: yes, useful




Reinout

--
Reinout van Rees http://reinout.vanrees.org/
rei...@vanrees.org http://www.nelen-schuurmans.nl/
"Learning history by destroying artifacts is a time-honored atrocity"

[emo...@mozilla.com]

Thank you for looking into this!

Another benefit alongside ease of install and discoverability, is that it makes the requirements files of projects using Django more self-documenting (in that it avoids stray dependency entries that appear to be unused, but actually aren't - and so need a comment to explain what uses them). See also the related open ticket https://code.djangoproject.com/ticket/27682.

Re the single package entries, I think there is still value in having them, so long as they are named by feature rather than by package name. eg `Django[crypto]` vs `Django[bcrypt]`. This allows the particular choice of library to be a Django implementation detail that can be adjusted without needing requirements file changes.

Ed