[Django] #30933: Users need to be directed to install 32-bit Python for GeoDjango
Django
------------------------------------------------+------------------------
Reporter: Adam Starrh | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Documentation | Version: 2.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------------+------------------------
I'm setting GeoDjango up on Windows and discovered that GDAL is (at least
currently) incompatible with 64-bit Python. Using 32-bit Python and 32-bit
OSGeo4Win, I was finally able to get it working. I suggest this be
mentioned in the docs to circumvent future headaches for other users.
Pull request:
https://github.com/django/django/pull/11995
--
Ticket URL: <https://code.djangoproject.com/ticket/30933>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Adam Starrh | Owner: nobody
Cleanup/optimization |
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Adam Starrh):
Basically, this has happened to me a few times over the years. As a casual
programmer, when following the setup docs I habitually install the 64-bit
versions of everything because I figure they should match the OS/chipset.
Inevitably, something fails mysteriously and I spend a lot of time trying
to figure out the problem before converting everything over to 32-bit
where things end up running more smoothly. Because it isn't an obvious
problem, and because it seems to manifest itself in different ways, it
takes a while to realize each time.
I was unable to run migrations after following the setup instructions on a
fresh installation of Windows 10 and choosing the 64-bit versions of
everything. I started over, this time choosing the 32-bit installations of
python and OSGeo4Win, and everything worked smoothly.
This is not the first time I've had this experience. It would really be
helpful as a novice programmer to at least be advised that the 64-bit
version of Python may be problematic and will likely require steps that
are outside the scope of the installation instructions if the user is to
ultimately get it working.
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Adam Starrh | Owner: nobody
Cleanup/optimization |
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Accepted
installation, windows |
Has patch: 0 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Nick Pope):
* keywords: => geodjango, installation, windows
* needs_docs: 0 => 1
* stage: Unreviewed => Accepted
Old description:
> I'm setting GeoDjango up on Windows and discovered that GDAL is (at least
> currently) incompatible with 64-bit Python. Using 32-bit Python and
> 32-bit OSGeo4Win, I was finally able to get it working. I suggest this be
> mentioned in the docs to circumvent future headaches for other users.
>
> Pull request:
> https://github.com/django/django/pull/11995
New description:
~~I'm setting GeoDjango up on Windows and discovered that GDAL is (at
least currently) incompatible with 64-bit Python. Using 32-bit Python and
32-bit OSGeo4Win, I was finally able to get it working. I suggest this be
mentioned in the docs to circumvent future headaches for other users.~~
~~Pull request:
https://github.com/django/django/pull/11995~~
----
The installation instructions for dependencies of !GeoDjango on Windows
are outdated.
Issues that need to be addressed include:
- It seems that there may be some as yet undetermined issue wit 64-bit
setups.
- Not referencing only 9.x versions of PostgreSQL which are > 3 years old.
- Consider referring to other geospatial database backends. Why should
Windows users be forced to use PostGIS?
- Replace outdated instructions for installation of psycopg2. We have
binary wheels now that are more up to date.
- Using !OSGeo4W can lead to a newer version of GEOS than is supported by
Django with no option to downgrade.
- Consider installing individual dependencies for GDAL, GEOS & Proj.
--
Comment:
So as I said on the PR, I do not feel that we should be encouraging users
to install 32-bit. It seems like a backward step and I would rather we
address making this work for 64-bit.
(I will admit that I don't use Windows, but I will endeavour to help where
I can.)
So my understanding is that the documentation for installing dependencies
for !GeoDjango on Windows is at
https://docs.djangoproject.com/en/stable/ref/contrib/gis/install/#windows.
The first requirement there is to install Python on Windows by following
https://docs.djangoproject.com/en/stable/howto/windows/. Django 2.2
doesn't currently support the recently release 3.8, so let's go with 3.7.5
available from https://www.python.org/downloads/release/python-375/.
I note that the next steps suggests is that Windows users ought to be
using PostgreSQL & PostGIS. This seems a bit restrictive as I'm sure the
other geospatial database backends will work. But let's run with it. ''(I
note that the documentation here ought to be updated to drop references to
9.x - version 10, 11 and 12 are available, so 9.6 is >3 years old.)''
Django 2.2 only supports up to 11, so let's go with 11.5, available from
https://www.enterprisedb.com/downloads/postgres-postgresql-downloads. I
note that EnterpriseDB have stopped providing 32-bit builds for Windows
for PostgreSQL >= 11. I will assume at this stage that the instructions
for PostGIS are the same or similar to those described.
According to the documentation, we will also need the latest version of
psycopg2 from http://www.stickpeople.com/projects/python/win-psycopg/.
Unfortunately that page no longer seems to work for me and times out. The
Internet Archive Wayback Machine gives
[http://web.archive.org/web/20190326183249/http://www.stickpeople.com/projects/python
/win-psycopg/index.html this version] of the page, but the latest version
there is only built for Python 3.6 and is psycopg2 2.6.2 when 2.8.4 is
available. ''I would suggest that this documentation is updated to point
to https://www.lfd.uci.edu/~gohlke/pythonlibs/#psycopg or, better yet,
recommend installing [https://pypi.org/project/psycopg2-binary/#files
psycopg2-binary from PyPI] which wasn't available when this was written."
Next we come to !OSGeo4W. The instructions point us to
https://trac.osgeo.org/osgeo4w/ and using the 64-bit link we get an
installer. I note the following restrictions under the "Notes" heading:
- "32bit/64bit don't contain the same set of packages/versions (compare)
(purple is 32 bit only)"
- "Don't blindly do a full install in Advanced mode. Some components
require third party additions that need to be installed manually (see the
wiki page of the package). Without the additions the components might
renderer the installation unuseable."
So after all of that I would expect that you are using the following
installers:
- https://www.python.org/ftp/python/3.7.5/python-3.7.5-amd64.exe
-
https://get.enterprisedb.com/postgresql/postgresql-11.5-2-windows-x64.exe
-
https://files.pythonhosted.org/packages/e7/0c/aca85d48bd72557251ee87ac5b8566ad17eb197725247357d90d0891216e/psycopg2_binary-2.8.4-cp37
-cp37m-win_amd64.whl
- http://download.osgeo.org/osgeo4w/osgeo4w-setup-x86_64.exe
For !OSGeo4W I presume that you are using "Express Install" as instructed
and not "Advanced mode". I don't know if that could cause any problems.
There is also a problem that you have stumbled into that !OSGeo4W installs
a newer version of GDAL than Django 2.2 officially supports. This does
make me wonder whether we'd be better off documenting installation of the
actual dependencies of !GeoDjango - that is GDAL, GEOS and Proj - although
some of those just point back to !OSGeo4W :(
We may also be bumping into issues with some packages compiled with MSVC
and others with MinGW64. Do you have a specific error message or
traceback, by the way?
Based on the above, I've updated this ticket to reflect the need for a
wider review of the installation of dependencies for GeoDjango on Windows.
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:2>
Django
-------------------------------------+-------------------------------------
Cleanup/optimization |
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Accepted
installation, windows |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by felixxm):
Nick, I still don't see where is an issue, except that
`http://www.stickpeople.com/projects/python/win-psycopg/` link is
outdated.
There is also a problem that you have stumbled into that OSGeo4W installs
a newer version of GDAL than Django 2.2 officially supports
I don't think that is true, `OSGeo4W` contains: `gdal200dll`,
`gdal201dll`, `gdal202dll`, `gdal203dll` all of them are supported by
Django 2.2.
It can be hard to fix an issue without details.
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:3>
Django
-------------------------------------+-------------------------------------
Cleanup/optimization |
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Accepted
installation, windows |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson):
* cc: Carlton Gibson (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:4>
Django
-------------------------------------+-------------------------------------
Cleanup/optimization |
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Accepted
installation, windows |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Old description:
> ~~I'm setting GeoDjango up on Windows and discovered that GDAL is (at
> least currently) incompatible with 64-bit Python. Using 32-bit Python and
> 32-bit OSGeo4Win, I was finally able to get it working. I suggest this be
> mentioned in the docs to circumvent future headaches for other users.~~
>
> ~~Pull request:
> https://github.com/django/django/pull/11995~~
>
> ----
>
> The installation instructions for dependencies of !GeoDjango on Windows
> are outdated.
>
> Issues that need to be addressed include:
>
> - It seems that there may be some as yet undetermined issue wit 64-bit
> setups.
> - Not referencing only 9.x versions of PostgreSQL which are > 3 years
> old.
> - Consider referring to other geospatial database backends. Why should
> Windows users be forced to use PostGIS?
> - Replace outdated instructions for installation of psycopg2. We have
> binary wheels now that are more up to date.
> - Using !OSGeo4W can lead to a newer version of GEOS than is supported by
> Django with no option to downgrade.
> - Consider installing individual dependencies for GDAL, GEOS & Proj.
New description:
~~I'm setting GeoDjango up on Windows and discovered that GDAL is (at
least currently) incompatible with 64-bit Python. Using 32-bit Python and
32-bit OSGeo4Win, I was finally able to get it working. I suggest this be
mentioned in the docs to circumvent future headaches for other users.~~
~~Pull request:
https://github.com/django/django/pull/11995~~
----
The installation instructions for dependencies of !GeoDjango on Windows
are outdated.
Issues that need to be addressed include:
- It seems that there may be some as yet undetermined issue wit 64-bit
setups.
- Not referencing only 9.x versions of PostgreSQL which are > 3 years old.
- Consider referring to other geospatial database backends. Why should
Windows users be forced to use PostGIS?
- Replace outdated instructions for installation of psycopg2. We have
binary wheels now that are more up to date.
- ~~Using OSGeo4W can lead to a newer version of GEOS than is supported by
Django with no option to downgrade.~~
- ~~Consider installing individual dependencies for GDAL, GEOS & Proj.~~
--
Comment (by Nick Pope):
Replying to [comment:3 felixxm]:
> Nick, I still don't see where is an issue, except that
`http://www.stickpeople.com/projects/python/win-psycopg/` link is
outdated.
That and referring to PostgreSQL 9.x seems overly restrictive. There
haven't been a great many updates to this section of the documentation
other than cosmetic in years and my impression from my comment above is
that it could be made easier.
>> There is also a problem that you have stumbled into that OSGeo4W
installs a newer version of GDAL than Django 2.2 officially supports
> I don't think that is true, `OSGeo4W` contains: `gdal200dll`,
`gdal201dll`, `gdal202dll`, `gdal203dll` all of them are supported by
Django 2.2.
I was alluding to the comments starting at
https://github.com/django/django/pull/10962#issuecomment-548004287, but if
it installs the older versions too, then ...
> It can be hard to fix an issue without details.
Indeed. I asked for some more details at the end...
Anyway. As I said, I don't use Windows. And I feel like I'm being
unhelpful here, so I'll back off.
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:5>
Django
-------------------------------------+-------------------------------------
Cleanup/optimization |
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Accepted
installation, windows |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by felixxm):
Replying to [comment:5 Nick Pope]:
> Anyway. As I said, I don't use Windows. ...
Me neither, but it seems to me that without details we'll not be able to
fix this ticket.
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:6>
Django
-------------------------------------+-------------------------------------
Cleanup/optimization |
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Accepted
installation, windows |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Carlton Gibson):
OK, I will have a look at this, and see if I can step through it to find
improvements. (I'll leave it unassigned in case anyone else wants to do
that too.)
> I would rather we address making this work for 64-bit.
Absolutely. We're just bottling up issues for later otherwise.
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:7>
Django
-------------------------------------+-------------------------------------
Cleanup/optimization |
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Accepted
installation, windows |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Adam Starrh):
Hello,
When following the instructions in the documentation, only the latest
version of the gdal dll is placed in the /bin/ directory via the OSGeo4Win
installer. There are no older versions available.
I don't have immediate access to the traces but the general result was
that the utility that searches for the .dll was unable to find it, even
when it was present. I should note that I identified two separate issues
while setting this up.
The first was a repeat of an issue I encountered earlier:
https://code.djangoproject.com/ticket/28237
In this case, `gdal204.dll` was missing from Django's `libgdal.py` file
and I had to enter it manually. However, this time, it continued to return
the same GDALexception as before,
which led me to the second issue. I tried setting the GDAL_LIBRARY_PATH
manually in the settings and when attempting to run the migrations, it
reported: "this is not a valid win32 application". This tipped me off to
the issue and I rolled python and OSGeo4Win back to the 32-bit versions,
and that resolved it. This was the 4th or so time over the years that I
have chosen 64-bit python in the installation process and had to go back
and convert to the 32-bit. This is why I am suggesting an amendment to the
instructions at this step in the process, as it creates hours of headaches
each time.
Aside from these issues with GDAL, the instructions as-written work pretty
well for getting casual coders like myself up and running quickly. The
techs all work together nicely and having the process simple and
straightforward is part of what I believe makes Django so accessible. The
OSGeo4Win is certainly a convenient package that enables users to skip a
lot of steps installing individual dependencies that they would have to
endure otherwise. If a full revision is necessary, I'd hope that this is
kept in mind.
64-bit Python isn't necessary for most users and creates a lot of these
kinds of problems, so much so that Windows users seem to typically avoid
using it. If someone has a good reason to opt for 64-bit Python, then
hopefully they are prepared to handle these kinds of issues independently.
The same goes for anyone with a preference toward another geospatial db
library. Having more options included in the instructions makes the
process more confusing for new users and more complicated to support.
The only other real trip-up is the outdated link. At one time psycopg2
wouldn't install via pip, and this was the preferred workaround. That is
no longer an issue as the package will successfully install via pip, so
long as the user has Visual Studio C++ installed (which should probably be
mentioned).
I am away from the system in question until early next week. I can post
the full tracebacks at that time and can also try the alternate gdal
version suggested in the PR if it would still be helpful.
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:8>
Django
-------------------------------------+-------------------------------------
Type: | Smith
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Accepted
installation, windows |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by David Smith):
* owner: nobody => David Smith
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Adam Starrh | Owner: David
Type: | Smith
Cleanup/optimization | Status: assigned
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Ready for
installation, windows | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: Adam Starrh | Owner: David
Type: | Smith
Cleanup/optimization | Status: assigned
Severity: Normal | Resolution:
Keywords: geodjango, | Triage Stage: Ready for
installation, windows | checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Carlton Gibson):
This will be fixed for Django 3.1, which has compatibility with GDAL 3 and
3.1 — These are automatically installed by the latest installers.
From [https://github.com/django/django/pull/13045#issue-432688043 the PR
description]:
> One of the trip-ups is that OSGeo4W installs a specific version of GDAL
(currently 3.0), unless we go the 'advanced' route. Therefore to support
older versions (e.g. Django 3.0) we would need to explore this. I've
included a screenshot of what we would need to navigate (this screen is
about half a dozen "next"s in to the install process)
Given the imminence of Django 3.1, the difficulty of _describing_ the
advanced steps, and the time this has been outstanding, we'll not try to
resolve the issue for 3.0 and below with GDAL 2.x. Anyone hitting this
will need to explore the advanced install if they're not able to update to
Django 3.1.
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: Adam Starrh | Owner: David
Type: | Smith
Component: Documentation | Version: 2.2
Severity: Normal | Resolution: fixed
Keywords: geodjango, | Triage Stage: Ready for
installation, windows | checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson <carlton@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"7f4db2d82b1385d1026c441e25abd4cb19fcccc6" 7f4db2d8]:
{{{
#!CommitTicketReference repository=""
revision="7f4db2d82b1385d1026c441e25abd4cb19fcccc6"
Fixed #30933 -- Updated instructions for installing GeoDjango on Windows.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: Adam Starrh | Owner: David
Type: | Smith
Cleanup/optimization | Status: closed
Severity: Normal | Resolution: fixed
Keywords: geodjango, | Triage Stage: Ready for
installation, windows | checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Carlton Gibson <carlton.gibson@…>):
In [changeset:"928f6a90e404cc2433399e84ba73d83ace5eac2f" 928f6a90]:
{{{
#!CommitTicketReference repository=""
revision="928f6a90e404cc2433399e84ba73d83ace5eac2f"
[3.1.x] Fixed #30933 -- Updated instructions for installing GeoDjango on
Windows.
Backport of 7f4db2d82b1385d1026c441e25abd4cb19fcccc6 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30933#comment:14>