Using setuptools to make django-admin.py runnable on Windows (#21340)

[Rémi Rampin]

Hi developers,

On Windows, running the django-admin.py tool is painful[1], because
.py scripts are not "executable". You might be able to run it using
the full path (if Python is the default handler for .py files, which
it really shouldn't be). Most probably you'll need to copy it to your
project directory and prefix it with "python " each time.

setuptools has a neat workaround for scripts on Windows, which works
great: it creates a wrapper binary that it puts on the PATH. I know
from previous threads that Django chose to move away from setuptools
and back to distutils, however it is easy to conditionally use this
feature if setuptools are available. That way, Windows developers that
have setuptools installed (which should be, like, all of them) will be
able to run django-admin.py easily.

I submitted a pull request on Github[2] a month ago, then opened a
ticket[3] later on. Apart from an uncommented update from akaariai, I
didn't get any feedback.

I'd really like to see this small change accepted. It is fairly minor
but would work towards restoring the portability that is a strength or
Python.

Thanks for your input/reviews/time,
-- Remram

[1] http://stackoverflow.com/q/19593404/711380
[2] https://github.com/django/django/pull/1812
[3] https://code.djangoproject.com/ticket/21340

TL;DR: what happened to my patch?

[Florian Apolloner]

Hi,

I am pretty much against setuptools and given that pip is somewhat becoming the defacto-standard to install stuff; I'd ask Donald what can be done here (cc'ed him). I don't think it's a good idea to fix this in Django since this is imo a problem in Python itself.

Regards,
Florian

[Donald Stufft]

The recommended build tool at the moment is setuptools.

It's up to the individual project to decide if they think the install story for setutpools pre 3.4 is appropriate for them. This'll get better in general in the future with MSI installers for setuptools and pip

[Waylan Limberg]

On Sunday, November 24, 2013, Rémi Rampin wrote:

Hi developers,

On Windows, running the django-admin.py tool is painful[1], because
.py scripts are not "executable". You might be able to run it using
the full path (if Python is the default handler for .py files, which
it really shouldn't be). Most probably you'll need to copy it to your
project directory and prefix it with "python " each time.

There are a few things I do whenever I install python on windows.

1) first, make sure the .py extension is associated with python.exe.

2) add the python directory to my path.

3) add the scripts directory to my path.

Then, when any packages install scripts (like django-admin.py), they ‘just work’.

If you have multiple versions on python installed, then you can copy python.exe to pythonX.X.exe and make sure the defult version (the version associated with .py) is listed first on your path. The order of dirs on your path in key here, but you can list all the versions on your path. Then you can just type `python3.3` for example, and you'll get that version just like on unix systems.

Of course, on unix based systems we all use virtualenv. There are a few windows virtualenv clones implemented for windows either as batch scripts or powershell scripts. You might want to look into those. Although, I haven't used them myself, so I can't speak to their completeness, reliability or usefulness. I always have git installed which comes with mssys and gives me a bash shell on windows - the best solution IMO (although virtualenv can be a little janky).

Of course, this has entered django-users territory. And additional discussion about how to work around windows limitations should probably be discussed there. My point is that with a correctly configured system, this is not a problem on windows at all.

If there are any changes to make to Django, perhaps the install docs could suggest the additions to the path I mention about as specific to installing on windows.

--
Waylan Limberg

[VernonCole]

Waylan is correct, but does not go far enough.

When one installs Python 3.3, the Python Launcher for Windows is also installed. I install Python 3 even on systems where I do not (at present) plan to use it, just to get the launcher. The launcher is the program which should be associated with .py files on Windows. It is responsible for reading the #! line at the top of your Python file and running the correct version of Python. The launcher works so well at handling different versions that I keep wanting to install it on my Linux boxes as well.

So, I would agree that the Windows installation instructions might indeed be altered -- to include installing Python 3.3 (or later), and associating the c:\windows\py.exe with .py files.

If you plan to use only a single version of Python, putting that directory (and its "scripts" directory) in your PATH is a good idea. But, I feel that virtualenvwrapper-win is the best way to go.  It works almost exactly like virtualenvwrapper for Linux, and if used with PyCharm 3.0 one can almost forget which OS one is using.

I am going to investigate using the launcher to pick the correct virtualenv.  Meanwhile -1 on the patch.

a

[Remram]

November 24 14:37, Florian Apolloner

I am pretty much against setuptools and given that pip is somewhat becoming the defacto-standard to install stuff

I completely agree to this. And setuptools is an important part of pip... In fact, pip has a strong dependency on setuptools, and automagically translates calls to distutils to setuptools to get stuff to work. [1]

November 24 14:55, Donald Stufft

The recommended build tool at the moment is setuptools.

I agree with this guy! This is very much true outside of Django. See [2] for instance...

November 24 18:58, waylan

Of course, on unix based systems we all use virtualenv. There are a few windows virtualenv clones implemented for windows either as batch scripts or powershell scripts. You might want to look into those.

Virtualenv, as in the official tool distributed on virtualenv.org, works completely fine on Windows. And of course we use it, in pretty much the same way we do on UNIX-like systems, with Django as well as anything else. Except that, while other packages work, the django-admin.py tool doesn't.

It seems to me that you are misrepresenting the current state of Python's packaging libraries and installation tools, and not only on Windows (see my first point). I understand that, for a project as big and important as Django, changes in packaging can't be done lightly, especially when they only seem to benefit these annoying Windows users; however I really can't find sense in the arguments you are presenting here. Of course, you'll do as you please, and as you seem to think this discussion has nothing to do on django-developers I'll go away (but I'll keep using my patch).

Thanks again for your continued work

Remram

[1] http://www.pip-installer.org/en/latest/logic.html#setuptools-pkg-resources

[2] https://python-packaging-user-guide.readthedocs.org/en/latest/current.html#installation-tool-recommendations

PS: I apologize for the delayed response as I was travelling

[Florian Apolloner]

Hi Remram,

On Wednesday, December 4, 2013 4:56:55 PM UTC+1, Remram wrote:

November 24 14:37, Florian Apolloner

I am pretty much against setuptools and given that pip is somewhat becoming the defacto-standard to install stuff

I completely agree to this. And setuptools is an important part of pip... In fact, pip has a strong dependency on setuptools, and automagically translates calls to distutils to setuptools to get stuff to work. [1]

To my understanding of https://github.com/pypa/pip/blob/develop/pip/req.py#L633 pip will use setuptools for installing -- so why do you need setuptools in Django itself? Also (refering to your first post), why is associating stuff with .py files a bad idea?

Cheers,
Florian

[Remram]

December 4 12:43, Florian Apolloner

To my understanding of https://github.com/pypa/pip/blob/develop/pip/req.py#L633 pip will use setuptools for installing -- so why do you need setuptools in Django itself?

Hi Florian,

The point here is to use options that distutils doesn't offer, like entry_points, if they are available.

Also (refering to your first post), why is associating stuff with .py files a bad idea?

I personally have .py files associated with my text editor. Defaulting them to the system-wide Python 2.7 might do the correct thing in some cases...

Associating .py files with Python is not enough -- only binaries are searched for in the PATH when typing a path-less command name (.com, .exe, .bat and a few others). Doing this, then installing the wrapper from Python 3, then changing the PATHEXT, and adding the 'Scripts' directories to the PATH might work, but can this really be the recommended way (just for Django)?

-- 

Remram

[Florian Apolloner]

On Wednesday, December 4, 2013 8:24:49 PM UTC+1, Remram wrote:

December 4 12:43, Florian Apolloner

To my understanding of https://github.com/pypa/pip/blob/develop/pip/req.py#L633 pip will use setuptools for installing -- so why do you need setuptools in Django itself?

The point here is to use options that distutils doesn't offer, like entry_points, if they are available.

Ah, I assumed that the workaround you mentioned would be setuptools just wrapping everything in 'scripts' -- didn't know that you'll have to use 'entry_points' then.
 

Also (refering to your first post), why is associating stuff with .py files a bad idea?

I personally have .py files associated with my text editor. Defaulting them to the system-wide Python 2.7 might do the correct thing in some cases...

Oh, that makes sense I guess :)

Associating .py files with Python is not enough -- only binaries are searched for in the PATH when typing a path-less command name (.com, .exe, .bat and a few others).

 

Okay, that's just horrible!

Which downsides does using 'entry_points' have for a *nix based system? I'd rather not support disutils and setuptools; this just calls out for pain. My last worry (probably needlessly) is that we might run into issues with one of the hundred setuptools versions out there; all together there is distribute, old setuptools, new setuptools and whatever distris like debian packaged currently -- I don't think most users upgrade setuptools that often, so are there any issues to be expected with stoneold setuptools? [Granted, that's not really our problem, but if we know of some issues we could warn the users]

Cheers,
Florian

[Florian Apolloner]

Oh, one last thing (a few actually):

• How does 'entry_points' work with wheels?
  
• Do we need an extra wheel for windows just to get the wrappers right?
  
• Can we generate a windows wheel from *nix, or do we need a windows box?

Cheers,
Florian

[Donald Stufft]

On Dec 4, 2013, at 4:31 PM, Florian Apolloner <f.apo...@gmail.com> wrote:

Oh, one last thing (a few actually):

• How does 'entry_points' work with wheels? 

• Do we need an extra wheel for windows just to get the wrappers right? 
  
• Can we generate a windows wheel from *nix, or do we need a windows box?

entry points are kinda wonky with pip 1.4, pip 1.5 makes them sane. You would not need a Windows specific Wheel with pip 1.5

Cheers,
Florian

On Wednesday, December 4, 2013 10:24:09 PM UTC+1, Florian Apolloner wrote:

On Wednesday, December 4, 2013 8:24:49 PM UTC+1, Remram wrote:

December 4 12:43, Florian Apolloner

To my understanding of https://github.com/pypa/pip/blob/develop/pip/req.py#L633 pip will use setuptools for installing -- so why do you need setuptools in Django itself?

The point here is to use options that distutils doesn't offer, like entry_points, if they are available.

Ah, I assumed that the workaround you mentioned would be setuptools just wrapping everything in 'scripts' -- didn't know that you'll have to use 'entry_points' then.
 

Also (refering to your first post), why is associating stuff with .py files a bad idea?

I personally have .py files associated with my text editor. Defaulting them to the system-wide Python 2.7 might do the correct thing in some cases...

Oh, that makes sense I guess :) 

Associating .py files with Python is not enough -- only binaries are searched for in the PATH when typing a path-less command name (.com, .exe, .bat and a few others).

 

Okay, that's just horrible!

Which downsides does using 'entry_points' have for a *nix based system? I'd rather not support disutils and setuptools; this just calls out for pain. My last worry (probably needlessly) is that we might run into issues with one of the hundred setuptools versions out there; all together there is distribute, old setuptools, new setuptools and whatever distris like debian packaged currently -- I don't think most users upgrade setuptools that often, so are there any issues to be expected with stoneold setuptools? [Granted, that's not really our problem, but if we know of some issues we could warn the users]

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f3195fb1-1505-41ce-b34a-0cf147f33e1f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

[Florian Apolloner]

On Wednesday, December 4, 2013 11:20:39 PM UTC+1, Donald Stufft wrote:

entry points are kinda wonky with pip 1.4, pip 1.5 makes them sane. You would not need a Windows specific Wheel with pip 1.5

Is there a test-pypi where I could upload Django packages to test this?

[Donald Stufft]

Yes, but it’s down right now :( 

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a0af4ac2-bff1-4b7f-af4a-31aa0dd9d120%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

[Florian Apolloner]

Just so we are all on the same page here (summarizing discussions from IRC etc):

 * We are not going to support setuptools and distutils, this makes the setupprocess difficult to debug and test imo.
 * Given Donald's "okay" we might switch to setuptools completely
 * There seems to be a bug in pip when installing a wheel, which renders the django-admin.exe unusable on windows, help welcome
 * PR for this issue is now at: https://github.com/django/django/pull/2116 [The original didn't work at all]

Cheers,

Florian

On Sunday, November 24, 2013 7:18:15 PM UTC+1, Remram wrote:

[Elena Williams]

Dear Florian,

Thanks so much for your update/feedback.

I updated the update here: https://github.com/elena/django-news-podcast/issues/1

Thanks again!

Elena