[Distutils] .exe for script in Windows doesn't work

[cool-RR]

Hi guys,

I made a release of my project today, and now I see there's something wrong with it.

When you install it on Windows using the windows binaries (that I made with `bdist_msi` under Python 2.7), the .exe scripts in the "Scripts" folder don't work. When you launch them, nothing happens. You can launch the associated *-script.pyw files and those *do* work.

I investigated a little and possibly I figured it out. The shebang line says "#!c:\Python27\pythonw.exe". This is the interpreter with which I *created* the binaries. But why should it be the interpreter with which the *user* works? The user could have "c:\Python26\pythonw.exe" as his interpreter or "c:\Programs\Python25\pythonw.exe" or "g:\Pypy\pypy.exe" as his interpreter.

I tried editing the shebang line with notepad, and this caused the exe files to start working! (Of course this is not a real solution, I can't ask my users to do that.)

Is this a bug in distribute?

I'd appreciate if you can reply soon since I already started the release process and I don't want to confuse users.

Thanks,

Ram.

[P.J. Eby]

At 09:31 PM 1/12/2011 +0200, cool-RR wrote:
>Hi guys,
>
>I made a release of my project today, and now I see there's
>something wrong with it.
>
>When you install it on Windows using the windows binaries (that I
>made with `bdist_msi` under Python 2.7), the .exe scripts in the
>"Scripts" folder don't work. When you launch them, nothing happens.
>You can launch the associated *-script.pyw files and those *do* work.
>
>I investigated a little and possibly I figured it out. The shebang
>line says "#!c:\Python27\pythonw.exe". This is the interpreter with
>which I *created* the binaries. But why should it be the interpreter
>with which the *user* works? The user could have
>"c:\Python26\pythonw.exe" as his interpreter or
>"c:\Programs\Python25\pythonw.exe" or "g:\Pypy\pypy.exe" as his interpreter.
>
>I tried editing the shebang line with notepad, and this caused the
>exe files to start working! (Of course this is not a real solution,
>I can't ask my users to do that.)
>
>Is this a bug in distribute?

It's a limitation of setuptools+bdist_msi - when you build a
bdist_wininst, setuptools writes a generic #! line, but when you
build a bdist_msi, it doesn't realize you're building a distribution.

>I'd appreciate if you can reply soon since I already started the
>release process and I don't want to confuse users.

As a quick workaround, you could try setting sys.executable to
'pythonw.exe' in your setup.py, *before* setuptools is imported, then
build the .msi.

_______________________________________________
Distutils-SIG maillist - Distut...@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[cool-RR]

On Thu, Jan 13, 2011 at 2:25 AM, P.J. Eby <p...@telecommunity.com> wrote:

At 09:31 PM 1/12/2011 +0200, cool-RR wrote:

Hi guys,

I made a release of my project today, and now I see there's something wrong with it.

When you install it on Windows using the windows binaries (that I made with `bdist_msi` under Python 2.7), the .exe scripts in the "Scripts" folder don't work. When you launch them, nothing happens. You can launch the associated *-script.pyw files and those *do* work.

I investigated a little and possibly I figured it out. The shebang line says "#!c:\Python27\pythonw.exe". This is the interpreter with which I *created* the binaries. But why should it be the interpreter with which the *user* works? The user could have "c:\Python26\pythonw.exe" as his interpreter or "c:\Programs\Python25\pythonw.exe" or "g:\Pypy\pypy.exe" as his interpreter.

I tried editing the shebang line with notepad, and this caused the exe files to start working! (Of course this is not a real solution, I can't ask my users to do that.)

Is this a bug in distribute?

It's a limitation of setuptools+bdist_msi - when you build a bdist_wininst, setuptools writes a generic #! line, but when you build a bdist_msi, it doesn't realize you're building a distribution.

So which project exactly is responsible for this limitation? Distribute?

Do you think there's a good reason not to have the MSI put the path to python.exe itself into the shebang upon installation? 

 

I'd appreciate if you can reply soon since I already started the release process and I don't want to confuse users.

As a quick workaround, you could try setting sys.executable to 'pythonw.exe' in your setup.py, *before* setuptools is imported, then build the .msi.

I'd be reluctant to do that, since many users don't have the Python folder on their PATH, so it won't work for them at all.

Ram.

[P.J. Eby]

Setuptools.

>Do you think there's a good reason not to have the MSI put the path
>to python.exe itself into the shebang upon installation?

I don't know how to have bdist_msi do that. Do you?

>As a quick workaround, you could try setting sys.executable to
>'pythonw.exe' in your setup.py, *before* setuptools is imported,
>then build the .msi.
>
>
>I'd be reluctant to do that, since many users don't have the Python
>folder on their PATH, so it won't work for them at all.

That's not how it works. When the #! line doesn't contain an
absolute path, the .exe stub looks in the parent directory relative
to where the script was installed. So, as long as the .exe and .pyw
are in an immediate subdirectory of the Python directory (usually
Scripts), it will work correctly.

[cool-RR]

On Thu, Jan 13, 2011 at 5:11 PM, P.J. Eby <p...@telecommunity.com> wrote:

At 09:54 AM 1/13/2011 +0200, cool-RR wrote:

On Thu, Jan 13, 2011 at 2:25 AM, P.J. Eby <<mailto:p...@telecommunity.com>p...@telecommunity.com> wrote:
At 09:31 PM 1/12/2011 +0200, cool-RR wrote:
Hi guys,

I made a release of my project today, and now I see there's something wrong with it.

When you install it on Windows using the windows binaries (that I made with `bdist_msi` under Python 2.7), the .exe scripts in the "Scripts" folder don't work. When you launch them, nothing happens. You can launch the associated *-script.pyw files and those *do* work.

I investigated a little and possibly I figured it out. The shebang line says "#!c:\Python27\pythonw.exe". This is the interpreter with which I *created* the binaries. But why should it be the interpreter with which the *user* works? The user could have "c:\Python26\pythonw.exe" as his interpreter or "c:\Programs\Python25\pythonw.exe" or "g:\Pypy\pypy.exe" as his interpreter.

I tried editing the shebang line with notepad, and this caused the exe files to start working! (Of course this is not a real solution, I can't ask my users to do that.)

Is this a bug in distribute?


It's a limitation of setuptools+bdist_msi - when you build a bdist_wininst, setuptools writes a generic #! line, but when you build a bdist_msi, it doesn't realize you're building a distribution.


So which project exactly is responsible for this limitation? Distribute?

Setuptools.

I don't understand; Isn't this one of the things that `distribute` has control of? I mean, I generate the MSI in a computer that has distribute installed. (IIUC, when I import `setuptools` on this computer, it uses the distribute code.) Am I wrong in one of my assumptions.

Do you think there's a good reason not to have the MSI put the path to python.exe itself into the shebang upon installation?

I don't know how to have bdist_msi do that.  Do you?

No, I have no idea how MSIs are generated at all. I didn't ask if you specifically know how to do it, just if you think there's a good reason not to put this feature. (e.g., it will cause a problem in some cases that I didn't think of.)

Who is maintaining bdist_msi?

 

As a quick workaround, you could try setting sys.executable to 'pythonw.exe' in your setup.py, *before* setuptools is imported, then build the .msi.


I'd be reluctant to do that, since many users don't have the Python folder on their PATH, so it won't work for them at all.

That's not how it works.  When the #! line doesn't contain an absolute path, the .exe stub looks in the parent directory relative to where the script was installed.  So, as long as the .exe and .pyw are in an immediate subdirectory of the Python directory (usually Scripts), it will work correctly.

Ah, not bad. Thanks.

Ram. 

["Martin v. Löwis"]

> Who is maintaining bdist_msi?

I am. Unfortunately, I don't even understand the problem you are talking
about, let alone being able to do anything about it.

Regards,
Martin

[cool-RR]

On Thu, Jan 13, 2011 at 11:26 PM, "Martin v. Löwis" <mar...@v.loewis.de> wrote:

> Who is maintaining bdist_msi?

I am. Unfortunately, I don't even understand the problem you are talking
about, let alone being able to do anything about it.

Regards,
Martin

I'll explain. (Please correct me if I'm making any mistakes here.)

When you install a Python package in Windows, an .exe file is generated for every script that your package defines. This .exe file sits alongside the .py scripts in the PythonXX\Scripts folder. The .exe file launches the Python interpreter with the Python script. The .exe file needs to know where your Python interpreter is, and it uses the shebang line on the script file to do it, like this: "#!c:\Python27\pythonw.exe"

Problem is, that instead of specifying the interpreter of the system on which you're installing, it's using the interpreter of the system which generated the bdist_msi. So since I generated MSI with Python 2.7, the shebang line showed a path to a Python 2.7 installation. This is a problem since the user might have Python 2.6 installed, or he can have Python installed in a non-default path, etc.

What seems right to me is that the MSI installer will figure out where the Python interpreter is on the end-user's system and then use that in the shebang line.

What do you think?

Ram.

["Martin v. Löwis"]

> I am. Unfortunately, I don't even understand the problem you are talking
> about, let alone being able to do anything about it.
>
> Regards,
> Martin
>
>
> I'll explain. (Please correct me if I'm making any mistakes here.)
>
> When you install a Python package in Windows, an .exe file is generated
> for every script that your package defines.

How is that generated (i.e. what piece of code specifically generates it)?

> What seems right to me is that the MSI installer will figure out where
> the Python interpreter is on the end-user's system and then use that in
> the shebang line.
>
> What do you think?

That will be difficult. It will require a custom action to be generated
when the MSI is produced, and that custom action must be scheduled into
the InstallExecute sequence.

[P.J. Eby]

At 10:45 PM 1/13/2011 +0100, Martin v. Löwis wrote:
>How is that generated (i.e. what piece of code specifically generates it)?

See setuptools.command.install_scripts.run() - it makes .exe and
-script.py/-script.pyw files. It has some hackery to detect
bdist_wininst and use a relative #! line in that case, but it doesn't
have equivalent hackery for bdist_msi.

[cool-RR]

On Fri, Jan 14, 2011 at 3:01 AM, P.J. Eby <p...@telecommunity.com> wrote:

At 10:45 PM 1/13/2011 +0100, Martin v. Löwis wrote:

How is that generated (i.e. what piece of code specifically generates it)?

See setuptools.command.install_scripts.run() - it makes .exe and -script.py/-script.pyw files.  It has some hackery to detect bdist_wininst and use a relative #! line in that case, but it doesn't have equivalent hackery for bdist_msi.

P.J., do you think it will suffice to patch Distribute here:

https://bitbucket.org/tarek/distribute/src/4ab9b96dc540/setuptools/command/install_scripts.py#cl-34

To detect bdist_msi as well?

Ram.

[P.J. Eby]

At 03:47 PM 2/14/2011 +0200, cool-RR wrote:
>On Fri, Jan 14, 2011 at 3:01 AM, P.J. Eby
><<mailto:p...@telecommunity.com>p...@telecommunity.com> wrote:
>At 10:45 PM 1/13/2011 +0100, Martin v. Löwis wrote:
>How is that generated (i.e. what piece of code specifically generates it)?
>
>
>See setuptools.command.install_scripts.run() - it makes .exe and

>-<http://script.py/-script.pyw>script.py/-script.pyw files. It has

>some hackery to detect bdist_wininst and use a relative #! line in
>that case, but it doesn't have equivalent hackery for bdist_msi.
>
>
>P.J., do you think it will suffice to patch Distribute here:
>

><https://bitbucket.org/tarek/distribute/src/4ab9b96dc540/setuptools/command/install_scripts.py#cl-34>https://bitbucket.org/tarek/distribute/src/4ab9b96dc540/setuptools/command/install_scripts.py#cl-34

>
>To detect bdist_msi as well?

No. If it were that simple, I'd already have done it.

[cool-RR]

On Mon, Feb 14, 2011 at 5:32 PM, P.J. Eby <p...@telecommunity.com> wrote:

At 03:47 PM 2/14/2011 +0200, cool-RR wrote:

On Fri, Jan 14, 2011 at 3:01 AM, P.J. Eby <<mailto:p...@telecommunity.com>p...@telecommunity.com> wrote:
At 10:45 PM 1/13/2011 +0100, Martin v. Löwis wrote:
How is that generated (i.e. what piece of code specifically generates it)?

See setuptools.command.install_scripts.run() - it makes .exe and -<http://script.py/-script.pyw>script.py/-script.pyw files.  It has some hackery to detect bdist_wininst and use a relative #! line in that case, but it doesn't have equivalent hackery for bdist_msi.

P.J., do you think it will suffice to patch Distribute here:

<https://bitbucket.org/tarek/distribute/src/4ab9b96dc540/setuptools/command/install_scripts.py#cl-34>https://bitbucket.org/tarek/distribute/src/4ab9b96dc540/setuptools/command/install_scripts.py#cl-34

To detect bdist_msi as well?

No.  If it were that simple, I'd already have done it.

So... Should I open a ticket on the Distribute project?

Ram.

[P.J. Eby]

At 06:49 PM 2/14/2011 +0200, cool-RR wrote:
>So... Should I open a ticket on the Distribute project?

If you like. But http://bugs.python.org/setuptools/issue112 already
exists, and I believe the Distribute project still shadows
setuptools' bug tracker. (I could be wrong about that.)