Shebang line on Windows?

[Walter Hurry]

I use FreeBSD or Linux, but my son is learning Python and is using
Windows.

My question is this: Would it be good practice for him to put #!/usr/bin/
env python at the top of his scripts, so that if made executable on *nix
they will be OK? As I understand it this will have no effect on Windows
itself.

[Dave Angel]

In Python 3.3 under Windows, the shebang line is useful. You run a
program called py, which examines the shebang, then loads the
appropriate version. I don't know much more, as I don't run Windows.

--
DaveA

[Zachary Ware]

Adding the shebang line on Windows would be excellent practice. In
fact, Python 3.3 and later ships with the Python Launcher for Windows
[1] which is very effective at reading the shebang line and executing
the script with the proper Python installation. It makes using Python
2.x and 3.x on the same Windows machine much less painful, as well as
not having to add anything to the PATH and being able to launch
whichever interpreter you want with a single command.

[1] http://docs.python.org/3/using/windows.html#python-launcher-for-windows

[James Harris]

On Feb 22, 6:40 pm, Zachary Ware <zachary.ware+pyl...@gmail.com>
wrote:

> On Fri, Feb 22, 2013 at 12:16 PM, Walter Hurry <walterhu...@lavabit.com> wrote:

> > I use FreeBSD or Linux, but my son is learning Python and is using
> > Windows.
>
> > My question is this: Would it be good practice for him to put #!/usr/bin/
> > env python at the top of his scripts, so that if made executable on *nix
> > they will be OK? As I understand it this will have no effect on Windows
> > itself.
>
> Adding the shebang line on Windows would be excellent practice.

A word of warning unless this has since been resolved: Whenever I have
tried adding the shebang line on Windows and running it on Unix the
latter has complained about the carriage return at the end of the
line. This means that Unix does not work when invoked as follows.
(And, yes, the file has had chmod +x applied.)

./program.py

It is, of course, OK when run as

python program.py

but that removes some of the benefit of the shebang line.

James

[MRAB]

Just use Unix line endings. Python will accept them, and any decent editor
on Windows will accept them. (Notepad doesn't.)

[Sells, Fred]

When moving from windows to unix you need to run "dos2unix" on any programs that use shebang (at least with python 2.6) that is installed on some platforms but must be installed on others like CentOs but it is in their repository.

--
http://mail.python.org/mailman/listinfo/python-list

[Dave Angel]

> -----Original Message-----
> From: Python-list [mailto:python-list-bounces+frsells=adventis...@python.org] On Behalf Of James Harris
> Sent: Friday, February 22, 2013 5:53 PM
> To: pytho...@python.org
> Subject: Re: Shebang line on Windows?
>

> On Feb 22, 6:40 pm, Zachary Ware <zachary.ware+pyl...@gmail.com>
>

> A word of warning unless this has since been resolved: Whenever I have tried adding the shebang line on Windows and running it on Unix the latter has complained about the carriage return at the end of the line. This means that Unix does not work when invoked as follows.
> (And, yes, the file has had chmod +x applied.)
>
> ./program.py
>
> It is, of course, OK when run as
>
> python program.py
>
> but that removes some of the benefit of the shebang line.
>
> James

> --
> http://mail.python.org/mailman/listinfo/python-list
>

(Fixing top-posted response, so it comes after the part it is quoting)

On 02/25/2013 07:35 AM, Sells, Fred wrote:
> When moving from windows to unix you need to run "dos2unix" on any
> programs that use shebang (at least with python 2.6) that is
> installed on some platforms but must be installed on others like
> CentOs but it is in their repository.
>

It's not Python that needs dos2unix, it's bash or equivalent. For some
reason, bash shebang processing still isn't tolerant of a trailing cr on
the line. Python doesn't care.

If someone is maintaining sources that need to run on both, it's easier
to maintain them using Unix-style newlines. All it usually requires is
a decent Windows text editor that honors the existing newline
convention. Or better that can be configured to always use simple
newlines at end of each line.


--
DaveA

[Chris Gonnerman]

On 02/25/2013 06:35 AM, Sells, Fred wrote:

When moving from windows to unix you need to run "dos2unix"   on any programs that use shebang (at least with python 2.6)   that is installed on some platforms but must be installed on others like CentOs but it is in their repository.

Or edit it in Vim and do

:se ff=unix

and then save it.

dos2unix is handy if you don't plan to edit the file for any other reason.  I'm assuming other editors provide similar features, but I've been a vi/vim user FOREVER.

Or, borrowed from a Stack Overflow thread here: http://stackoverflow.com/questions/800030/remove-carriage-return-in-unix

sed 's/\r\n$/\n/' mymodule.py > mymodule-unix.py

-----Original Message-----
From: Python-list [mailto:python-list-bounces+frsells=adventis...@python.org] On Behalf Of James Harris
Sent: Friday, February 22, 2013 5:53 PM
To: pytho...@python.org

Subject: [Python] Re: Shebang line on Windows?

On Feb 22, 6:40 pm, Zachary Ware <zachary.ware+pyl...@gmail.com>
wrote:

A word of warning unless this has since been resolved: Whenever I have tried adding the shebang line on Windows and running it on Unix the latter has complained about the carriage return at the end of the line. This means that Unix does not work when invoked as follows.
(And, yes, the file has had chmod +x applied.)

  ./program.py

It is, of course, OK when run as

  python program.py

but that removes some of the benefit of the shebang line.

James
--
http://mail.python.org/mailman/listinfo/python-list

[Chris Angelico]

On Tue, Feb 26, 2013 at 12:28 AM, Chris Gonnerman <ch...@gonnerman.org> wrote:
> On 02/25/2013 06:35 AM, Sells, Fred wrote:
>
>> When moving from windows to unix you need to run "dos2unix" on any
>> programs that use shebang (at least with python 2.6) that is installed on
>> some platforms but must be installed on others like CentOs but it is in
>> their repository.
>
> Or edit it in Vim and do
>
> :se ff=unix
>
> and then save it.

Or manage your files in git and set the core.autocrlf option to always
commit with Unix newlines (and you can optionally check files out with
DOS newlines, if you wish). Strongly recommended for cross-platform
work, as it "just happens" - no need to explicitly convert the file.

ChrisA

[Michael Torrie]

On 02/25/2013 06:14 AM, Dave Angel wrote:
> It's not Python that needs dos2unix, it's bash or equivalent. For some
> reason, bash shebang processing still isn't tolerant of a trailing cr on
> the line. Python doesn't care.

Actually, the shell isn't involved in parsing the shebang line at all.
That's actually done in the kernel by the program loader. So it's the
kernel that has a problem with it; wonder if Linus would accept a patch
to ignore the tailing CR?

[D'Arcy J.M. Cain]

So much the wrong solution. First of all, I don't think that Linus is
on the bash development team so he can't help there. Also, bash is not
the only shell in the world. And, Linux is not the only operating
system in the world. There are still a lot of Unix systems (the system
that Linux is a clone of) out there. FreeBSD, NetBSD, Solaris, Mac
OSX, etc. You can't expect all of them to bend over backwards for
every Windows wart out there.

I don't run Windows myself so I can't test it but doesn't Python on
Windows work fine with Unix style EOL? So why not strip out the CR and
run the same file everywhere?

--
D'Arcy J.M. Cain <da...@druid.net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.
IM: da...@Vex.Net, VOIP: sip:da...@Vex.Net

[David Robinow]

On Mon, Feb 25, 2013 at 12:29 PM, D'Arcy J.M. Cain <da...@druid.net> wrote:
> ...

> I don't run Windows myself so I can't test it but doesn't Python on
> Windows work fine with Unix style EOL? So why not strip out the CR and
> run the same file everywhere?

That's the ideal solution, but so many Windows tools default to text
mode that it's easy to create the wrong type file, especially for
beginners who are moving there code to Linux for the first time. I've
done it myself, even though I should no better by now.
In addition, the error message is rather obscure (if I remember right
- I'm not near my Ubuntu at the moment and Cygwin handles this fine,
as it should)
I'm afraid this problem won't go away. People are just going to have
to learn from experience.

[Michael Torrie]

On 02/25/2013 10:29 AM, D'Arcy J.M. Cain wrote:
> So much the wrong solution. First of all, I don't think that Linus is
> on the bash development team so he can't help there. Also, bash is not
> the only shell in the world.

Ooops you didn't read what I said. The shebang parsing is not done by
bash. It's done by the kernel. So it has nothing to do with bash or
any shell for that matter.

> And, Linux is not the only operating
> system in the world. There are still a lot of Unix systems (the system
> that Linux is a clone of) out there. FreeBSD, NetBSD, Solaris, Mac
> OSX, etc. You can't expect all of them to bend over backwards for
> every Windows wart out there.

Yup. This is true. My suggestion was tongue in cheek.

[Michael Torrie]

On 02/25/2013 10:29 AM, D'Arcy J.M. Cain wrote:

> I don't run Windows myself so I can't test it but doesn't Python on
> Windows work fine with Unix style EOL? So why not strip out the CR and
> run the same file everywhere?

As has been said on this thread, python is perfectly happy on windows
with the Unix-style EOL. And if you use a good programmer's editor, it
will happily keep unix-style line endings. Use a crappier,
windows-centric editor, and you'll see the CR's creeping back in.

[Steven D'Aprano]

On Mon, 25 Feb 2013 12:29:58 -0500, D'Arcy J.M. Cain wrote:

> On Mon, 25 Feb 2013 10:18:44 -0700
> Michael Torrie <tor...@gmail.com> wrote:
>> On 02/25/2013 06:14 AM, Dave Angel wrote:
>> > It's not Python that needs dos2unix, it's bash or equivalent. For
>> > some reason, bash shebang processing still isn't tolerant of a
>> > trailing cr on the line. Python doesn't care.
>>
>> Actually, the shell isn't involved in parsing the shebang line at all.
>> That's actually done in the kernel by the program loader. So it's the
>> kernel that has a problem with it; wonder if Linus would accept a patch
>> to ignore the tailing CR?
>
> So much the wrong solution. First of all, I don't think that Linus is
> on the bash development team so he can't help there.

/facepalm

Perhaps you forgot to read Michael's comment before criticising it?

The bash dev team is irrelevant, because this is not a bash problem. It
is the kernel, not bash, that reads the shebang line. So yes, Linus
Torvalds could fix this problem if he chose.

> Also, bash is not
> the only shell in the world. And, Linux is not the only operating
> system in the world. There are still a lot of Unix systems (the system
> that Linux is a clone of) out there. FreeBSD, NetBSD, Solaris, Mac OSX,
> etc. You can't expect all of them to bend over backwards for every
> Windows wart out there.

Nobody is asking anyone to support "every Windows wart out there".
Windows-style line separators are not a wart, it is a convention used by
many, many tools, operating systems, data formats (e.g. email), etc. It
is an old, old convention, going back to teletype days and so predating
not just Windows but also Unix. So in fact it is *Unix* that broke the
convention, and Unix line separators which is the "wart" (or at least a
regression).


--
Steven

[Michael Torrie]

On 02/25/2013 05:52 PM, Steven D'Aprano wrote:
> Nobody is asking anyone to support "every Windows wart out there".
> Windows-style line separators are not a wart, it is a convention used by
> many, many tools, operating systems, data formats (e.g. email), etc. It
> is an old, old convention, going back to teletype days and so predating
> not just Windows but also Unix. So in fact it is *Unix* that broke the
> convention, and Unix line separators which is the "wart" (or at least a
> regression).

That's really interesting. I didn't know that before. It does make
sense. As much as I love unix, it really originated as a hack in many
senses. With that in mind I think Linux should allow a trailing CR in
the shebang line, even if other unix OS's don't. Of course it's a minor
thing, and there are ways of dealing with it.

This is a reminder to me how much we Linux users look at Windows as a
quaint anomaly with it's apparently backwards ways of doing things (like
backslash directory separators, like CP/M did), but forget it is still
the dominant platform out there for general purpose computing. So it
really could be argued that Linux indeed is the backward OS when it
comes to these kind of incompatibilities (though I still think I like it
better!)

[Dave Angel]

On 02/25/2013 09:08 PM, Michael Torrie wrote:
> <snip>

> This is a reminder to me how much we Linux users look at Windows as a
> quaint anomaly with it's apparently backwards ways of doing things (like
> backslash directory separators, like CP/M did),

Actually the reason MSDOS used backslash was because it had already used
the forward slash for a switch-character. Then for version 2, with hard
disks being supported for the first time, they used the backslash
instead. At the time I talked them into supporting a "switchchar" call
to change to using the dash for switch character, and slash for
subdirectories.

But the idea was never publicized, so it never caught on. And future
versions of utilities generally paid no attention to the value of
switchchar.

By the time Windows split off from MSDOS (NT 3.1), the support in the OS
for both slash and backslash was well established. But the utilities
never grew up.

Yes, using the slash as a switch-character was inherited from CP/M,
through QDOS, then MSDOS.


On some of the old teletypes, if the data was coming in fast enough, you
could see the first character of the next line printed before the typing
element reached the left margin. So newline was then spelled CR/LF/NULL
or even CR/LF/NULL/NULL

Buffering? What's that?

--
DaveA

[MRAB]

On 2013-02-26 02:08, Michael Torrie wrote:
> On 02/25/2013 05:52 PM, Steven D'Aprano wrote:

>> Nobody is asking anyone to support "every Windows wart out there".
>> Windows-style line separators are not a wart, it is a convention used by
>> many, many tools, operating systems, data formats (e.g. email), etc. It
>> is an old, old convention, going back to teletype days and so predating
>> not just Windows but also Unix. So in fact it is *Unix* that broke the
>> convention, and Unix line separators which is the "wart" (or at least a
>> regression).
>

> That's really interesting. I didn't know that before. It does make
> sense. As much as I love unix, it really originated as a hack in many
> senses. With that in mind I think Linux should allow a trailing CR in
> the shebang line, even if other unix OS's don't. Of course it's a minor
> thing, and there are ways of dealing with it.
>

> This is a reminder to me how much we Linux users look at Windows as a
> quaint anomaly with it's apparently backwards ways of doing things (like

> backslash directory separators, like CP/M did), but forget it is still
> the dominant platform out there for general purpose computing. So it
> really could be argued that Linux indeed is the backward OS when it
> comes to these kind of incompatibilities (though I still think I like it
> better!)
>

That reminds me of the time I was making PPD (PostScript Printer
Description) files. They worked on both Windows and MacOS.

Then Apple released MacOS X, which complained when they were installed.

It turned out that MacOS X didn't like the line endings. It insisted on
CR only, despite the fact that the PPD specification said that the line
endings could be CR, LF, or CR/LF, and that they had followed the
specification previously!

[Anssi Saari]

Michael Torrie <tor...@gmail.com> writes:

> Actually, the shell isn't involved in parsing the shebang line at all.
> That's actually done in the kernel by the program loader. So it's the
> kernel that has a problem with it; wonder if Linus would accept a patch
> to ignore the tailing CR?

Worth a try in my opinion. There's some historical information about the
shebang at http://www.in-ulm.de/~mascheck/various/shebang/ There's a
table which says Linux since 2.4.0 removes trailing whitespace from the
shebang line. I guess Linux doesn't count CR as whitespace in this
context.