How to run script from interpreter?

[Rolander, Dan]

What is the best way to run a python script from within the interpreter?
What command should I use?

Thanks,
Dan

[Rolander, Dan]

[JustinHJ]

say you have a script in a file called test.py just type

import test

(you don't need the .py)

As for doing stuff in the script it will execute any commands it encounters.
It's neat to have all your stuff in functions and then have commands at the
bottom to call the functions as needed. This is like a simulated main() from
C.

if __name__ == '__main__': # only run if main program
DoStuff( )

Rolander, Dan <Dan.Ro...@marriott.com> wrote in message
news:mailman.979855413...@python.org...

[Joe Smith]

I like to do:
if __name__ == '__main__':
main( sys.argv[1:] )

This way I can similate everything because I can pass main whatever list of
arguments that I want.

[Dan Rolander]

This doesn't work for me. I know how to import and how to use the if
__name__ trick, but how can I execute a script from the interactive Python
interpreter? That is, can I load a script and have it run from inside the
interpreter? (I'm sure this is easy I'm just not seeing it.)

Dan

> --
> http://mail.python.org/mailman/listinfo/python-list
>

[Steve Holden]

"Dan Rolander" <dan.ro...@marriott.com> wrote in message
news:mailman.979898969...@python.org...

> This doesn't work for me. I know how to import and how to use the if
> __name__ trick, but how can I execute a script from the interactive Python
> interpreter? That is, can I load a script and have it run from inside the
> interpreter? (I'm sure this is easy I'm just not seeing it.)
>
> Dan

Well, if you've written your program as a .py file, the usual technique is
to enter an import statement to the interactive interpreter. The code is
executed as the import proceeds. The __name__ == _"__main__" trick is
usually used to run test code in a module intended primarily for import by
other code rather than for execution as a program.

Of course, this only works once, because further imports will assume they
can use teh already-imported module. To get subsequent modifications of the
script to work you have to use reload(mdulename), which will force
execution.

It's a slight pain that the first import is different from all the others,
but never mind.

regards
Steve

[Alex Martelli]

"Dan Rolander" <dan.ro...@marriott.com> wrote in message
news:mailman.979898969...@python.org...

> This doesn't work for me. I know how to import and how to use the if
> __name__ trick, but how can I execute a script from the interactive Python
> interpreter? That is, can I load a script and have it run from inside the
> interpreter? (I'm sure this is easy I'm just not seeing it.)

Suppose you have, somewhere on the path Python searches, a
file foo.py containing the single Python statement:

print "I'm being executed"

When, at the interactive interpreter prompt, you type an import
statement:

>>> import foo
I'm being executed
>>>

module foo.py has been loaded and, since this was the first time
it was imported in this session, its module-level code was executed.

Another import statement will NOT repeat execution of the module's
code...:

>>> import foo
>>>

No execution in this case, because it was not the first import
in this session!

You can, however, force such re-execution by reloading the
module object after having imported it:

>>> reload(foo)
I'm being executed
<module 'foo' from 'foo.pyc'>
>>>

The reload function also returns the new module-object, which
is why we see this second line here (the interpreter is showing
it to us) -- if you don't want this little bother just assign it:

>>> _=reload(goo)
I'm being executed
>>>

Alex

[Moshe Zadka]

On Fri, 19 Jan 2001, "Dan Rolander" <dan.ro...@marriott.com> wrote:
> This doesn't work for me. I know how to import and how to use the if
> __name__ trick, but how can I execute a script from the interactive Python
> interpreter? That is, can I load a script and have it run from inside the
> interpreter? (I'm sure this is easy I'm just not seeing it.)

execfile(script_name)

E.g.,

execfile("test.py")
--
Moshe Zadka <s...@zadka.site.co.il>
This is a signature anti-virus.
Please stop the spread of signature viruses!
Fingerprint: 4BD1 7705 EEC0 260A 7F21 4817 C7FC A636 46D0 1BD6

[Mikael Olofsson]

On 19-Jan-01 Dan Rolander wrote:
> This doesn't work for me. I know how to import and how to use the if
> __name__ trick, but how can I execute a script from the interactive Python
> interpreter? That is, can I load a script and have it run from inside the
> interpreter? (I'm sure this is easy I'm just not seeing it.)

Let's see... I think it was here somewhere... Ah, yes: In Python Library
Reference, 2.3 Built-in Functions, at

http://www.python.org/doc/current/lib/built-in-funcs.html

execfile (file[, globals[, locals]])
This function is similar to the exec statement, but parses a file
instead of a string. It is different from the import statement in
that it does not use the module administration -- it reads the file
unconditionally and does not create a new module.2.8

The arguments are a file name and two optional dictionaries. The
file is parsed and evaluated as a sequence of Python statements
(similarly to a module) using the globals and locals dictionaries
as global and local namespace. If the locals dictionary is omitted
it defaults to the globals dictionary. If both dictionaries are
omitted, the expression is executed in the environment where
execfile() is called. The return value is None.

HTH

/Mikael

-----------------------------------------------------------------------
E-Mail: Mikael Olofsson <mik...@isy.liu.se>
WWW: http://www.dtr.isy.liu.se/dtr/staff/mikael
Phone: +46 - (0)13 - 28 1343
Telefax: +46 - (0)13 - 28 1339
Date: 19-Jan-01
Time: 12:35:02

/"\
\ / ASCII Ribbon Campaign
X Against HTML Mail
/ \

This message was sent by XF-Mail.
-----------------------------------------------------------------------

[Rolander, Dan]

See, I knew it was something simple. Thanks for pointing me in the right
direction!

One more thing, is there a way to pass a command line parameter to the
script named in execfile? Say if I have a script to test which expects a
value in sys.argv[1]?

Thanks,
Dan

-----Original Message-----
From: Mikael Olofsson [mailto:mik...@isy.liu.se]
Sent: Friday, January 19, 2001 6:52 AM
To: Dan Rolander
Cc: pytho...@python.org
Subject: Re: How to run script from interpreter?

[Mikael Olofsson]

On 19-Jan-01 Rolander, Dan wrote:
> See, I knew it was something simple. Thanks for pointing me in the right
> direction!
>
> One more thing, is there a way to pass a command line parameter to the
> script named in execfile? Say if I have a script to test which expects a
> value in sys.argv[1]?

I'm afraid that's beyond me. You'll have to wait for someone else to
jump in.

/Mikael

-----------------------------------------------------------------------
E-Mail: Mikael Olofsson <mik...@isy.liu.se>
WWW: http://www.dtr.isy.liu.se/dtr/staff/mikael
Phone: +46 - (0)13 - 28 1343
Telefax: +46 - (0)13 - 28 1339
Date: 19-Jan-01

Time: 16:03:21

[Fredrik Lundh]

Dan Rolander wrote:
> One more thing, is there a way to pass a command line parameter to the
> script named in execfile? Say if I have a script to test which expects a
> value in sys.argv[1]?

did you try putting it in sys.argv[1] ?

import sys
sys.argv = ["spam.py", "one", "two", "three"]
execfile("spam.py")

Cheers /F

[JustinHJ]

A cool thing I found out solving my own problem is that a module can be a
variable. So you want to run a module called 'bobsmodule'

str = 'bobsmodule'

fp, pathname, description = imp.find_module( modname )

print 'importing module', modname
LoadedModule = imp.load_module( modname, fp, pathname, description)
LoadedModule.Afunctioninbobsmodule(i)

This gives you flexibility if you need it.

[Fredrik Lundh]

JustinHJ wrote:
> A cool thing I found out solving my own problem is that a module can be a
> variable. So you want to run a module called 'bobsmodule'
>
> str = 'bobsmodule'
> fp, pathname, description = imp.find_module( modname )
> print 'importing module', modname
> LoadedModule = imp.load_module( modname, fp, pathname, description)
> LoadedModule.Afunctioninbobsmodule(i)

or better:

LoadedModule = __import__('bobsmodule')

Cheers /F

[Rolander, Dan]

That works great! I did try to sys.argv[1] = "one" but that didn't work. I
realize now that I needed to set the whole list.

Thanks for your help!
Dan

-----Original Message-----
From: Fredrik Lundh [mailto:fre...@effbot.org]
Sent: Friday, January 19, 2001 12:53 PM
To: pytho...@python.org
Subject: Re: How to run script from interpreter?

Cheers /F

--
http://mail.python.org/mailman/listinfo/python-list

[onlyv...@gmail.com]

try using execfile(filename)

[Terry Reedy]

or in 3.x
with open(filename) as f: exec f

These both assume that you want to run the script in the same process as
the interpreter and within the module containing the statement. This is
rare. People usually either want to import into a separate module or run
in a separate process. For the latter, use the subprocess module and the
same command line that you would use in a console.

--
Terry Jan Reedy

[Mark H Harris]

On 5/28/14 2:44 AM, onlyv...@gmail.com wrote:
> On Friday, January 19, 2001 1:22:23 AM UTC+5:30, Rolander, Dan wrote:
>> What is the best way to run a python script from within the interpreter?
>> What command should I use?
>>
>

> try using execfile(filename)
>

What type of script? python? bash? tcl? other?

If you want to use python as a shell-glue you can try using system.

>>> from os import system
>>> def <function_name>([parms])
>>> .... blah blah
>>> .... rc = system("<your_script_name")
>>>

When you call function_name within the python interpreter
your_script_name will be called using system.


OTOH, if you are wanting to run a python script within the interpreter
then just import the names you want from your_script.py file and then
call the name... like main, forinstance.

>>> from my_script import main
>>> main([parms])

Within your_script.py define a main function:

def main([parms]):
blah blah
return rc

-----

OTOH, just write the script.py file (top down procedural) and then
import it:

>>> import my_script.py


marcus

[Steven D'Aprano]

On Wed, 28 May 2014 11:39:23 -0500, Mark H Harris wrote:

> On 5/28/14 2:44 AM, onlyv...@gmail.com wrote:
>> On Friday, January 19, 2001 1:22:23 AM UTC+5:30, Rolander, Dan wrote:
>>> What is the best way to run a python script from within the

.................................^^^^^^^^

>>> interpreter? What command should I use?
>>>
>>>
>> try using execfile(filename)
>>
>>
> What type of script? python? bash? tcl? other?

Most Python scripts are Python. The remainder are usually Python.

> If you want to use python as a shell-glue you can try using system.
>
> >>> from os import system
> >>> def <function_name>([parms])
> >>> .... blah blah
> >>> .... rc = system("<your_script_name")

os.system is cool for quick and dirty calls to an external command. But
for serious work, the subprocess module is better.



--
Steven

[Mark H Harris]

On 5/28/14 10:22 PM, Steven D'Aprano wrote:
>> If you want to use python as a shell-glue you can try using system.
>>
>> >>> from os import system
>> >>> def <function_name>([parms])
>> >>> .... blah blah
>> >>> .... rc = system("<your_script_name")
>
>
> os.system is cool for quick and dirty calls to an external command. But
> for serious work, the subprocess module is better.
>

... yup, particularly for non trivial network related stuff.


Neither here nor there, but I just learned the ";" character command
today for the Julia REPL, and got to thinking that python should have a
similar way for the REPL to drop into "shell" mode for system commands.

So, I might code a clear screen in python:

def cls()
rc = system("clear")


or in Julia

function cls()
run(`clear`)
end

... but on Julia we can also do this:

; clear

On the Julia REPL the ";" character drops the julia prompt into shell. I
think the IDLE REPL should have a system shell mode. What say you?

marcus

[Steven D'Aprano]

On Thu, 29 May 2014 15:26:37 -0500, Mark H Harris wrote:

> I think the IDLE REPL should have a system shell mode. What say you?

I don't use IDLE, so I don't really care what you do to it. But speaking
generally, -1 on that. IDLE is primarily aimed at beginners, and
beginners have enough trouble distinguishing the system shell separate in
their heads from the Python shell, the last thing they need is something
that blurs the difference.

There is a power-user shell that does exactly what you want, turned up to
eleven: IPython. IPython allows you to run system commands by prefixing
them with ! and has an entire subsystem of user-configurable "magic"
commands using % prefix.

(By the way, ; won't work for a Python shell, because ;spam already is
valid Python syntax: it's an empty statement followed by the statement
spam, separated by a semicolon.)




--
Steven D'Aprano
http://import-that.dreamwidth.org/

[Chris Angelico]

On Fri, May 30, 2014 at 10:33 AM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> (By the way, ; won't work for a Python shell, because ;spam already is
> valid Python syntax: it's an empty statement followed by the statement
> spam, separated by a semicolon.)

That's not really a problem, though. It's not going to stop you from
doing something actually *useful*, and the fact that the semicolon
could be syntactically valid isn't going to come into it, because the
REPL would swallow it anyway.

But I agree that IDLE does not need a "drop to shell" mode.

ChrisA

[Steven D'Aprano]

On Fri, 30 May 2014 10:46:34 +1000, Chris Angelico wrote:

> On Fri, May 30, 2014 at 10:33 AM, Steven D'Aprano
> <steve+comp....@pearwood.info> wrote:
>> (By the way, ; won't work for a Python shell, because ;spam already is
>> valid Python syntax: it's an empty statement followed by the statement
>> spam, separated by a semicolon.)
>
> That's not really a problem, though. It's not going to stop you from
> doing something actually *useful*, and the fact that the semicolon could
> be syntactically valid isn't going to come into it, because the REPL
> would swallow it anyway.

The point is that *syntactically valid* Python statements should not
behave differently when running inside the shell or not. I thought
that ;statement was syntactically valid -- but I was wrong. The vanilla
CPython interactive interpreter gives a syntax error, as do IronPython
and Jython. I misinterpreted what I saw in IPython -- it's already doing
magic at the command line:

In [4]: len []
------> len([])
Out[4]: 0

In [5]: ;len []
------> len("[]")
Out[5]: 2

[Chris Angelico]

On Fri, May 30, 2014 at 11:49 AM, Steven D'Aprano

<steve+comp....@pearwood.info> wrote:
> On Fri, 30 May 2014 10:46:34 +1000, Chris Angelico wrote:
>
>> On Fri, May 30, 2014 at 10:33 AM, Steven D'Aprano
>> <steve+comp....@pearwood.info> wrote:
>>> (By the way, ; won't work for a Python shell, because ;spam already is
>>> valid Python syntax: it's an empty statement followed by the statement
>>> spam, separated by a semicolon.)
>>
>> That's not really a problem, though. It's not going to stop you from
>> doing something actually *useful*, and the fact that the semicolon could
>> be syntactically valid isn't going to come into it, because the REPL
>> would swallow it anyway.
>
> The point is that *syntactically valid* Python statements should not
> behave differently when running inside the shell or not. I thought
> that ;statement was syntactically valid -- but I was wrong. The vanilla
> CPython interactive interpreter gives a syntax error, as do IronPython
> and Jython.

Huh. I responded to your post on the unchecked assumption that ";spam"
was syntactically valid. I'm still of the opinion that technically
valid (but not useful) constructs are allowed to be swallowed by an
interactive interpreter; for instance, it's perfectly valid to write
this:

>>> x = (1
.
imag)

But quite a few interpreters (though not Python) take a dot on its own
line to mean "flush the buffer, cancel my current command". I couldn't
find a way to do this in Python, but if someone proposed adding it,
the fact that the above is syntactically legal shouldn't be a proposal
blocker. It's on par with creating a file with a name beginning with a
hyphen, and then fiddling around with various commands as you try to
manipulate it (tip: "rm ./-r" works); programs will happily interpret
"-r" as an option rather than a file name, without being concerned
that it's technically legal.

ChrisA

[Steven D'Aprano]

On Fri, 30 May 2014 12:04:27 +1000, Chris Angelico wrote:

> On Fri, May 30, 2014 at 11:49 AM, Steven D'Aprano
> <steve+comp....@pearwood.info> wrote:
>> On Fri, 30 May 2014 10:46:34 +1000, Chris Angelico wrote:
>>
>>> On Fri, May 30, 2014 at 10:33 AM, Steven D'Aprano
>>> <steve+comp....@pearwood.info> wrote:
>>>> (By the way, ; won't work for a Python shell, because ;spam already
>>>> is valid Python syntax: it's an empty statement followed by the
>>>> statement spam, separated by a semicolon.)
>>>
>>> That's not really a problem, though. It's not going to stop you from
>>> doing something actually *useful*, and the fact that the semicolon
>>> could be syntactically valid isn't going to come into it, because the
>>> REPL would swallow it anyway.
>>
>> The point is that *syntactically valid* Python statements should not
>> behave differently when running inside the shell or not. I thought that
>> ;statement was syntactically valid -- but I was wrong. The vanilla
>> CPython interactive interpreter gives a syntax error, as do IronPython
>> and Jython.
>
> Huh. I responded to your post on the unchecked assumption that ";spam"
> was syntactically valid. I'm still of the opinion that technically valid
> (but not useful) constructs are allowed to be swallowed by an
> interactive interpreter;

Heavens no. If it's legal in a script when running non-interactively, it
should be legal when running interactively.

I give a grudging exception to things like the REPL ending a function
definition at the first empty line, even though it breaks copy-and-paste
of working code:

py> def example():
... do_this()
...
py> do_that()
File "<stdin>", line 1
do_that()
^
IndentationError: unexpected indent


That's a *serious* PITA when copying (say) classes into the interactive
interpreter, because you have to delete all the blank lines (or stick
comments # in them) before pasting. But I can't see any practical
alternative.

> for instance, it's perfectly valid to write
> this:
>
>>>> x = (1
> .
> imag)

Absolutely it is valid Python code, so your REPL better accept it as
valid Python code.

> But quite a few interpreters (though not Python) take a dot on its own
> line to mean "flush the buffer, cancel my current command".

I've never come across that. I'm very glad I haven't. Who came up with
that broken interface?

> I couldn't
> find a way to do this in Python, but if someone proposed adding it, the
> fact that the above is syntactically legal shouldn't be a proposal
> blocker.

Damn straight it ought to. If I proposed making:

x = (1
+
y)


do something other than set x to 1+y, I would completely deserve to be
dragged out into the street and beaten with halibuts. Replace the + with
a dot, and there is no difference at all. If you want to kill the current
buffer, just hit Ctrl-C like you're meant to.

> It's on par with creating a file with a name beginning with a
> hyphen, and then fiddling around with various commands as you try to
> manipulate it (tip: "rm ./-r" works); programs will happily interpret
> "-r" as an option rather than a file name, without being concerned that
> it's technically legal.

I don't see any connection between the two.

[Chris Angelico]

-r is a valid file name, just as . is a valid line of input. But with
the rm command, you can't provide it with -r as a file name - you have
to use ./-r or somesuch, because the parser gets to it first. That's
the connection.

ChrisA

[Steven D'Aprano]

The analogy doesn't quite match, because rm, like all shell commands,
does it's own argument parsing, whereas Python operators do not. But
putting that aside, the parser's rule is "arguments starting with a dash
are treated as options". That's *standard behaviour*. If you want to
override that standard behaviour, you have to tweak the command:

rm -f ./-r /tmp/*

Some commands accept a single dash to mean "no more options following
this". Assuming rm does the same:

rm -f - -r /tmp/*

Either way, it is the standard behaviour as defined by rm, and so
expected. That makes this the equivalent of Python's rule that dot
behaves somewhat like an operator, and that a dot on its own is legal any
place a bare binary operator is legal.

The analogy to "the REPL is allowed to modify standard syntax to make a
bare dot special" would be something like:

the shell is allowed to pre-process the command line, so
that -r on a line on its own like this:

rm -f \
-r \
/tmp/*

which otherwise would have been treated as `rm -f -r /tmp/*`,
is instead turned into `rm -f ./-r /tmp/*`.

Reading the rm man page won't tell you this, because it happens outside
of rm, just as reading the Python documentation won't tell you about the
tricks your hypothetical REPL does to otherwise legal syntax. rm doesn't
know what tricks your hypothetical shell might do, and neither can the
Python parser tell what tricks your hypothetical REPL might do. Both see
only what the shell give them.

This is why I'm so adamant that, while REPLs may be permitted to
introduce *new* syntax which is otherwise illegal to the Python parser,
(e.g. like IPython's %magic and !shell commands) they *must not* change
the meaning of otherwise legal Python syntax. It's simply a bad idea to
have legal Python code mean different things depending on what
environment you're running it in.

(As I mentioned before, the annoying "blank line closes all open blocks"
rule gets an exception simply because I don't believe there is any
practical alternative.)

[Chris Angelico]

On Fri, May 30, 2014 at 9:27 PM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> This is why I'm so adamant that, while REPLs may be permitted to
> introduce *new* syntax which is otherwise illegal to the Python parser,
> (e.g. like IPython's %magic and !shell commands) they *must not* change
> the meaning of otherwise legal Python syntax. It's simply a bad idea to
> have legal Python code mean different things depending on what
> environment you're running it in.

Hmm. I'm not sure that "raises SyntaxError" is any less a part of the
language's promise than "evaluates to twice the value of x" is. Would
you, for instance, permit the REPL to define a new __future__
directive, on the basis that it's invalid syntax currently?

>>> from __future__ import magic_command_history
SyntaxError: future feature magic_command_history is not defined

I don't think SyntaxError equates to "invitation to make changes".
Also, consider:

>>> for i in range(5): i*3+2

2
5
8
11
14

If you do this in a script, it's perfectly legal, but won't print
anything. So the REPL is already "chang[ing] the meaning of otherwise
legal Python syntax", and what's more, it's making None into a special
case:

>>> for i in range(5): None if i%2 else i

0
2
4

Practicality beats purity. If it's more useful to the end user for
something valid-but-illogical to have a new bit of meaning in
interactive mode, I say go for it.

ChrisA

[Terry Reedy]

On 5/30/2014 7:46 AM, Chris Angelico wrote:

> Hmm. I'm not sure that "raises SyntaxError" is any less a part of the
> language's promise than "evaluates to twice the value of x" is.

Of course it is. A real SyntaxError cannot be caught immediately.* When
new syntax features are added, breaking the raising of a SyntaxError is
*not* a consideration. On the other hand, we do not freely change
AbcError to XyzError even when the latter would be more 'correct'.

* IE, you cannot type

>>> try: ..
except SyntaxError: print('caught')

because the SyntaxError during parsing rather than during execution.

> Also, consider:
>
>>>> for i in range(5): i*3+2
>
> 2
> 5
> 8
> 11
> 14
>
> If you do this in a script, it's perfectly legal, but won't print

> anything. So the REPL is already "chang[ing] the meaning of otherwise
> legal Python syntax",

It does not change what is calculated. It adds introspection output as a
convenience. The minus is that one may forget that it *is* an addition
and use the shortcut when writing a script. (I know I have, occasionally.)

> and what's more, it's making None into a special case:

Which it is, as the 'zero' of the set of Python objects.

>>>> for i in range(5): None if i%2 else i
>
> 0
> 2
> 4
>
> Practicality beats purity. If it's more useful to the end user for
> something valid-but-illogical to have a new bit of meaning in
> interactive mode, I say go for it.

I disagree.

--
Terry Jan Reedy

[Steven D'Aprano]

On Fri, 30 May 2014 21:46:55 +1000, Chris Angelico wrote:

> On Fri, May 30, 2014 at 9:27 PM, Steven D'Aprano
> <steve+comp....@pearwood.info> wrote:
>> This is why I'm so adamant that, while REPLs may be permitted to
>> introduce *new* syntax which is otherwise illegal to the Python parser,
>> (e.g. like IPython's %magic and !shell commands) they *must not* change
>> the meaning of otherwise legal Python syntax. It's simply a bad idea to
>> have legal Python code mean different things depending on what
>> environment you're running it in.
>
> Hmm. I'm not sure that "raises SyntaxError" is any less a part of the
> language's promise than "evaluates to twice the value of x" is. Would
> you, for instance, permit the REPL to define a new __future__ directive,
> on the basis that it's invalid syntax currently?
>
>>>> from __future__ import magic_command_history
> SyntaxError: future feature magic_command_history is not defined

"from __future__ import ..." is an instruction to the compiler, hence a
positive language feature. (That's why in source files any such import
must occur at the top of the file, before any line of code.) It's a
little unfortunate that bogus __future__ imports raise SyntaxError
directly rather than some subclass of it, but given that it occurs at
compile time, and how rarely one is in a position to try catching the
exception, its understandable that the core devs didn't bother making it
a separate subclass.

(On the other hand, "import __future__" is a regular import that can
occur any time you like.)

No, a REPL cannot legitimately invent new language features that change
Python's semantics, any more than they can legitimately redefine the plus
operator + to perform subtraction. But they can legitimately add features
to the shell, provided those features don't affect Python code. What's
the difference?

Consider entering this part of an interactive IPython session:


In [14]: len []
-------> len([])
Out[14]: 0

In [15]: n = len []
------------------------------------------------------------
File "<ipython console>", line 1
n = len []
^
SyntaxError: invalid syntax


Why is [14] permitted, but not [15]? Because [14] is a shell feature, but
[15] is Python code. If [15] were permitted, then we would be tempted to
use it inside functions:

def spam(a):
n = len a
...

and either be surprised at the syntax error, or (worse!) be surprised
that the function runs interactively inside IPython but fails in other
shells and non-interactively. If [15] were allowed, we would no longer be
running Python code.

Of course, a naive user will be confused that len [] is permitted in
IPython, but that's okay since it's a power-user's shell aimed at power-
users. If the vanilla Python shell introduced such power-user convenience
features by default, I would be very disappointed.

Before you ask, there is no absolutely hard and fast line between "shell
feature" and "Python code", but the more closely your shell features
resemble Python code, the harder it will be for users (power or not) to
keep them separate in their head and the more likely they will be
confused. E.g. suppose my shell decided to allow lines like

var = len []

to define the variable var and set it to 0. I'd argue that crosses the
line from "useful convenience feature for power-users" to "dangerously
confusing misfeature" because it looks too much like Python code. (I'm
already dubious about len [] on its own.) A better interface is to have a
clear separation between shell commands and Python, and IPython commonly
usually uses prefix sigils such as ; % and ! for that.

More here, including some cases where I think IPython crosses that line:

http://ipython.org/ipython-doc/dev/interactive/reference.html


You raise the issue of the vanilla Python shell printing the result of
expressions. That would be the P in REPL, yes? :-) It would be a funny
REPL that *didn't* print evaluated expressions.

(Not entirely unheard of though -- Forth, for example, doesn't print the
values you push onto the stack unless you pop them from the stack first.
But it does print "ok" after each Read-Eval cycle when working
interactively, so I guess it still counts as a REPL.)

If we wanted to be pedantic, then yes there are semantic differences
between code running in a REPL and the same running non-interactively.
The vanilla REPL sets the magic variable _ to the result of the last
evaluated expression (IPython has a bazillion variations of that). The
REPL defines sys.ps1 and sys.ps2, when running non-interactively they
don't exist. PYTHONSTARTUP isn't loaded outside of the REPL. But my
argument is that these are a different kind of semantic difference than
changing how Python expressions are parsed. Not all semantic differences
are equal:

(1) in the REPL, evaluating "(spam . attr)" on its own has the
side-effect of printing the value of spam.attr

(2) in the REPL, evaluating "(spam . attr)" does not perform a
lookup of attr on spam

are very different kinds of behavioural changes.

[For brevity I left out the newlines.]

> I don't think SyntaxError equates to "invitation to make changes".

No, not in general. But SyntaxError does give a convenient opportunity to
add shell features. If a line of text L is valid Python code, then the
REPL needs to treat it as valid Python code. If L is invalid Python code,
i.e. raises SyntaxError, then the REPL *may* use it as a shell feature,
but whether it should or not depends on the details of L.


--
Steven

[Chris Angelico]

On Sat, May 31, 2014 at 5:28 AM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> Before you ask, there is no absolutely hard and fast line between "shell
> feature" and "Python code", but the more closely your shell features
> resemble Python code, the harder it will be for users (power or not) to
> keep them separate in their head and the more likely they will be
> confused. E.g. suppose my shell decided to allow lines like
>
> var = len []
>
> to define the variable var and set it to 0. I'd argue that crosses the
> line from "useful convenience feature for power-users" to "dangerously
> confusing misfeature" because it looks too much like Python code. (I'm
> already dubious about len [] on its own.) A better interface is to have a
> clear separation between shell commands and Python, and IPython commonly
> usually uses prefix sigils such as ; % and ! for that.

I think this is the nub of the issue. You believe that anything that's
syntactically legal in a script MUST (a) be syntactically legal, and
(b) have the same semantics (modulo the printing part mentioned below)
in the interactive interpreter. I'm a little less strict, and would be
okay with some things that make little sense being disallowed. There
are already a few such things (you mention the blank line issue), and
the question is more whether they're recommended or not, than whether
they're allowed or not. (Python 3's super() is a piece of magic, and
it's better that it be magical, but it isn't a precedent to be
followed.) I can accept that the desirable line is further over than I
was putting it; practicality is only so-much of an argument.

> You raise the issue of the vanilla Python shell printing the result of
> expressions. That would be the P in REPL, yes? :-) It would be a funny
> REPL that *didn't* print evaluated expressions.

Yes of course, but there's a difference between these:

>>> 1 + 2
3
>>> while True:
input("Echo? ")
if _=="Echo!": break

The first one is using the REPL exactly the way everyone would expect
it to be used. A single expression is entered, and it is printed.
Failing to do that is, indeed, failing on the P of REPL. (That said, a
Read-Eval Loop, sans Print, is still of value. I used one for over a
decade with REXX, manually prefixing things with "say" when I wanted
them printed, err I mean said.) But the second one is looking a lot
more like a script, and if you're writing code like that, you really
should think about assigning the input to a variable and explicitly
printing it, not fiddling with automatic display and the _ capture.

A "top-level expression", if you like, should definitely be printed.
An inner expression isn't so fundamental. Since Python has been
written to print them, we can make use of it; but a REPL that didn't
print inner expressions is still a useful tool. Printing inner
expressions is sliding toward the line of dangerously confusing
misfeatures that you mentioned; you could compose a massively long
function, and deep inside it, call something and have its return value
printed - and then paste that into a script and find that it's not
printing any more. Or, possibly worse, come to python-list with the
question "How do I fix this?", not even knowing what the problem is,
and having us all assume it's a script.

It's a grey area. What's a convenient and useful feature, and what's a
problem? Is supporting "print 1+2" in Python 3 on one side or the
other of that line?

ChrisA