Execute commands from file

[tmp123]

Hello,

Thanks for your time.

We have very big files with python commands (more or less, 500000
commands each file).

It is possible to execute them command by command, like if the
commands was typed one after the other in a interactive session?

( Better using command flags than with an small script like "while 1:
input()" )

Thanks a lot.

[Marc 'BlackJack' Rintsch]

In <1179320782....@u30g2000hsc.googlegroups.com>, tmp123 wrote:

> We have very big files with python commands (more or less, 500000
> commands each file).
>
> It is possible to execute them command by command, like if the
> commands was typed one after the other in a interactive session?

Take a look at the `code` module in the standard library:

In [31]: code?
Type: module
Base Class: <type 'module'>
String Form: <module 'code' from '/usr/lib/python2.4/code.pyc'>
Namespace: Interactive
File: /usr/lib/python2.4/code.py
Docstring:
Utilities needed to emulate Python's interactive interpreter.

Ciao,
Marc 'BlackJack' Rintsch

[Steve Holden]

tmp123 wrote:
> Hello,
>
> Thanks for your time.
>
> We have very big files with python commands (more or less, 500000
> commands each file).
>

Those are BIG programs. Presumably other programs are writing them?

> It is possible to execute them command by command, like if the
> commands was typed one after the other in a interactive session?
>

You need to look for "pdb", the interactive Python debugger. This is
capable of single-step operations, and supports breakpoints.

> ( Better using command flags than with an small script like "while 1:
> input()" )
>
> Thanks a lot.
>

You are pretty much going to have to run pdb then trigger your code by
calling a pdb method with a function in your code as an argument, if I
am remembering correctly how it works.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

[Martin Blume]

"tmp123" schrieb >

> We have very big files with python commands
> (more or less, 500000 commands each file).
>
> It is possible to execute them command by command,

inp = open(cmd_file)
for line in inp:
exec line

might help. You don't get quite the same feeling as

"like if the commands was typed one after the other

in a interactive session", but perhaps this helps.

Warning: the code above is without any error checks.
You might also run into security problems, the example
above assumes you trust your input.

HTH. YMMV.
Martin

[Steve Holden]

The problem with this approach is that each line executes without any
connection to the environment created by previous lies.

Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment hasn't affected
the environment for the print statement.

[i3dmaster]

cat file:

x = 100
print x

cat file.py:
#!/usr/bin/python2.4

import os.path
import sys

file, ext = os.path.splitext(sys.argv[0])
f = open(file,'rb')
for i in f:
exec i

>./file.py
100

Don't see the problem though.

[Douglas Woodrow]

On Thu, 17 May 2007 00:30:23, i3dmaster <i3dm...@gmail.com> wrote

>f = open(file,'rb')
>for i in f:
> exec i

Why are you opening the file in binary mode?

--
Doug Woodrow

[Steve Holden]

No, because there isn't one. Now try adding a function definition and
see how well it works.

[Maric Michaud]

Steve Holden a écrit :

> i3dmaster wrote:
>> On May 16, 1:05 pm, Steve Holden <s...@holdenweb.com> wrote:
>>> Martin Blume wrote:
>>>> "tmp123" schrieb >
>>>>> We have very big files with python commands
>>>>> (more or less, 500000 commands each file).
>>>>> It is possible to execute them command by command,
>>>> inp = open(cmd_file)
>>>> for line in inp:
>>>> exec line

>>> The problem with this approach is that each line executes without any
>>> connection to the environment created by previous lies.
>>>
>>> Try it on a file that reads something like
>>>
>>> xxx = 42
>>> print xxx
>>>

>> cat file:
>>
>> x = 100
>> print x
>>
>> cat file.py:
>> #!/usr/bin/python2.4
>>
>> import os.path
>> import sys
>>
>> file, ext = os.path.splitext(sys.argv[0])
>> f = open(file,'rb')
>> for i in f:
>> exec i
>>
>>> ./file.py
>> 100
>>
>> Don't see the problem though.
>>

> No, because there isn't one. Now try adding a function definition and
> see how well it works.
>
> regards
> Steve

This is just a problem with indentation and blocks of code, the
followong will do :

commands = open("commands")
namespace, block = {}, ""
for line in commands :
line=line[:-1]
if not line : continue
if line[0].isspace() :
block += '\n' + line
continue
else :
if block.strip() :
exec block in namespace
block = line

exec block in namespace
print dict((k, v) for k, v in namespace.items() if k != "__builtins__")

with commands containing :

"""

x = 5

def toto(arg) :
print arg

def inner() :
print arg*arg

inner()

toto(x)
"""

output :
5
25
{'x': 5, 'toto': <function toto at 0x01D30C70>}

(sorry Steve for the private mail)

--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 4 26 88 00 97
Mobile: +33 6 32 77 00 21

[Martin Blume]

"Steve Holden" schrieb

> >>
> >> Try it on a file that reads something like
> >>
> >> xxx = 42
> >> print xxx
> >>
> >> and you will see NameError raised because the assignment
> >> hasn't affected the environment for the print statement.
> >>

> > [...]

> >
> No, because there isn't one. Now try adding a function
> definition and see how well it works.
>

C:\temp>more question.py
xxx=42
print xxx
def sowhat():
print xxx

print xxx

C:\temp>c:\programme\python\python
Python 2.4 (#60, Nov 30 2004, 11:49:19)
[MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license"
for more information.
>>> exec open("question.py").read()
42
42
>>> sowhat()
42
>>> xxx
42

Seems to work great to me.

OTOH, this doesn't:
>>> inp=open("question.py")
>>> for l in inp:
... exec l
...
42
Traceback (most recent call last):
File "<stdin>", line 2, in ?
File "<string>", line 1
def sowhat():
^
SyntaxError: unexpected EOF while parsing

So it seems to depend on the way the file is read.

Regards
Martin

[i3dmaster]

On May 17, 3:02 am, Douglas Woodrow <newsgro...@nospam.demon.co.uk>
wrote:
> On Thu, 17 May 2007 00:30:23, i3dmaster <i3dmas...@gmail.com> wrote

>
> >f = open(file,'rb')
> >for i in f:
> > exec i
>
> Why are you opening the file in binary mode?
>
> --
> Doug Woodrow

'b' is generally useful on systems that don't treat binary and text
files differently. It will improve portability.

[Steve Holden]

It depends on the way the lines of the file are executed, not how they
are read. And you may remember the original poster was proposing this:

inp = open(cmd_file)
> for line in inp:
> exec line

As for your first example, why not just use execfile() ?

[Douglas Woodrow]

On Fri, 18 May 2007 04:45:30, Dennis Lee Bieber <wlf...@ix.netcom.com>
wrote
>On 17 May 2007 13:12:10 -0700, i3dmaster <i3dm...@gmail.com> declaimed
>the following in comp.lang.python:

>
>> 'b' is generally useful on systems that don't treat binary and text
>> files differently. It will improve portability.
>

> "b" is needed for binary files on systems that /do/ treat binary
>differently from text. And it does add to portability only in that it
>has no effect on those that treat all files the same.
>
> However, as I recall the thread, the intent is to process text lines
>from a file -- and using "b" is going to affect how the line endings are
>being treated.

Yes that was my understanding too, Dennis, and the reason I queried it
in the first place. I had to remove the "b" option in order to get the
sample code to work under Windows, because the standard line termination
under Windows is carriage return + linefeed (\r\n).

Of course if I manually edit the command file so that it only has a
linefeed character at the end of each line, the binary mode works.

So I think i3dmaster's method is only portable as long as the command
file is created with unix-style line termination.

--
Doug Woodrow

[Martin Blume]

"Steve Holden" schrieb
> >
> > [ difference between exec open(fname).read()
> > and for line in open(fname): exec line ]

> >
> > So it seems to depend on the way the file is read.
> >
> It depends on the way the lines of the file are executed,
> not how they are read.
>

Could you elaborate a little bit more on the difference?
I assumed that because read() reads the whole file, the
body of my function sowhat() is present, so that it can
be parsed while the invocation of exec is still running.
If it is read and exec'd line by line, the definition of
the function is still left open at the moment exec() ends,
causing the "EOF" error. Hence my statement, "it depends
on the way the file is read".

> And you may remember the original poster was
> proposing this:
>
> inp = open(cmd_file)
> for line in inp:
> exec line
>
> As for your first example, why not just use execfile() ?
>

I assume that
execfile(fname)
is equivalent to
exec open(fname).read() ?

Regards
Martin

[Steve Holden]

Martin Blume wrote:
> "Steve Holden" schrieb
>>> [ difference between exec open(fname).read()
>>> and for line in open(fname): exec line ]
>>>
>>> So it seems to depend on the way the file is read.
>>>
>> It depends on the way the lines of the file are executed,
>> not how they are read.
>>
> Could you elaborate a little bit more on the difference?
> I assumed that because read() reads the whole file, the
> body of my function sowhat() is present, so that it can
> be parsed while the invocation of exec is still running.
> If it is read and exec'd line by line, the definition of
> the function is still left open at the moment exec() ends,
> causing the "EOF" error. Hence my statement, "it depends
> on the way the file is read".
>

I simply meant that the whole source has to be presented to the exec
statement and not chunked into lines.

Clearly I could read all the source in with

lines = open(cmd_file).readlines()

but if you then proceed to try and execute the source line by line as in

for l in lines:
exec l

you will hit problems because of the disjoint nature of the execution
which will breal up indented suites and so on.

I was probably just a little over-zealous in pursuing correct English
usage, in which case please accept my apology.

>
>> And you may remember the original poster was
>> proposing this:
>>
>> inp = open(cmd_file)
>> for line in inp:
>> exec line
>>
>> As for your first example, why not just use execfile() ?
>>
> I assume that
> execfile(fname)
> is equivalent to
> exec open(fname).read() ?
>

Pretty much.

[Martin Blume]

"Steve Holden" schrieb

>
> I simply meant that the whole source has to be presented
> to the exec statement and not chunked into lines.
>

That's what I meant: With exec open(f).read() it is not
broken into several exec invocations.

>
> I was probably just a little over-zealous in pursuing
> correct English usage, in which case please accept
> my apology.
>

The apology is on my part, I didn't explain my thinking
clearly enough.

Thanks for your explanations. Makes my newbie understanding
of Python much more robust.

Regards
Martin