Python Bytecode
Daniel Nuriyev
through the Python code etc. but haval (it's a pity) to waste time. Is
there any information about the bytecode?
Thank You all
Aahz
Yes. But if you want any of it, you'll probably have to be more
specific.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
Project Vote Smart: http://www.vote-smart.org/
Michael Hudson
> I do not find any info about the Python bytecode.
Doesn't this:
http://www.python.org/doc/current/lib/bytecodes.html
count?
> I certainly can go through the Python code etc. but haval (it's a
> pity) to waste time. Is there any information about the bytecode?
Only the above, AFAIK, and you have to have a pretty good idea of how
the interpreter works to make sense of any description you find.
Cheers,
M.
--
I've even been known to get Marmite *near* my mouth -- but never
actually in it yet. Vegamite is right out.
UnicodeError: ASCII unpalatable error: vegamite found, ham expected
-- Tim Peters, comp.lang.python
Terry Reedy
"Michael Hudson" <m...@python.net> wrote
...
http://www.python.org/doc/current/lib/bytecodes.html
Thanks. I read this, found a couple of typos, and sent note to
python-doc (don't have Source Forge login).
Can you or someone answer a question regarding the following from the
above:
"
JUMP_IF_TRUE delta
If TOS is true, increment the byte code counter by delta. TOS is left
on the stack.
JUMP_IF_FALSE delta
If TOS is false, increment the byte code counter by delta. TOS is not
changed.
"
Why isn't TOS removed? Or, at least, why isn't there alternative
bytecode that does so? Seems like most (all?) usages remove it anyway
on both branches. For example (from your python-dev post)
>>> def f():
... if a:
... print 1
...
>>> import dis
>>> dis.dis(f)
2 0 LOAD_GLOBAL 0 (a)
3 JUMP_IF_FALSE 9 (to 15)
6 POP_TOP
3 7 LOAD_CONST 1 (1)
10 PRINT_ITEM
11 PRINT_NEWLINE
12 JUMP_FORWARD 1 (to 16)
15 POP_TOP
16 LOAD_CONST 0 (None)
19 RETURN_VALUE
Folding the two POP_TOPs into the jump would shorten both code length
and execution time. For an 'if' without 'elif' or 'else', it would
also eliminate the need to JUMP_FORWARD over the now not-there second
POP_TOP, giving even more saving.
Terry J. Reedy
Neal Norwitz
> <emailed and posted>
> "Michael Hudson" <m...@python.net> wrote ...
> http://www.python.org/doc/current/lib/bytecodes.html
>
> Thanks. I read this, found a couple of typos, and sent note to
> python-doc (don't have Source Forge login).
>
> Can you or someone answer a question regarding the following from the
> above:
> "
> JUMP_IF_TRUE delta
> If TOS is true, increment the byte code counter by delta. TOS is left on
> the stack.
>
> JUMP_IF_FALSE delta
> If TOS is false, increment the byte code counter by delta. TOS is not
> changed.
> "
>
> Why isn't TOS removed? Or, at least, why isn't there alternative
> bytecode that does so? Seems like most (all?) usages remove it anyway
> on both branches. For example (from your python-dev post)
I think it was a mistake or limitation from a long time ago.
Not sure, I wasn't around then.
However, the problem is known. When I was playing with removing it,
4 out of the six uses could be modified easily. But there
were 2 places (I think it was with code like: if x or y and z:)
which couldn't be fixed easily. The compiler is being rewritten
and hopefully fixing this will be easy once the rewrite is complete.
Neal
Michael Hudson
> Can you or someone answer a question regarding the following from the
> above:
> "
> JUMP_IF_TRUE delta
> If TOS is true, increment the byte code counter by delta. TOS is left
> on the stack.
>
> JUMP_IF_FALSE delta
> If TOS is false, increment the byte code counter by delta. TOS is not
> changed.
> "
>
> Why isn't TOS removed?
Haven't thought about this in a while, but I think it's for chained
comparisons, e.g. code like:
if 10 < a < 20:
...
> Or, at least, why isn't there alternative bytecode that does so?
Avoiding proliferation of opcodes? Hard to say whether this would be
worth it without trying, and I'm unlikely to get around to trying it
soon...
> Seems like most (all?) usages remove it anyway on both branches.
Not quite all, I think.
Cheers,
M.
--
>> REVIEW OF THE YEAR, 2000 <<
It was shit. Give us another one.
-- NTK Now, 2000-12-29, http://www.ntk.net/
Tim Peters
> Can you or someone answer a question regarding the following from the
> above:
> "
> JUMP_IF_TRUE delta
> If TOS is true, increment the byte code counter by delta. TOS is left
> on the stack.
>
> JUMP_IF_FALSE delta
> If TOS is false, increment the byte code counter by delta. TOS is not
> changed.
> "
>
> Why isn't TOS removed?
[Michael Hudson]
> Haven't thought about this in a while, but I think it's for chained
> comparisons, e.g. code like:
>
> if 10 < a < 20:
> ...
That may be its only use; as Terry suggested later, in other cases both
tails of the possible branch have to waste time with a POP_TOP. I've been
complaining about this for years, but it's never bubbled to the top of the
stack. Note that any change here would require bumping the PVM version
number.
> ...
> Hard to say whether this would be worth it without trying,
It would be a small win (since it does so little "real work", POP_TOP is
almost pure eval-loop overhead).
> and I'm unlikely to get around to trying it soon...
You're in good company <wink>.
Christos TZOTZIOY
<tjr...@udel.edu> might have written:
>Can you or someone answer a question regarding the following from the
>above:
>"
>JUMP_IF_TRUE delta
>If TOS is true, increment the byte code counter by delta. TOS is left
>on the stack.
>
>JUMP_IF_FALSE delta
>If TOS is false, increment the byte code counter by delta. TOS is not
>changed.
>"
As Tim, Neal and Michael have also noticed, there are two cases
(shortcut 'and' - 'or' comparisons and multiple comparisons, eg. 0<x<10)
where the TOS value must not be popped.
There are two quick-and-dirty ways to overcome this possibility, both
introducing new opcodes (affecting opcodes.h, ceval.c, compile.c):
1: introduce JUMP_IF_FALSE_POP and JUMP_IF_TRUE_POP (both doing a test
and then a pop). They can be used in the following compile.c functions:
com_list_if, com_assert_stmt, com_if_stmt, com_while_stmt,
com_try_except. The relevant POP_TOP's can be safely removed then, and
so can two JUMP_FORWARD's.
2: change JUMP_IF_FALSE and JUMP_IF_TRUE to always pop (and remove all
JUMP_IF related POP_TOP's), but then you introduce two opcodes:
STORE_TRUE and STORE_FALSE (STORE_TRUE would PUSH(Py_True);
Py_INCREF(Py_True); and samewise STORE_FALSE), which should be inserted
at the end of com_test and com_and_test respectively.
The slightly more tough point here is com_comparison, where if you reach
the "if (anchor)" point, instead of ROT_TWO and then POP_TOP, you must
POP_TOP and then PUSH_FALSE (since you get to that point *always* after
a JUMP_IF_FALSE) to let the False value pass through to the next opcode.
I agree (without any thorough scientific backing :) that the savings
should not be that great; I'll give it (method 2) a try though, and
report back... won't make any promises :)
PS An opcode optimizing pass would be interesting... stuff like
STORE_FAST n and then immediate LOAD_FAST n, which seem often to be used
in loops with calculations using intermediate variables, are good
candidates, but in many cases (like my example) would need new opcodes
(STORE_FAST_NOPOP).
--
TZOTZIOY, I speak England very best,
Real email address: 'dHpvdEBzaWwtdGVjLmdy\n'.decode('base64')
Michael Hudson
> As Tim, Neal and Michael have also noticed, there are two cases
> (shortcut 'and' - 'or' comparisons and multiple comparisons, eg. 0<x<10)
> where the TOS value must not be popped.
> There are two quick-and-dirty ways to overcome this possibility, both
> introducing new opcodes (affecting opcodes.h, ceval.c, compile.c):
>
> 1: introduce JUMP_IF_FALSE_POP and JUMP_IF_TRUE_POP (both doing a test
> and then a pop). They can be used in the following compile.c functions:
> com_list_if, com_assert_stmt, com_if_stmt, com_while_stmt,
> com_try_except. The relevant POP_TOP's can be safely removed then, and
> so can two JUMP_FORWARD's.
This is indeed the easy option.
> 2: change JUMP_IF_FALSE and JUMP_IF_TRUE to always pop (and remove all
> JUMP_IF related POP_TOP's), but then you introduce two opcodes:
> STORE_TRUE and STORE_FALSE (STORE_TRUE would PUSH(Py_True);
> Py_INCREF(Py_True); and samewise STORE_FALSE), which should be inserted
> at the end of com_test and com_and_test respectively.
> The slightly more tough point here is com_comparison, where if you reach
> the "if (anchor)" point, instead of ROT_TWO and then POP_TOP, you must
> POP_TOP and then PUSH_FALSE (since you get to that point *always* after
> a JUMP_IF_FALSE) to let the False value pass through to the next opcode.
Err, don't really follow that (but didn't try too hard).
Can't you get the same effect by selectively emitting a DUP_TOP before
the JUMP_IF_* opcodes? This may be hard to graft into the way the
compiler currently works, I guess.
> I agree (without any thorough scientific backing :) that the savings
> should not be that great; I'll give it (method 2) a try though, and
> report back... won't make any promises :)
Look forward to seeing it!
> PS An opcode optimizing pass would be interesting...
There are a couple of these around, you know -- one in bytecodehacks,
one by Skip Montanaro, but both are somewhat out of date wrt modern
Python.
Cheers,
M.
--
ZAPHOD: You know what I'm thinking?
FORD: No.
ZAPHOD: Neither do I. Frightening isn't it?
-- The Hitch-Hikers Guide to the Galaxy, Episode 11
Christos TZOTZIOY
<m...@python.net> might have written:
>> I agree (without any thorough scientific backing :) that the savings
>> should not be that great; I'll give it (method 2) a try though, and
>> report back... won't make any promises :)
>
>Look forward to seeing it!
Well, I implemented four new opcodes (changing opcodes.h, ceval.c and
dis.py) in order to be able to play freely with compile.c. I can post
the diff outputs for anyone interested (if also it's ok in the
newsgroup, they're not long anyway), although the compile.c is not
"final" --the version I have now has only com_list_if, com_assert_stmt,
com_if_stmt and com_while_stmt touched-- but the build is stable and
passes all tests (at least test.testall, plus some checks of my own).
Bytecodes of .py[co] files are somewhat shortened, but pystone timings
are only marginally better, as expected.
At least the bytecode seems a little bit more tidy :)