Backwards emulation rather than backwards compatibility?
Antaeus Feldspar
scripts!" vs. "You're holding back the language!" and I can understand
both sides. While I do not think there is a perfect solution, I was
struck the other day by a possible way to bridge the gap.
Could the python interpreter be equipped with a mode that would allow it
to emulate past interpreters? Thus, if someone needed access to a
script that was designed for 1.5.2, they could pass a "--1.5.2" argument
to the interpreter and have it run under 1.5.2's rules; more to the
point, if a script's author wanted to be sure that their script could
run on anything from 1.5.2 to 2.3, they could do it all with a single
interpreter, rather than have to have a separate interpreter for
checking each compatibility.
-jc
Martin v. Loewis
> Could the python interpreter be equipped with a mode that would allow
> it to emulate past interpreters?
No, that won't be possible. It is easy enough to install multiple
versions of the interpreter, though.
Regards,
Martin
Kragen Sitaker
> Could the python interpreter be equipped with a mode that would allow it
> to emulate past interpreters? Thus, if someone needed access to a
> script that was designed for 1.5.2, they could pass a "--1.5.2" argument
You'd really want it to be a per-module declaration, at which point it
becomes 'from __future__ import ...'.
Tim Churches
I posted this some time ago:
> I sometimes use a statistics package called Stata. It has been around
> for well over a decade, with major upgrades to the core language released
> about every two years or so - hence there are three or four versions in
> common use. In order to allow users who are still running Stata V4
> to write routines which will behave correctly in Stata V7 (bit not vice
> versa), Stata has a "version" statement. If I am running Version 7, I can put
> something like
> "USE VERSION 4" (I don't recall the exact syntax) at the top of my
> programs and the program will be executed as if it were running in Stata V4.
> Hence, a user who still runs Stata V4 can put "USE VERSION 4" in every program and be
> sure these will work correctly when run in all subsequent versions of Stata.
>
> This is the equivalent, I think, of having a "from __past__ import *"
> statement in Python, or more precisely, a "from __v1.5.2__ import *" statement.
>
> Obviously it is impossible to retrospectively provide support for such a
> statement, but if there were a "from __v2.2.2__ import *" statement, one could put
> that at the beginning of every Python module one writes and be certain that the Bool
> type or any other newfangled feature would not trip up one's code at any time in the
> future, no matter how recent the version of Python used to execute it. Similarly a user of a
> recent version could verify that their code runs OK with older versions by using such a
> statement, which would of course disable all new language features and library modules, as well
> as making any new language behaviour revert to the old.
>
> However, it may not be possible to avoid bloat in the Python core because successive
> versions of Python would need to support more and more old behaviour, but if this backwards support
> could be relegated to libraries, it shouldn't be too bad. Bloat in the Python core in
> order to support historical behaviour would be a lot less desirable, though.
>
> And of course, such support for historical features and behaviour may also
> result in ever increasing intervals between releases of new versions of Python, as the
> developers struggle to accomodate ever more historical baggage.
>
> Hence I don't think that any of the above is a good idea.
>
> Tim C
But Martin v. Loewis points out:
> ...It is easy enough to install multiple
> versions of the interpreter, though.
Indeed it is (unlike many other languages which make having
multiple versions installed a nightmare), and these days the
extra disc space used by multiple installations is not a concern. So...
How about a Python meta-bytecode-compiler/interpreter which examines
Python source code and/or bytecode and automatically sends the code to
the appropriate installed version of Python, resetting PYTHONPATH and
any other aspects of the environment appropriately?
Parsing the entire source and trying to infer which version of Python is
appropriate would probably be too slow (and a big job to write...),
so maybe an #ifdef-style directive could be put at the top of each
Python script which triggers the appropriate behaviour, such as
import __v1.5.2__
The module __v1.5.2__ would do nothing, but that line in the source
would direct the meta-Python to use the correct version.
I'm not sure how such a meta-interpreter would deal with already
compiled
bytecode - is there a version stamp in the bytecode?
If someone wants to write such a thing, I'd be happy to help test it...
Tim C
John Roth
"Martin v. Loewis" <mar...@v.loewis.de> wrote in message
news:m3it5cp...@mira.informatik.hu-berlin.de...
Oh, come on now. Anything is possible, the question is
whether it is worth doing and (this being a volunteer effort)
whether anyone wants to do it.
Frankly, I think it's an excellent idea, although it would
have a cost.
To delimit the debate: either a specific statement at the
front of the module "Requires 2.1" or the code in the
compiled module indicates the language level needed.
Also, it should be possible to mix levels in one program,
the limitation being that any one module is interpreted
under one set of rules. No mixing is allowed within a
module - that way lies insanity for the next person who
has to deal with the resulting mess. It also means that
the interpreter does not have to do very much different
when compiling a module: if you want to compile a module
under a back level, Python automatically invoked the
appropriate level of the compiler, producing the
appropriate level of compiled module.
Frankly, I wouldn't think it would be that hard,
although I have absolutely no knowledge of the
internals of the python interpreter. Everything, after all,
is an object, so an int created under 2.0 would not
necessarilly have the same execution routines that an
int created under 2.3 would have.
Globals would have to be dealt with, and so would
imports from back level modules (it might be useful
to insure that they didn't import up level modules.)
John Roth
>
Christopher Browne
> I'm not sure how such a meta-interpreter would deal with already
> compiled bytecode - is there a version stamp in the bytecode?
Wouldn't it be easier to simply change the first line to:
#!/usr/bin/python1.5
????
--
(concatenate 'string "chris" "@cbbrowne.com")
http://www.cbbrowne.com/info/
Rules of the Evil Overlord #71. "If I decide to test a lieutenant's
loyalty and see if he/she should be made a trusted lieutenant, I will
have a crack squad of marksmen standing by in case the answer is no."
<http://www.eviloverlord.com/>
Chris Liechti
news:mailman.1022364102...@python.org:
...
> But Martin v. Loewis points out:
>> ...It is easy enough to install multiple
>> versions of the interpreter, though.
>
> Indeed it is (unlike many other languages which make having
> multiple versions installed a nightmare), and these days the
> extra disc space used by multiple installations is not a concern. So...
>
> How about a Python meta-bytecode-compiler/interpreter which examines
> Python source code and/or bytecode and automatically sends the code to
> the appropriate installed version of Python, resetting PYTHONPATH and
> any other aspects of the environment appropriately?
>
> Parsing the entire source and trying to infer which version of Python is
> appropriate would probably be too slow (and a big job to write...),
> so maybe an #ifdef-style directive could be put at the top of each
> Python script which triggers the appropriate behaviour, such as
>
> import __v1.5.2__
>
> The module __v1.5.2__ would do nothing, but that line in the source
> would direct the meta-Python to use the correct version.
>
> I'm not sure how such a meta-interpreter would deal with already
> compiled
> bytecode - is there a version stamp in the bytecode?
>
> If someone wants to write such a thing, I'd be happy to help test it...
that idea might sound nice, but it's too late! the problem are _old_
scripts that nobody wants to touch and so there is no possibility to
include such an import statement (and even add a empty _v*_.py file to
satisfy the import). for the same reason a "from __past__ import x" is
useless. if an old script needs python 1.5.2, just change the first line to
#/usr/bin/env python15 (on unix, windows in an other plagu^H^H^H^Hroblem)
if you're right now writing a module that has to be backward compatible,
thats no problem, 'cause you can choose to only use features that exist in
all python versions from 1.5.2 to 2.2.1.
the problem is to write a _forward compatible_ script - but we all don't
know what the distant future wil bring up...
chris
--
Chris <clie...@gmx.net>
Neil Hodgson
> Also, it should be possible to mix levels in one program,
> the limitation being that any one module is interpreted
> under one set of rules.
Some of the language changes affect communication between modules. How do
you deal with changes in the language that affect the interface provided by
one module and consumed by another with a different language level? Say I
have some 1.52 code that uses introspection and point it at a 2.2 module
that uses the new property support? Or a 1.52 module asked to deal with
Unicode?
While most dependencies are of the form of new code using old code, it is
common for this use to include subclassing old code and other forms where
'use' is really bidirectional.
To me, providing a language level feature looks like a huge time sink,
with a need for exhaustive pairwise (at least!) cross level testing.
Neil
Tim Churches
>
> Quoth Tim Churches <tc...@optushome.com.au>:
> > I'm not sure how such a meta-interpreter would deal with already
> > compiled bytecode - is there a version stamp in the bytecode?
>
> Wouldn't it be easier to simply change the first line to:
> #!/usr/bin/python1.5
Yeah, but what about the value of environment variables such as
PYTHONPATH, PYTHONSTARTUP and PYTHONHOME? I would want these to be
automatically set to the appropriate values.
On Windows, I gather that these values are read from the Registry
under HKEY_LOCAL_MACHINE/Software/Python
Tim C
Chris Liechti
news:uf06o3f...@news.supernews.com:
> "Martin v. Loewis" <mar...@v.loewis.de> wrote in message
> news:m3it5cp...@mira.informatik.hu-berlin.de...
>> Antaeus Feldspar <feld...@ix.netcom.com> writes:
>>
>> > Could the python interpreter be equipped with a mode that would
> allow
>> > it to emulate past interpreters?
>>
>> No, that won't be possible. It is easy enough to install multiple
>> versions of the interpreter, though.
>>
>> Regards,
>> Martin
>
> Oh, come on now. Anything is possible, the question is
> whether it is worth doing and (this being a volunteer effort)
> whether anyone wants to do it.
what ever you define as "possible" - if the result is something so
complicated that nobody realy knows whats going on, would you say that it
is a favorable target?
> Frankly, I think it's an excellent idea, although it would
> have a cost.
>
> To delimit the debate: either a specific statement at the
> front of the module "Requires 2.1" or the code in the
> compiled module indicates the language level needed.
no it would require a statement "must not have newer features than 2.1"
it's the new features in the future that cause the problems.
(for me "requires" means "at least", "that or a newer one")
e.g. if you just say that some script requires 2.1 you can still use
"yield" as a name but it will fail in 2.3 becuase that name becomes a
keyword. so you effectively need to specify that the scripts needs 2.1 but
not newer, not older.
and this is best done using "#!/usr/bin/env python2.1"
no new invention needed, its already there...
chris
--
Chris <clie...@gmx.net>
Martin v. Loewis
> How about a Python meta-bytecode-compiler/interpreter which examines
> Python source code and/or bytecode and automatically sends the code to
> the appropriate installed version of Python, resetting PYTHONPATH and
> any other aspects of the environment appropriately?
On Linux, this feature is available today: the binfmt_misc mechanism
allows easy dispatching to interpreters based on the magic inside the
file. Most of the recent Python version vary in file magic, for the
byte code.
Regards,
Martin
Martin v. Loewis
> > > Could the python interpreter be equipped with a mode that would
> > > allow it to emulate past interpreters?
> > No, that won't be possible. It is easy enough to install multiple
> > versions of the interpreter, though.
> Oh, come on now. Anything is possible, the question is whether it is
> worth doing and (this being a volunteer effort) whether anyone wants
> to do it.
To fully emulate a past implementation, you'd have to know *all* it's
features. If you fix a bug, you somehow need to be able to preserve
the old library behaviour. I see no way to do this, except
incorporating old copies of the library. Same for the interpreter
itself, so you would also need to include copies of the old
interpreter.
Doing so woul mean that the directory structure of the library will
need to change - such changes are detectable by applications, so full
backwards compatibility would not achieved.
Likewise, the mere introduction of that "mode" would cause that Python
changes its behaviour, and might cause existing applications to break.
So if you want "full emulation", I maintain that this is not possible
to implement.
If you want "partial emulation" - this feature is available today; new
versions execute much old code correctly.
Regards,
Martin
John La Rooy
I think it would be more practical to develop tools to help identify and perhaps even fix old code.
Writing good unit tests will allow developers to be confident that a module is functioning correctly.
John
John Roth
> "John Roth" <john...@ameritech.net> writes:
>
> > > > Could the python interpreter be equipped with a mode that would
> > > > allow it to emulate past interpreters?
>
> > > No, that won't be possible. It is easy enough to install multiple
> > > versions of the interpreter, though.
>
> > Oh, come on now. Anything is possible, the question is whether it is
> > worth doing and (this being a volunteer effort) whether anyone wants
> > to do it.
>
> To fully emulate a past implementation, you'd have to know *all* it's
> features. If you fix a bug, you somehow need to be able to preserve
> the old library behaviour. I see no way to do this, except
> incorporating old copies of the library. Same for the interpreter
> itself, so you would also need to include copies of the old
> interpreter.
Do you really want to preserve incorrect behavior? Code that
depends on bugs is arguably incorrect code.
> Doing so woul mean that the directory structure of the library will
> need to change - such changes are detectable by applications, so full
> backwards compatibility would not achieved.
You'd certainly need something to identify versions. This could be
handled by a compatibility module.
>
> Likewise, the mere introduction of that "mode" would cause that Python
> changes its behaviour, and might cause existing applications to break.
"Might" isn't the same as 'would.'
> So if you want "full emulation", I maintain that this is not possible
> to implement.
>
> If you want "partial emulation" - this feature is available today; new
> versions execute much old code correctly.
However, that's not what the original poster asked for.
John Roth
John Roth
"Chris Liechti" <clie...@gmx.net> wrote in message
news:Xns921A11F2F5EE...@62.2.16.82...
More FUD. The fact that nobody wants to touch them doesn't
mean that nobody will touch them. What it means is that people
generally don't want to make changes because they were written
in an opaque manner, without clear objectives. That has nothing
to do with a mechanical addition of a first line.
John Roth
John Roth
"Neil Hodgson" <nhod...@bigpond.net.au> wrote in message
news:NyVH8.177231$o66.5...@news-server.bigpond.net.au...
> John Roth:
>
> > Also, it should be possible to mix levels in one program,
> > the limitation being that any one module is interpreted
> > under one set of rules.
>
> Some of the language changes affect communication between modules.
How do
> you deal with changes in the language that affect the interface
provided by
> one module and consumed by another with a different language level?
Say I
> have some 1.52 code that uses introspection and point it at a 2.2
module
> that uses the new property support?
It must return what the 1.5.2 code expects.
> Or a 1.52 module asked to deal with
> Unicode?
It would probably fail, since 1.5.2 doesn't support Unicode. Of course,
there are some methods on a unicode string (such as __len__) that
would work, but in general, if the method didn't exist, why is the old
code calling it?
Neil Hodgson
> > Unicode?
>
> It would probably fail, since 1.5.2 doesn't support Unicode. Of course,
> there are some methods on a unicode string (such as __len__) that
> would work, but in general, if the method didn't exist, why is the old
> code calling it?
Because it was handed an argument and the assumptions it makes for that
argument are that it is of a type that support the operations the old code
will apply to the argument. In the current situation with one language level
used, much code is unconcerned by actual types and gets by with relying on
which methods are present. When I added some support for Unicode file names,
much code including glob, worked quite well even though it was not expecting
Unicode.
With your multiple level concept, the exact semantics that you desire
from passing Unicode objects between levels has to be defined. One technique
is to say that since level 1.52 does not support Unicode, then they must be
converted to narrow strings by cross level marshalling code. Another
approach would be to say that they are treated as objects of a user defined
class by the 1.52 level code.
Interesting questions involving levels include whether operations are
completely determined by the lexical (module) context they are executed in
or by the home context where the object was created.
Neil
Chris Barker
Christopher Browne wrote:
> Wouldn't it be easier to simply change the first line to:
> #!/usr/bin/python1.5
Only on *nix....
I'd love to see a platform independent way of doing this. Maybe all we
need to do is have the Python interpreter front-end look for that
classic *nix line, and then dispatch the script to the correct
interpreter for you. And have it work the same way on all platforms.
Yes, it's too late to have code like this in old scripts, but it only
gets worse with each new version. This was brought up two versions ago
(2.0). Also, while someone would have to go back and add a line to all
their old scripts, this is a whole lot less work than checking each for
incompatability with new features. OF course, that being said, RedHat
seems to be incapable of adding a "1.5" to all their config scripts, so
maybe there is more to it than I imagine.
All we need is a very simple dispatching front end, that looks for a key
at the top of the __main__ script, and invokes the correct interpreter.
A sysadmin could decide how many versions they want to support, but it
should be easy to have a half a dozen non each machine. This really is a
problem that needs to be addressed, and it just seems so easy!
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Dave Kuhlman
on today:
I typically start my scripts with a first line like the following:
#! /usr/bin/env python
Quite a few files in Python2.2.1/Lib do likewise.
But, when I tried to add an option to be passed to the Python
interpreter by changing the first line to:
#!/usr/bin/env python -t
I get an error message:
/usr/bin/env: python -t: No such file or directory
Shouldn't I be able to pass a flag to the Python interpreter?
Looking at the env man page did not help. And, quotes around
"python -t" did not help. Is there another "trick"?
So I guess I am just asking for a solution to one more problem in
addition to the one Chris is describing.
I'm on Linux, by the way.
- Dave
Chris Barker wrote:
--
Dave Kuhlman
dkuh...@rexx.com
http://www.rexx.com/~dkuhlman
Oleg Broytmann
> I typically start my scripts with a first line like the following:
>
> #! /usr/bin/env python
>
> Quite a few files in Python2.2.1/Lib do likewise.
>
> But, when I tried to add an option to be passed to the Python
> interpreter by changing the first line to:
>
> #!/usr/bin/env python -t
>
> I get an error message:
>
> /usr/bin/env: python -t: No such file or directory
>
> Shouldn't I be able to pass a flag to the Python interpreter?
> Looking at the env man page did not help. And, quotes around
> "python -t" did not help. Is there another "trick"?
There is no.
Parsing the #! header is the job of the OS kernel, and all unicies do it
in the very simple way - they split the header into an interpreter
(/usr/bin/env, in this case) and its flags (python -t). Flags ARE NOT
SUPPOSED to have spaces, and are apssed to the interpreter as is.
So the only way is to write
#! /usr/local/bin/python -O
as I always do.
If you want to distribute the script and worry about portability problem
- write setup.py and use "script" option. Distutils rocks!
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ p...@phd.pp.ru
Programmers don't die, they just GOSUB without RETURN.
Berthold Höllmann
> On Tue, May 28, 2002 at 02:43:53PM -0700, Dave Kuhlman wrote:
> > I typically start my scripts with a first line like the following:
> >
> > #! /usr/bin/env python
> >
> > Quite a few files in Python2.2.1/Lib do likewise.
> >
>
> Parsing the #! header is the job of the OS kernel, and all unicies do it
> in the very simple way - they split the header into an interpreter
> (/usr/bin/env, in this case) and its flags (python -t). Flags ARE NOT
> SUPPOSED to have spaces, and are apssed to the interpreter as is.
>
> So the only way is to write
>
> #! /usr/local/bin/python -O
>
> as I always do.
>
> If you want to distribute the script and worry about portability problem
> - write setup.py and use "script" option. Distutils rocks!
Only as long as you don't fiddle with the installation to support
multiple architectures. If you install Python using different
argunents for the configure options --prefix and --exec-prefix to put
all achitecture independent files into one directory, installing
scripts using distutils becomes a problem. Distutils will replace the
#! line with the path to the interpreter for the architecture you are
doing the install, but installs the script into the acrchitecture
independent bin directory. That is a bad idea. Installing them into
the architecture dependent directory is not much better when the
script is subject to changes because the install has to be done for
all architectures you support, but the idea of the
--prefix/--exec-prefix was to make support for different architectures
easier.
Greetings
Berthold
--
bh...@web.de / http://starship.python.net/crew/bhoel/
It is unlawful to use this email address for unsolicited ads
(USC Title 47 Sec.227). I will assess a US$500 charge for
reviewing and deleting each unsolicited ad.
James T. Dennis
> "Martin v. Loewis" <mar...@v.loewis.de> wrote in message
> news:m3it5cp...@mira.informatik.hu-berlin.de...
>> Antaeus Feldspar <feld...@ix.netcom.com> writes:
>>
>>> Could the python interpreter be equipped with a mode that would
> allow it to emulate past interpreters?
Why emulate, why not exec the old interpreter with the open filehandle
of the script that's to be interpreted?
>> No, that won't be possible. It is easy enough to install multiple
>> versions of the interpreter, though.
Ever since I saw "from __future__ import" I've thought that Python
was crying out for a "from __past__ import"
... though I have no real idea what that would mean.
(The big issue would be when you want to mix modules; run a 1.5.2
module in a 2.x interpreter, etc. If there was a good "conduit" to
pass data between the two interpreters it might work (though we'd
have two processes, and the "newer" version would be responsible for
pickling up objects and passing them through some IPC to be unpickled
on the other side; and some objects (generators) would have to be
passed as proxies (the other side would call on them; pass the arguments
back to the other intpreter and get the results back.
This is probably technical possible, but I'm not sure the idea is worth
any effort; it probably would have inevitable limitations that would make
it less than transparent --- at which point upgrading the programs and
modules to be compatible with the new version looks more attractive.