Add .par functionality (like Java .jar)
Johann Höchtl
logged in, i post it here:
Summary:Add .par functionality (like Java .jar)
Detailed Description:
This feature would make people extremely happy to be
able to download a single file, double-klick (either on
windows or gnome/kde) and see what python can do for them.
(Assuming, that a python interpreter is installed)
PEP 273 does only adress loading modules from a zip
file, not the actual bootstraping of the application.
The pypar project is aiming into the right direction ...
Paul Moore
> As SourceForge doesn't allow to add a featur erequest without beeing
> logged in, i post it here:
Why not log in? It's not hard...
> Summary:Add .par functionality (like Java .jar)
>
> Detailed Description:
> This feature would make people extremely happy to be
> able to download a single file, double-klick (either on
> windows or gnome/kde) and see what python can do for them.
>
> (Assuming, that a python interpreter is installed)
This has come up a number of times. Most of the pieces are already in
place. You can write an import hook now, to allow import from Zip
files. PEP 273 formalises this and makes it part of Python, but that's
all.
Loading data from a zip file is a little harder, but here's a class
that (mostly) works.
class Resource:
def __init__(self, filename):
self.zip = ZipFile(filename)
def __del__(self):
self.zip.close()
def read(self, filename):
return self.zip.read(filename)
def open(self, filename):
return StringIO(self.zip.read(filename))
Just use as
z = Resource("test.zip")
Then to get the complete data from a particular file in the zip, just
do data = z.read("filename") or to get a file-like object, just do f =
z.open("filename"). You have problems with stuff that needs a *real*
file, but there isn't much around that you can't work round one way or
another...
The main lack is the ability to handle compiled extensions, but you're
never going to get that without OS support. Just ship the necessary
.dll/.so files separately, and manipulate sys.path, if you need to.
For true one-file-to-run functionality like JAR files, all you really
need is to package the main driver script plus the zip file
together. As zip files can have arbitrary content tacked on the front,
just concatenating the script and the zip file should
work. Unfortunately, the Python interpreter chokes on trailing garbage
:-(
You can work around *this* on Windows by embedding a CTRL-Z character
(Python opens scripts in text mode, and CTRL-Z is EOF in text mode)
but I don't know how to do it on Unix.
So the only remaining pieces of the puzzle are:
1. Find a way of combining a driver script plus a zip file.
2. Formalise this, if you want, to come up with a "standard".
There are a number of options for part 1 (custom executable with
Python embedded, encode the zipfile as text and decode on the fly at
script startup, etc etc). Gordon McMillan's Installer does all this,
but it takes it a few steps further, by searching out dependencies,
rather than relying on an installed Python interpreter. So there are
ideas there, if you want to look.
It would be nice to have an option in the existing Python interpreter,
but I'm not sure it's needed. With PEP 273, all you need may be
python -c "import sys; sys.path.insert(sys.argv[1]); import Main; Main.main()" myfile.zip
Part 2 is the important one. If you really care about making this a
*standard* feature, you have to persuade people to do it in the same
way each time, rather than reinventing the wheel. Personally, I would
like to see one-file Python applications which worked like jar files
become common - it irks me that people view this as something clever
that only Java can do - but I'm not motivated enough to push what is
essentially a "political" issue rather than a technical one.
So, in summary: You have my support, but it's not so much a feature
request as an (informational) PEP describing a standardised
technique. If the standard becomes popular, I can see Python growing a
command line switch to simplify the process, just like Java did
(executable Jars came with JDK 1.2, IIRC, but jarfile technology was
there from day 1...)
Hope this helps,
Paul.
Kragen Sitaker
> The main lack is the ability to handle compiled extensions, but you're
> never going to get that without OS support. Just ship the necessary
> .dll/.so files separately, and manipulate sys.path, if you need to.
Maybe on Windows, but you don't need OS support on Linux. The
.so-loading equipment on Linux is purely user-level code --- it just
uses mmap() to map in the files. This is done in elf/dl-open.c and
elf/dl-load.c in glibc.
Johann Höchtl
I was not thinking about any external ressources a python program will
eventually require like bitmapped data for graphics or architecture
dependent libraries (.so, dll).
> The main lack is the ability to handle compiled extensions, but you're
> never going to get that without OS support. Just ship the necessary
> .dll/.so files separately, and manipulate sys.path, if you need to.
>
> For true one-file-to-run functionality like JAR files, all you really
> need is to package the main driver script plus the zip file
> together. As zip files can have arbitrary content tacked on the front,
> just concatenating the script and the zip file should
> work. Unfortunately, the Python interpreter chokes on trailing garbage
> :-(
>
> You can work around *this* on Windows by embedding a CTRL-Z character
> (Python opens scripts in text mode, and CTRL-Z is EOF in text mode)
> but I don't know how to do it on Unix.
>
> So the only remaining pieces of the puzzle are:
>
> 1. Find a way of combining a driver script plus a zip file.
> 2. Formalise this, if you want, to come up with a "standard".
>
I spent some time to see how it is done in java. There, a standardized
file in a standardized directory within the jar(zip)-archive is read and
interpreted. This file can carry a bunch of information, for example the
startup-class or the CLASS-PATH which serves as the startpoint of execution.
http://java.sun.com/j2se/1.3/docs/guide/jar/jar.html
Once this file is loaded, it can import dependencies without knowing
whether it was started from the file system or jusing the 'jar-lever'.
I think you mean package the main .py- file with all the other modules
which are now in a zip file together and let the main.py file care about
it's imports.
IMHO this is bad design as the startup.py has to care whether is is
loading it's modules from the directory (and relying on standard python
mechanism) or from a python-zip package.
The consequence is that already python(.exe) has to have the knowledge
to inspect the archive and see if there is a startup-script in it.
I am a python beginner and i don't yet know if it would be a problem to
name this file __init__.py like packages in a directory.
> There are a number of options for part 1 (custom executable with
> Python embedded, encode the zipfile as text and decode on the fly at
> script startup, etc etc). Gordon McMillan's Installer does all this,
> but it takes it a few steps further, by searching out dependencies,
> rather than relying on an installed Python interpreter. So there are
> ideas there, if you want to look.
>
I visited the site. Packaging the whole interpreter is maybe still
necessary for MS-Windows, on UNIX-Systems python can already be assumed
to be available without having especially asked for it. (I wouldn't
wonder if SuSE or Slackware 8 give a dependency warning when i try to
deselect python) Reminds me of the time three years ago or so, when java
applications came with JRE pre-packaged. Nowadays it's assumed to be
installed.
> It would be nice to have an option in the existing Python interpreter,
> but I'm not sure it's needed. With PEP 273, all you need may be
>
> python -c "import sys; sys.path.insert(sys.argv[1]); import Main; Main.main()" myfile.zip
That's already near at the goal, but it's not self-contained. You have
to put this line into a unix semi-executable +x or create a shortcut in
a GUI.
>
> Part 2 is the important one. If you really care about making this a
> *standard* feature, you have to persuade people to do it in the same
> way each time, rather than reinventing the wheel. Personally, I would
> like to see one-file Python applications which worked like jar files
> become common - it irks me that people view this as something clever
> that only Java can do - but I'm not motivated enough to push what is
> essentially a "political" issue rather than a technical one.
>
> So, in summary: You have my support, but it's not so much a feature
> request as an (informational) PEP describing a standardised
> technique. If the standard becomes popular, I can see Python growing a
> command line switch to simplify the process, just like Java did
> (executable Jars came with JDK 1.2, IIRC, but jarfile technology was
> there from day 1...)
>
> Hope this helps,
Thank's a lot for your explanations. I am still a Python beginner. After
a year of language (re-)search I think I finally ended up with Python
and I'm happy that I didn't get stuck with Java ....
> Paul.
>
>
Paul Moore
> I think you mean package the main .py- file with all the other modules
> which are now in a zip file together and let the main.py file care
> about it's imports.
>
> IMHO this is bad design as the startup.py has to care whether is is
> loading it's modules from the directory (and relying on standard
> python mechanism) or from a python-zip package.
No it doesn't. Just add the zip file to sys.path. That's easy, and can
be done in the driver code.
> The consequence is that already python(.exe) has to have the knowledge
> to inspect the archive and see if there is a startup-script in it.
As I said, to make this "built in" python.exe would need to know about
the concept of "executable archives", just like java.exe knows about
executable jarfiles. But it's trivial to write a module (which would
have to be installed for this to work, obviously...) which
encapsulates the process, and which can be run via the -c argument to
python.exe.
Here's a proof of concept implementation:
--- put this in par.__init__.py somewhere on sys.path...
"""Support Python Archive (par) files
This module provides basic support for executable "archives" of Python
code and associated data.
Python archive file format:
Python archive files are basically just zip files. The only "special"
aspect is that the file must have a "main.py" file which is the entry
point to the archive.
Usage:
python -c "import par; par.run()" parfile arg arg arg...
"""
import zipfile as _zipfile
import sys as _sys
import os as _os
def run(parfile = None):
"""Run a par file. Use sys.argv to get a filename if none is given."""
# Get the name of the archive, and shift sys.argv one place
# to remove the "-c" entry. Special case for run being passed
# no filename. If a filename is passed, don't change sys.argv.
if parfile is None:
parfile = _sys.argv[1]
del _sys.argv[0]
# Register the parfile import hook (all the work is done on import)
import par.importer
# Install the parfile at the start of sys.path
_sys.path.insert(0, parfile)
# Get the entry point as metadata, and the content of the entry point
# Then run the entry point in the top level namespace
main = _zipfile.ZipFile(parfile).read("main.py")
main = main.replace("\r", "")
runcode(main, _os.path.join(parfile, "main.py"))
def runcode(code, filename):
"""Compile and run code, as if it came from filename, in the toplevel namespace"""
toplevel = _sys._getframe()
while toplevel.f_back:
toplevel = toplevel.f_back
codeobj = compile(code, filename, "exec")
exec codeobj in toplevel.f_globals, toplevel.f_locals
----------------------------------------------------
This code relies on a par.importer module, which is just an import
hook (using Gordon McMillan's iu.py import hook package) to allow
zipfiles as part of sys.path. When this is supported natively (Python
2.3) the par.importer module is no longer needed, and you have a
simple 35-line (or so) module.
> I am a python beginner and i don't yet know if it would be a problem
> to name this file __init__.py like packages in a directory.
That sounds reasonable. As you can see, the name is hard-coded in the
package above. You could do all sorts of things, such as having a
MANIFEST file containing the name of the startup file, just like Java
does. But that's trivial details...
> That's already near at the goal, but it's not self-contained. You have
> to put this line into a unix semi-executable +x or create a shortcut
> in a GUI.
But you have to do that with JAR files, too!!! OS-level magic to make
particular filetypes executable, and the command line to run to
execute them, is highly system-dependent (filetype associations on
Windows, binfmt_misc magic on Linux, whatever else...)
If you care that much, write a custom executable and concatenate it
onto the front of the zipfile. The zip format copes with this, and
many executable formats (certainly Windows, and I believe Linux too)
don't mind. But that involves C coding, which is too hard for an
evening's quick hack :-)
> Thank's a lot for your explanations. I am still a Python
> beginner. After a year of language (re-)search I think I finally ended
> up with Python and I'm happy that I didn't get stuck with Java ....
No problem. Like many things with Python, the reason that there is no
"standard" way of doing this sort of thing is that it's pretty trivial
to put something together for your own use, but almost impossible to
persuade the rest of the community that your 10-line hack is so much
better than theirs as to be acceptable as a "standard" :-)
The PEP process should be good for this sort of thing, but it does
need a fairly committed champion to push anything through. Heck, maybe
I'll try pushing this one, just to see if I can work out how to make
the process smoother... :-)
Paul.
Anurag Uniyal
will do the job
I am distributing python application using it
and it works fine!
Paul Moore <gus...@morpheus.demon.co.uk> wrote in message news:<u1oxr6...@morpheus.demon.co.uk>...