bootstrapping on machines without Python
Jonathan Hartley
the idea of writing, in C or similar, a compiled stand-alone
executable 'bootstrapper', which:
1) downloads and install a Python interpreter if none exists
2) runs the application's Python source code using this interpreter.
An application source code could be distributed with this small exe to
wrap its top level 'main.py', and the application could then be run on
machines which do not (yet) have Python installed.
The goal of this is to provide a way for end-users to download and
double-click a Python application, without having to know what Python
is or whether (an appropriate version of) it is installed. This method
has some disadvantages compared to using py2exe et al, but it has the
advantage that a small application would remain small: there is no
need to bundle a whole interpreter with every application. From an
advocacy point of view, it also must help that merely running such an
application would seamlessly put a usable Python interpreter installed
and on the PATH on Windows boxes.
Even my very limited understanding of the issues is enough to see that
the idea is far from trivial. I'm aware that great minds are working
on related and overlapping problems, so I thought I'd ask whether many
people have done this already, and if not, is there any value in
taking the trivial first step described above? ie. Ignore all
complications, and write a simple C program to download & install an
interpreter, then use that to run my Python.
In the long run, to be useful for real projects, the bootstrapper
would need to manage some nasty details:
* different versions of the interpreter for different applications
* download required packages
* manage conflicting versions of packages
I'm hoping that these thorny problems will be solved, if they aren't
already, by the great minds working on packaging, etc.
I'm very aware that I don't know anything at all about this, compared
to many on the list. Be gentle with me. :-)
Tim Golden
> While examining py2exe et al of late, my thoughts keep returning to
> the idea of writing, in C or similar, a compiled stand-alone
> executable 'bootstrapper', which:
> 1) downloads and install a Python interpreter if none exists
> 2) runs the application's Python source code using this interpreter.
Thinking aloud about what you're describing here... Assuming Windows,
as your starting point is py2exe and I imagine that most Unix-like
boxes already have Python on them.
Step 1: The user downloads a tiny myapp.exe which is basically a zip of the
myapp package / files plus an .exe header which will...
Step 2a: ... download a minimised? (ie custom-built) Python interpreter
and stdlib bundle which is just enough to run the app. Or...
Step 2b: ... download and install the official python.org Windows
installer for version x.y identified as the highest known to run
this app if...
Step 2bi) ... that version of the interpreter isn't already installed
and registered on that machine (at least: for that user).
My step 2a seems to be little better than a bundled py2exe, so I can
only assume you mean Step 2b, especially given your comment below
about ending up with an installation of Python where one wasn't
before. This, though, seems fraught with accusations of backdoor
installations etc.
Have I misunderstood you? For the Record, I'm entirely in favour of moves
to make Python use on Windows more seamless, attractive, consistent,
etc. I was going to comment positively on your recent PyChooser widget
and to plug a similar but unpublished one of my own. But I'm not sure
if this particular proposal has enough wings to make it fly.
TJG
Martin P. Hellwig
> While examining py2exe et al of late, my thoughts keep returning to
> the idea of writing, in C or similar, a compiled stand-alone
> executable 'bootstrapper', which:
> 1) downloads and install a Python interpreter if none exists
> 2) runs the application's Python source code using this interpreter.
I can see two distinctive scenarios:
- Local temporarily installation
- System wide installation (possible conflicts with older versions)
Anyway this reminds me of the web installers where you only download the
installer which download the rest from the net, like they used to be
available from adobe and for java.
It should be doable, for example you have an executable which has only
one function; download a second stage installer from a fixed location.
That second stage installer is a bit bigger and knows how to check for
dependencies for that particular platform and install other dependencies.
So you can chain a lot of installers after each other, for example the
second stage installer can be a python installer then you could do all
the remaining installing from the convenience of python.
--
MPH
http://blog.dcuktec.com
'If consumed, best digested with added seasoning to own preference.'
M.-A. Lemburg
> While examining py2exe et al of late, my thoughts keep returning to
> the idea of writing, in C or similar, a compiled stand-alone
> executable 'bootstrapper', which:
> 1) downloads and install a Python interpreter if none exists
> 2) runs the application's Python source code using this interpreter.
>
> An application source code could be distributed with this small exe to
> wrap its top level 'main.py', and the application could then be run on
> machines which do not (yet) have Python installed.
>
> The goal of this is to provide a way for end-users to download and
> double-click a Python application, without having to know what Python
> is or whether (an appropriate version of) it is installed. This method
> has some disadvantages compared to using py2exe et al, but it has the
> advantage that a small application would remain small: there is no
> need to bundle a whole interpreter with every application.
There are two major issues with such an approach:
* users will not necessarily like it if a small application
downloads a several MB from the web somewhere without asking
* installing applications on Windows typically requires admin
rights or at least user interaction
I think a good solution to the problem is wrapping the application's
main.py in a batch file which pops up a window to let the user know
that s/he will need to install Python first and point him/her to the
www.python.org web site, if necessary.
Another aspect to consider is whether a py2exe wrapped Python
application is really too big at all.
We ship a rather complex application using py2exe (mxODBC Connect Server)
and the resulting installed footprint is just around 14 MB - that's
roughly the size of a typical YouTube video.
By comparison, a full Python 2.5 installation is around 84 MB and
includes lots of stuff normally not needed for a small
Python application.
> From an
> advocacy point of view, it also must help that merely running such an
> application would seamlessly put a usable Python interpreter installed
> and on the PATH on Windows boxes.
>
> Even my very limited understanding of the issues is enough to see that
> the idea is far from trivial. I'm aware that great minds are working
> on related and overlapping problems, so I thought I'd ask whether many
> people have done this already, and if not, is there any value in
> taking the trivial first step described above? ie. Ignore all
> complications, and write a simple C program to download & install an
> interpreter, then use that to run my Python.
>
> In the long run, to be useful for real projects, the bootstrapper
> would need to manage some nasty details:
> * different versions of the interpreter for different applications
> * download required packages
> * manage conflicting versions of packages
> I'm hoping that these thorny problems will be solved, if they aren't
> already, by the great minds working on packaging, etc.
>
> I'm very aware that I don't know anything at all about this, compared
> to many on the list. Be gentle with me. :-)
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Nov 13 2009)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
Thomas Heller
> Jonathan Hartley wrote:
>> While examining py2exe et al of late, my thoughts keep returning to
>> the idea of writing, in C or similar, a compiled stand-alone
>> executable 'bootstrapper', which:
>> 1) downloads and install a Python interpreter if none exists
>> 2) runs the application's Python source code using this interpreter.
>>
>> An application source code could be distributed with this small exe to
>> wrap its top level 'main.py', and the application could then be run on
>> machines which do not (yet) have Python installed.
>>
>> The goal of this is to provide a way for end-users to download and
>> double-click a Python application, without having to know what Python
>> is or whether (an appropriate version of) it is installed. This method
>> has some disadvantages compared to using py2exe et al, but it has the
>> advantage that a small application would remain small: there is no
>> need to bundle a whole interpreter with every application.
>
> There are two major issues with such an approach:
>
> * users will not necessarily like it if a small application
> downloads a several MB from the web somewhere without asking
>
> * installing applications on Windows typically requires admin
> rights or at least user interaction
>
> I think a good solution to the problem is wrapping the application's
> main.py in a batch file which pops up a window to let the user know
> that s/he will need to install Python first and point him/her to the
> www.python.org web site, if necessary.
Here is a batch file that simulates a similar approach,
for demonstration purposes it doesn't download and install python,
instead it simply copies 'c:\python25\python.exe' to the current directory
under a different name (if it is not yet present), and executes the script with it.
Batchfile experts should also be able to improve other things:
<snip>
@echo off && if exist py_exe.exe (py_exe.exe -x %0 %* && goto exit) else (goto download)
# The python script
import sys
print "Hi, this is Python", sys.version
sys.exit()
# rest of the batch file
"""
:download
echo Simulating download...
copy c:\python25\python.exe py_exe.exe
echo Done.
REM start the script again after downloading
%0 %*
:exit
rem """
</snip>
Thomas
mma...@gmx.net
Jonathan Hartley <tar...@tartley.com> wrote:
> Even my very limited understanding of the issues is enough to see that
> the idea is far from trivial.
[...]
> In the long run, to be useful for real projects, the bootstrapper
> would need to manage some nasty details:
> * different versions of the interpreter for different applications
> * download required packages
> * manage conflicting versions of packages
An easy approach could be to use the portable apps version of Python.
(I assume that you use Windows.)
After installing portable Python on your computer, your application
and any additional libraries are put into the site-packages / Scripts
folders. Next, the install folder is zipped and provided for download.
A batch script, which is downloaded and run by the user, downloads the
zip file and extracts it. The script executes python.exe with your
Python script as command line argument.
In this scenario, the user would not need Administrator privileges.
You could also get around compatibility problems by providing
preferred versions of additional libraries in the site-packages folder
of portable Python.
However, the download size would probably exceed 100 MB. I also do not
know if there are licensing issues using portable Python like this.
Martin
Jonathan Hartley
Hi Tim. Thanks for that.
Yes, you have understood pretty much perfectly. I was envisaging
something like 2b.
I'm very much enamored of creating cross-platform apps, but I'm
focused on deployment on Windows first, because this is where most
users are, and also where the problem seems to need the most work.
Your input is very much appreciated. It may be that you are right that
I haven't thought this through clearly enough.
Incidentally, I'd love to see your approach of dealing with the
problem that pychoose addresses - Since creating it, I'm gripped with
paranoia that it is failing to take account of lots of things which
will trip me up later (or worse- trip other people up too, if anyone
uses it)
Jonathan Hartley
> > Even my very limited understanding of the issues is enough to see that
> > the idea is far from trivial.
Thanks heaps for the input from everyone. Martin Lemburg's 'chained'
approach does sound like the smart way to do it, and Thomas does
demonstrate pretty much the simplest possible example of what I'm
thinking of. Martin Manns' portable Python sounds useful too, and is
not disimilar to a suggestion made by Michael Foord off list.
However, the problems that Tim, Martin, Marc and Martin point out do
seem very real. I think users could be placated about 'backdoor
installation' if we tell them what's going on, with an OK button. But
I confess I wasn't aware that a full Python install is quite so large
compared to the bundle produced by py2exe et al.
Perhaps this idea is overly idealistic then - I was maybe transfixed
by the 'needless consistency' of sharing a single interpreter between
many applications.
Thanks for helping me straighten out my thoughts on the subject.
Best!
Jonathan