"static" installation from SCM checkout

[m h]

Hey Folks-

This might be semi weird. I've got a client that wants to be able to
run an application without installing per se. They've currently got
everything checked into an SCM, and want to be able to pull it down
and type "make serve" and have the server start running. They are
doing pure WSGI right now but would like to move to a web framework.

Has anyone done something like this with Pylons? (ie have Pylons and
it's dependencies in SCM, rather than having to install them). It
looks like PIP with it's support for "editable" packages might work.
Any suggestions?

thanks

-matt

[Jason S.]

On Oct 12, 6:37 pm, m h <sesqu...@gmail.com> wrote:
> Has anyone done something like this with Pylons? (ie have Pylons and
> it's dependencies in SCM, rather than having to install them).

If they really want third-party packages to be in the SCM as well, you
could put an entire virtualenv directory in the SCM. Use the --no-site-
packages option to virtualenv to make it self-contained:

virtualenv --no-site-packages $DIRNAME

From $DIRNAME, run "source ./bin/activate", then install things with
easy_install or pip, then put in your app directories, then put it all
in the SCM. If you use easy_install use the -Z option to prevent it
from zipping anything. pip has zip/unzip commands. Make sure to use
the python executable that's in $DIRNAME/bin rather than the system
python.

Jason

[Jonathan Vanasco]

i don't know about things like Paste, but i personally do handle all
of my non-pylons includes by putting them in version control.

i toss them all in namespace ##APP##/lib/externals , and add that to
the python search path in the __init__.py

you might be able to put paste, pylons, etc in the same dir, and then
just add that command to the ENV

[Mike Orr]

On Mon, Oct 12, 2009 at 4:37 PM, m h <sesq...@gmail.com> wrote:
>
> Hey Folks-
>
> This might be semi weird.  I've got a client that wants to be able to
> run an application without installing per se.  They've currently got
> everything checked into an SCM,

Does SCM mean version control system?

> and want to be able to pull it down
> and type "make serve" and have the server start running.  They are
> doing pure WSGI right now but would like to move to a web framework.
>
> Has anyone done something like this with Pylons?  (ie have Pylons and
> it's dependencies in SCM, rather than having to install them).  It
> looks like PIP with it's support for "editable" packages might work.
> Any suggestions?

I would hesitate to put the entire virtualenv into version control
because the Python executable is a binary, there are symlinks from the
lib directory to the system Python, and also .so files (binary) in the
lib directory. These all will work only on the same computer, or at
least one with identical paths, OS, and Python version.

You can set up a "make" command to create a virtualenv on the
destination system and install the application into it (using editable
as you said).

--
Mike Orr <slugg...@gmail.com>

[m h]

On Tue, Oct 13, 2009 at 11:04 AM, Mike Orr <slugg...@gmail.com> wrote:
>
> On Mon, Oct 12, 2009 at 4:37 PM, m h <sesq...@gmail.com> wrote:
>>
>> Hey Folks-
>>
>> This might be semi weird.  I've got a client that wants to be able to
>> run an application without installing per se.  They've currently got
>> everything checked into an SCM,
>
> Does SCM mean version control system?

Yes.

>
>> and want to be able to pull it down
>> and type "make serve" and have the server start running.  They are
>> doing pure WSGI right now but would like to move to a web framework.
>>
>> Has anyone done something like this with Pylons?  (ie have Pylons and
>> it's dependencies in SCM, rather than having to install them).  It
>> looks like PIP with it's support for "editable" packages might work.
>> Any suggestions?
>
> I would hesitate to put the entire virtualenv into version control
> because the Python executable is a binary, there are symlinks from the
> lib directory to the system Python, and also .so files (binary) in the
> lib directory.  These all will work only on the same computer, or at
> least one with identical paths, OS, and Python version.
>
> You can set up a "make" command to create a virtualenv on the
> destination system and install the application into it (using editable
> as you said).
>

I want to avoid having to download anything from pypi (or the network
other than the SCM). So I would think my options are to stuff it all
in the SCM as virtualenv, or have a way to do local installs.

cheers,

-matt

[Mike Orr]

You can put the source tarballs in a SCM directory, and use
pip/easy_install options to look only in that directory.

--
Mike Orr <slugg...@gmail.com>

[Wyatt Lee Baldwin]

On Oct 13, 10:41 am, Mike Orr <sluggos...@gmail.com> wrote:
> On Tue, Oct 13, 2009 at 10:38 AM, m h <sesqu...@gmail.com> wrote:
>
> > On Tue, Oct 13, 2009 at 11:04 AM, Mike Orr <sluggos...@gmail.com> wrote:

That's what I would do and use Fabric to deploy it all with just a few
lines of code.

[Jason S.]

On Oct 13, 12:04 pm, Mike Orr <sluggos...@gmail.com> wrote:

> I would hesitate to put the entire virtualenv into version control
> because the Python executable is a binary, there are symlinks from the
> lib directory to the system Python, and also .so files (binary) in the
> lib directory.  These all will work only on the same computer, or at
> least one with identical paths, OS, and Python version.

Good point! I haven't actually done this. But if you wanted the
convenience of virtualenv installation on the client side, you could
use an exclude list in the SCM (e.g. .git/info/exclude) to prevent any
of the binaries from creeping in, then either overlap the deployment
with a virtualenv on the server (to use its binary versions) or have
no virtualenv on the server and set PYTHONPATH.

(Mike's idea of pushing source tarballs to the server and installing
them there sounds like it would work too.)

The "pip bundle" command looks promising, but the docs still
explicitly say it's not stable.

[m h]

Just a follow up on what I did to get it working. Working here means
to be able to checkout a project and run a command to start the server
(no installs or hitting external sites to download stuff). This must
work on XP, but main development is going on in OSX and linux.

* Created a virtualenv for pylons dependencies
* Created a pylons project using said env
* Removed references to simplejson in Pylons and WebError requires.txt
files. (I know that my XP box has Python 2.6 on it which includes
json). Since simplejson has c libraries, the virtualenv version won't
work on windows (it's linux specific).
* Created a simplejson.py stub in virtualenv site-packages that has
the following "from json import *"
* Created a batch file that runs the server :

SET PYTHONPATH=pylonsenv\lib\python2.6\site-packages;pylons_server\
python pylonsenv\bin\paster serve pylons_server\development.ini

Now it works on my linux dev box and the ms deploy box. I realize
this might be a contrived example, but it's what the doctor... (er
client) ordered.

cheers,
-matt

[m h]

Whoops, forgot a step

On Mon, Oct 19, 2009 at 11:47 AM, m h <sesq...@gmail.com> wrote:
> Just a follow up on what I did to get it working.  Working here means
> to be able to checkout a project and run a command to start the server
> (no installs or hitting external sites to download stuff).  This must
> work on XP, but main development is going on in OSX and linux.
>
> * Created a virtualenv for pylons dependencies
> * Created a pylons project using said env

* Copied pkg_resources.py into virtualenv site-packages