buildout and paver
percious
So, I started playing with paver yesterday, and I was pleasantly
surprised how well it meets my needs. I started off by trying to
figure out how to convert repoze.bfg's buildout.cfg over to a paver
file, because I wanted to run bfg inside a venv, but I didn't want
some wacky buildout configuration that is in my mind non-sensicle.
For feedback, I offer these comments.
__file__ issues
--------------------------
One of the things the original setup.py script does is to read the
readme and changes files from the packages root directory, and inject
them into the setup function. This doesn't work with paver, because
it looks like __file__ points to something in the paver module, not
where the pavement.py script is running, which is problematic for this
use case.
zc.buildout
----------------
It seems like it would not be too too much effort to make paver read
in buildout.cfg files and generically map the usage to virtualenvs.
The repoze.bfg pavement.py script I wrote illustrates how one might go
about using zc.recipes in their paver script. It actually pulls the
necessary recipes down from the index provided, loads them into the
sys.path, and then executes them. It also creates a "fake_buildout"
which it passes to the recipes.
I could imagine a pavement plugin that reads a buildout.cfg, and then
figures out which recipes are needed, and builds a "fake_buildout"
object automatically, before running everything in sequence. I wonder
if anyone else has also contemplated this?
Anyway, here is a link to the script I created for repoze.bfg. I'm
sure it could be cleaned up, but it's a start:
http://svn.repoze.org/repoze.bfg/branches/paver-branch/pavement.py
Mark Ramm
percious
lxml4venv which
will install lxml into any virtualenv you want to use. Then you can
simply include the lxml4venv
in your setup.py ABYU.
The only problem is that this solution links your package to having to
run in a venv, and I don't
have a solid solution for that.... yet.
cheers.
-chris
On Oct 15, 11:06 am, "Mark Ramm" <mark.mchristen...@gmail.com> wrote:
> I am interested in this because the thing that makes bfg hard to install
> (chameleon, and lxml) is also very useful to TurboGears since
> chameleon.genshi is more than 10x faster than genshi in the benchmarks I've
> seen for comon use cases.
>
> And beyond that lxml is useful in a lot of other places. Deliverance makes
> extensive use of lxml, and it's html parser is fast and pretty darned
> effective.
>
Kevin Dangoor
> I am interested in this because the thing that makes bfg hard to
> install (chameleon, and lxml) is also very useful to TurboGears
> since chameleon.genshi is more than 10x faster than genshi in the
> benchmarks I've seen for comon use cases.
>
> And beyond that lxml is useful in a lot of other places.
> Deliverance makes extensive use of lxml, and it's html parser is
> fast and pretty darned effective.
>
> But generally, I think it's been a paver goal to be "buildout"
> compatable, but nobody has had time to work on that, and I'm not
> even 100% clear that we all know what it means.
We certainly don't all know what it means, but I definitely had a very
specific concept in mind.
What ultimately happened is that I got derailed from that course,
because I preferred venv's simplicity, the fact that venv works with
WingIDE trivially, the fact that venv lets you install *anything*
(even hard-to-egg stuff like Twisted) and the fact that PoachEggs will
update your requirements for you. I also like that pyinstall seeks to
simplify and make more transparent the installation process.
Unfortunately, I ran into issues with virtualenv on SuSE 64-bit (which
includes a distutils.cfg) and I haven't been completely successful at
having PoachEggs manage my egg versions :(
That's why we haven't seen a new Paver release with the virtualenv/
poacheggs goodies.
As far as buildout goes, my original plan was for Paver to participate
in the buildout in much the same way that it works with distutils/
setuptools now. Paver, and your pavement.py, would drive the whole
buildout process. buildout recipes would be usable as Paver tasks.
Rather than using a buildout.cfg, you would define your requirements
in a more Paver-esque manner.
In some ways, I'm at a fork in the road. I can either go back to
buildout and do what I just described, or I can work on getting the
virtualenv/pyinstall stuff working the way I want it to. My
inclination is the latter, and that would mean that I personally don't
have much interest in putting the work in to make buildout recipes work.
> But if it just means that paver can reuse existing buildout scripts
> in paver, I think that would be very cool. I think the system
> isolation stuff in buildout isn't as nessisary now that we have
> virtual env, and it is probably a good thing to break the hard
> connection between the buildout recipes and using buildout for
> isolation.
>
Hard to say offhand how difficult it would be to make buildout recipes
work in Paver... I'm guessing that there's quite a lot of setup that
would need to be undertaken to simulate a buildout environment. I
could be wrong.
Kevin
percious
> Hard to say offhand how difficult it would be to make buildout recipes
> work in Paver... I'm guessing that there's quite a lot of setup that
> would need to be undertaken to simulate a buildout environment. I
> could be wrong.
>
> Kevin
use them to install stuff. The link above shows how I did it.
I don't think it's terribly obtuse to replicate the buildout object
that recipies use to do what they do. There are a few default
configuration items, and then the rest of the creation of the object
is loaded directly from buildout.cfg using (i believe) ConfigObj.
I think all that would be required would be to:
a) Figure out how the defaults should work in the buildout
b) Find all of the recipes in the buildout, and pull them down from
either the index, or a pre-determined place.
Thanks Kevin for writing paver, it will definitely solve some
technical problems I have at work. I am interested in possibly
contributing my ideas for a buildout.cfg->virtualenv tool.
cheers.
-chris
Kevin Dangoor
> As I said, I was able to successfully load zc.recipies into paver and
> use them to install stuff. The link above shows how I did it.
Oh! I missed that! Sorry.
That's great! Very simple. That should definitely be included with
Paver in some form.
> I don't think it's terribly obtuse to replicate the buildout object
> that recipies use to do what they do. There are a few default
> configuration items, and then the rest of the creation of the object
> is loaded directly from buildout.cfg using (i believe) ConfigObj.
>
> I think all that would be required would be to:
> a) Figure out how the defaults should work in the buildout
> b) Find all of the recipes in the buildout, and pull them down from
> either the index, or a pre-determined place.
I wonder if it would be better to be able to turn a buildout.cfg into
pavement config sections instead?
> Thanks Kevin for writing paver, it will definitely solve some
> technical problems I have at work. I am interested in possibly
> contributing my ideas for a buildout.cfg->virtualenv tool.
I'm glad it's working out for you, Chris. It certainly solves a
category of problems I had been having.
I'm completely swamped at the moment, so I'd love to see any ideas
you've got. And I'll try to read them a bit more carefully :)
Kevin