Future development of Paver
Almad
Hans Lellelid
> As I view and use Paver, it's kind of "console endpoint" for my apps
> - I want to invoke every functionality I can through paver. For my
> company, I maintain our "internal standard library" of specific paver
> tasks (they are
> at https://github.com/ella/citools/blob/master/citools/pavement.py
> [1] ).
> <snip>
Thanks, Almad for taking over the project. I know that volunteering
for an effort like this is no small commitment, and I appreciate your
time. We use Paver to manage build tasks for our applications --
especially tasks like building RPMs. Typically those sort of tasks
involve lots of copying of files, potentially parsing files and
replacing values in them (e.g. putting a version number into our spec
files), and then running external commands to build RPMs or run other
build scripts (e.g. Ant). Our needs may be different from the way that
most people are using Paver, but from our perspective I think we'd like
to see:
(1) More/better filesystem utilities. Things like <FileSet> or
<FilterChain> in Ant would be really powerful concepts for us in Paver.
(We also use Ant for some builds and find this to be an invaluable way
of creating reusable pieces of a build script.) Specifically, we often
need to work with collections of files (copying/parsing/etc.). A
pythonic implementation of these ideas seems tractable; if we are able
to devote time to implementing this, we'll contribute it back.
(2) Less built-in magic. E.g. the setuptools wrapping is not useful
for us (we would rather have a separate setup.py for
setuptools/distribute); we have had specific bugs with that
"transparent" wrapping that Paver does such that we don't feel safe
relying on it to be a 100% identical implementation. (Unfortunately, I
don't remember the specifics.) And it wasn't that beneficial for us in
the first place.
(3) Better commandline options/argument support. I'm just thinking
out loud, but allowing dependency tasks to also have commandline
options/args that are parsed would be really helpful. E.g. it'd be nice
to have a "setup" dependency task that takes a "--clean" option, so I
could just call paver make --clean and have that passed through to dep
task. Hope that makes sense. I think also just better
optparse/argparse support would be nice for tasks.
Obviously, these are just general thoughts and I realize that since I'm
not really volunteering time to help implement these that it's unlikely
for these to get implemented. It might be useful for discussion,
though, about the future of the project.
Thanks again for your time & involvement in this open-source project.
Hans
Felix Ingram
(3) Better commandline options/argument support. I'm just thinking out loud, but allowing dependency tasks to also have commandline options/args that are parsed would be really helpful. E.g. it'd be nice to have a "setup" dependency task that takes a "--clean" option, so I could just call paver make --clean and have that passed through to dep task. Hope that makes sense. I think also just better optparse/argparse support would be nice for tasks.
jhermann
task library") that some plugin support (via setuptools entry points,
like Paste does it) would be great, since then a simple easy_install /
pip would suffice to add an egg containing external tasks and
commands.
Towards project management, I see two things as essential:
* clear documentation on how to provide such external tasks or
contribute to the core, to keep the entry barrier low (i.e. describe
to people the needed git use-cases, including full commands). Too bad
we don't have a really standard DVCS in open source circles... Anyway,
making contribution easy will prevent Almad from being burdened with
all the work alone.
* a roadmap with ideas (e.g. from this thread) and things people
actually work on. Two effects: you might get more volunteers, and it
shows activity in the project, which is very important for adoption by
even more people.
Kevin Dangoor
Very good ideas, I'd like to add (also in reference to the "external
task library") that some plugin support (via setuptools entry points,
like Paste does it) would be great, since then a simple easy_install /
pip would suffice to add an egg containing external tasks and
commands.
FWIW, I didn't take that approach on purpose. I didn't want "paver help" to get stuck displaying all tasks that you have installed on your system. So, getting a task to show up for a specific pavement was a matter of:
install the external tasks
add an import line
That one extra step gives you control of which tasks appear in which pavements.
Towards project management, I see two things as essential:
* clear documentation on how to provide such external tasks or
contribute to the core, to keep the entry barrier low (i.e. describe
to people the needed git use-cases, including full commands). Too bad
we don't have a really standard DVCS in open source circles... Anyway,
making contribution easy will prevent Almad from being burdened with
all the work alone.
While hg is still big in Python circles, I think that git has largely won, mostly due to GitHub. GitHub's pull requests are quite nice.
Kevin
Kevin Dangoor
work: http://mozilla.com/
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com
Michele Mattioni
On top of that, if the code sits on a git server is possible to use hg
as a client
http://hg-git.github.com/
HIH,
Michele.
Almad
> On Wed, Nov 17, 2010 at 5:16 AM, jhermann <Juergen...@1und1.de> wrote:
> > Very good ideas, I'd like to add (also in reference to the "external
> > task library") that some plugin support (via setuptools entry points,
> > like Paste does it) would be great, since then a simple easy_install /
> > pip would suffice to add an egg containing external tasks and
> > commands.
>
> FWIW, I didn't take that approach on purpose. I didn't want "paver help" to
> get stuck displaying all tasks that you have installed on your system. So,
> getting a task to show up for a specific pavement was a matter of:
>
> install the external tasks
> add an import line
>
> That one extra step gives you control of which tasks appear in which
> pavements.
Those are good points. Additionally, I consider entry_point a bit like playing
with fire, as every installed package modifies behavior of whole system, which
can be quite awful.
(I recall some sphinx bug where Django, sphinx and PIL can interact in a funky
way just by being installed.)
However, it's up to plugin's authors. Perhaps we could just think about some
syntactic suger / standard way to declare plugins usage, something like
@using(['paver.debian', 'paver.rpm']), and decalre in docs some "standard
way".
> > Towards project management, I see two things as essential:
> > * clear documentation on how to provide such external tasks or
> > contribute to the core, to keep the entry barrier low (i.e. describe
> > to people the needed git use-cases, including full commands). Too bad
> > we don't have a really standard DVCS in open source circles... Anyway,
> > making contribution easy will prevent Almad from being burdened with
> > all the work alone.
Good point, writing on my list.
> While hg is still big in Python circles, I think that git has largely won,
> mostly due to GitHub. GitHub's pull requests are quite nice.
Yes. I regret it, as I think hg is much more accessible to new users, as well
as non-programmers (coders etc.)...but working with github is much smoother.
Also, handling the remotes is kinda smoother with git, which is important, as
pushing around on lot of places is much more usual.
> Kevin
Almad
Almad
On Tuesday 16 of November 2010 18:22:14 Hans Lellelid wrote:
> We use Paver to manage build tasks for our applications --
> especially tasks like building RPMs. Typically those sort of tasks
> involve lots of copying of files, potentially parsing files and
> replacing values in them (e.g. putting a version number into our spec
> files), and then running external commands to build RPMs or run other
> build scripts (e.g. Ant).
Would it be possible to provide rpm-build scripts? We're doing something
similar with debian, althrough we are planning to switch to stdeb.
I think having easy and standard way to do versioning and builds for common
systems (that would probably be mainly be .deb and .rpm, as .exe is well-
covered with distutils...perhaps some template generation for source-based
distros, putting together ebuild/PKGBUILDs should be easy) would be nice for
package authos.
> Our needs may be different from the way that
> most people are using Paver, but from our perspective I think we'd like
> to see:
>
> (1) More/better filesystem utilities. Things like <FileSet> or
> <FilterChain> in Ant would be really powerful concepts for us in Paver.
> (We also use Ant for some builds and find this to be an invaluable way
> of creating reusable pieces of a build script.) Specifically, we often
> need to work with collections of files (copying/parsing/etc.). A
> pythonic implementation of these ideas seems tractable; if we are able
> to devote time to implementing this, we'll contribute it back.
That would be awesome. I admit I don't know ant features much, during my brief
Java time I was just using it through some IDE generators.
> (2) Less built-in magic. E.g. the setuptools wrapping is not useful
> for us (we would rather have a separate setup.py for
> setuptools/distribute); we have had specific bugs with that
> "transparent" wrapping that Paver does such that we don't feel safe
> relying on it to be a 100% identical implementation. (Unfortunately, I
> don't remember the specifics.) And it wasn't that beneficial for us in
> the first place.
I think this is major architecture decision. I will dig deeper in how hard it
would be to make wrapping optional, but I do believe that providing setuptools
"default" significantly lowers entry barrier.
> (3) Better commandline options/argument support. I'm just thinking
> out loud, but allowing dependency tasks to also have commandline
> options/args that are parsed would be really helpful. E.g. it'd be nice
> to have a "setup" dependency task that takes a "--clean" option, so I
> could just call paver make --clean and have that passed through to dep
> task. Hope that makes sense. I think also just better
> optparse/argparse support would be nice for tasks.
Definitely. I personally woud like to see conditional post-dependencies
implemented, so doing some argparse on the way might make sense.
> Obviously, these are just general thoughts and I realize that since I'm
> not really volunteering time to help implement these that it's unlikely
> for these to get implemented. It might be useful for discussion,
> though, about the future of the project.
> Thanks again for your time & involvement in this open-source project.
Thanks for them.
I have not done anything yet, so save Your thanks until later ;) Paver is a
side project for me too, but I am willing to do the housekeeping work and help
it shape.
As for plans, I am used to use Trac roadmaps and github support for those kind
of things is...strange. But perhaps I'll try to fill ideas on wiki there and
create few tickets and see if it will be bearable in the long run.
> Hans
Thanks for Your feedback,
Almad
mpa
On Nov 16, 8:49 am, Almad <b...@almad.net> wrote:
> Hi,
>
> as a probable maintainer, I'd like to ask the community what they expect
> Paver to become and how they are using it.
does the actual compiling, and Paver handles shell-script kinds of
things, like building documentation, running debuggers, downloading
firmware to the development system, etc.
For me, Paver is kind of a Bash alternative. At work, I mostly work on
Windoze, but often switch between Cygwin and native Windows, so Paver
makes a nice "cross-platform" scripting environment. It's also nice to
know that there's a straightforward path to running on both Linux and
OS X.
I don't use or expect Paver to handle build tasks directly -- I think
purpose-made tools like Make, SCons and (lately) Waf for that. But in
the future, I could see Paver taking on many of the tasks that ./
configure performs in packages that use Autotools.
>
> As I view and use Paver, it's kind of "console endpoint" for my apps - I
> want to invoke every functionality I can through paver. For my company, I
> maintain our "internal standard library" of specific paver tasks (they are
> athttps://github.com/ella/citools/blob/master/citools/pavement.py).
>
> For some of these areas, I'd like to see some support directly in paver, but
> I'd like to know how others are solving those problems and what are Your
> opinions on what should be added to python standard library and what should
> be kept separatly (and perhaps to have "blessed libaries" on paver's github
> that will be just kept, developed and released independenly on Paver).
>
> My thoughts on what should be done, in random order:
>
> - Python 3 support. I have not much experience with it, but I want to gain
> it anyways, so let's go for it ;) However, anyone experienced with 2to3 and
> friends would be welcomed.
yet though.
It might be a problem if Python 3 were *required*.
> - distutils2 - keep an eye on it and provide future compatibility
distribute.
> - with that, we have big "support combination matrix", I should set up some
> buildbots for that.
>
> - As a django app developer, I'd like to add "django" task as an entry
> point to manage.py
> - Any other frameworks to support?
>
> - Better coop with virtualenv; currently, system-wide installed paver is
> not picking up venv, even when it's sourced (imo). I'd like to add
> "autoactivate" option if virtualenv is detected, as IMO it's becoming
> default way to develop in python).
little cumbersome. I think it needs better documentation.
>
> - Invent pip support. Paver-enabled apps are now not installable via pip
> install -e repo; find a good way to enable it.
> - And yeah, fix docs, that's up to me ;)
>
> Generally, I like the idea of keeping basic frameworks small & clean and
> keep apps/task libraries separate...
I think waf gets this right.
> so, for example, I am not so sure about
> having VCS-specific thing in stdlib (for example, as we're on github, we
> should add git...but git api is fairly unstable between versions, should we
> re-release paver with every git change?).
>
> So, dear co-users. Feedback? Thoughts? Suggestions? Forks?
One problem I have with Paver is the need to "install" it; this makes
bootstrapping our dev environments much more complicated -- right now
I have to use shell scripts, which just seems wrong!
I'd much rather see Paver distributed as waf is, in a single minimal
file which you include directly in projects, along with optional
modules, so that the only external dependency for a project is Python.
I think there's some ability to do this now, so maybe it wouldn't be
too difficult.
Thanks for taking this forward (provided you decide to do that). Mr.
Dangoor did a pretty fantastic job making Paver, and I haven't missed
frequent maintenance too much, but there are lots of good directions
it can still go.
thanks
Michael
mpa
On Nov 17, 5:59 am, Kevin Dangoor <k...@blazingthings.com> wrote:
> While hg is still big in Python circles, I think that git has largely won,
> mostly due to GitHub. GitHub's pull requests are quite nice.
and in fact pretty much everything else too. It works very well for
me, and I rarely miss git. Maybe git has won, but I wouldn't
necessarily say that hg has lost :) --mpa