State of the Patchbot
Rob Beezer
tool. Thanks, Robert, for your work on it.
However, I have found myself wrestling with it a bit. I probably have
some things confused below - corrections, clarifications or comments
appreciated.
1. If the patchbot passes all tests for a ticket - then good. But if
it fails, there can be a variety of reasons, like missing dependencies
(on dependent tickets?) so a reviewer should not assume the absence of
a green disk is a ticket that is not really ready for review. The
patchbot is experimental still, no?
2. The release manager would like apply/depends information in the
Trac description entry (at the top). The patchbot does not read
these?
3. In the presence of several patches on a ticket, or other
complicated situations, the patchbot uses the the last "Depends" and
"Apply" strings it sees in the *comments*?
4. Patchbot will begin at the latest stable release, so dependencies
need to be stated on dependent tickets back as far as the last stable
release?
5. Looks like Martin D. is adding a "Dependencies" field to Trac.
How about an "Apply" field, and this will all be in one place?
6. Is there a way to trigger the patchbot on a specific ticket?
Otherwise - what type of edit triggers a run, and what sort of delay
should be expected?
7. Documentation is at: http://wiki.sagemath.org/buildbot
Thanks,
Rob
Robert Bradshaw
> First, the patchbot is an excellent idea and will be a very useful
> tool. Thanks, Robert, for your work on it.
Thank you. I haven't gotten to do much with it lately, but hope to get
back on it (in particular, get it merged into Sage itself) early next
month. I'd also love to set up a notebook server based on the
patchbot, so you could select any (successfully-built) ticket you want
to try it out (see #9967, and even cooler would be a mechanism to
easily add your manual tests as doctests, but one thing at at
time...).
I've also been thinking about how best to handle spkgs, and Dag
Seljebotn recently pointed me to Nix http://nixos.org/nix/
https://github.com/dagss/scidist/blob/master/ideas.rst . The basic
idea is that each component is compiled in a directory based on the
hash of its sources and all of its dependencies, and symlinks are used
to keep track of the "current" copy in use. The key insight that I
think makes this a great solution for spkgs is that the version of
spkgs used can be configured either as a single txt file or a set of
symlinks and managed by version control (ideally, the same repo as the
Sage library itself. Thus patches to the sage library can explicitly
depend on spkgs, re-building after applying them can automatically
install the spkgs (assuming an internet connection), and un-installing
an spkg is as simple as popping a patch off your queue (the symlinks
are re-assigned, the old spkg is still built, ready to be used again.
Of course this would be a fair amount of work, both updating the spkg
system (whether we use Nix or just the ideas) and better supporting
Python (probably via PYTHONPATH or some other special import hooks
rather than installing stuff directly via setup.py, similar to
setup.py develop where things can be edited/installed inplace). Of
course this would be great for the patchbot, but that's just the tip
of the iceburg.
> However, I have found myself wrestling with it a bit. I probably have
> some things confused below - corrections, clarifications or comments
> appreciated.
>
> 1. If the patchbot passes all tests for a ticket - then good. But if
> it fails, there can be a variety of reasons, like missing dependencies
> (on dependent tickets?) so a reviewer should not assume the absence of
> a green disk is a ticket that is not really ready for review. The
> patchbot is experimental still, no?
The colored blurb is just a useful data point--if it's green chances
are all is well, but if not it's worth seeing what went wrong. It
doesn't automatically bump the status to "needs work." Sometimes
there's spurious timeouts or missing dependancies (and a red blurb can
be a good indication to the author about these things as well).
> 2. The release manager would like apply/depends information in the
> Trac description entry (at the top). The patchbot does not read
> these?
It actually reads the rss for lots of stuff, i.e.
http://trac.sagemath.org/sage_trac/ticket/10865?format=rss , where
description changes are diff links. It also parses out stuff mostly
line-by-line, making bulleted lists a pain (though this could be
fixed).
> 3. In the presence of several patches on a ticket, or other
> complicated situations, the patchbot uses the the last "Depends" and
> "Apply" strings it sees in the *comments*?
Yep. There's no "last" description unless you want to follow all the
diffs. Actually, first it looks at the list of patches attached,
usually no "apply" strings are necessary in the comments, they're only
needed when things get to complicated, and even then it "resets" the
patch list so patches added after that point get automatically
appended (possibly replacing others).
The way to look at it is that it makes a guess, which is right a good
percentage of the time (essentially always if there are only a couple
of patches), and most critically *you* don't have to guess at what
it's thinking 'cause it's super easy to see what it guessed and
provide corrections only if needed.
> 4. Patchbot will begin at the latest stable release, so dependencies
> need to be stated on dependent tickets back as far as the last stable
> release?
Currently, yes.
> 5. Looks like Martin D. is adding a "Dependencies" field to Trac.
+1
> How about an "Apply" field, and this will all be in one place?
This was discussed, but it's unclear how to best do it. Editing a list
of patch names in a text field is both cumbersome and tedious--and the
whole point of the patchbot is to do what a machine can do to free up
humans doing what only a human can do.
What we'd want is something like a check box (or drop-down list of
repos) next to the list of patches, assuming the order doesn't get
messed up, or an ajaxy way to sort the list.
> 6. Is there a way to trigger the patchbot on a specific ticket?
> Otherwise - what type of edit triggers a run, and what sort of delay
> should be expected?
Any change to the ticket triggers a refresh, any change to the
patch-list or dependencies triggers a rerun. As does changing the
buildbot (its system, name, sage version, ...) There's a way to
manually trigger this as well that I used for debugging, but I can't
remember it off the top of my head (look at the source...) It's
actually idle most of the time, and sleeps for a default of 180
seconds or so while there's nothing to do, so that's the expected
delay (though interestingly sage development tends to happen in
bursts).
> 7. Documentation is at: http://wiki.sagemath.org/buildbot
Note that the difficulty with applying patches to the non-main repo is
that it would require a whole different system than just switching
between branches, not to mention the danger of the scripts repo
getting into an inconsistant state. Of course, IMHO having to manage
several distinct repos for a single project is a pain for the
developer as well as the system.
- Robert
Rob Beezer
Thank-you very much for the long and informative reply. It will be
helpful for me, and I hope for other developers.
Rob
Maarten Derickx
>
> I've also been thinking about how best to handle spkgs, and Dag
> hash of its sources and all of its dependencies, and symlinks are used
> to keep track of the "current" copy in use. The key insight that I
> think makes this a great solution for spkgs is that the version of
> spkgs used can be configured either as a single txt file or a set of
> symlinks and managed by version control (ideally, the same repo as the
> Sage library itself. Thus patches to the sage library can explicitly
> depend on spkgs, re-building after applying them can automatically
> install the spkgs (assuming an internet connection), and un-installing
> an spkg is as simple as popping a patch off your queue (the symlinks
> are re-assigned, the old spkg is still built, ready to be used again.
> Of course this would be a fair amount of work, both updating the spkg
> system (whether we use Nix or just the ideas) and better supporting
> Python (probably via PYTHONPATH or some other special import hooks
> rather than installing stuff directly via setup.py, similar to
> setup.py develop where things can be edited/installed inplace). Of
> course this would be great for the patchbot, but that's just the tip
> of the iceburg.
thinking about how to make the sage spkg system to be able to
uninstall spkgs. I guess the not reinventing the wheel idea suggests
that we should first look for the most suited cross platform package
management system and build the sage spkg system on top of that.
Something interesting I found when looking for other package manage
systems is http://0install.net/ which shares a lot of nice features
with nix (multiple versions installed at the same time, non
destructive updates, uninstall) and has the advantage of having a
python interface.
Let me know when you plan to concretely start on doing things with the
sage spkg system, cause I'm also very interested in participating in
that.
Kind Regrads.
Maarten Derickx
>
> - Robert