Group collaboration on a branch of Sage
David Roe
David
Jason Grout
> Is there an easy way to collaborate on an experimental set of patches on
> top of Sage? The sage-combinat model is very close to what I'm looking
> for, but it seems difficult to set up all the infrastructure: we have to
> have a server analogous to http://sage.math.washington.edu:2144/, set up
> the initial repositories correctly.... Is there an easier way for a
> project with only a couple people? Is there a nice way to do this with
> Sage on github?
You took the words right out of my mouth. Just fork Sage on github.
Then you can submit pull requests to each other, designate one person's
branch as the "blessed" branch, and coordinate from there.
I'd say git and github is your ticket to ride.
Jason
William Stein
Note that it is not obvious or documented how to effectively do this
with the Sage library with other people, and still properly submit
stuff for inclusion in Sage, but it should be possible.
I setup something exactly like the above for trac #12545:
http://trac.sagemath.org/sage_trac/ticket/12545
with my github repo here:
https://github.com/williamstein/sagelib/commits/lfun
From memory, I basically did the following:
1. Got a git repo as follows:
$ cd SAGE_ROOT/devel
$ git clone git://github.com/williamstein/sagelib.git sage-git
$ rm sage
$ ln -s sage-git sage
$ export MAKE="make -j4"
$ sage -br # wait an hour?
2. Branch my repo for this particular project:
$ git branch lfun
$ git checkout lfun # switch to this branch
3. Whenever I am doing with some work and do "git commit -a" (say),
then I'll do this:
# makes a flattened patch of everything I've done.
git diff ecaff3a3d14614bd01a724c8b26bd874bb6bf483 >
~/patches/trac_12545-cumulative.patch
# copies that flattened patch to the web -- there is a link to this
patch on the trac ticket.
scp ~/patches/trac_12545-cumulative.patch sage.math.washington.edu:patches/
# pushes my repo to github
git push was lfun
REMARKS:
* So far I've failed to have anybody else collaborate with me on this
code, but I hold out hope it will happen, so I can see if that would
actually work. I'm very optimistic though, given my recent
experience with github.
* I've tried using my sage-git "branch" and branching it in order to
test out various trac tickets when refereeing, etc., It seems much
more cumbersome than using Mercurial queues for this, but that may be
because the patches are in hg format, etc., so this isn't so
surprising.
Anyway, git is awesome.
>
> Jason
>
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org
Keshav Kini
+1, you are most welcome to use the sagemath/sagelib repo on github! I
find the sage-combinat patch queue a little bit terrifying, to be
honest, but maybe that's just me.
However, please do keep in mind that the sagemath/sagelib repo on github
is just following Jeroen's Sage Library SPKGs right now. As such, I have
no control over what commits go into it. Therefore, when we eventually
move to git, the git repo we use might look pretty different from what
github:sagemath/sagelib is right now. I might need to delete it and
start over with a new git repository, or at least perhaps delete some
branches, depending on what exactly we decide to do during the
switchover. At that point you may need to do some hairy stuff with git
to get your commits into the new repo.
I will of course try to avoid doing this, but I can make no guarantees.
William already ran into an inconvenience when I had to rewrite some
history on the repo and then bug him about rebasing some of his commits.
If you're fully proficient with git I'm sure you can keep up, though.
Technical details of the current situation follow, for those who know
more about git and/or DVCS.
----
Jeroen creates each new development release by applying a string of
positively reviewed patches to the previous stable release, rather than
applying *newly* positively reviewed patches to the previous development
release.
Thus the history tree of sagemath/sagelib on github looks like a bunch
of stable releases along one branch (named "release") and a bunch of
branches splitting from it at intervals and containing substantial
repetition of commits from the "release" branch. These abortive branches
will never be merged into the main trunk by Jeroen, but will just sit
there. I intend to eventually *delete* them, once we move to git. So you
will have to do rebasing and *not* merging, for now, if you want to
produce patches that can be submitted on trac, and if you want to avoid
having upstream commits in your local history which have been deleted by
upstream (me).
However, once we switch to git, I would encourage you to merge rather
than rebase.
-Keshav
----
Join us in #sagemath on irc.freenode.net !
Keshav Kini
> Note that it is not obvious or documented how to effectively do this
> with the Sage library with other people, and still properly submit
> stuff for inclusion in Sage, but it should be possible.
>
> I setup something exactly like the above for trac #12545:
> http://trac.sagemath.org/sage_trac/ticket/12545
>
> with my github repo here:
> https://github.com/williamstein/sagelib/commits/lfun
>
> From memory, I basically did the following:
>
> 1. Got a git repo as follows:
>
> $ cd SAGE_ROOT/devel
> $ git clone git://github.com/williamstein/sagelib.git sage-git
Of course, other users should first fork the repo on github and then
replace "williamstein" with their github username.
> $ rm sage
> $ ln -s sage-git sage
> $ export MAKE="make -j4"
> $ sage -br # wait an hour?
>
> 2. Branch my repo for this particular project:
>
> $ git branch lfun
> $ git checkout lfun # switch to this branch
Before this, please checkout the version of Sage you are running. If
your SAGE_ROOT is an installation of Sage 4.8, for example, please do
$ git checkout 4.8
$ git checkout -b lfun # shorthand for `git branch lfun && git checkout
lfun`
If you do not do this, you may run into problems in Sage due to your
library being "too new" compared to the other SPKGs installed in Sage.
The best thing to do is to *always* be running the latest development
version, IMO. Then you don't need to worry about this stuff. Not to
mention that creating your patch will be easier - no rebasing necessary,
if you work fast enough and get your code done before Jeroen releases
the next development release!
> 3. Whenever I am doing with some work and do "git commit -a" (say),
> then I'll do this:
>
> # makes a flattened patch of everything I've done.
> git diff ecaff3a3d14614bd01a724c8b26bd874bb6bf483 >
> ~/patches/trac_12545-cumulative.patch
Note: ecaff3a3d14614bd01a724c8b26bd874bb6bf483 doesn't exist upstream
anymore. It was a commit I foolishly made to include a github-specific
README into the repo on github - from now on that README lives in the
wiki on the github project page, and I will no longer make any commits
to the github repo which aren't in Jeroen's SPKGs. Please instead do
$ git diff master > ~/patches/whatever.patch
This will produce a patch against whatever the latest development
release was when you created your fork on github (assuming you didn't do
anything to "master" in your local repo). If much time has passed since
then, you will need to rebase your patch, but I won't go into that here.
> # copies that flattened patch to the web -- there is a link to this
> patch on the trac ticket.
> scp ~/patches/trac_12545-cumulative.patch sage.math.washington.edu:patches/
>
> # pushes my repo to github
> git push was lfun
Note: William has done
$ git remote add was github.com:williamstein/sagelib
which creates "was" as a short name for a remote repository,
specifically his github fork of the Sage library. You can do something
similar.
> REMARKS:
>
> * So far I've failed to have anybody else collaborate with me on this
> code, but I hold out hope it will happen, so I can see if that would
> actually work. I'm very optimistic though, given my recent
> experience with github.
>
> * I've tried using my sage-git "branch" and branching it in order to
> test out various trac tickets when refereeing, etc., It seems much
> more cumbersome than using Mercurial queues for this, but that may be
> because the patches are in hg format, etc., so this isn't so
> surprising.
What do you mean by "branching it"? `git branch`? Yes, it is pretty
terrible right now, but that's because we're using patches (not really
because they are in hg format). Fetching from other git repos to do
reviews is much, much easier :)
> Anyway, git is awesome.
Yup!
William Stein
Amazingly, after all we went through, I still have a messed up repo
(no master branch). Sigh.
deep:d wstein$ git branch
12594
github-master
lfun
* trac_8393
I guess I should just start over.
>> # copies that flattened patch to the web -- there is a link to this
>> patch on the trac ticket.
>> scp ~/patches/trac_12545-cumulative.patch sage.math.washington.edu:patches/
>>
>> # pushes my repo to github
>> git push was lfun
>
> Note: William has done
>
> $ git remote add was github.com:williamstein/sagelib
>
> which creates "was" as a short name for a remote repository,
> specifically his github fork of the Sage library. You can do something
> similar.
>
>> REMARKS:
>>
>> * So far I've failed to have anybody else collaborate with me on this
>> code, but I hold out hope it will happen, so I can see if that would
>> actually work. I'm very optimistic though, given my recent
>> experience with github.
>>
>> * I've tried using my sage-git "branch" and branching it in order to
>> test out various trac tickets when refereeing, etc., It seems much
>> more cumbersome than using Mercurial queues for this, but that may be
>> because the patches are in hg format, etc., so this isn't so
>> surprising.
>
> What do you mean by "branching it"? `git branch`?
Yes, precisely.
> Yes, it is pretty
> terrible right now, but that's because we're using patches (not really
> because they are in hg format). Fetching from other git repos to do
> reviews is much, much easier :)
It's difficult for me because they are patch in hg format, but I can't
export a patch out of git in hg format. It's possible to import the
code in from the trac ticket, but then if I make changes I have to
export them as a git diff, then import them into another hg repo, then
export them again. As far as I can tell -- no easy way to referee a
trac ticket right now using git. I bet it would be possible for a
git wizard to setup some scripts to make this easy. I think doing
so is an important first step toward moving Sage developers to git.
>
>> Anyway, git is awesome.
>
> Yup!
>
> -Keshav
>
> ----
> Join us in #sagemath on irc.freenode.net !
>
Jonathan Bober
It's difficult for me because they are patch in hg format, but I can't
export a patch out of git in hg format. It's possible to import the
code in from the trac ticket, but then if I make changes I have to
export them as a git diff, then import them into another hg repo, then
export them again. As far as I can tell -- no easy way to referee a
trac ticket right now using git. I bet it would be possible for a
git wizard to setup some scripts to make this easy. I think doing
so is an important first step toward moving Sage developers to git.
Keshav Kini
> Amazingly, after all we went through, I still have a messed up repo
> (no master branch). Sigh.
>
> deep:d wstein$ git branch
> 12594
> github-master
> lfun
> * trac_8393
>
> I guess I should just start over.
You almost never need to start over with git. Just `git fetch` from
upstream, create a master branch, check it out, and `git reset <commit>`
where <commit> is wherever you want master to be pointing.
>> Yes, it is pretty
>> terrible right now, but that's because we're using patches (not really
>> because they are in hg format). Fetching from other git repos to do
>> reviews is much, much easier :)
>
> It's difficult for me because they are patch in hg format, but I can't
> export a patch out of git in hg format. It's possible to import the
> code in from the trac ticket, but then if I make changes I have to
> export them as a git diff, then import them into another hg repo, then
> export them again. As far as I can tell -- no easy way to referee a
> trac ticket right now using git. I bet it would be possible for a
> git wizard to setup some scripts to make this easy. I think doing
> so is an important first step toward moving Sage developers to git.
I see that as a waste of time. We should create scripts to facilitate a
new workflow, not the old one. The average Sage developer should not be
encouraged to use git just yet, I think. That is because, as you have
noticed, it is downright painful to use git to work on Sage right now.
Basically the only advantage right now is the ease of collaboration,
which Jason mentioned above.
But the solution is not to create scripts that automate this painful
process. What we need to do is set up an infrastructure to support a
git-based workflow, not create scripts that dumb git down to support our
current workflow. (This will be covered in the Workflow SEP.)
For non-average Sage developers who for whatever reason want to use git
and want to use it *now*, there is a small script displayed on `the
github README`_ which may be of use. See also `a recent sage-devel
thread`_ (started by you, incidentally) for more advice on how to
interoperate with our current system using git.
.. _the github README: https://github.com/sagemath/sagelib/wiki
.. _a recent sage-devel thread:
https://groups.google.com/d/topic/sage-devel/mwY_ju2QWAQ/discussion
> then import them into another hg repo, then export them again
I have no idea why you need to do this, or think you need to do this.
Can you explain? Have you seen the script posted in the above two links?
If so, does it not work? As far as I can see, if you do use the git
repository of the Sage library, you should have no need to use the
Mercurial repository of the Sage library.
Martin Albrecht
please excuse my ignorance especially if this has been covered before, but as
far as I can you can do the fork & pull-request development model with
mercurial as well, e.g. http://bitbucket.org supports it. I have never used it
though and I never used the equivalent on github.
https://confluence.atlassian.com/display/BITBUCKET/Forking+a+bitbucket+Repository
Cheers,
Martin
--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
_www: http://martinralbrecht.wordpress.com/
_jab: martinr...@jabber.ccc.de
Jason Grout
> Hi,
>
> please excuse my ignorance especially if this has been covered before, but as
> mercurial as well, e.g. http://bitbucket.org supports it. I have never used it
> though and I never used the equivalent on github.
>
> https://confluence.atlassian.com/display/BITBUCKET/Forking+a+bitbucket+Repository
>
You're right. In fact, bitbucket supports git as well. In many ways,
bitbucket is nicer than github, for example, the issue tracker is
better, in my opinion (bitbucket has priorities for tickets, for
example), and the display on bitbucket is much nicer (the information is
much more densely displayed, and helpful tree diagrams are out to the
side of commits), and bitbucket has threaded comments.
There are some things github does better, though. I can't recall them
off the top of my head, though.
Jason
Keshav Kini
> Hi,
>
> please excuse my ignorance especially if this has been covered before, but as
> far as I can you can do the fork & pull-request development model with
> mercurial as well, e.g. http://bitbucket.org supports it. I have never used it
> though and I never used the equivalent on github.
You're absolutely right. William has been learning git and is quite
enthusiastic about it, and Jason likes git enough to have moved the Sage
Notebook and the Sage Cell Server to github. IPython is also on github,
and Fernando has strongly recommended we consider moving Sage to github
(and has been doing so for at least 8 months, as far as I can recall).
The main thing I want for Sage's development process is a push/pull
architecture of some kind. I don't mind if that ends up meaning that we
finally start using Mercurial in the way it was "meant" to be used,
rather than switching to git. But the wind seems to be blowing in git's
direction.
I prefer git over Mercurial myself, FWIW, for various reasons, chief
among them being 1) that git has had lightweight branching at the core
of its philosophy from the beginning, whereas Mercurial added them as an
afterthought quite late in its history; and 2) that git's storage system
is a lot more transparent and easy to hack on than Mercurial's - logical
objects are snapshots, and delta compression is done transparently by
the git garbage collector / storage optimizer, whereas in Mercurial,
delta compression is built into the file format of Mercurial storage
objects. Downsides include that git itself is a bit hard to extend,
unlike Mercurial, where you can pretty easily write extensions. However,
git also exposes a bunch of semi-"internal" functionality through the
command line, so most things you would want to do can be done with a
shell script.
kcrisman
On Mar 1, 9:20 am, Keshav Kini <keshav.k...@gmail.com> wrote:
> > Hi,
>
> > please excuse my ignorance especially if this has been covered before, but as
> > far as I can you can do the fork & pull-request development model with
>
> You're absolutely right. William has been learning git and is quite
> enthusiastic about it, and Jason likes git enough to have moved the Sage
> Notebook and the Sage Cell Server to github. IPython is also on github,
> and Fernando has strongly recommended we consider moving Sage to github
> (and has been doing so for at least 8 months, as far as I can recall).
> The main thing I want for Sage's development process is a push/pull
> architecture of some kind. I don't mind if that ends up meaning that we
> finally start using Mercurial in the way it was "meant" to be used,
years, and probably since when Sage switched from darcs to hg.
Is doing all this stuff in Mercurial an option, especially for folks
like sage-combinat who have spent a lot of time taking people who are
*not* hard-core programmers, but want to do research math, up to speed
in hg? This seems like a potential halfway point, assuming that there
is a site which would want to host this all - Sage seems so huge, I'm
kind of surprised even github would want to host it for free, though I
guess it's GPL...
Keshav Kini
I am very far from convinced that sage-combinat developers have been
learning anything substantial about *Mercurial*. Going by what I see on
the wiki pages about combinat, what a lot of time has been spent on is
taking people who are not hard-core programmers and teaching them how to
plant in and help to fertilize a colorful garden of *patch files*.
Learning git will be about equal to learning "real" Mercurial in terms
of head-scratching required, in my estimation (though of course I could
be wrong). Furthermore I intend to write a script called
$SAGE_LOCAL/bin/sage-dev which will walk people through the process of
using git to work on Sage - basically a "wizard", asking yes/no
questions and soliciting input with ample explanatory text - which I
believe will be pretty intuitive and simple, though we'll see how true
that is once I set out concrete design goals and ask for feedback.
For what it's worth, I have emailed the main sage-combinat developers a
little while ago and asked them for an overview of what I should keep in
mind when designing this SEP, so don't worry too much about
sage-combinat :) They will be kept informed at every step and included
in our planning discussions.
As it happens, the Sage library isn't that huge. The git repo of the
Sage library is just 30 MB (incidentally, only about half the size of
our Mercurial repository). Github offers 300 MB of space to non-paying
members, and even that is just a "soft limit" (i.e. if you have a
reasonable excuse, you may use more than 300 MB). We are of course not
going to be using github to host the src/ directories in our SPKGs of
external programs. However, hosting the spkg-install files, SPKG.txt,
etc. is something that might be on the table, though probably farther in
the future than switching to git and a push-pull system for the Sage
library proper.
Fernando Perez
> IPython is also on github,
> and Fernando has strongly recommended we consider moving Sage to github
> (and has been doing so for at least 8 months, as far as I can recall).
If that's how you understood my posts, I wasn't clear enough then.
I've told the sage team about how *our* experience in ipython with
git/github has been on the net extremely positive, despite some real
drawbacks (chiefly their sub-par bug tracker). Our development pace
has greatly accelerated and I'm convinced the fluidity of the github
workflow has been a factor there (though certainly not the only one,
since not every project that moves to github automatically picks up
speed).
But I have never said "sage should move to git/github", because I
don't actually know the sage development process well enough to
meaningfully make such a statement. I know sage uses patches, there's
the management of SPKG files, and an elaborate workflow that has
evolved over the years. I have no clue if any or all of this would
map well or not to git/github. That is a decision that only you guys
can make, and it would be foolish for me to propose or try to push you
in any particular direction.
Besides, I should add that I have no stake whatsoever in whether Sage
moves to git, github, bitbucket or anything else: I'm not affiliated
with any of those organizations other than by having an account with
them and liking github for my projects. If one of them helps Sage get
better, I'll be happy; if a self-hosted solution is better for Sage,
then so be it. I have no horse in this race, really.
Cheers,
f
Keshav Kini
I didn't mean to imply you were suggesting we move to github at all. I meant that you were suggesting we *consider* moving to github, based on your own experience. Is that still wrong?
Also, besides your posts I was also remembering something you said to William (and everyone else) at Sage Days 29 last year about how you could vouch for github working well for IPython, if William wanted a testimonial about it. Assuming my memory is correct. Sorry for confusing people.
-Keshav
William Stein
Just for the record: it is definitely not impossible that we could pay. I already pay for trac (in very substantial money for hardware and time to admin it).
> reasonable excuse, you may use more than 300 MB). We are of course not
> going to be using github to host the src/ directories in our SPKGs of
> external programs. However, hosting the spkg-install files, SPKG.txt,
> etc. is something that might be on the table, though probably farther in
> the future than switching to git and a push-pull system for the Sage
> library proper.
>
> -Keshav
>
> ----
> Join us in #sagemath on irc.freenode.net !
>
Fernando Perez
> I didn't mean to imply you were suggesting we move to github at all. I meant
> that you were suggesting we *consider* moving to github, based on your own
> experience. Is that still wrong?
No, that's certainly correct. Given our positive experiences with it,
both in ipython and the other scientific python projects, I think it
deserves consideration for sage. I just wanted to make sure it was
clear that I'm not trying to push you guys in any particular
direction. Not that I would succeed anyway ;)
But by all means, if there's anything from our experience so far that
can help sage make these decisions, ask away and I'll do my best to
provide relevant info.
best,
f
Fernando Perez
> Also, besides your posts I was also remembering something you said to
> William (and everyone else) at Sage Days 29 last year about how you could
> vouch for github working well for IPython, if William wanted a testimonial
> about it.
Oh certainly, and my posts here are sort of that testimonial.
Best,
f
Keshav Kini
> You're right. In fact, bitbucket supports git as well. In many ways,
> bitbucket is nicer than github, for example, the issue tracker is
> better, in my opinion (bitbucket has priorities for tickets, for
> example), and the display on bitbucket is much nicer (the information
> is much more densely displayed, and helpful tree diagrams are out to
> the side of commits), and bitbucket has threaded comments.
Atlassian is the developer of a quite nice bug tracking system called
JIRA. Maybe they have been applying their expertise to bitbucket's
collaborative systems. The issue tracker is definitely a lot better than
github's issue tracker.
> There are some things github does better, though. I can't recall them
> off the top of my head, though.
I find github to be a lot faster than bitbucket, for some reason, though
maybe it's some consequence of being in Singapore.
The main thing that github "does better" is probably to have a userbase
about ten times the size of bitbucket, which means it's much more likely
to find your current collaborators already on github than to find them
already on bitbucket, and the same for discovering potential new
collaborators.
Another feature of github that I really notice to be missing on
bitbucket is the network graph. Take a look at, for example,
http://github.com/sagemath/sagenb/network/ . Nothing like this seems to
exist on bitbucket, and it's really useful for keeping an eye on what
other people are doing on the codebase.
There are also various other minor wins for github, such as the ability
to use reStructuredText on your wiki pages.
Robert Bradshaw
+1 (I do think we should have scripts and/or a sage command so the
average developer doesn't even have to know that git (or whatever else
we are settling on) is being used under the hood to submit new code
and review/extend submitted code. They should be clearly defined in
terms of git commands, so the semi-advanced user can use git directly
with little/no hassle.)
As for issue tracking, I've heard a lot of complaints about the github
one (especially before they launched their new system) and my only
comment here is while it's nice to have issue tracking and source code
repositories synchronized, there's no need to do so as long as we can
link between the two (e.g. use standard naming conventions) Certainly
using trac as a transient source code repository is rather ugly.
- Robert
Keshav Kini
Right, `your post`_ about that a little while ago is definitely
informing my thoughts on all of this.
.. _your post:
https://groups.google.com/d/msg/sage-notebook/P6M-_ajbXJQ/m4oG29xbi5QJ
> As for issue tracking, I've heard a lot of complaints about the github
> one (especially before they launched their new system) and my only
> comment here is while it's nice to have issue tracking and source code
> repositories synchronized, there's no need to do so as long as we can
> link between the two (e.g. use standard naming conventions) Certainly
> using trac as a transient source code repository is rather ugly.
I don't propose to get rid of our Trac instance, though I could be
persuaded otherwise - I don't really have a feel for what the various
popular issue tracker software packages are and what features they have.
I personally have a passing acquaintance with bugzilla (e.g. Mozilla),
debbugs (e.g. Debian), JIRA (e.g. MusicBrainz), redmine (e.g. Quassel
IRC), trac (e.g. Sage), Roundup (e.g. python), google code (e.g.
cyanogenmod), github (e.g. sagenb), and bitbucket (e.g. evil-mode) issue
trackers. Of these, it seems to me that redmine, trac, google code, and
github are pretty lightweight; bugzilla, debbugs, and Roundup are also
pretty lightweight but also have the advantage / disadvantage that they
are strongly email-based (especially in the case of debbugs, where the
UI even sort of suffers for that fact); and JIRA is more heavyweight and
featureful, though it is also a paid, hosted service and not open
source. But I haven't put much thought into which of these categories is
best for Sage. Maybe someone else has?
I do notice that David Roe's description for Review Days includes
something like a feature request for better code review procedures, such
as line comments. Github's tracker can do that without any further
configuration, if we move to github, which is nice. Trac might be able
to, if we do some hacking on it to make it understand where our
repositories and people's forks are - I haven't looked into this.
Assuming we do stay with Trac, I'm still undecided about whether we
should post github-user/branch names on tickets in a special field, or
whether we should automatically associate ticket #n with branches called
github-user/trac-n. I'm leaning towards the former, to allow people to
write code without first worrying about the trac ticket number, and also
reduce the burden on boxen of finding these branches periodically.
Robert Bradshaw
Yep. Note that code reviewing and issue tracking are separable tasks,
and URLs are, well, universal (as well as easy to copy/paste and even
generate).
> Trac might be able
> to, if we do some hacking on it to make it understand where our
> repositories and people's forks are - I haven't looked into this.
>
> Assuming we do stay with Trac, I'm still undecided about whether we
> should post github-user/branch names on tickets in a special field, or
> whether we should automatically associate ticket #n with branches called
> github-user/trac-n. I'm leaning towards the former, to allow people to
> write code without first worrying about the trac ticket number, and also
> reduce the burden on boxen of finding these branches periodically.
We do both :). If the field is empty, try to discover a "default"
location. I'm not worried about the load--it'll probably be light and
if need be we can parallelize the work (my desktop is "keeping up"
with all posted patches on trac with only 3 cores). More importantly,
developer-cpu-seconds are much more valuable than machine-cpu-seconds,
and I'll spend many of the latter to save any of the former :).
- Robert
Keshav Kini
> On Fri, Mar 2, 2012 at 1:06 AM, Keshav Kini <kesha...@gmail.com> wrote:
>> I do notice that David Roe's description for Review Days includes
>> something like a feature request for better code review procedures, such
>> as line comments. Github's tracker can do that without any further
>> configuration, if we move to github, which is nice.
>
> Yep. Note that code reviewing and issue tracking are separable tasks,
> and URLs are, well, universal (as well as easy to copy/paste and even
> generate).
Very good point. However, we seem to be using Trac mostly for code
review and less for issue tracking - as you might expect due to our
policy of all code changes needing to come from a trac ticket. We mark
tickets as "positive review", "needs review", "needs work", and "new",
and care a lot about which of these states (of code review) the ticket
is in.
On the other hand, We also have "issue tracking"-related fields such as
component, owner, priority, milestone, etc., but we don't really seem to
care at all about these. component is mainly used to set owner, and
owner is ignored. priority is vaguely important in that we don't put out
a stable release of Sage unless there are no open "blocker" tickets.
Milestone is basically useless as it's almost always set to the next
planned stable release.
Moving our code review to github would mean that we wouldn't really be
using trac for much anyway, in which case why not just move our issue
tracker there as well? (Possible answer: github's issue tracker is not
extensible enough; see below.) Maybe my perception of our use of Trac is
incorrect, though.
>> Trac might be able
>> to, if we do some hacking on it to make it understand where our
>> repositories and people's forks are - I haven't looked into this.
>>
>> Assuming we do stay with Trac, I'm still undecided about whether we
>> should post github-user/branch names on tickets in a special field, or
>> whether we should automatically associate ticket #n with branches called
>> github-user/trac-n. I'm leaning towards the former, to allow people to
>> write code without first worrying about the trac ticket number, and also
>> reduce the burden on boxen of finding these branches periodically.
>
> We do both :). If the field is empty, try to discover a "default"
> location.
I guess so. I'm wary of doing too much "trying to discover", though. IMO
automated tools should follow some easily predictable routine which is
well-documented. In this case it's certainly simple enough to explain
what Trac would be doing. But for example take the patchbot - it
actually tries to do some kind of (very basic) natural language
processing to figure out what to do! I hope we can avoid doing anything
on that level of "trying to discover" (and I wish the patchbot had a way
to override that stuff too, perhaps a custom field on trac where you
could input instructions in a standardized format). For example, if two
people have branches named trac-n, the script should just do nothing,
not try to pick the most recently updated one, or look for branch names
in the comments, or whatever :)
> I'm not worried about the load--it'll probably be light and
> if need be we can parallelize the work (my desktop is "keeping up"
> with all posted patches on trac with only 3 cores). More importantly,
> developer-cpu-seconds are much more valuable than machine-cpu-seconds,
> and I'll spend many of the latter to save any of the former :).
Sure, I agree with that.
Martin Albrecht
On Friday 02 Mar 2012, Keshav Kini wrote:
[snip]
> > There are some things github does better, though. I can't recall them
> > off the top of my head, though.
>
> I find github to be a lot faster than bitbucket, for some reason, though
> maybe it's some consequence of being in Singapore.
>
> The main thing that github "does better" is probably to have a userbase
> about ten times the size of bitbucket, which means it's much more likely
> to find your current collaborators already on github than to find them
> already on bitbucket, and the same for discovering potential new
> collaborators.
I think this point is moot. Sage contributors definitely use Mercurial because
that's what we currently have to use. I don't know how many of us use git.
I have no horse in the race of which company we could choose to host our
repository and I am glad that we are revamping the development process. I
found myself not contributing bugfixes because I figured it's not worth the
trouble, where trouble is not writing & testing the code.
However, I have to admit that I am rather against this switch-to-git
direction. I see no convincing advantage but a lot of disadvantages, such as:
- learn a new revision control system (although we probably all have to deal
with git at some point)
- we'd ship git in that case and drop mercurial? So no more revision control
in and from Python, which I think is pretty nice.
- it feels to me like: let's switch software to address a social problem.
Robert Bradshaw
It would be much less useful if it simply refused to do anything in
the face of ambiguity, and it does have an override. It's a balance
between the cost of guessing wrong (very low), the value of guessing
correctly (high) and the cost of manual specification (medium). For
something like a compiler the costs are very different :).
> For example, if two
> people have branches named trac-n, the script should just do nothing,
> not try to pick the most recently updated one, or look for branch names
> in the comments, or whatever :)
Or one could preemptively test both, assuming the cycles are
available. For someone at the command line trying to review/pull a
ticket, it should ask.
>> I'm not worried about the load--it'll probably be light and
>> if need be we can parallelize the work (my desktop is "keeping up"
>> with all posted patches on trac with only 3 cores). More importantly,
>> developer-cpu-seconds are much more valuable than machine-cpu-seconds,
>> and I'll spend many of the latter to save any of the former :).
>
> Sure, I agree with that.
>
> -Keshav
>
> ----
> Join us in #sagemath on irc.freenode.net !
>
Robert Bradshaw
<martinr...@googlemail.com> wrote:
> Hi,
>
> On Friday 02 Mar 2012, Keshav Kini wrote:
> [snip]
>> > There are some things github does better, though. I can't recall them
>> > off the top of my head, though.
>>
>> I find github to be a lot faster than bitbucket, for some reason, though
>> maybe it's some consequence of being in Singapore.
>>
>> The main thing that github "does better" is probably to have a userbase
>> about ten times the size of bitbucket, which means it's much more likely
>> to find your current collaborators already on github than to find them
>> already on bitbucket, and the same for discovering potential new
>> collaborators.
>
> I think this point is moot. Sage contributors definitely use Mercurial because
> that's what we currently have to use. I don't know how many of us use git.
>
> I have no horse in the race of which company we could choose to host our
> repository and I am glad that we are revamping the development process. I
> found myself not contributing bugfixes because I figured it's not worth the
> trouble, where trouble is not writing & testing the code.
Exactly the issue I want to solve.
> However, I have to admit that I am rather against this switch-to-git
> direction. I see no convincing advantage but a lot of disadvantages, such as:
>
> - learn a new revision control system (although we probably all have to deal
> with git at some point)
>
> - we'd ship git in that case and drop mercurial? So no more revision control
> in and from Python, which I think is pretty nice.
>
> - it feels to me like: let's switch software to address a social problem.
In terms of git vs. mercurial, I think it's become clear (much more so
than 5 years ago) that the momentum and mindshare has moved to git,
especially among the scientific programming community. There are other
technical advantages (e.g. faster branching) but I think these are
secondary to the social considerations.
Of course moving to a new code-review platform is the real motivation,
and I think the platform should advise what revision control system to
use (as we're shaking things up anyways) as much as the revision
control system place constraints on the platform. You're right, this
is a technical solution to a social problem, but one of the most
effective ways to get good behavior is to build/adopt a system that
facilitates and encourages it.
- Robert
Keshav Kini
> On Fri, Mar 2, 2012 at 1:45 AM, Keshav Kini <kesha...@gmail.com> wrote:
>> For example, if two
>> people have branches named trac-n, the script should just do nothing,
>> not try to pick the most recently updated one, or look for branch names
>> in the comments, or whatever :)
>
> Or one could preemptively test both, assuming the cycles are
> available. For someone at the command line trying to review/pull a
> ticket, it should ask.
I wasn't talking about the patchbot there, actually, just a trac
extension to grok what was "going on" in people's code wrt a ticket and
display something about it on the trac ticket page.
I would actually like the patchbot to NOT do continuous builds and
tests, once we move to a push/pull system. There should be a big button
on each ticket which says "Test Me!", which will cause the patchbot to
fetch whatever branch is specified on the ticket (in some "branch"
field) and put the pair "(ticket, commit-id)" into a queue. (The
patchbot wouldn't actually test commit-id, but would attempt to
temporarily merge commit-id into master, and then test the resulting
commit or return a failure to automatically merge / report that the
branch needed merging / rebasing.)
One reason for this: once we switch to a push/pull system, commits will
hopefully be a lot more frequent, and I question whether we would even
want the patchbot to test each and every commit over the whole doctest
suite. Of course, it would be fine for the patchbot to at least notice
that its last test was out of date and mention this on the trac ticket
somewhere - maybe we could have a little "patchbot info area" <div>
instead of the current patchbot icon.
What do you think?
Ivan Andrus
I'm not involved with sage-combinat so maybe this will be addressed in their work flow, but I have a question as to how one uses a version of sage with many patches applied.
For example, lets say that while using Sage I have found 4 (unrelated) bugs, one of which has a fix on trac that is being reviewed (or whatever). Two of the others I fix myself and put up for review. With a queue-based approach it is entirely trivial to have an instance of sage running that includes any combination of these fixes that I can use to test or (perhaps more importantly) do my actual calculations and research. And at any point I can update any of the fixes and go right back to work.
I assume (since it's git) that there is some magic that will let me specify N branches and then merge them into an unnamed branch that I can then use to build and run the version I'm interested in. Is this in fact possible? Or rather, how much more difficult will this be than using queues?
-Ivan
Jason Grout
git checkout -b mybranch
git pull trac-request-1
git pull trac-request-2
git pull trac-request-3
where trac-request-* are the URLS for the pull requests you'd like to
apply (and that could include multiple commits in a single pull
request---they are branches, not just single patches).
Jason
William Stein
It will be named, not unnamed. You could call it "unnamed" if you want.
Julien Puydt
> I assume (since it's git) that there is some magic that will let me
> specify N branches and then merge them into an unnamed branch that I
> can then use to build and run the version I'm interested in. Is this
> in fact possible? Or rather, how much more difficult will this be
> than using queues?
Create a branch (from an unpatched one, for example) and cherry-pick the
patches you want in.
I insist on "cherry-pick" : that's the name of the git command you
would use, not just an english verb!
Snark on #sagemath
Robert Bradshaw
> Le samedi 03 mars, Ivan Andrus a écrit:
>> I assume (since it's git) that there is some magic that will let me
>> specify N branches and then merge them into an unnamed branch that I
>> can then use to build and run the version I'm interested in. Is this
>> in fact possible? Or rather, how much more difficult will this be
>> than using queues?
>
> Create a branch (from an unpatched one, for example) and cherry-pick the
> patches you want in.
Or merge the N branches in; cherry-picking is for when you only want
part of a branch, and is a lot messier.
- Robert
Robert Bradshaw
> Robert Bradshaw <robe...@math.washington.edu> writes:
>> On Fri, Mar 2, 2012 at 1:45 AM, Keshav Kini <kesha...@gmail.com> wrote:
>>> For example, if two
>>> people have branches named trac-n, the script should just do nothing,
>>> not try to pick the most recently updated one, or look for branch names
>>> in the comments, or whatever :)
>>
>> Or one could preemptively test both, assuming the cycles are
>> available. For someone at the command line trying to review/pull a
>> ticket, it should ask.
>
> I wasn't talking about the patchbot there, actually, just a trac
> extension to grok what was "going on" in people's code wrt a ticket and
> display something about it on the trac ticket page.
Listing them all (and their state) is better than "just do nothing."
> I would actually like the patchbot to NOT do continuous builds and
> tests, once we move to a push/pull system. There should be a big button
> on each ticket which says "Test Me!", which will cause the patchbot to
> fetch whatever branch is specified on the ticket (in some "branch"
> field) and put the pair "(ticket, commit-id)" into a queue. (The
> patchbot wouldn't actually test commit-id, but would attempt to
> temporarily merge commit-id into master, and then test the resulting
> commit or return a failure to automatically merge / report that the
> branch needed merging / rebasing.)
I think that each patchbot could, according to its configuration,
periodically pull a newer version of master (and run tests against
that to isolate faults before trying to apply tickets). This is
essentially what we do now, but "master" is whatever sage version (or
alpha) you're running and doesn't change. (This also bleeds into the
discussion about being able to upgrade sage and all its dependencies
by syncing a single repository...)
> One reason for this: once we switch to a push/pull system, commits will
> hopefully be a lot more frequent, and I question whether we would even
> want the patchbot to test each and every commit over the whole doctest
> suite.
Setting a ticket to "needs review" is basically this button, I don't
think we need a separate one. We wouldn't test "every commit," it's
the tips of what gets pushed (or, possibly, when the master progresses
forward enough). As for wanting to test aboundently, there are two
possible concerns:
(1) the utility of running tests, and
(2) the cost of running tests.
In terms of utility, even if a ticket is not "done" it can be useful
to see what its status is, especially if it affects distant parts of
the library (e.g. breaks monsky-washnitzer...) or for collaboration
(or if a lot of time has passed) do see what is broken or if it no
longer applies.
As for cost, one thing that was far from obvious before actually
writing the patchbot is that testing everything is surprisingly cheap.
I mean so cheap that a single computer can keep up with everything and
still sit idle most of the time. If this made things an order of
magnitude more expensive than we could still keep up by using less
than a dozen cores (on sage.math, or a handful of developers willing
to run niced processes on their machine). We have almost the opposite
problem: I'm willing to donate CPU cycles and there's nothing for my
machine to do right now. Of course there are environmental/financial
costs to running lots of tests too, but even by that metric it's
extremely cheap to run all the tests (especially if you live in an
environment that requires regular heating, as a computer is just as
efficient as any other electric heater but does something useful on
the side).
The other thing to keep in mind is that the patchbot is structured so
the client decides what tickets to accept (and what the relative
priority of each should be). This way high priority tasks (e.g.
needs-review tickets with no background) never have to take a back
seat to lower-priority ones; the lowest priority ones would only
happen when there's "nothing better" to do. Also, clients can decide
what threshold they're willing to contribute at. (This would be
especially useful for an expensive but unique system that might want
to only check positively reviewed tickets once.)
> Of course, it would be fine for the patchbot to at least notice
> that its last test was out of date and mention this on the trac ticket
> somewhere - maybe we could have a little "patchbot info area" <div>
> instead of the current patchbot icon.
There's only so much info you can pack there (or elsewhere on the
page). We could stick a little bit more, but this involves modifying
trac and has diminishing utility over just putting detailed status on
a separate page that's only a click away.
- Robert
Keshav Kini
> On Friday 02 Mar 2012, Keshav Kini wrote:
>> The main thing that github "does better" is probably to have a userbase
>> about ten times the size of bitbucket, which means it's much more likely
>> to find your current collaborators already on github than to find them
>> already on bitbucket, and the same for discovering potential new
>> collaborators.
>
> I think this point is moot. Sage contributors definitely use Mercurial because
> that's what we currently have to use. I don't know how many of us use git.
>
> I have no horse in the race of which company we could choose to host our
> repository and I am glad that we are revamping the development process. I
> found myself not contributing bugfixes because I figured it's not worth the
> trouble, where trouble is not writing & testing the code.
>
> However, I have to admit that I am rather against this switch-to-git
> direction. I see no convincing advantage but a lot of disadvantages, such as:
>
> - learn a new revision control system (although we probably all have to deal
> with git at some point)
I think the majority of Sage developers have not invested much time in
learning Mercurial. What they may have learned from the Sage Developers'
Guide is going to be useless once we switch to a push/pull system,
whether we stay with Mercurial or switch to git.
> - we'd ship git in that case and drop mercurial? So no more revision control
> in and from Python, which I think is pretty nice.
There are interfaces to git which can be used from Python, such as
GitPython_. There's also Dulwich_, which is a reimplementation of much
of the functionality of git in pure Python. Is that what you mean?
.. _GitPython: https://github.com/gitpython-developers/GitPython
.. _Dulwich: https://github.com/jelmer/dulwich
By the way, git, unlike Mercurial, exposes a huge amount of internal
functionality on the command line through its so-called "plumbing
scripts". It's usually not difficult to do what you want with a shell
script.
> - it feels to me like: let's switch software to address a social
> problem.
As Robert said, you're more or less right about this :)
Keshav Kini
> On Fri, Mar 2, 2012 at 5:35 AM, Keshav Kini <kesha...@gmail.com> wrote:
>> I would actually like the patchbot to NOT do continuous builds and
>> tests, once we move to a push/pull system. There should be a big button
>> on each ticket which says "Test Me!", which will cause the patchbot to
>> fetch whatever branch is specified on the ticket (in some "branch"
>> field) and put the pair "(ticket, commit-id)" into a queue. (The
>> patchbot wouldn't actually test commit-id, but would attempt to
>> temporarily merge commit-id into master, and then test the resulting
>> commit or return a failure to automatically merge / report that the
>> branch needed merging / rebasing.)
>
> I think that each patchbot could, according to its configuration,
> periodically pull a newer version of master (and run tests against
> that to isolate faults before trying to apply tickets). This is
> essentially what we do now, but "master" is whatever sage version (or
> alpha) you're running and doesn't change. (This also bleeds into the
> discussion about being able to upgrade sage and all its dependencies
> by syncing a single repository...)
>
>> One reason for this: once we switch to a push/pull system, commits will
>> hopefully be a lot more frequent, and I question whether we would even
>> want the patchbot to test each and every commit over the whole doctest
>> suite.
>
> Setting a ticket to "needs review" is basically this button, I don't
> think we need a separate one.
The difference is that tickets do not automatically get set back to
"doesn't need review" when you push new commits, nor does the patchbot
get notified, necessarily, when you push new commits. This should
fundamentally be a "request action" type of thing, not "set status" type
of thing, IMO.
> We wouldn't test "every commit," it's
> the tips of what gets pushed (or, possibly, when the master progresses
> forward enough).
I don't want pushing my commits to lead to side-effects. People should
be encouraged to push their commits somewhere public as soon as possible
so people can see what they are doing and collaborate.
> As for wanting to test aboundently, there are two
> possible concerns:
>
> (1) the utility of running tests, and
> (2) the cost of running tests.
>
> In terms of utility, even if a ticket is not "done" it can be useful
> to see what its status is, especially if it affects distant parts of
> the library (e.g. breaks monsky-washnitzer...) or for collaboration
> (or if a lot of time has passed) do see what is broken or if it no
> longer applies.
True.
> As for cost, one thing that was far from obvious before actually
> writing the patchbot is that testing everything is surprisingly cheap.
> I mean so cheap that a single computer can keep up with everything and
> still sit idle most of the time. If this made things an order of
> magnitude more expensive than we could still keep up by using less
> than a dozen cores (on sage.math, or a handful of developers willing
> to run niced processes on their machine). We have almost the opposite
> problem: I'm willing to donate CPU cycles and there's nothing for my
> machine to do right now.
Wow, I had no idea. That's certainly good to hear.
> The other thing to keep in mind is that the patchbot is structured so
> the client decides what tickets to accept (and what the relative
> priority of each should be). This way high priority tasks (e.g.
> needs-review tickets with no background) never have to take a back
> seat to lower-priority ones; the lowest priority ones would only
> happen when there's "nothing better" to do. Also, clients can decide
> what threshold they're willing to contribute at. (This would be
> especially useful for an expensive but unique system that might want
> to only check positively reviewed tickets once.)
In that case maybe we could set manually requested tests at a higher
priority and automatic tests at a lower priority. The automatic tests
would be silent if successful and result in a warning comment to the
ticket if something went wrong, whereas the manually requested tests
could post to the ticket a link to a full log of the doctest.
I would want to avoid automatic doctests marking anything as "failing",
though. The current red/green/yellow status should be based on the last
manually-requested doctest, not the last doctest. Perhaps green and red
for "last manually requested doctest passed" and "last manually
requested doctest failed", respectively, with an overlaid question mark
if the branch set on the ticket is not currently on the commit which was
last manually tested. We could make the question mark green or red
depending on the status of the latest autotest... I'm almost literally
painting a bikeshed here, haha.
>> Of course, it would be fine for the patchbot to at least notice
>> that its last test was out of date and mention this on the trac ticket
>> somewhere - maybe we could have a little "patchbot info area" <div>
>> instead of the current patchbot icon.
>
> There's only so much info you can pack there (or elsewhere on the
> page). We could stick a little bit more, but this involves modifying
> trac and has diminishing utility over just putting detailed status on
> a separate page that's only a click away.
Right, that's fine too, of course.
Robert Bradshaw
Nor should it, if this is still the correct state (which it often is).
> nor does the patchbot
> get notified, necessarily, when you push new commits.
But there's no reason it couldn't.
> This should fundamentally be a "request action" type of thing, not "set status" type
> of thing, IMO.
Can you think of any case where you would want to change the status to
"needs review" and not activate the patchbot?
>> We wouldn't test "every commit," it's
>> the tips of what gets pushed (or, possibly, when the master progresses
>> forward enough).
>
> I don't want pushing my commits to lead to side-effects. People should
> be encouraged to push their commits somewhere public as soon as possible
> so people can see what they are doing and collaborate.
I agree, which is fine as long as the patchbot is side effect free
(other than providing additional information, perhaps even in an
(easily ignored, but usually useful) email for failures).
>> As for wanting to test aboundently, there are two
>> possible concerns:
>>
>> (1) the utility of running tests, and
>> (2) the cost of running tests.
>>
>> In terms of utility, even if a ticket is not "done" it can be useful
>> to see what its status is, especially if it affects distant parts of
>> the library (e.g. breaks monsky-washnitzer...) or for collaboration
>> (or if a lot of time has passed) do see what is broken or if it no
>> longer applies.
>
> True.
>
>> As for cost, one thing that was far from obvious before actually
>> writing the patchbot is that testing everything is surprisingly cheap.
>> I mean so cheap that a single computer can keep up with everything and
>> still sit idle most of the time. If this made things an order of
>> magnitude more expensive than we could still keep up by using less
>> than a dozen cores (on sage.math, or a handful of developers willing
>> to run niced processes on their machine). We have almost the opposite
>> problem: I'm willing to donate CPU cycles and there's nothing for my
>> machine to do right now.
>
> Wow, I had no idea. That's certainly good to hear.
Yeah, it was a big surprise to me as well.
>> The other thing to keep in mind is that the patchbot is structured so
>> the client decides what tickets to accept (and what the relative
>> priority of each should be). This way high priority tasks (e.g.
>> needs-review tickets with no background) never have to take a back
>> seat to lower-priority ones; the lowest priority ones would only
>> happen when there's "nothing better" to do. Also, clients can decide
>> what threshold they're willing to contribute at. (This would be
>> especially useful for an expensive but unique system that might want
>> to only check positively reviewed tickets once.)
>
> In that case maybe we could set manually requested tests at a higher
> priority and automatic tests at a lower priority. The automatic tests
> would be silent if successful and result in a warning comment to the
> ticket if something went wrong, whereas the manually requested tests
> could post to the ticket a link to a full log of the doctest.
>
> I would want to avoid automatic doctests marking anything as "failing",
> though. The current red/green/yellow status should be based on the last
> manually-requested doctest, not the last doctest. Perhaps green and red
> for "last manually requested doctest passed" and "last manually
> requested doctest failed", respectively, with an overlaid question mark
> if the branch set on the ticket is not currently on the commit which was
> last manually tested.
I can't think of any case that the results of the last manually
requested run would be preferable to a more recent automatically
triggered result.
> We could make the question mark green or red
> depending on the status of the latest autotest... I'm almost literally
> painting a bikeshed here, haha.
:)
>>> Of course, it would be fine for the patchbot to at least notice
>>> that its last test was out of date and mention this on the trac ticket
>>> somewhere - maybe we could have a little "patchbot info area" <div>
>>> instead of the current patchbot icon.
>>
>> There's only so much info you can pack there (or elsewhere on the
>> page). We could stick a little bit more, but this involves modifying
>> trac and has diminishing utility over just putting detailed status on
>> a separate page that's only a click away.
>
> Right, that's fine too, of course.
Which is how it is now.
- Robert
Keshav Kini
Of course, but... well, see below.
>> nor does the patchbot
>> get notified, necessarily, when you push new commits.
>
> But there's no reason it couldn't.
>
>> This should fundamentally be a "request action" type of thing, not "set status" type
>> of thing, IMO.
>
> Can you think of any case where you would want to change the status to
> "needs review" and not activate the patchbot?
No, but I *can* think of cases where I would want to activate the
patchbot and not change the status to "needs review" - for example, at
some of the times when I have pushed new commits to a branch which I've
already marked as "needs review". This will no doubt be a common
practice, just as it is common to push more commits to open pull
requests in github's workflow, even if nobody has indicated they don't
like the current commits.
>>> We wouldn't test "every commit," it's
>>> the tips of what gets pushed (or, possibly, when the master progresses
>>> forward enough).
>>
>> I don't want pushing my commits to lead to side-effects. People should
>> be encouraged to push their commits somewhere public as soon as possible
>> so people can see what they are doing and collaborate.
>
> I agree, which is fine as long as the patchbot is side effect free
> (other than providing additional information, perhaps even in an
> (easily ignored, but usually useful) email for failures).
Well, I was considering the computation time required for a doctest
suite runthrough as a "side effect". But we truly have as much of a glut
of CPU cycles as you say... :) And again we can make these automatic
runs have lower priority anyway.
>> I would want to avoid automatic doctests marking anything as "failing",
>> though. The current red/green/yellow status should be based on the last
>> manually-requested doctest, not the last doctest. Perhaps green and red
>> for "last manually requested doctest passed" and "last manually
>> requested doctest failed", respectively, with an overlaid question mark
>> if the branch set on the ticket is not currently on the commit which was
>> last manually tested.
>
> I can't think of any case that the results of the last manually
> requested run would be preferable to a more recent automatically
> triggered result.
Here's what I had in mind: I push a commit. It works! The patchbot gives
its OK to my branch. I push another commit. Oops, I broke it. I realize
this soon thereafter, and begin working on a fix. Meanwhile, the
patchbot sees my broken commit and tests it. When the tests fail, my
branch is marked as broken. I finally finish fixing the problem, and
push my fix, but now the patchbot happens to be overloaded and my new
branch head is stuck at the end of a long testing queue. My branch is
continues to be marked broken. Everyone is ignoring my ticket even
though it is ready for review! Even if the ticket displays something
like "test result out of date", a good test result that's out of date
inspires more confidence than a bad test result that's out of date, and
would have attracted more reviewers. Oh, how I wish I could have stopped
that automatic test of my broken commit!
... OK, I can see my scenario is not exactly a big deal, now that I
write it out like that...
By the way, if tickets instead just display "pending" whenever their
tests are out of date, without disclosing what the latest (now
out-of-date) result was, it seems possible that patchbot results will
become hidden by a "pending" mark much or most of the time, especially
if I commit faster than the patchbot can test my commits. This makes the
indicator much less useful, so I assume it will not be set up that way.
Robert Bradshaw
In which case the patchbot will still be triggered due to needs review
+ out of date tests.
:). In this case you quickly fire up/tweek your own patchbot client
with an absurdity high priority to test that one ticket. In any case,
better than a good test result on a bad branch.
> By the way, if tickets instead just display "pending" whenever their
> tests are out of date, without disclosing what the latest (now
> out-of-date) result was, it seems possible that patchbot results will
> become hidden by a "pending" mark much or most of the time, especially
> if I commit faster than the patchbot can test my commits. This makes the
> indicator much less useful, so I assume it will not be set up that way.
Yeah, I agree it would be useful to have some kind of qualified "the
last iteration of this branch passed/failed" status when it becomes
out of date due to new commits. Generally peoples workflow seems to be
commit, commit, commit, push, [lots of time passes], commit, commit,
push [more time passes.
- Robert