Some thought on trac and the Sage development process - feedback wanted
mabshoff
I will do a 15 minute talk about trac and the Sage development
process. I figured that because
a) not everybody will be there
b) I might miss some issues and will probably be completely wrong on
others
c) we might not have a lot of time after to discuss the issues
it would be a good idea to get some input from sage-devel. The
following is not very structured and written down in "stream of
consciousness" mode. So rip it apart and let me know how to do this
better. Once we get closer I will obviously turn this into a proper
presentation, but it should also end up somewhere in the
documentation.
*Better Sage Project Management With The Help of Trac*
# Opening Remarks #
The is more about than just the use of trac to help out with Sage
development, but also about work flow in general and how we can
make developing Sage more efficient now that more and more
developers are joining Sage.
# The Situation #
* "everything you work on is a ticket" rule seems to work out well.
* the number of open tickets keeps on rising (from about 220 to
slightly under 300 at the moment), but turnover is rather high.
* Bug Days have closed up to 70 ticker, the lower bounds seems to
be about 40.
* The old cruft, i.e. tickets closed a long the way, that was left
in trac seems to have all been closed.
* If you do development for Sage or consider yourself a user how
wants to report issues get account for Sage's trac installation:
either email William Stein or alternatively contact him on
#sage-devel on freenode. Preferably get a tray account with the
same name as your google group nickname, so that one know how to
contact you.
# Milestones vs. Releases #
* Milestones are usually goals to be met while working toward a
release. In Sage's trac we use milestones instead of releases,
but unless somebody volunteers we will stick with the current
model
* finely grained releases are good, release early and often
* it is realistic to make a big release and schedule at least one
bugfix release after that to sort out the inevitable "doctest X
is broken on distribution Y and compiler Z" problem
# Opening Tickets #
* Before opening a ticket make sure that nobody else has opened
a ticket about the same or closely related issue.
* It is better to open several specific tickets than one that is
very broad.
* Be precise: If foo doesn't work on OSX, but is fine on Linux
mention that in the title, also use the keyword option to make
searches pick up the issue
* be careful with the priority: "blocker" should be used sparingly,
don't be surprised that such a ticker is reclassified
## Working On Tickets ##
* tickets are kind of like homicides: They either get solved in 48
hours, otherwise they have a tendency to become cold cases. This
is mostly an issue with defects, there are many enhancements
possible for Sage, but too few developers to implement all the
good ideas. But it is useful to keep ideas in a central place
because in the google groups they tend to get lost once they drop
off the first page
* If you are a developer be nice and try to solve a stale/old ticket
every once in a while.
* I and mhansen (and some other people I properly forgot to mention)
regularly do triage, especially before Bug Days. Triage in this
context means that we look at new bugs and classify them according
to out perceived priority.
* every bug fixed should result in a doctest: Example Zombie det()
problem with LinBox, considered fixed twice, but reopened in both
cases.
# Assigning Tickets #
* each ticket should have a milestone assigned
* defect vs. enhancement vs. task
* if you are unsure whom to assign to assign to somebody or was
* certain categories have default people to assign to
* if you have been assigned a ticket you should either accept it
or assign it back to "somebody" or "tbd"
# Closing Tickets #
* if you have a solution/patch attach it to the ticket and indicate
that there is a solution available by adding "[with patch]" to the
title. It might also be a good idea to reassign the ticket to the
current bugfix release if it is scheduled for some milestone that
is far in the future.
* Do not close the ticket, but let William close it once the patch
has actually been merged. In the past patches have been lost due
to the fact that somebody closed the ticket and William never saw
the patch. Another possibility is especially during Bug Days or
interactive discussions via IRC that you can close it after you
have been encouraged to do so.
* Somewhat on an exception is wontfix or duplicate tickets, but it
is also a good idea to check with somebody in IRC. Alternatively
write an email to sage-devel or William directly so he can react
in case a mistake was made.
# Desirable Trac Features #
* more statistics
* a default CC to a google group sage-trac, this requires that a
google group is created [done] and that somebody with admin
access to sagemath.org enables smtp for trac [not done yet]
* loads more interesting bits at http://trac-hacks.org/ - but we
should not merge in too many extensions because it makes
maintainability more difficult once we upgrade to newer trac
releases. I maintain several phpBB installations and not
adding a wild bunch of mods proved to be a smart choice.
Chris Chiasson
* every bug fixed should result in a doctest: Example Zombie det()
problem with LinBox, considered fixed twice, but reopened in both
cases.
^^^ this should be first (even though this is an unordered list)
* tickets are kind of like homicides: They either get solved in 48
hours, otherwise they have a tendency to become cold cases.
^^^ you may want to preface this with something like "sensitive people
should cover their ears"
On Sep 23, 11:00 am, mabshoff <Michael.Absh...@fsmath.mathematik.uni-
> * loads more interesting bits athttp://trac-hacks.org/- but we
William Stein
>
> since you asked for reactions:
>
> * every bug fixed should result in a doctest: Example Zombie det()
> problem with LinBox, considered fixed twice, but reopened in both
> cases.
>
> ^^^ this should be first (even though this is an unordered list)
I strongly agree with this. Moreover, in the above instance the
fact that det was broken was only found because I retroactively
decided to put in a doctest to verify that the problem was really
fixed, and doing doctesting on some architectures showed that
it wasn't fixed on all architectures.
> * tickets are kind of like homicides: They either get solved in 48
> hours, otherwise they have a tendency to become cold cases.
>
> ^^^ you may want to preface this with something like "sensitive people
> should cover their ears"
Also, I don't agree with it. That is only the case if the ticket isn't
assigned to a milestone. Moreover, often the solution can be much
clearer a month later.
William
Robert Bradshaw
> # Closing Tickets #
>
> * if you have a solution/patch attach it to the ticket and indicate
> that there is a solution available by adding "[with patch]" to the
> title. It might also be a good idea to reassign the ticket to the
> current bugfix release if it is scheduled for some milestone that
> is far in the future.
> * Do not close the ticket, but let William close it once the patch
> has actually been merged. In the past patches have been lost due
> to the fact that somebody closed the ticket and William never saw
> the patch. Another possibility is especially during Bug Days or
> interactive discussions via IRC that you can close it after you
> have been encouraged to do so.
How hard would it be to make a new status for "fixed" (as opposed to
"committed/released") rather than modifying the title or something
equally hackish?
> # Desirable Trac Features #
>
> * more statistics
> * a default CC to a google group sage-trac, this requires that a
> google group is created [done] and that somebody with admin
> access to sagemath.org enables smtp for trac [not done yet]
> * loads more interesting bits at http://trac-hacks.org/ - but we
> should not merge in too many extensions because it makes
> maintainability more difficult once we upgrade to newer trac
> releases. I maintain several phpBB installations and not
> adding a wild bunch of mods proved to be a smart choice.
* Mercurial bundle browsing. I don't know how hard this would be to
do, but I might end up doing some trac hacking as part of my TA
appointment here so I could perhaps look into this.
- Robert
Jason Grout
> On Sep 23, 2007, at 9:00 AM, mabshoff wrote:
>
>> # Closing Tickets #
>>
>> * if you have a solution/patch attach it to the ticket and indicate
>> that there is a solution available by adding "[with patch]" to the
>> title. It might also be a good idea to reassign the ticket to the
>> current bugfix release if it is scheduled for some milestone that
>> is far in the future.
>> * Do not close the ticket, but let William close it once the patch
>> has actually been merged. In the past patches have been lost due
>> to the fact that somebody closed the ticket and William never saw
>> the patch. Another possibility is especially during Bug Days or
>> interactive discussions via IRC that you can close it after you
>> have been encouraged to do so.
>
> How hard would it be to make a new status for "fixed" (as opposed to
> "committed/released") rather than modifying the title or something
> equally hackish?
>
How hard would it be to add a few states to track the development of a
patch (I'm modeling this after Drupal ticket tracking):
"states" of a ticket:
active
active (needs more info)
patch (code needs work)
patch (code needs review)
patch (code needs to be committed)
fixed
duplicate
invalid
worksforme
wontfix
bydesign
where the last 6 states are "resolved" states (and the last two states
may mean the same thing, but it sounds nice to say "bydesign" than the
flat refusal "wontfix")
-Jason
mabshoff
hopefully I incorporated all feed back:
*Better Sage Project Management With The Help of Trac - rev 2*
# Opening Remarks #
The is more about than just the use of trac to help out with Sage
development, but also about work flow in general and how we can
make developing Sage more efficient now that more and more
developers are joining Sage.
# The Situation #
* The use of trac seems to work out well. It seems that we do not
lose bug reports and issues keep getting added to make sure they
do not disappear. "everything you work on is a ticket" rule
has been accepted by the vast majority of developers.
* The number of open tickets keeps on rising (from about 220 to
slightly under 300 at the moment), but turnover is rather high.
The increased numbers of bugs found and enhancements requested
is a good sign because it shows that Sage is being used by more
and more people and that people care enough to report problems.
Many times I have looked for software to do a specific task
and encountered bugs. I ususally do neither attempt to fix
the problem nor report it unless it is something I see potential
in.
* Bug Days have closed up to 70 ticker, the lower bounds seems to
be about 40. It will be interesting to see how many tickets will
be closed/opened during SD5.
* The old cruft, i.e. tickets closed along the way, that were left
in trac seems to have all been closed.
* If you do development for Sage or consider yourself a user who
wants to report issues get an account for Sage's trac installation:
either email William Stein or alternatively contact him on
#sage-devel on freenode. Preferably get a tray account with the
same name as your google group nickname, so that one know how to
contact you.
# Releases, Milestones vs. Releases #
* Milestones are usually goals to be met while working toward a
release. In Sage's trac we use milestones instead of releases,
but unless somebody volunteers to clean up all the old milestones
we will stick with the current model. It doesn't make a whole lot
of a difference if we use milestone instead of release.
* finely grained releases are good, release early and often is the
way to go, especially as more and more patches are coming in.
* it is realistic to make a big release and schedule at least one
bugfix release after that to sort out the inevitable "doctest X
is broken on distribution Y and compiler Z" problem. If we ever
get access to a compile farm the number of those issues will
hopefully decrease, but given the number of compilers and
operating systems out there one has to be realistic to expect
problems.
# Opening Tickets #
* Before opening a ticket make sure that nobody else has opened
a ticket about the same or closely related issue.
* It is better to open several specific tickets than one that is
very broad.
* Be precise: If foo doesn't work on OSX, but is fine on Linux
mention that in the title, also use the keyword option to make
searches pick up the issue
* be careful with the priority: "blocker" should be used sparingly,
don't be surprised that such a ticker is reclassified. On the other
hand tickets that one might consider to be of non-blocker quality
might be promoted. When in doubt write an email to sage-devel or
ask around on #sage-devel.
# Working On Tickets #
* Every bug fixed should result in a doctest: Example Zombie det()
problem with LinBox, considered fixed twice, but reopened in both
cases.
* Cooperative debugging via IRC is faster by at least a magnitude. If
you haven't learned how to use IRC please do so. If you have
problems to use IRC because of firewalls, but you do have an
account
on sage.math you can use irssi via ssh there. If you have a flaky
connection (hello malb ;) - you can use it together with screen.
* This is mostly an issue with defects, there are many enhancements
possible for Sage, but too few developers to implement all the
good ideas. But it is useful to keep ideas in a central place
because in the google groups they tend to get lost once they drop
off the first page
* If you are a developer be nice and try to solve a stale/old ticket
every once in a while.
* mhansen and I (and some other people I properly forgot to mention)
regularly do triage, especially before Bug Days. Triage in this
context means that we look at new bugs and classify them according
to out perceived priority.
# Assigning Tickets #
* each ticket should have a milestone assigned
* defect vs. enhancement vs. task
* if you are unsure whom to assign to assign to somebody or was
* certain categories have default people to assign to
* if you have been assigned a ticket you should either accept it
or assign it back to "somebody" or "tbd"
# Closing Tickets #
* if you have a solution/patch attach it to the ticket and indicate
that there is a solution available by adding "[with patch]" to the
title. It might also be a good idea to reassign the ticket to the
current bugfix release if it is scheduled for some milestone that
is far in the future. The "[with patch]" crutch should be solved in
a more elegant way - see below.
* Do not close the ticket, but let William close it once the patch
has actually been merged. In the past patches have been lost due
to the fact that somebody closed the ticket and William never saw
the patch. Another possibility is especially during Bug Days or
interactive discussions via IRC that you can close it after you
have been encouraged to do so.
* Somewhat on an exception is wontfix or duplicate tickets, but it
is also a good idea to check with somebody in IRC. Alternatively
write an email to sage-devel or William directly so he can react
in case a mistake was made.
# Desirable Trac Features #
* more statistics
* a default CC to a google group sage-trac, this requires that a
google group is created [done] and that somebody with admin
access to sagemath.org enables smtp for trac [not done yet]
* mercurial bundle browsing: This is currently not support by
the mercurial plugin for trac, but Robert Bradshaw might end up
doing some trac hacking as part of his TA appointment at UW so
he could perhaps look into this.
* IRC logs: A trac extension to log IRC channels exists. We should
consider using it, but make it clear in #sage-devel that the
discussion is logged. We should also consider adding #sage-support
so that the signal/noise ratio for development stays high on
#sage-devel, but sage newbies could ask questions about Sage use.
This would also potentially be a way for people to move up in
the food chain from sage newbie to experienced user to eventually
sage developer.
* loads more interesting bits at http://trac-hacks.org/ - but we
should not merge in too many extensions because it makes
maintainability more difficult once we upgrade to newer trac
releases. I maintain several phpBB installations and not
adding a wild bunch of mods proved to be a smart choice.
* Jason Grout has suggested new "states" of tickets to make
the system more finely grained:
active
active (needs more info)
patch (code needs work)
patch (code needs review)
patch (code needs to be committed)
fixed
duplicate
invalid
worksforme
wontfix
bydesign
The names have been modeled after Drupal's ticket tracking system
[end of rev2]
Let me know if you have any more suggestions/fixes/criticism. The trac
talk at SD5 has been scheduled as the last talk on Monday, so we
should be able to get a discussion going afterwards without being
forced to stop due to other talks.
Cheers,
Michael