New Django ticket system workflow ideas

[Adrian Holovaty]

Hello all,

I'd really like to streamline the way we interact with our ticket system, Trac.

We've fallen behind on managing tickets, which is not only frustrating
to contributors but a hindrance to Django's growth, as solid
contributions are made but eventually get lost in the sea of
half-baked ideas.

The major problem, aside from a general lack of time on core
developers' part to manage tickets, is that our current Trac set up
does not mesh well with the workflow we've established over the past
year and a half.

I'll give a personal example. Occasionally I'll have some free time in
the evening to close a few Django tickets, but I won't know where to
start. I'll inevitably look at the "Tickets with patches" report page,
which is generated by listing all the open tickets with "[patch]" in
the title. I'll choose one and have a look, but in many cases the
patch has a fatal flaw -- maybe it lacks documentation, lacks unit
tests or makes a design decision that isn't necessarily in-line with
the rest of Django's philosophy (or needs to be discussed). At this
point, I can't do much other than leave a comment explaining the
patch's shortcomings. And then -- here's the tragedy -- the ticket
goes back into the sea of tickets, with *no visible way of
distinguishing it from the rest*.

Simply put, we need a way to tell Trac which "stage" of the process
each ticket is in.

I propose, then, to start using a tagging system that tells Trac
*exactly* what we're waiting on for each ticket. Here are a few ideas:

* If a ticket does *not* have a patch, a committer should read the
ticket and add either a "[design-q]" or "[ok-needs-patch]" prefix to
the ticket's title. [design-q] means "this concept requires a design
decision," which should be discussed either in the ticket comments or
on django-developers. [ok-needs-patch] means the idea is good as
stated and we need a patch that implements it.

* If a ticket *does* have a patch, its title should have a "[patch]"
prefix, as before. A committer should read the patch. If the patch is
good, commit it! If the patch is not good, insert one or more of the
following in the ticket's title:

[needs-docs] -- The existing patch needs documentation.

[needs-tests] -- The existing patch needs unit tests.

[needs-better-patch] -- The existing patch is too hackish or
otherwise unacceptable.

We'll also create some extra Trac reports that isolate all of the
tickets of a certain status.

(Clearly it'd be best if we didn't have to use silly prefixes in the
ticket titles to designate ticket status. I'm not familiar enough with
Trac to know whether it can be hacked to add separate fields for this
purpose.)

Once we get this system going, things will be much easier for the
commiters, contributors will get a better feeling for the status of
their tickets, and it'll expose small to-do tasks for people who are
looking for a way to contribute to Django.

ALSO, while we're talking about changing Trac, I believe we should
remove the following fields from the ticket interface, as I don't
think any of the core devs (including I) use them:

* Priority -- Used only by ticket submitters who want to add
artificial urgency.
* Severity -- Used only by ticket submitters who want to add
artificial urgency.
* Assign to -- Eh, we don't really use this.
* Milestone -- We don't really use this, either.

I never use "Keywords" either, but I know some contributors enter
keywords just for their own information management purposes, so it's
cool by me if we leave that in.

Thoughts?

Adrian

--
Adrian Holovaty
holovaty.com | djangoproject.com

[Malcolm Tredinnick]

On Sat, 2007-01-13 at 21:09 -0600, Adrian Holovaty wrote:
> Hello all,
>
> I'd really like to streamline the way we interact with our ticket system, Trac.
>
> We've fallen behind on managing tickets, which is not only frustrating
> to contributors but a hindrance to Django's growth, as solid
> contributions are made but eventually get lost in the sea of
> half-baked ideas.
>
> The major problem, aside from a general lack of time on core
> developers' part to manage tickets, is that our current Trac set up
> does not mesh well with the workflow we've established over the past
> year and a half.
>
> I'll give a personal example. Occasionally I'll have some free time in
> the evening to close a few Django tickets, but I won't know where to
> start. I'll inevitably look at the "Tickets with patches" report page,
> which is generated by listing all the open tickets with "[patch]" in
> the title. I'll choose one and have a look, but in many cases the
> patch has a fatal flaw -- maybe it lacks documentation, lacks unit
> tests or makes a design decision that isn't necessarily in-line with
> the rest of Django's philosophy (or needs to be discussed). At this
> point, I can't do much other than leave a comment explaining the
> patch's shortcomings. And then -- here's the tragedy -- the ticket
> goes back into the sea of tickets, with *no visible way of
> distinguishing it from the rest*.

Trac's non-feature of not automatically sending changes to the reporter
(which follows from not requiring a valid reporter email) doesn't help
here, either. Even I've been bitten by this as a bug reporter: I file a
ticket and it's only by blind luck that I notice it needs follow up
information. Having to remember to also add myself to the CC field is a
bit of a Trac-specific thing, so it's something I forget (and possibly
others, too, who use multiple bug tracking systems). Not sure there's a
lot we can do about that, though. :-(

> Simply put, we need a way to tell Trac which "stage" of the process
> each ticket is in.
>
> I propose, then, to start using a tagging system that tells Trac
> *exactly* what we're waiting on for each ticket. Here are a few ideas:
>
> * If a ticket does *not* have a patch, a committer should read the
> ticket and add either a "[design-q]" or "[ok-needs-patch]" prefix to
> the ticket's title. [design-q] means "this concept requires a design
> decision," which should be discussed either in the ticket comments or
> on django-developers. [ok-needs-patch] means the idea is good as
> stated and we need a patch that implements it.
>
> * If a ticket *does* have a patch, its title should have a "[patch]"
> prefix, as before. A committer should read the patch. If the patch is
> good, commit it! If the patch is not good, insert one or more of the
> following in the ticket's title:
>
> [needs-docs] -- The existing patch needs documentation.
>
> [needs-tests] -- The existing patch needs unit tests.
>
> [needs-better-patch] -- The existing patch is too hackish or
> otherwise unacceptable.
>
> We'll also create some extra Trac reports that isolate all of the
> tickets of a certain status.
>
> (Clearly it'd be best if we didn't have to use silly prefixes in the
> ticket titles to designate ticket status. I'm not familiar enough with
> Trac to know whether it can be hacked to add separate fields for this
> purpose.)

Why don't you want to use the keywords field for this? It can still mix
and match with people wanting to use their own keywords; we just claim a
few short ones for triage purposes.

Like you, I'd be reluctant to overload the title field like this because
it cuts down on the amount of space that can be used for a descriptive
title in summary reports. If we can do Trac reports on reg-exps on the
keyword field, that would be nice.

>
> Once we get this system going, things will be much easier for the
> commiters, contributors will get a better feeling for the status of
> their tickets, and it'll expose small to-do tasks for people who are
> looking for a way to contribute to Django.
>
> ALSO, while we're talking about changing Trac, I believe we should
> remove the following fields from the ticket interface, as I don't
> think any of the core devs (including I) use them:
>
> * Priority -- Used only by ticket submitters who want to add
> artificial urgency.
> * Severity -- Used only by ticket submitters who want to add
> artificial urgency.

Mostly agree. We don't use them because we they are meaningless in so
many cases at the moment. The only time these might be useful is coming
up to a release: being able to tag the "release critical" bugs for
easier hunt-and-destroy. Not a big issue at the moment, since we don't
really have that many tickets, so a quick scan of a couple of pages
works. But maybe keeping priority would be useful (and telling
submitters to keep their mitts off that field). If we got rid of these
it wouldn't be the end of the world.

> * Assign to -- Eh, we don't really use this.

It doesn't work too well at the moment because a tickets get assigned en
masse to you or Jacob, for the most part. However, it is useful to know
when somebody is actively working on a ticket (or to be able to look at
all the tickets I am working on). So "assigned" is a useful field, but I
would prefer the initial owner was something more generic, rather than
"adrian" or "jacob" (only because it's currently hard to tell if the
Adrian or Jacob entities are working on the ticket or just getting
screwed by auto-assignation).

> * Milestone -- We don't really use this, either.
>
> I never use "Keywords" either, but I know some contributors enter
> keywords just for their own information management purposes, so it's
> cool by me if we leave that in.
>
> Thoughts?

I like these ideas. A consequence here is something we've talked about
previously, too: we (developers and triage volunteers) can then loosely
commit to following up every new ticket with a triage annotation within
a given time period (something like 3 days). If we find we aren't doing
that, then we have to work out why we are falling behind. There are
enough people with clues around now that we should be able to keep up.

Cheers,
Malcolm

[Honza Král]

I think it would be useful to concentrate all the discussion in one
place, the mailing list. It would be easier for developers and users
to keep track of things...

[ok-needs-patch] means the idea is good as
> stated and we need a patch that implements it.
>
> * If a ticket *does* have a patch, its title should have a "[patch]"
> prefix, as before. A committer should read the patch. If the patch is
> good, commit it! If the patch is not good, insert one or more of the
> following in the ticket's title:
>
> [needs-docs] -- The existing patch needs documentation.
>
> [needs-tests] -- The existing patch needs unit tests.
>
> [needs-better-patch] -- The existing patch is too hackish or
> otherwise unacceptable.
>
> We'll also create some extra Trac reports that isolate all of the
> tickets of a certain status.
>
> (Clearly it'd be best if we didn't have to use silly prefixes in the
> ticket titles to designate ticket status. I'm not familiar enough with
> Trac to know whether it can be hacked to add separate fields for this
> purpose.)
>
> Once we get this system going, things will be much easier for the
> commiters, contributors will get a better feeling for the status of
> their tickets, and it'll expose small to-do tasks for people who are
> looking for a way to contribute to Django.

I am looking forward to that ;)

>
> ALSO, while we're talking about changing Trac, I believe we should
> remove the following fields from the ticket interface, as I don't
> think any of the core devs (including I) use them:
>
> * Priority -- Used only by ticket submitters who want to add
> artificial urgency.
> * Severity -- Used only by ticket submitters who want to add
> artificial urgency.

this MIGHT be useful, if only few people had rights to change that, so
that people willing to help will know what really is the priority... I
agree that it is useless as it is now

for example if you don't have the time, but recognize a real problem
that should be addressed.

> * Assign to -- Eh, we don't really use this.
> * Milestone -- We don't really use this, either.
>
> I never use "Keywords" either, but I know some contributors enter
> keywords just for their own information management purposes, so it's
> cool by me if we leave that in.

I try and enter some keywords in order to help trac search (I have NO
idea whether trac somehow leverages the keywords for search but it
only seems logical ;) )

>
> Thoughts?

with the amount of django developers here, wouldn't it be simpler to
rewrite the ticket handling system in django to provide for better
flexibility?
if we stick with just the ticket system it should be fairly simple to
develop and to integrate it with TRAC...
the only problem would be importing the old tickets and that doesn't
seem unsolvable either (trac db schema is pretty straightforward)...

I personally am willing to help with this in order to improve the
patch/ticket handling process...

>
> Adrian
>
> --
> Adrian Holovaty
> holovaty.com | djangoproject.com
>
> >
>

--
Honza Král
E-Mail: Honza...@gmail.com
ICQ#: 107471613
Phone: +420 606 678585

[Gary Wilson]

You also might want to take a look at:
http://trac.edgewall.org/wiki/TracTicketTriage
where the Trac team lays out how they use milestones, keywords, and
ticket queries to help with ticket workflow.

There is talk of a real ticket workflow system for Trac
http://trac.edgewall.org/wiki/WorkFlow
but, contrary to what's mentioned on that and a few other pages, it
doesn't seem that it will be going in to 0.11. There was a branch in
the sandbox that was deleted about a month ago due to a lack of code.

> (Clearly it'd be best if we didn't have to use silly prefixes in the
> ticket titles to designate ticket status. I'm not familiar enough with
> Trac to know whether it can be hacked to add separate fields for this
> purpose.)
>
> Once we get this system going, things will be much easier for the
> commiters, contributors will get a better feeling for the status of
> their tickets, and it'll expose small to-do tasks for people who are
> looking for a way to contribute to Django.
>
> ALSO, while we're talking about changing Trac, I believe we should
> remove the following fields from the ticket interface, as I don't
> think any of the core devs (including I) use them:
>
> * Priority -- Used only by ticket submitters who want to add
> artificial urgency.
> * Severity -- Used only by ticket submitters who want to add
> artificial urgency.
> * Assign to -- Eh, we don't really use this.
> * Milestone -- We don't really use this, either.

I agree with the removal of all but Milestone. I think the Milestones,
when used, are a nice way of organizing the tickets that need to be
completed. For the big ticket stomping that Jacob has mentioned, what
better than to start assigning tickets to Milestones (fix = milestone
set to 1.0, reject = closed as wontfix, or defer = set milestone to
1.1, 1.2, none, etc.). This not only serves as a way to find tickets
to work on, pushing towards the next milestone, but also gives you
pretty charts:
http://trac.edgewall.org/milestone/0.11

I think it's also cool how the Trac devs use the milestone wiki areas
(editable only by admins by default, but I think this might be
configurable) for notes about what features are planned for that
release and also notes about what has been completed already.

> I never use "Keywords" either, but I know some contributors enter
> keywords just for their own information management purposes, so it's
> cool by me if we leave that in.

Keywords could be useful for creating a ticket workflow (as opposed to
putting keywords in the ticket's title). They are also good for
grouping tickets related to a bigger concept requiring many tickets.
For example, if there were 3 tickets for ousting oldforms from the
admin interface, comments app, and generic views, these three tickets
could all be keyworded with "newforms" for tracking what has already
been done or what needs to be done.

[Michael Radziej]

Hi,

here are a few further suggestions for the ticket workflow.

First, Malcolm has brought up the idea about using Trac's keywords. I've
looked up the URL from his email, and I'd like these keywords:

>From adrian:

- design-q (or rather: "needs-design-decision")
- ok-needs-patch (or rather: needs-patch + additional keyword "approved")
- needs-docs
- needs-tests
- needs-better-patch

>From trac:

- needs-review - peer review requested
- help-wanted - original contributor is stuck
- needs-info - waiting on information from the reporter
- to-be-deleted - "noise" tickets to be deleted one day

Perhaps in addition:

- needs-discussion - ticket is being discussed in mailing list

Using "needs-*" for anything that blocks a ticket keeps the queries simple.

Now when you deal with tickets, filter away everything that has a
"needs-*" keyword, and order them by age, so you avoid the "last in,
first off" syndrom that is the source of all evil (tm). Reporters and
ticket triage team are responsible for removing the "needs-*" keywords
when the information has arrived. (Hopefully this can be automated at
least partly.)

But there's room for improvement:

Adrian Holovaty wrote:
> ALSO, while we're talking about changing Trac, I believe we should
> remove the following fields from the ticket interface, as I don't
> think any of the core devs (including I) use them:
>
> * Priority -- Used only by ticket submitters who want to add
> artificial urgency.
> * Severity -- Used only by ticket submitters who want to add
> artificial urgency.
> * Assign to -- Eh, we don't really use this.
> * Milestone -- We don't really use this, either.

These fields could actually help to get you a nicely organised queue of
tickets. You shouldn't wade through a list of minor improvements to find
a bug that means data corruption, so I'd rather suggest to start using
these fields. To give you a few ideas:

Priority - owned by the core developer (and perhaps the whole core
developer team) to prioritize their own work. Adrian might want to set
the priority for newforms stuff to higher, for example. The priority is
also a bit of information for the reporter how fast they could expect
the ticket to be fixed.

Severity - owned by the triage team. If used systematically and with
some criteria (like, blocker: stops django from running at all, or data
corruption, etc.), it helps users to find current important issues quickly.

Assign to - owned by the core developer team. It marks who is working at
the stuff. A ticket should initially be assigned to nobody, and only set
by the individual developer himself when they really work on it or want
to continue after the requested information, decision or patch has been
provided.

With this in place, a developer can simply also ignore all tickets that
are assigned to somebody else, deal with assigned tickets first
(accelerating everything already touched) and sort assigned tickets by
priority (to avoid minor stuff blocking important stuff).

The Milestone is great in pre-release times. It's owned by the release
manager and should be empty by default.

Michael

[Michael Radziej]

Hi,

Michael Radziej wrote:
> First, Malcolm has brought up the idea about using Trac's keywords. I've
> looked up the URL from his email, and I'd like these keywords:

Oops: it wasn't brought up by Malcolm but by Gary. Sorry about that!

Michael

[SmileyChris]

It has always frustrated me that email addresses are just given away to
spammers. Is there anything that can be done to hide or obscure
commenter/submitter emails and "CC" emails?

[Tim]

I wonder if any of you have seen "Dr. Project". It's basically some
additional functionality built on top of Trac (includes a tag cloud and
multi-project support).

https://www.drproject.org/

This may be a source of inspiration or even a solution.

[Gary Wilson]

FYI, this is being talked about too:
http://trac.edgewall.org/ticket/153
http://trac.edgewall.org/ticket/1459

[Jan Claeys]

Op zondag 14-01-2007 om 12:40 uur [tijdzone -0800], schreef SmileyChris:

> It has always frustrated me that email addresses are just given away
> to spammers. Is there anything that can be done to hide or obscure
> commenter/submitter emails and "CC" emails?

By mailing this list, you have just sent your e-mail address to all the
harvesting-bots that subscribe to it... :-P

--
Jan Claeys

[SmileyChris]

Granted, Jan, but a human still had to sign up their spam bot.

In the case of the web site, all the email addresses publicly
accessible to any and every spam-spider. It's just not good etiquette.

[Michael Radziej]

Hi Tim,

Tim schrieb:

>
> I wonder if any of you have seen "Dr. Project". It's basically some
> additional functionality built on top of Trac (includes a tag cloud and
> multi-project support).
>
> https://www.drproject.org/

Hmm, browsing through the docs the only stuff that tries to explain the
differenes to trac is the white paper. None of these look interesting
for django, e.g., we really don't need another mailing list, or the
ability to quicky deploy other projects.

What exactly are you referring to?

Michael

[Michael Radziej]

Hi,

after thinking about drproject: I'd rather switch to something
full-fledged like Bugzilla (http://www.bugzilla.org/). But I guess that
is out of question. The integration of tickets with svn and all this is
really too cool, and while bugzilla provides a much better ticketing
system than trac, it doesn't provide any of that. Sigh.

Michael

[Adrian Holovaty]

Yeah, moving to a whole other bug-tracking system (or writing one
ourselves, as somebody else suggested) isn't really an option. We've
just got to streamline Trac for our needs.

[d...@simon.net.nz]

I'm willing to put some time into triage, if we get some guidelines
sorted.

--Simon

[Michael Radziej]

si...@simon.net.nz schrieb:

>
> I'm willing to put some time into triage, if we get some guidelines
> sorted.

I'd join in. Especially sorting out all the old stuff.

Michael

--
noris network AG - Deutschherrnstraße 15-19 - D-90429 Nürnberg -
Tel +49-911-9352-0 - Fax +49-911-9352-100

http://www.noris.de - The IT-Outsourcing Company

[Russell Keith-Magee]

On 1/14/07, Malcolm Tredinnick <mal...@pointy-stick.com> wrote:
>
> Trac's non-feature of not automatically sending changes to the reporter
> (which follows from not requiring a valid reporter email) doesn't help
> here, either.

I'd always wondered about this.. er... feature. Is it possible to
filter the submitter field and automatically add a CC, or
automatically add a CC if the user is logged in?

> > (Clearly it'd be best if we didn't have to use silly prefixes in the
> > ticket titles to designate ticket status. I'm not familiar enough with
> > Trac to know whether it can be hacked to add separate fields for this
> > purpose.)
>
> Why don't you want to use the keywords field for this? It can still mix
> and match with people wanting to use their own keywords; we just claim a
> few short ones for triage purposes.

Ideally, I'd prefer to see proper checkboxes for design approval,
docs, tests, etc, but if this is difficult to do with Trac, and you
can filter tickets by keyword, I'm +1 to using keywords rather than
title.

> > * Assign to -- Eh, we don't really use this.
>
> It doesn't work too well at the moment because a tickets get assigned en
> masse to you or Jacob, for the most part. However, it is useful to know
> when somebody is actively working on a ticket (or to be able to look at
> all the tickets I am working on). So "assigned" is a useful field, but I
> would prefer the initial owner was something more generic, rather than
> "adrian" or "jacob" (only because it's currently hard to tell if the
> Adrian or Jacob entities are working on the ticket or just getting
> screwed by auto-assignation).

+1 to this sentiment and solution.

> > * Milestone -- We don't really use this, either.

-1 on removing this one. For me, 'milestone' represents what should be
the only required priority-based field. We don't really need to track
severity or priority - severe bugs get fixed quickly as a result of
the problem getting mentioned on the mailing lists (or being raised
privately as a security issue), and priority sorts itself out in that
wonderful open source way. The only thing left is to identify what is
standing in the way of the version X release.

Yours,
Russ Magee %-)

[Gary Wilson]

Russell Keith-Magee wrote:
> On 1/14/07, Malcolm Tredinnick <mal...@pointy-stick.com> wrote:
> >
> > Trac's non-feature of not automatically sending changes to the reporter
> > (which follows from not requiring a valid reporter email) doesn't help
> > here, either.
>
> I'd always wondered about this.. er... feature. Is it possible to
> filter the submitter field and automatically add a CC, or
> automatically add a CC if the user is logged in?

I must have missed this comment. Trac has got this too. It can notify
owner, reporter, and commenters.

In trac.ini:

[notification]
always_notify_reporter = true
always_notify_owner = true
always_notify_updater = true

I would be +1 for turning on notify reporter and updater, since then I
wouldn't have to add myself to the CC.

[medhat]

I think this should be really high on the priority list of the core
django team because, as many have said, everybody here is a volunteer,
and personally many times I would be thinking of investing some time to
implement a feature in django thinking that it might be useful for
other people (instead of just implementing a workaround that will only
work for me) but then I get discouraged because I know that if I create
a ticket with a patch, most probably my work will get lost among all
the other tickets, and then I spend my time on something else that I
would consider worthwhile. If I feel that the time I will spend doing
something will not be in vain, I think I will be willing to spend more
time on sharing ideas and creating patches.

I am not trying to accuse anybody of any wrongdoing, I am just stating
a fact about how I decide how to spend my spare time (and probably
others do the same thing)

On a more positive note: Are you guys aware of this:
http://trac.edgewall.org/wiki/TracTicketsCustomFields this might help
in classifying tickets without overloading the title or keyword fields
(abd having some checkboxes will make it easier for people to know all
the options instead of just *knowing* that I need to add [patch] or any
other magic string to the title or keyword fields)

Also, I haven't looked at this, but if trac would allow some fields to
only be edited by core developers, I would think that at least one of
priority and severity could help in planning and directing people for
where they should spend their time.

Just my 2 cents :-)

--
Thanks,
Medhat

[Phil Powell]

A quick tip (which might be obvious, but might help a few people): I
use the RSS feed for specific tickets to keep track of their progress.
That way, if comments are added, or new patches are supplied, it
automatically pops up in "Django tickets" in my RSS reader.

-Phil