ANN: Improving our decision-making and committer process

121 views
Skip to first unread message

Jacob Kaplan-Moss

unread,
Sep 29, 2010, 1:59:30 PM9/29/10
to django-d...@googlegroups.com
Hi folks --

Starting today, we're going to be making some minor but significant changes
to the way the Django core committer team "does business."

Put simply, we're changing the way we make decisions so that we can make
important policy and design calls more quickly and easily. In a nutshell:

* We're substantially reducing the number of decisions that Adrian and I
need to make as BDFLs, thus removing us as a decision-making bottleneck.

* We're cutting back on our reliance on consensus for decision making,
thus ensuring that decisions can be made in a timely manner.

* Finally -- and I think most importantly -- we're relaxing our policy on
adding new core committers.

I've just made a couple of changes [1] to our "Contributing to Django"
guide [2] which explain these changes; read them if you want the short
explanation of the new rules.

[1] http://code.djangoproject.com/changeset/13962
[2] http://docs.djangoproject.com/en/dev/internals/contributing/

For those who've been following our development process closely -- and
calling for these kind of changes for quite some time -- I'll explain some
of the background and ramification in more detail below.

Some background
---------------

The discussion leading to these changes began with conversations Russ had
at DjangoCon. I wasn't there, but he reported that there was a repeated and
general dissatisfaction with the activity and responsiveness of the core
team. I don't think this came as a surprise to any of us. I, at least, have
felt like we've been lax in our responsibilities lately.

Now, I don't really care to examine the merits of individual complaints or
make excuses/explanations for the lack of time and attention the core
team's been paying. Any way you cut it, it's clear that the core team --
and Adrian and I in particular -- have some work to do to regain the
confidence and trust of the community.

It's clear this won't happen overnight, and I don't expect the changes
we're making today to magically grant us that confidence back. Instead, I
see these changes as the first step in a general process towards more
openness and inclusiveness in Django's development process.

Making decisions
----------------

One of the main reasons we've ended up in this situation is our decision
making process. When we first came up with Django's development process,
Django was a tiny project with a handful of developers and a
not-much-larger group of users. In that context, decision by consensus made
some sense, but it simply doesn't scale as the number of participants increases.

In particular, our reliance on consensus means that any developer can
accidentally execute a veto simply by not having time to participate in
discussions!

We've always had the "out" of appealing to Adrian or I for a BDFL decision.
However, that, too, did't scale as the number of decisions-to-be-made
increased (and our own spare time decreased).

So we've adopted a new, simple, streamlined voting process. This deals with
situations where we can't reach a quick consensus, and it cuts out the
requirement that Adrian and/or I rule on each and every decision.

This new process allows a proposal to be carried if:

* There are at least 3 "+1" votes from core committers.
* There are no "-1" votes from core committers.
* A BDFL hasn't issued a ruling on the issue.

Since this process allows any core committer to veto a proposal, any "-1"
votes (or BDFL vetos) should be accompanied by an explanation that explains
what it would take to convert that "-1" into at least a "+0".

More details are in the contributor doc, but in essence this means that we
can move forward without needing to wait for the entire core team to weigh
in. It does maintain our fairly skeptical attitude towards feature-creep by
allowing any one contributor to object if something rubs him/her the wrong
way, but it also forces that contributor to explain how she/he may be
overruled.

With luck, we'll see the pace of development speed up a bit as a result.

I should stress that this voting procedure DOES NOT replace community
discussion, nor does it remove entirely the desire for consensus. In other
words, this new process is a new way to "break ties" without having to ask
for a BDFL ruling. Whenever possible, we should continue to strive to reach
consensus on issues, and use the voting process only as a fallback where
consensus can't be reached for whatever reason. I, at least, plan to vote
"-1" if I see community discussion being skipped, or if I see no attempt at
consensus before a vote.

Committers
----------

It's been clear for quite some time that there's a really right bottleneck
at the top of the project: we simply don't have enough committers. The
issue of adding new ones comes up fairly regularly, but has never come to
any conclusion.

We've deliberately set the bar for new contributors quite high: new
committers required unanimous approval of all existing committers. We've
resisted lowering that bar out of a fear that the quality of Django would
degrade; we're quite proud that many users of Django run the latest trunk
revision in production.

However, like other decisions, requiring consensus simply stops scaling.
Over the last year we've discussed a number of candidates for commit
access, but many times failed to reach consensus often simply because not
every committer was familiar with the candidate's work. Looking back, we
all agree that we've failed to give the commit bit to some very deserving
people. It's not that these folks failed to clear the bar; it's that we set
the bar at an unobtainable height.

So we're lowering the bar a bit: from now on, we'll decide on new
committers using the same voting procedure described above. In essence,
instead of unanimity we'll now simply require 3 champions and no strong
votes against.

This doesn't represent a huge change from our existing approach. No core
committer will be committing big features without some sign-off and review
from at least one other committer. No bug fixes will go in if we're not
sure we've gotten it right. We'll continue to encourage new committers to
"start small" and work up towards larger commits as the rest of the team
gets to know their work.

All that we're doing is making a change in our "default" opinion about
proposed committers: instead of assuming that un-received votes are "no"
votes, we're assuming that most committers will be happy to see new blood,
forcing ourselves to articulate objections if they're serious.

We've already started discussing a group of new committers under this
policy, and the new process seems to be working out well. I expect we'll
see some new names on the core committer list shortly. Actually, they won't
be new names at all: anyone who's been around for a while will recognize
these names.

[If, when we add these new members, you don't see *your* name and think you
should have, please let one of us know so we can discuss it with you and
probably nominate you. Please do ask: we won't be able to intuit who's
interested.]

What's next
-----------

Again, this is just the start of an ongoing process.

For example, one of the things we haven't done well is to articulate
exactly what our criteria for the commit bit ought to be. Our "we know it
when we see it" policy doesn't set clear goals for community members who'd
like to work towards commit, and we need to do a better job mentoring
potential and new committers.

We're going to continue to work on these areas, and on improving our
process in general. If you have other suggestions please feel free to make
'em here -- we're listening.

Thank you!

Jacob

PS: And if anyone feels uncomfortable raising process issues -- related or
unrelated to the above -- please always feel free to contact me privately.

Ivan Sagalaev

unread,
Sep 29, 2010, 5:32:42 PM9/29/10
to django-d...@googlegroups.com
Hello Jacob and everyone.

On 09/29/2010 09:59 PM, Jacob Kaplan-Moss wrote:
> Starting today, we're going to be making some minor but significant changes
> to the way the Django core committer team "does business."

That's about time :-). Congratulations and thank you!

I have a comment and a suggestion:

> This new process allows a proposal to be carried if:
>
> * There are at least 3 "+1" votes from core committers.
> * There are no "-1" votes from core committers.
> * A BDFL hasn't issued a ruling on the issue.

This doesn't explain what's one is supposed to think in situation when a
proposal is ignored by core devs, which may happen for all sorts of
reasons. I suspect that "less than 3 +1 votes" means the same as any -1
vote but I think that an explicit clarification would be nice.

---

My suggestion is about this unfortunate ticket status -- 'Accepted'.
This now works as a sort of a dusty shelf: when anyone of the core team
looks at the patch and decides that there's nothing wrong with it he
puts it on that shelf where the ticket has all the chances to lie for
months or even years. And if the author of the patch tries from time to
time pitching it to get it into trunk he can easily fall into all sorts
of "not-a-good-times": conferences, feature freezes, hot discussions on
other topics etc.

My proposal is simple: 'Accepted' status shouldn't exist. If the patch
is good it should be committed right away. If it's not there have to be
an explanation why it needs improvement or why it's going to be
wontfixed. Simple waiting doesn't really improve quality of the patch.

What do you think?

Chuck Harmston

unread,
Sep 29, 2010, 5:40:46 PM9/29/10
to django-d...@googlegroups.com
In my world, the "accepted" status should only be used in one circumstance: when a person is actively developing under or maintaining a patch for the ticket. It's an indicator that someone has taken ownership of a ticket, to prevent duplication of effort, etc. For example, I accepted ticket #25 during the DjangoCon sprints to prevent other developers at the sprints from attempting to develop against the ticket.

Then again, my world also has purple skies and orange grass, so take it with a grain...errr, a bushel of salt. ;)




--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Ivan Sagalaev

unread,
Sep 29, 2010, 6:02:27 PM9/29/10
to django-d...@googlegroups.com
On 09/30/2010 01:40 AM, Chuck Harmston wrote:
> In my world, the "accepted" status should only be used in one
> circumstance: when a person is actively developing under or maintaining
> a patch for the ticket. It's an indicator that someone has taken
> ownership of a ticket, to prevent duplication of effort, etc. For
> example, I accepted ticket #25 during the DjangoCon sprints to prevent
> other developers at the sprints from attempting to develop against the
> ticket.

I believe the field 'Assigned to' works just fine for this purpose, no?

Luke Plant

unread,
Sep 29, 2010, 7:26:13 PM9/29/10
to django-d...@googlegroups.com
On Thu, 2010-09-30 at 01:32 +0400, Ivan Sagalaev wrote:

> My suggestion is about this unfortunate ticket status -- 'Accepted'.
> This now works as a sort of a dusty shelf: when anyone of the core team
> looks at the patch and decides that there's nothing wrong with it he
> puts it on that shelf where the ticket has all the chances to lie for
> months or even years. And if the author of the patch tries from time to
> time pitching it to get it into trunk he can easily fall into all sorts
> of "not-a-good-times": conferences, feature freezes, hot discussions on
> other topics etc.
>
> My proposal is simple: 'Accepted' status shouldn't exist. If the patch
> is good it should be committed right away. If it's not there have to be
> an explanation why it needs improvement or why it's going to be
> wontfixed. Simple waiting doesn't really improve quality of the patch.
>
> What do you think?

This doesn't account for these facts:

1) Accepted != Ready for checkin.
2) A lot of triage is done by people without commit access.

Sometimes a ticket stays at 'Accepted' for a long time because it
doesn't actually have anyone motivated enough to write even a patch, or
tests etc, which means that it is de-facto low priority, and we
shouldn't feel guilty about this kind. The ones in the 'Ready for
checkin' queue are the ones that deserve to be checked in, and it
currently has only 35 in it, compared to 1226 in 'Accepted'.

However, there are many 'Accepted' tickets which actually ought to be in
'Ready for checkin', or need some feedback to say what should be done to
move them on. This is not something that can only be done by
committers, but how to get help with this massive task is still a big
problem. It's quite possible for someone using Django to submit a
completely valid bug report, with a good patch and tests etc., and for
it not to get reviewed because no-one else in the community cares about
that particular bug, and I for one don't know what to do about that.

The core team do agree that the "it's not a good time" response has been
an unhelpful sticking point, and we probably need to make it easier for
people to discuss things at any point in time.

Luke

--
"Defeat: For every winner, there are dozens of losers. Odds are
you're one of them." (despair.com)

Luke Plant || http://lukeplant.me.uk/

Russell Keith-Magee

unread,
Sep 29, 2010, 7:46:05 PM9/29/10
to django-d...@googlegroups.com
On Thu, Sep 30, 2010 at 7:26 AM, Luke Plant <L.Pla...@cantab.net> wrote:
> On Thu, 2010-09-30 at 01:32 +0400, Ivan Sagalaev wrote:
>
>> My suggestion is about this unfortunate ticket status -- 'Accepted'.
>> This now works as a sort of a dusty shelf: when anyone of the core team
>> looks at the patch and decides that there's nothing wrong with it he
>> puts it on that shelf where the ticket has all the chances to lie for
>> months or even years. And if the author of the patch tries from time to
>> time pitching it to get it into trunk he can easily fall into all sorts
>> of "not-a-good-times": conferences, feature freezes, hot discussions on
>> other topics etc.
>>
>> My proposal is simple: 'Accepted' status shouldn't exist. If the patch
>> is good it should be committed right away. If it's not there have to be
>> an explanation why it needs improvement or why it's going to be
>> wontfixed. Simple waiting doesn't really improve quality of the patch.
>>
>> What do you think?
>
> This doesn't account for these facts:
>
> 1) Accepted != Ready for checkin.
> 2) A lot of triage is done by people without commit access.
>
> Sometimes a ticket stays at 'Accepted' for a long time because it
> doesn't actually have anyone motivated enough to write even a patch, or
> tests etc, which means that it is de-facto low priority, and we
> shouldn't feel guilty about this kind.  The ones in the 'Ready for
> checkin' queue are the ones that deserve to be checked in, and it
> currently has only 35 in it, compared to 1226 in 'Accepted'.

This is an important stat -- but it glosses over the fact that 1226
"accepted" tickets doesn't necessarily clarify how many of these have
viable patches -- or patches at all.

Accepted tickets can be:

* Purely accepted, indicating that someone has verified that the
problem exists, but not how to solve it

* Accepted with a patch that is wrong in some way (e.g., fixing the
symptom, not the problem)

* Accepted with a patch that is missing documentation or tests

* Accepted with a valid patch, just awaiting review by someone else.

A ticket in the first three conditions patently isn't ready for
checkin. A ticket in the last condition *may* be ready for checkin; we
ask for independent verification before it gets moved to RFC.

So - IMHO "Accepted" plays a very important role in our triage system.
I don't think it can be eliminated without overloading a different
area of the ticket system.

> However, there are many 'Accepted' tickets which actually ought to be in
> 'Ready for checkin', or need some feedback to say what should be done to
> move them on.  This is not something that can only be done by
> committers, but how to get help with this massive task is still a big
> problem. It's quite possible for someone using Django to submit a
> completely valid bug report, with a good patch and tests etc., and for
> it not to get reviewed because no-one else in the community cares about
> that particular bug, and I for one don't know what to do about that.

There was some work done at DjangoCon to try and provide a voting
extension to Trac. The aim here would be to provide a way for people
to register "I've been bitten by this bug", so we can get a list of
issues ordered by how many people are being affected. This will
hopefully provide some direction to triage (and committer) activity so
that the "most important" tickets get addressed first.

Yours,
Russ Magee %-)

Waylan Limberg

unread,
Sep 29, 2010, 9:14:09 PM9/29/10
to django-d...@googlegroups.com

So, in other words, accepted simply means that the ticket reports a
valid bug or feature request that is considered worth fixing, but
offers to indication as to the status of any patches for committing.
Obviously, some seem to imply the later meaning into "Accepted" which
could raise the question regarding whether it is named correctly (I
say it is fine). But, more importantly, is there a place were each
status is simply defined?

Sure there is this:
http://docs.djangoproject.com/en/dev/internals/contributing/#ticket-triage

But that hardly makes clear exactly what "accepted" actually means.
The text in that section is helpful to understanding the basic
process, but if someone changes the status of my ticket, there's no
definitive place to go and see exactly what that status means.

In fact, in reading the list over the last few years, I have the
impression that this is a problem that is repeated constantly. People
don't understand what the various statuses mean and get frustrated
when things do not happen as they expected. I think perhaps clearer
documentation would help in this case.

--
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Russell Keith-Magee

unread,
Sep 30, 2010, 12:13:13 AM9/30/10
to django-d...@googlegroups.com

A reasonable suggestion if ever I've heard one. I've opened a ticket
to track the idea:

http://code.djangoproject.com/ticket/14360

and I've put it on the 1.3 milestone; if anyone wants to take a swing
a clarifying the language before I (or someone else on the core team)
gets a chance to look at it, feel free.

Yours,
Russ Magee %-)

Ivan Sagalaev

unread,
Sep 30, 2010, 4:11:36 AM9/30/10
to django-d...@googlegroups.com
On 09/30/2010 03:46 AM, Russell Keith-Magee wrote:
> Accepted tickets can be:
>
> * Purely accepted, indicating that someone has verified that the
> problem exists, but not how to solve it
>
> * Accepted with a patch that is wrong in some way (e.g., fixing the
> symptom, not the problem)
>
> * Accepted with a patch that is missing documentation or tests
>
> * Accepted with a valid patch, just awaiting review by someone else.
>
> A ticket in the first three conditions patently isn't ready for
> checkin. A ticket in the last condition *may* be ready for checkin; we
> ask for independent verification before it gets moved to RFC.
>
> So - IMHO "Accepted" plays a very important role in our triage system.

To me this shows exactly the opposite: a status that has so many
meanings doesn't really mean anything useful :-). I'm not new in Django
development yet as you see I managed to misunderstand its meaning in my
previous mail.

Anyway, what I'm saying is that we have this huge gap between someone
writing a good patch (with docs & tests) and the moment it gets into
trunk. Current mechanism of volunteer triagers who are supposed to move
such tickets from 'accepted' into 'ready for checkin' doesn't seem to work.

So let me mend my suggestion to better fit reality. Instead of removing
'accepted' altogether (though I still think it's a good idea) I think
each committer has to do this routinely. I know that every programmer
would rather code instead of applying patches and running tests but the
development process simply won't scale otherwise.

BTW, Bazaar guys use the approach with a "champion of the week". Every
week a person from core committers volunteers to actively do triaging
and committing new and old tickets in the tracker. May be such explicit
approach would work for Django too. Especially since we're about to have
more committers than we do now.

Russell Keith-Magee

unread,
Sep 30, 2010, 5:22:29 AM9/30/10
to django-d...@googlegroups.com
On Thursday, September 30, 2010, Ivan Sagalaev

<man...@softwaremaniacs.org> wrote:
> On 09/30/2010 03:46 AM, Russell Keith-Magee wrote:
>
> Accepted tickets can be:
>
>  * Purely accepted, indicating that someone has verified that the
> problem exists, but not how to solve it
>
>  * Accepted with a patch that is wrong in some way (e.g., fixing the
> symptom, not the problem)
>
>  * Accepted with a patch that is missing documentation or tests
>
>  * Accepted with a valid patch, just awaiting review by someone else.
>
> A ticket in the first three conditions patently isn't ready for
> checkin. A ticket in the last condition *may* be ready for checkin; we
> ask for independent verification before it gets moved to RFC.
>
> So - IMHO "Accepted" plays a very important role in our triage system.
>
>
> To me this shows exactly the opposite: a status that has so many meanings doesn't really mean anything useful :-). I'm not new in Django development yet as you see I managed to misunderstand its meaning in my previous mail.

Accepted means exactly one thing - the issue has been verified as a
real problem (or acknowledged as a reasonable feature request), but
isn't yet ready for inclusion in trunk.

If viewed in combination with the other flags on a ticket, you can
work out why a ticket isn't ready for inclusion in trunk - it needs
documentation, it needs tests, it needs a patch, or the patch that
exists needs improvement.

As Waylan noted, this probably indicates that our documentation needs
to be improved.

> Anyway, what I'm saying is that we have this huge gap between someone writing a good patch (with docs & tests) and the moment it gets into trunk. Current mechanism of volunteer triagers who are supposed to move such tickets from 'accepted' into 'ready for checkin' doesn't seem to work.

The officially documented practice is that the "triage team" moves
tickets to RFC. However, this hasn't been actual practice for some
time. The effective practice is that anyone other than the submitted
of a patch can m ove a ticket to RFC if they feel confident that it is
ready.

This is certainly an area where the documentation needs to be
improved, in order to reflect common practice.

What is also needed is a whole lot more people volunteering. Any
suggestions on how to get more people doing the entirely unglamorous,
but completely necessary work will be gratefully accepted.

> So let me mend my suggestion to better fit reality. Instead of removing 'accepted' altogether (though I still think it's a good idea) I think each committer has to do this routinely. I know that every programmer would rather code instead of applying patches and running tests but the development process simply won't scale otherwise.

So, let me be clear on what you're proposing: you acknowledge that the
triage process is backlogged. Your proposal is to put extra workload
onto the core team - the one group that is already a bottleneck in the
process.

During the 1.2 release cycle, I had to spend almost 2 weeks checking
tickets in the unreviewed state in order to check that no release
blocking bugs had been reported. This amounts to something like 50
hours of development effort - 50 hours that I didn't spend closing
bugs. If you estimate that it takes an hour to close a ticket, that
means 1.2 has 50 more bugs in it than it would have had if someone not
on the core team had performed the triage task.

As I write this, there are 215 tickets in the unreviewed state. There
is plenty of triage work to do, and absolutely no reason that the core
team has to be the group to do it.

> BTW, Bazaar guys use the approach with a "champion of the week". Every week a person from core committers volunteers to actively do triaging and committing new and old tickets in the tracker. May be such explicit approach would work for Django too. Especially since we're about to have more committers than we do now.

The core team volunteers the time that they have available. Some weeks
that means a lot of time; some weeks it means no time. Sometimes time
is spent closing tickets, and sometimes - when it's clear that there
is a backlog of unreviewed tickets blocking a release, for example -
the core team will do triage.

The core team isn't wanting for things to do. There a plenty of
tickets in RFC; there is lots of mailing list traffic that needs
attention; there are new features that need to be designed, built and
reviewed; and yes, there is triage. Forcing core developers to do
triage exclusively simply takes energy away from other, equally
important tasks that form part of the development process.

On top of that, I can't speak for anyone else in the core team, but I
fail to see see how a formal "this week is my week" arrangement will
increase the time I have available to volunteer to the project. If
anything, I see it decreasing the quantity of volunteered time,
because if I don't think I will be able to dedicate an entire week, I
probably won't put my hand up for fear of raising false expectations.

Yours,
Russ Magee %-)

Ivan Sagalaev

unread,
Sep 30, 2010, 11:07:27 AM9/30/10
to django-d...@googlegroups.com
On 09/30/2010 01:22 PM, Russell Keith-Magee wrote:
> So, let me be clear on what you're proposing: you acknowledge that the
> triage process is backlogged. Your proposal is to put extra workload
> onto the core team - the one group that is already a bottleneck in the
> process.

Pretty much. I just don't put it the way you do.

Triage process is backlogged not because of some strange reason. People
need motivation to do daunting work and I can't imagine what could
motivate enough volunteers to triage other people bugs regularly. And no
amount documentation will fix this. A committer on the other hand is not
just a volunteer. Being a committer of a high-profile open-source is a
nice ego boost and committers receive more moral compensation than
casual contributors. They are higher in an implicit project hierarchy
and it matters to people.

As for bottleneck, I propose this only in light of Jacob's decision to
increase the number of committers. This is certainly a prerequisite.
It's not a question that committers are overloaded and this why this
conversation has started.

To put it *really* short: I'm not trying to find extra work for
committers, I'm bluntly saying that no one else would ever do it. So we
have to find ways to do it. And having more committers and a simpler
decision process does help here.

> If you estimate that it takes an hour to close a ticket

This is BTW another, purely technical, problem. It shouldn't take an
hour to close a ticket if it has proper tests. I was testing patches
during various sprints and one big thing that I learned is that our
whole test suit needs a deep refactoring to be simpler and faster. But
this is another story...

Tai Lee

unread,
Oct 1, 2010, 2:44:27 AM10/1/10
to Django developers
On Sep 30, 7:22 pm, Russell Keith-Magee <russ...@keith-magee.com>
wrote:

> What is also needed is a whole lot more people volunteering. Any
> suggestions on how to get more people doing the entirely unglamorous,
> but completely necessary work will be gratefully accepted.

I'd like to suggest (1) easy to find and use pre-defined searches to
find tickets at each stage of triage, (2) a clearer indication of the
next steps and the person responsible for it whenever a ticket is
reviewed, and (3) tickets that have been sitting on DDN for a while be
put to a vote.

The "stage" is not just unreviewed, DDN, accepted, RFC, someday/maybe,
and fixed on a branch. The "accepted" stage itself is broken down into
has patch, needs docs, needs tests, patch needs improvement.

As an example, http://code.djangoproject.com/ticket/13291 and
http://code.djangoproject.com/ticket/12398 both have a patch with docs
and tests, and were marked as "accepted" by a core committer (not that
it matters who reviewed them), without providing any actual feedback
and without stating that they need a better patch. These tickets are
fairly trivial and have been in limbo for 6-8 months.

As the original reporter and developer, I assumed that since these
tickets were reviewed by a core committer and "accepted" without any
feedback about needing a better patch, that they would be committed
shortly, or at least be moved to RFC shortly. These tickets are ideal
candidates for anyone doing triage (not only core committers) to
review and move to RFC, but they've been forgotten about by the
reviewer, assumed no further action was required by the developer, and
ignored in the sea of "accepted" tickets by everyone else.

So for (1), I'd like to see the how to contribute documentation asking
for people to triage specific sets of tickets with specific goals. For
example, to review tickets that are almost RFC and just waiting on
review (with a link to tickets that are accepted, with a patch that
has docs and tests and doesn't need improvement), or to review tickets
that just need docs or test (with links to appropriate searches), etc.

And for (2), I'd like to see tickets remember who reviewed them (who
accepted it, or who decided it needed a better patch, etc.) and for
the system to ping both the original reporter (or recently active
developer) and the reviewer after a period of inactivity. This should
hopefully catch tickets which are initially "accepted" and then
forgotten about.

Lastly I also have several tickets that have been DDN for 1-2 years,
so for (3), the reporter and recently active developers could be
notified that the DDN ticket will be put to a vote in say 1 or 2 weeks
time and allow them time to prepare any arguments or improved patch
that may sway the decision, or try to reach a consensus on django-
developers prior to the vote.

I've been working with Django for several years now and I have tried
to contribute useful tickets with patches including docs and tests
when possible, and it has at times been extremely discouraging to see
this effort go unnoticed for literally years without review (beyond a
passing glance to "accept" a ticket, in an attempt to reduce the
"unreviewed" backlog).

I've also felt that at busy times, some tickets that had merit but
didn't have an immediately apparent and elegant solution were rejected
simply save time leading up to a release.

I was pleased to see the recent announcement from Jacob about the
changes to the decision making process, but I think we also need to
improve the triage process so that tickets requiring action (including
decisions) can be directed to or found by the people responsible for
making those next steps happen.

Cheers.
Tai.

Jacob Kaplan-Moss

unread,
Oct 1, 2010, 11:46:29 AM10/1/10
to django-d...@googlegroups.com
On Fri, Oct 1, 2010 at 1:44 AM, Tai Lee <real....@mrmachine.net> wrote:
> I'd like to suggest (1) easy to find and use pre-defined searches to
> find tickets at each stage of triage, (2) a clearer indication of the
> next steps and the person responsible for it whenever a ticket is
> reviewed, and (3) tickets that have been sitting on DDN for a while be
> put to a vote.

This is completely right-on; I agree entirely. Some work has already
been done in these areas:

For (1) check out http://code.djangoproject.com/wiki/Reports (it's
linked in the nav). If there's anything missing there, please feel
free to add it -- it's a wiki page. Let me know if you need help
figuring out the linked query syntax.

Work on (2) began at some point over at
http://code.djangoproject.com/wiki/TicketChangeHelp. It's pretty rough
still, but once it gets better I'd love to link sections from the
ticket page and/or move it into the docs directly.

Ideally, I'd like each ticket page to have a "what's next?" box on it
indicating where the ticket is and how to move it along. Technically
this isn't hard -- we've got plenty of folks in our community who
could write a Trac plugin -- but getting the language right is
important. If we get that worked out to some extent I'll figure out
what the next steps are technically.

For (3), well, feel free to bring things up here or on IRC, I s'pose.
I'd really appreciate it if we didn't get 'em all brought up at once,
of course. But yes, if there are tickets marked DDN that you feel are
critical, please feel free to ask for a yay/nay.

Hope this gives you and/or others who want to help improve process a
place to channel that desire!

Jacob

Tai Lee

unread,
Oct 4, 2010, 8:16:44 PM10/4/10
to Django developers
Hi Jacob,

Thanks for your feedback.

> For (1) check out http://code.djangoproject.com/wiki/Reports(it's
> linked in the nav). If there's anything missing there, please feel
> free to add it -- it's a wiki page. Let me know if you need help
> figuring out the linked query syntax.

I wasn't able to edit this page. Other wiki pages have an "Edit this
page" and "Attach file" buttons at the bottom, but not the Reports
page.

I was just going to add a link to tickets that are Accepted with a
patch that doesn't need docs, tests or improvement as tickets that
just need a fresh pair of eyes to review and either bump up to Ready
for checkin or back down to Patch needs improvement (with feedback).

I also noticed that 3 of the 4 existing links under "Tickets needing
some work" are incorrect. They indicate that they are accepted
tickets, but only one of these links is filtering by triage stage.

> Work on (2) began at some point over athttp://code.djangoproject.com/wiki/TicketChangeHelp. It's pretty rough
> still, but once it gets better I'd love to link sections from the
> ticket page and/or move it into the docs directly.

This is good, but it looks like mostly copy and paste from the
"Contributing to Django" docs (duplication of data, also the Wiki is
not authoritative, especially when not authored by a core committer)
for the descriptions of each triage stage and patch flag, with the
addition of next steps. I'm also not sure if this Wiki page is linked
to from anywhere (e.g. actual ticket pages)?

Perhaps we'd be better off simply adding the next steps information to
the "Contributing to Django" docs directly, and to also update the
diagram there to include the various patch flags (has patch, needs
docs, needs tests, patch needs improvement) between the Accepted and
Ready for checkin triage stages?

At the moment all the docs / diagram say is that tickets which have a
patch with docs and tests are moved to ready for checkin, but a quick
search on trac revealed 358 tickets with this combination of triage
stage and patch flags. These tickets should all be either "patch needs
improvement" with feedback, or ready for checkin, right?

Perhaps we need another triage stage for these tickets, "Needs final
review" or something?

This is where I personally feel one of the big problems is with
perceived responsiveness of the core committers. These tickets are
sitting there as Accepted, and don't appear to require any further
action from the community or patch author (don't need a patch, docs,
tests or improved patch), and they are ignored by people wanting to
contribute and core committers alike.

> Ideally, I'd like each ticket page to have a "what's next?" box on it
> indicating where the ticket is and how to move it along. Technically
> this isn't hard -- we've got plenty of folks in our community who
> could write a Trac plugin -- but getting the language right is
> important. If we get that worked out to some extent I'll figure out
> what the next steps are technically.

Perhaps another boolean flag that simply says if the next step lies
with the patch author / community, or with the core committers?
Similar to regular support tickets which might say "waiting on
customer" or "waiting on support staff". This could help avoid
situations where the patch author thinks they've done everything they
need to do, satisfied all the requirements (docs, tests, code style),
and yet their ticket is never pushed up to Ready for checkin, which
only makes them less likely to contribute again in future and more
likely to simply fix or work around their problem locally instead and
save writing docs and tests for Django that go unused.

> For (3), well, feel free to bring things up here or on IRC, I s'pose.
> I'd really appreciate it if we didn't get 'em all brought up at once,
> of course. But yes, if there are tickets marked DDN that you feel are
> critical, please feel free to ask for a yay/nay.

Are only critical tickets eligible for a vote? I think there are a lot
of non-critical (even trivial) tickets that are stuck in DDN hell :)
Maybe we could organise regular DDN and Ready for checkin triage
sprints?

Also as DDN tickets must be decided by core committers, the community
and patch authors can't really do anything with these tickets and they
are stick in limbo. Perhaps there needs to be another flag to indicate
whether the core committers need more information or a proposal for
why a ticket should be included before they can make their decision,
or if it has been added to the queue of tickets to be decided on
(maybe with a position in the queue), and make sure that the queue is
worked through in order, and either sent back to the patch author for
more information or put to a vote if a decision can't be made?

Russell Keith-Magee

unread,
Oct 4, 2010, 8:48:18 PM10/4/10
to django-d...@googlegroups.com
On Tue, Oct 5, 2010 at 8:16 AM, Tai Lee <real....@mrmachine.net> wrote:
> Hi Jacob,
>
> Thanks for your feedback.
>
>> For (1) check out http://code.djangoproject.com/wiki/Reports(it's
>> linked in the nav). If there's anything missing there, please feel
>> free to add it -- it's a wiki page. Let me know if you need help
>> figuring out the linked query syntax.
>
> I wasn't able to edit this page. Other wiki pages have an "Edit this
> page" and "Attach file" buttons at the bottom, but not the Reports
> page.

It looks like that page has been locked down to prevent bad edits.
I'll put some wiki gardening on my todo list.

>> Work on (2) began at some point over athttp://code.djangoproject.com/wiki/TicketChangeHelp. It's pretty rough
>> still, but once it gets better I'd love to link sections from the
>> ticket page and/or move it into the docs directly.
>
> This is good, but it looks like mostly copy and paste from the
> "Contributing to Django" docs (duplication of data, also the Wiki is
> not authoritative, especially when not authored by a core committer)
> for the descriptions of each triage stage and patch flag, with the
> addition of next steps. I'm also not sure if this Wiki page is linked
> to from anywhere (e.g. actual ticket pages)?
>
> Perhaps we'd be better off simply adding the next steps information to
> the "Contributing to Django" docs directly, and to also update the
> diagram there to include the various patch flags (has patch, needs
> docs, needs tests, patch needs improvement) between the Accepted and
> Ready for checkin triage stages?

There have been a couple of suggestions recently that the contributing
guide should be distilled into a specific HOWTO for new users. I
suppose the idea here would be for the contribution guide to be the
letter of the law, and the HOWTO to be the spirit of the law -- a
lightweight guide pointing people at some specific activities they
could engage in as a new contributor.

> Perhaps we need another triage stage for these tickets, "Needs final
> review" or something?

That's essentially what RFC is. This is an area where our documented
procedures have lagged behind in-practice procedures, but we've
essentially accepted that once a patch has been written by one person,
and reviewed by another and found to be ok, then it's RFC, which means
a core committer needs to take a look.

>> Ideally, I'd like each ticket page to have a "what's next?" box on it
>> indicating where the ticket is and how to move it along. Technically
>> this isn't hard -- we've got plenty of folks in our community who
>> could write a Trac plugin -- but getting the language right is
>> important. If we get that worked out to some extent I'll figure out
>> what the next steps are technically.
>
> Perhaps another boolean flag that simply says if the next step lies
> with the patch author / community, or with the core committers?
> Similar to regular support tickets which might say "waiting on
> customer" or "waiting on support staff". This could help avoid
> situations where the patch author thinks they've done everything they
> need to do, satisfied all the requirements (docs, tests, code style),
> and yet their ticket is never pushed up to Ready for checkin, which
> only makes them less likely to contribute again in future and more
> likely to simply fix or work around their problem locally instead and
> save writing docs and tests for Django that go unused.

I don't know if another boolean flag is what is needed -- that strikes
me as more Trac gardening that can be mistriaged. Plus, the
information needed to determine the 'next step' is already there, you
just need to know how to read it. However, reading it requires a
little expertise and familiarity with the system.

A combination of Trac queries that can pull out tickets that need
attention, and an autogenerated text summary that clearly describes
the action required sounds like a valuable addition (i.e., distill the
Trac flags into a text statement of "This ticket needs external
review" or "This ticket isn't complete -- it's missing tests and/or
docs").

> Are only critical tickets eligible for a vote? I think there are a lot
> of non-critical (even trivial) tickets that are stuck in DDN hell :)
> Maybe we could organise regular DDN and Ready for checkin triage
> sprints?
>
> Also as DDN tickets must be decided by core committers, the community
> and patch authors can't really do anything with these tickets and they
> are stick in limbo. Perhaps there needs to be another flag to indicate
> whether the core committers need more information or a proposal for
> why a ticket should be included before they can make their decision,
> or if it has been added to the queue of tickets to be decided on
> (maybe with a position in the queue), and make sure that the queue is
> worked through in order, and either sent back to the patch author for
> more information or put to a vote if a decision can't be made?

As a general principle, DDN isn't usually a situation where more
information is needed; it's usually just time that is lacking.

Sprints definitely help push this sort of thing along. However, that's
more because we have two core developers in a room at the same time,
not because of the presence of the rest of the community (although the
community got lots of other great work done). Malcolm and I spend a
day at the recent sprints just doing DDN tickets, and we managed to
clear out quite a few DDNs. We made vague plans to hook up on
IRC/Skype as a semi-regular activity to try and clear out the backlog,
but we haven't made that happen yet.

Looks like another thing to put on my todo list... :-)

Yours,
Russ Magee %-)

Tai Lee

unread,
Oct 4, 2010, 9:58:19 PM10/4/10
to Django developers
Hi Russ,

On Oct 5, 11:48 am, Russell Keith-Magee <russ...@keith-magee.com>
wrote:

> There have been a couple of suggestions recently that the contributing
> guide should be distilled into a specific HOWTO for new users. I
> suppose the idea here would be for the contribution guide to be the
> letter of the law, and the HOWTO to be the spirit of the law -- a
> lightweight guide pointing people at some specific activities they
> could engage in as a new contributor.

I see. Sounds reasonable. I can see that it might be difficult to
extract the relevant information from the full contributing guide for
newbies, as well as for long time users wanting a quick reference.

> > Perhaps we need another triage stage for these tickets, "Needs final
> > review" or something?
>
> That's essentially what RFC is. This is an area where our documented
> procedures have lagged behind in-practice procedures, but we've
> essentially accepted that once a patch has been written by one person,
> and reviewed by another and found to be ok, then it's RFC, which means
> a core committer needs to take a look.

I'm happy for RFC to indicate that a ticket is ready for final review
by a core committer, after being reviewed by at least someone other
than the patch author. But, I think there's still a problem of
accepted tickets that have a patch with docs and tests and no feedback
indicating that it needs improvement, being lost in the shuffle, as
well as how and why we ever end up with tickets in this state and how
can the patch author get someone to review their work.

It could happen that a ticket is accepted on merit, then it gets a
patch with docs and tests, and then sits there getting no attention
because nobody knows it needs a review. The patch author could just
move it to RFC themselves as it had previously been accepted and he or
she subsequently added the required docs and tests, in order to get
somebody to look at it. But I think this is generally frowned upon and
might take up too much core committer attention before the ticket is
really ready.

It could also happen that a ticket is unreviewed and already has a
patch with docs and tests, if it is reviewed by a core committer (or
not) and "accepted" without feedback about any necessary improvements,
shouldn't it have simply gone straight to RFC instead? What should the
original patch author do when his or her patch with docs and tests is
reviewed by someone else and moved to "accepted" like this? There's no
feedback that they need to incorporate into the patch, and they can't
move their own work to RFC, even though someone else has reviewed and
"accepted" the patch?

I'd like to see the ticket system work smoothly in and of itself, and
for tickets to be reviewed without the patch authors having to ask
around on IRC or django-developers for someone to take a look. I don't
want to make the ticket system too complicated, but I think it is
already a bit muddled by determining whether or not a ticket needs
review by checking 5 or 6 different properties, instead of a single
clear indicator set by the patch author that he or she would like
somebody (not necessarily a core committer) to review the work. This
could also help in eliminating another problem of not knowing if a
ticket is still being worked on or if the patch author considers it
finished.

> As a general principle, DDN isn't usually a situation where more
> information is needed; it's usually just time that is lacking.
>
> Sprints definitely help push this sort of thing along. However, that's
> more because we have two core developers in a room at the same time,
> not because of the presence of the rest of the community (although the
> community got lots of other great work done). Malcolm and I spend a
> day at the recent sprints just doing DDN tickets, and we managed to
> clear out quite a few DDNs. We made vague plans to hook up on
> IRC/Skype as a semi-regular activity to try and clear out the backlog,
> but we haven't made that happen yet.

Are at least two core committers required for all DDN tickets? Can a a
single core committer make decisions on minor or trivial tickets, and
the reporter be given the option to request a vote or further
consideration if they feel the decision is wrong (possibly based on
lack of information)? Perhaps some of the people who are well known
and long standing in the community (but who don't have commit access)
could make some of these decisions?

Cheers.
Tai.

Russell Keith-Magee

unread,
Oct 4, 2010, 10:32:15 PM10/4/10
to django-d...@googlegroups.com
On Tue, Oct 5, 2010 at 9:58 AM, Tai Lee <real....@mrmachine.net> wrote:
> Hi Russ,
>
> On Oct 5, 11:48 am, Russell Keith-Magee <russ...@keith-magee.com>
> wrote:
>
>> > Perhaps we need another triage stage for these tickets, "Needs final
>> > review" or something?
>>
>> That's essentially what RFC is. This is an area where our documented
>> procedures have lagged behind in-practice procedures, but we've
>> essentially accepted that once a patch has been written by one person,
>> and reviewed by another and found to be ok, then it's RFC, which means
>> a core committer needs to take a look.
>
> I'm happy for RFC to indicate that a ticket is ready for final review
> by a core committer, after being reviewed by at least someone other
> than the patch author. But, I think there's still a problem of
> accepted tickets that have a patch with docs and tests and no feedback
> indicating that it needs improvement, being lost in the shuffle, as
> well as how and why we ever end up with tickets in this state and how
> can the patch author get someone to review their work.

You can get a list of tickets that meet that criterion very easily -
it's tickets in "accepted" with a "patch needs improvement" flag.

This doesn't solve the problem of people who review a ticket, but
don't flick the "patch needs improvement" flag or provide feedback on
what exactly is wrong... but no technological solution will fix those
problem. The best we can hope for here is for a second reviewer to
come along and clean up the ticket state.

The problems you describe here fall into a couple of categories:

* People doing triage, but not setting flags or giving feedback
correctly. This is a social problem; no technology is going to fix it
-- especially not adding *more* flags to our Trac configuration.
Improved documentation of what we expect *might* improve things here;
suggestions (and patches!) are welcome.

* Improving visibility of work that needs to be done. This is
somewhere that Trac gardening is needed -- the relevant queries aren't
hard to write; they just need to be made more prominent.

* Determining which ticket out of hundreds require attention. Two
ways to slice this that aren't currently available to us are ordering
by last modification date, and ordering by "popularity" (as determined
by users voting on tickets). Right now, the biggest impediment is that
our Trac install is *really* old, so all the nice new plugins
implementing these features won't run. We're in the process of
upgrading Trac; once that is done, we'll be able to add some of these
new features.

The only edge case I can see that you've identified that we don't
cover is people who upload incomplete patches which they *don't* want
reviewed yet. I don't see this as a major problem -- in my experience,
people upload patches because they think they're ready. Patches may
not always be correct, but I haven't seen a lot of people using Trac
as a temporary storage area for incomplete work. If someone uploads a
patch, they're generally looking for *some* feedback, even if it's
just "am I on the right track?"

>> As a general principle, DDN isn't usually a situation where more
>> information is needed; it's usually just time that is lacking.
>>
>> Sprints definitely help push this sort of thing along. However, that's
>> more because we have two core developers in a room at the same time,
>> not because of the presence of the rest of the community (although the
>> community got lots of other great work done). Malcolm and I spend a
>> day at the recent sprints just doing DDN tickets, and we managed to
>> clear out quite a few DDNs. We made vague plans to hook up on
>> IRC/Skype as a semi-regular activity to try and clear out the backlog,
>> but we haven't made that happen yet.
>
> Are at least two core committers required for all DDN tickets? Can a a
> single core committer make decisions on minor or trivial tickets, and
> the reporter be given the option to request a vote or further
> consideration if they feel the decision is wrong (possibly based on
> lack of information)? Perhaps some of the people who are well known
> and long standing in the community (but who don't have commit access)
> could make some of these decisions?

Are two core developers *required*? No. Having two developers is
really just a way of maintaining momentum; both in the since that pair
programming helps you maintain focus, and as a way to ensure that you
get a quick sanity check on an idea, rather than bouncing it into the
too-hard basket.

Could trusted non-core people do some of this DDN work? Yes, and they
already do. If you check the activity logs from yesterday, Eric
Holscher moved some of DDN tickets relating to the testing system, as
well as flicking a couple of tickets *to* DDN where he wasn't
comfortable making a call.

It's a little like jaywalking laws. Almost all cities have jaywalking
laws on the books. If you're visiting a city, you'll generally err on
the side of caution and not jaywalk at all. The longer you stay, the
better you will understand when the police *really* care about
jaywalking, when local traffic will be on the lookout etc. You won't
find "official" rules for when it's ok to jaywalk, but people do it
anyway, nobody is harmed, everyone gets where they need to go in a
timely fashion, and life goes on.

The same goes with Trac. We have an official set of Trac guidelines.
However, our Trac install is deliberately open ended so that anyone
can do anything to any ticket. We trust (and so far, that trust has
been generally well placed) that people won't start making decisions
arbitrarily. Trusted members of the community generally know when and
where they are trusted (like Eric, especially when it comes to
testing), and we rely on those people being a little bold and
jaywalking occasionally.

Yours,
Russ Magee %-)

Gabriel Hurley

unread,
Oct 5, 2010, 7:41:05 PM10/5/10
to Django developers
I think Russell put it very well in pointing out that one of the few
big technological impediments in Trac is that there isn't an easy way
to see the last-modification time of tickets. Particularly to the
point that it's hard to tell when a new patch is uploaded and an
accepted ticket might need new review. Could that be made easier
through technology, sure...

However, from my own observations, a lot of the community contributors
tend to be "subject matter experts" who focus in on certain areas.
I'll go through and gladly triage a whole set of tickets in an area I
know about regardless whether the patches are new or old. The point
being that even if the technology was there to make it easier, it's
still dependent on the actual volunteers themselves to take action.

On a separate note, a few posts back there was mention of the HOWTO
page that would encompass the spirit of contributing rather than the
law of it. I think that's a fantastic idea and I've started a new
thread regarding that issue.

Thanks!

- Gabriel

On Oct 4, 7:32 pm, Russell Keith-Magee <russ...@keith-magee.com>
wrote:
Reply all
Reply to author
Forward
0 new messages