Cutting down the review queue - Mentoring System
Axel
newbie on this group here, I had just read the article "Burning down
the Add-On review queues" on the mozilla blog at
http://blog.mozilla.com/addons/2009/11/13/burning-down-the-add-on-review-queues/
- and left a (rather verbose) comment on this, which I would like to
share and put up for discussion.
Please don't take offense if this happens to mirror some of the ideas
that I am sure some of you guys already have come up with, this is
just my independant approach which comes from my first year of fervent
Extension Development for Thunderbird, and I am still a very strong
believer in the Open Source development - I think more commercial
software should develop using the principles of open source, IMHO it
would increase the quality of the software written and also of the
process of "creating" software. Without further ado, here goes:
----------------
The reviewing queue is a bit like the door to the law in Kafka's short
story "Before the Law" - you never know when you're extension will be
let ;)
Anyway if you are really interested in cutting down the review queue I
would ask to also look at Fx's little brother Thunderbird, it seems
the review queue there suffers because of the new Fx Release.
My personal viewpoint on releasing is "release early and often" which
leaves my extension with 4 (!) updates that have not yet been
reviewed, the latest one offering an ever increasing feature set and
lots of desirable bug fixes - and just today I have released the 5th
one [which now includes support for 2 more hosts and 3 more locales,
compared to the last published version]; I do not know whether this
actually has again reset the waiting time for my queue - but in
general it seems to boil down to a publishing cycle of roughly 1
published update / month (usually I integrate several bugfixes during
that time, so it is not exclusively negative).
- all thanks to the diligence of the users who constantly come up with
ideas, requests and bug reports. There is an highly interesting
article "the cathedral and the bazaar" by Eric Stephen Raymond (see
catb.org) which points out the difference between commercial and open
source software release cycles, which comes to the conclusion that a
higher release frequency is indeed a good warranty for high software
quality standards - obviously it also generates a lot more
administrative work; I think one of the methods to cut down on the
amount of work would be fairly obvious - people who have reviewed an
extension before (and wish to do so) should be preferred as reviewers
for updates of this.
In my case I have a fairly complex extension with a lot of added value
for long term users, but this is not obvious at first glance to a
casual user - the more you use this extension the more useful features
you will discover; for reviewers, testing this functionality can of
course be a daunting task - you need to set up test folders, do drag /
copy operations on emails, customize the extension's interface and so
on and so on...
How much easier is it if you already use the extension and you might
even be interested in what's new - wouldn't it be great if the
reviewers would be notified of updates of any extension they have
installed on their system and have reviewed previously - surely
somebody who <emp>uses</emp> a piece of software daily is somewhat of
an expert.
I would propose a "mentoring system" where AMO reviewers could
voluntarily take some of their favorite extensions and be contactable
by the authors about important updates?
In emergencies (such as showstopper bug fixes) I have done so before,
using more oblique methods such as preoject owners mailing lists and
ICQ, but it would be great if there was an official method openly
available to developers of extensions that have already been
published.
The other thing that would surely diminish the problem of "stale"
updates would be a more obvious presentation of newer versions on the
AMO pages. I must say that it was slightly better in an earlier
incarnation of the AMO web site, now the link to the latest versions
hides shamefully at the bottom of the page, and its labeled
misguidingly "View Older Versions". This is not obvious to the end
users. IMHO there is probably more chance of the users looking at the
top 3 reviews and finding that their host version is supported or a
certain bug is fixed and then telling them where to download the
unreviewed version than them preiodically "poking" the versions page
to see whether there was a new version available. But you know this
already, as you're trying to cut down the queue - it is just not
obvious how it can be done without increasing manpower...
Also, there is a link for getting the latest published version for
each extension, would it be possible (and desirable) to craft one for
the latest "experimental" version as well? Or do you think that it
would open the door for all sorts of security risks?
I hope you found some food for thought here,
yours sincerely
Axel
Archaeopteryx
I don't think that Thunderbird add-ons suffer from Firefox betas, but
from fewer editors and the abstinence of some of from (i.e. me).
You also mention an assumption which causes problems: "release early and
often". Think about the consequences. It is little effort for you to
push your development build to public, but if every version would be
tested by an editor, think about how much time it costs. Furthermore, if
an editor sees the often releases by you, he can think like "No hurry
with that one, it is pretty probable that there will be a new version
tomorrow." Version history:
https://addons.mozilla.org/thunderbird/addons/versions/3254
About the proposition to let people already using the extension review
it _in already running environment_ - alone it's a No-Go. Some
developers do this, but what if the extension isn't working after
initial install?
Currently, editors can get notifications for add-on updates if they did
the latest review for them. A permanent subscription is planned, but has
low priority: https://bugzilla.mozilla.org/show_bug.cgi?id=502394
Keep on the good work with QuickFolders
Archaeopteryx