Knowledge Base Contributor Survey: results and analysis
Chris Ilias
improve the contributor experience.
<http://blog.mozilla.com/sumo/2008/05/02/knowledge-base-contributor-survey/>
<http://blog.mozilla.com/sumo/2008/05/20/reminder-for-our-knowledge-base-contributors/>
352 invites were sent
- 302 Contributors
- 10 Approvers
- 40 Locale leaders
60 people filled out the survey
- 42 Contributors
- 4 Approvers
- 14 Locale leaders
(NOTE: some of us have dummy accounts for testing purposes, which may
have been in one of the above groups)
_Results_
SurveyGizmo processed a page of results for me at:
<http://app.sgizmo.com/reports/15640/35352/PBMX0AMQN6077TGD144VZTTP2JF1BL/>
The following isn't meant to address everything; but highlight points
for discussion. I have not filed the appropriate bugs for these yet:
* Most people intended to contribute when they first signed up, but 25%
did not.
* 28% did not know they could edit pages.
-- Solution: Instead of (or in addition to) "Submit additional
feedback", include another link to edit/translate the article.
* 26% of *first* article edits do not get approved.
-- Edits that need style fixing are always kept and fixed before going live.
-- If an edit is not spam, it is usually rejected because the
information is not backed up with a reference, or site policy does not
allow us to include the info. How about we change "Edit Summary", to
"Describe your edit, and cite a reference".
* "Dashboard", "Modify existing articles", and "Article I monitor..."
are the top three used Contributor links, then it drops off and
gradually declines.
-- If "Modify..." is used a lot, perhaps people aren't finding the edit
this page link.
-- Follow-up question: what is the personal dashboard used for, and how
can we improve that.
* Almost 50/50 on people that use the page watch feature.
-- Don't know what to make of that.
* 45% have not read the Best practices page or the style guide. Most
found them easy to understand.
-- My immediate thought is to include links to these pages within the
editor, along with a page on KB policies; but I don't want to pack too
much into the editor. It is already crowded.
_Suggestion box for best practices and style guide_:
(Some suggestions have already been taken care of, or bugs have already
been filed.)
* Tell contributors to use staging copies for article discussion:
-- I can add it to the Improving articles page.
* 41% needed assistance from another contributor.
-- I think this is an area in which we can really leverage the community
aspect of contributing. I'd like to make more use of the Contributors
forum. We added a link to the forum in the Contributor tools box after
the survey.
* 21% did not find the article editor easy to use.
-- I've already filed a bug on making rearranging items in the editor.
Couple this with the links mentioned earlier, and it looks like the
editor could use a lot more focus.
-- Personally, I think we can make the content area larger (taller and
wider).
* 31% did not find tikiwiki markup easy to use.
-- Since then, I have created a markup chart. I think there is a bug on
creating article templates as well.
-- We need to review the quicktags, to see:
--- which tags are most used.
--- customize the quicktag images.
--- create quicktags for SHOWFOR.
--- look into a quick paste function for dynamic content.
* 45% did not know about the staging system.
-- David suggested that this result may be skewed because of
misunderstanding over the term "staging system". People may think it was
something different from the "review system" or "approval system" (Which
is I never ever use the term "Awesome Bar"). What do you think of making
the official term "Review System"? I also want to change the term
"Approver" to "Senior Editor".
-- But I still need to do a better job of "explaining the system". Any
ideas? Perhaps a 5 second page that shows up after clicking on [Save],
that gives a thank you for contributing, and tells the contributor that
the edit will require approval before being applied to the KB version.
* The majority of Contributors read the Firefox Support blog, then
there's a big drop to mozilla.support.planning. Just a little lower is
the Contributors forum, then another big drop to #sumo.
-- Simple answer: I should start using the blog more, even for the
little news.
* 26% don't have a bugzilla account, and 38% don't have a
wiki.mozilla.org account. (I grouped these for this report, because the
greater question is: How much do we need to keep everything on
support.mozilla.com?)
-- I've tried to use the support site for docs such as listing which
articles need updating for Firefox 3; but I want to avoid cluttering
support.mozilla.com with more categories. Looking at which pages we use
wiki.m.o for instead of support.mozilla.com, I'm comfortable with the
judgement calls that have been made about where to put each doc.
-- I'm actually surprised the bugzilla number is as low as it is. Moving
article tracking from bugzilla to support.mozilla.com is something that
we've already talked about and filed bugs for.
_Documentation feedback box_:
(there are bugs already filed for most feedback items. Some things that
have not been addressed...)
* "Tell contributors in the Best Practices article or elsewhere that
they can use the staging copy comments section for documenting or
discussing changes to an article."
-- The large majority of comments on production articles is still people
looking for help. And these days, almost all of it is from registered
users. I'm leaning toward disabling anonymous commenting altogether,
disabling article comments on production versions, and pointing
registered users to the staging copy. And that pointer should give the
same "Still need help?" link for registered users.
_Staging system feedback_:
(there are bugs already filed for most feedback items. Some things that
have not been addressed...)
* link to bug report in staging copy
-- We certainly have enough discussion in bug reports to warrant this.
Even if it's just the first comment.
* link to staging copy in production copy
-- Definitely! Love this idea. Of course, it should only display for
contributors.
* "We don't know when an article is published or refused."
-- Another very good point. If I reject an edit, or need further info
about sources, all I can do is post a comment saying so. We should have
a rejection notice, similar to these mailing lists.
General feedback box:
(there are bugs already filed for most feedback items. Some things that
have not been addressed:)
* "I'm concerned that Firefox Support KB seems to stress style over
substance. The Contributors pages should make it very clear that adding
new KB content or correcting misinformation is primary; how the content
is formatted is secondary."
-- That is true, and I'll take it a step further, and say approve the
content if the info is correct, then style it later. That was the
initial plan.
* "Interface for translators still seems a bit buggy".
-- This person is obviously referring to the interactive translation.
Now, locale leaders have to edit language.php. It would be great if
locale leaders could go to a URL and enter the translation in text
fields, instead of having to manually edit a text file.
* "This survey did not cover translation of existing articles in another
language."
-- Unfortunately true. We've done quite a few things in the past month
or two to help translators learn how to translate. Two big issues:
1)translators sometimes use "edit this page" instead of "translate this
page", and 2)we need to get rid of the language selector for non-admins.
* "Ability to _browse_ articles in the KB."
-- This is a general request that has been brought up in a few different
ways. When I first contacted Sheppy about the KB, he mentioned the
desire to "navigate without search". David McRitchie has complained
about the lack of TOC in the in-product start page. And since the
redesign, we no longer have a "more articles..." link on the start page,
that links to
<http://support.mozilla.com/tiki-wiki_rankings.php?locale=en-US&limit=500&categId=1>.
Trying to put the articles in to a hierarchy tree is doomed to failure.
There's too much overlap. That's why most organizing systems have
switched to tags. I don't know what we can do help with this request.
* "And give contributors more perks."
-- Any ideas on what perks I can give?