Use of article comments
Chris Ilias
comments have been support requests, even though the intent is to get
article feedback. We had a long discussion about this a year ago [1],
and out of that discussion came the "Need more Help?" text[2], a change
in wording[3], and we even hid it by default[4].
Last week, I got notifications for 314 article comments, 16 of which I
considered useful. That's ~5%. The problem has not gone away. It's so
bad, I provide a weekly report of article comments to Cheng for common
issues metrics. As we've seen from Hendrix feedback and recently with
the @mozilla.com email addresses, users will use almost any contact
method for support requests. Article comments are even worse, because
there's no way to reply directly to an anonymous commenter.
I think we would be doing more good for users and make better use of our
time by directing everyone to the support forum, and doing away with
article comments altogether. We can take any KB article feedback from
the support forum.
On a greater level, I thought the feedback UI for the Things application
was interesting. There's one feebdack mechanism for support/bugs/RFEs.
Here are screenshots:
<http://ilias.ca/screenshots/Thingsfeedback-bug.png>
<http://ilias.ca/screenshots/Thingsfeedback-rfe.png>
<http://ilias.ca/screenshots/Thingsfeedback-support.png>
[1]<http://groups.google.com/group/mozilla.support.planning/browse_frm/thread/392e534e811dec1a>
[2]<https://bugzilla.mozilla.org/show_bug.cgi?id=413086>
[3]<https://bugzilla.mozilla.org/show_bug.cgi?id=413808>
[4]<https://bugzilla.mozilla.org/show_bug.cgi?id=418199>
--
Chris Ilias <http://ilias.ca>
List-owner: support-firefox, support-thunderbird, test-multimedia
Keeper of the Knowledge Base: <https://support.mozilla.com/kb/>
Majken Connor
> support-planning mailing list
> support-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/support-planning
>
If we did want to try leaving the article feedback, I noticed that we
never did try adding text near the comment field that explicitly tells
people that we won't respond to comments and that they should ask a
question instead.
Knowing what problem people are trying to solve when reading an
article is good feedback, even if they don't give us specific feedback
on how they'd change the article. For instance if they leave a comment
asking how to do something that's already in the article we can see if
we need to edit that section to make it easier to find/understand. I'm
watching several articles that I've edited and on those ones at least
only a very small amount are actually unrelated to the article. When
people go directly to the forums we don't always get to know what
articles they actually ended up on.
We might also just want to let people leave an email address and when
we see a support comment send them a email saying "thanks for your
feedback, but it looks like you still have a problem you need to
solve. Please ask a question in the forums or live chat so we can help
you." Maybe we can make a way to duplicate the comment contents in the
new thread.
Some other things I noticed:
* "still need help" is in the light gray, we might want to look at
ways of making this stand out.
* We should do something better when someone clicks "no" to the solve
a problem question
* Maybe we only want to show the feedback form and link if someone
clicks "no" to the easy to understand question.
That last point had me going back and forth. If the article didn't
solve the problem then maybe people have feedback, too, but I think if
people click no it's more important to get them to an answer. We can
tell from the forums what existing answers people aren't finding, and
we'll want to decide ourselves the appropriate place for new answers,
so I think we can get away with skipping the feedback form for those
people. People who answer yes to both questions probably don't have
feedback to leave.
I guess it depends on what you're counting as useful, and if you're
automatically counting support requests as not useful. I think there
are still things we can do to make sure users get to ask a question if
we decide there's still value in having direct article feedback.
We could also change from the default article comment system to
creating forum threads with the article as the topic in a new article
feedback forum (or even in the regular support forum). That way we can
still reply to the support requests, but we still get some idea of
where the user has been and where they expected their answer. I think
this addresses all the problems with the current feedback system
without removing any obvious way for people to give us their
suggestions.
David Tenser
> On Mon, Feb 16, 2009 at 12:47 AM, Chris Ilias<n...@ilias.ca> wrote:
>> Ever since article commenting was made available, the majority of comments
>> have been support requests, even though the intent is to get article
>> feedback. We had a long discussion about this a year ago [1], and out of
>> that discussion came the "Need more Help?" text[2], a change in wording[3],
>> and we even hid it by default[4].
>>
>> Last week, I got notifications for 314 article comments, 16 of which I
>> considered useful. That's ~5%. The problem has not gone away. It's so bad, I
>> provide a weekly report of article comments to Cheng for common issues
>> metrics. As we've seen from Hendrix feedback and recently with the
>> @mozilla.com email addresses, users will use almost any contact method for
>> support requests. Article comments are even worse, because there's no way to
>> reply directly to an anonymous commenter.
>>
>> I think we would be doing more good for users and make better use of our
>> time by directing everyone to the support forum, and doing away with article
>> comments altogether. We can take any KB article feedback from the support
>> forum.
>>
>>
>
> never did try adding text near the comment field that explicitly tells
> people that we won't respond to comments and that they should ask a
> question instead.
I agree that this is something we should explore. The notification above
the text box today is almost illegible, as is the captcha description
below it.
I think the important thing here is that it needs to be clear that we're
not going to respond to this feedback, because if we do (e.g. by
providing an e-mail field for users to fill in), this essentially
becomes a new support channel that we _have_ to monitor for support
issues. If we can make it totally clear that we aren't going to respond
to the feedback, we're halfway there.
There's also a bug here: once you click "Submit additional feedback,"
the "Still need help? Ask a support question" text above it disappears.
We need to make sure that text is still visible (that's just a bug,
really) and then refer to that link in the text about what the feedback
field is for.
>
> Knowing what problem people are trying to solve when reading an
> article is good feedback, even if they don't give us specific feedback
> on how they'd change the article. For instance if they leave a comment
> asking how to do something that's already in the article we can see if
> we need to edit that section to make it easier to find/understand. '
Right. This is the main reason why I think keeping the feedback field is
a good idea, with the above suggestions implemented. If we remove it, we
lose a lot of valuable info about our articles.
>
> Some other things I noticed:
> * "still need help" is in the light gray, we might want to look at
> ways of making this stand out.
Seems odd that that question is gray and the rest of the text with the
same size is dark or links. Someone please file a bug for that.
> * We should do something better when someone clicks "no" to the solve
> a problem question
Agreed. I'm betting that the majority of the people that provide support
related "feedback" to the article are also answering No to that poll.
I still like what mconnor suggested of showing three relevant articles
(based on e.g. popularity combined with tags), along with a prominent
note that we also offer the ability to ask a new support question. We
just haven't given this a high priority -- so many things being done at
the same time, so little SUMO dev resources -- but I'd love to try to
get this into the 1.1 UX milestone; at least with the link to ask a
support question, in case the relevant article match requires a lot more
work.
> * Maybe we only want to show the feedback form and link if someone
> clicks "no" to the easy to understand question.
We'd then miss the people that actually found it easy to understand but
spotted some inaccuracies. Just as an example.
Chris Ilias
> On 09-02-16 08.20, Majken Connor wrote:
>
>> If we did want to try leaving the article feedback, I noticed that we
>> never did try adding text near the comment field that explicitly tells
>> people that we won't respond to comments and that they should ask a
>> question instead.
>
> I agree that this is something we should explore. The notification above
> the text box today is almost illegible, as is the captcha description
> below it.
>
> I think the important thing here is that it needs to be clear that we're
> not going to respond to this feedback, because if we do (e.g. by
> providing an e-mail field for users to fill in), this essentially
> becomes a new support channel that we _have_ to monitor for support
> issues. If we can make it totally clear that we aren't going to respond
> to the feedback, we're halfway there.
I think both of you are overestimating users. For example,
<http://hendrix.mozilla.org/> has "Please accept our thanks, but do not
expect a response to feedback submitted here." in big bold red text, yet
mozilla.feedback.firefox still has a lot of support requests.
Why is there reluctance to point those people to the support forum? KB
contributors should be pulling feedback from the support forum anyway.
It might be better if we did something like including a referrer URL in
the original post of each forum thread. Or maybe implement a way to tag
forum threads with "KB update". Or ask the user when posting to the
forum, what type of message they are sending (Firefox Help, KB feebdack,
and maybe even Firefox feedback).
Michael Connor
On 26-Feb-09, at 5:02 PM, Chris Ilias wrote:
> On 2/26/09 7:14 AM, _David Tenser_ spoke thusly:
>> On 09-02-16 08.20, Majken Connor wrote:
>>
>>> If we did want to try leaving the article feedback, I noticed that
>>> we
>>> never did try adding text near the comment field that explicitly
>>> tells
>>> people that we won't respond to comments and that they should ask a
>>> question instead.
>> I agree that this is something we should explore. The notification
>> above the text box today is almost illegible, as is the captcha
>> description below it.
>> I think the important thing here is that it needs to be clear that
>> we're not going to respond to this feedback, because if we do (e.g.
>> by providing an e-mail field for users to fill in), this
>> essentially becomes a new support channel that we _have_ to monitor
>> for support issues. If we can make it totally clear that we aren't
>> going to respond to the feedback, we're halfway there.
>
> I think both of you are overestimating users. For example, <http://hendrix.mozilla.org/
> > has "Please accept our thanks, but do not expect a response to
> feedback submitted here." in big bold red text, yet
> mozilla.feedback.firefox still has a lot of support requests.
Yes, well, that isn't "You will not get an answer" I stopped reading
that the first time I saw it after the first phrase...
-- Mike
Majken Connor
> On 2/26/09 7:14 AM, _David Tenser_ spoke thusly:
>>
>> On 09-02-16 08.20, Majken Connor wrote:
>>
>>> If we did want to try leaving the article feedback, I noticed that we
>>> never did try adding text near the comment field that explicitly tells
>>> people that we won't respond to comments and that they should ask a
>>> question instead.
>>
>> I agree that this is something we should explore. The notification above
>> the text box today is almost illegible, as is the captcha description below
>> it.
>>
>> I think the important thing here is that it needs to be clear that we're
>> not going to respond to this feedback, because if we do (e.g. by providing
>> an e-mail field for users to fill in), this essentially becomes a new
>> support channel that we _have_ to monitor for support issues. If we can make
>> it totally clear that we aren't going to respond to the feedback, we're
>> halfway there.
>
> I think both of you are overestimating users. For example,
> <http://hendrix.mozilla.org/> has "Please accept our thanks, but do not
> expect a response to feedback submitted here." in big bold red text, yet
> mozilla.feedback.firefox still has a lot of support requests.
Some people do submit their support issues as feedback. Many people
think if they're having a problem it must be a bug and so even though
they know we won't reply they think they're helping out by letting us
know about it, or they haven't found the support forums yet and again
just want to make sure we're aware of the problem. I'd be very curious
to track whether people who post article comments also end up asking a
question.
For these cases I think the answer isn't a negative reinforcement "you
won't get a response" but a positive reinforcement of where we want
them to go "we'll help you if you ask a question"
>
> Why is there reluctance to point those people to the support forum? KB
> contributors should be pulling feedback from the support forum anyway. It
> might be better if we did something like including a referrer URL in the
> original post of each forum thread. Or maybe implement a way to tag forum
> threads with "KB update". Or ask the user when posting to the forum, what
> type of message they are sending (Firefox Help, KB feebdack, and maybe even
> Firefox feedback).
I agree, we really need more super easy ways to mark things for KB
authors to pick up to add to articles, as well as easy ways for kb
authors to see these. I think this has been left out of planning as we
were thinking of it as part of a migration away from bugzilla for
article requests.
I don't think this functionality will be very simple as we'll also
need to be able to mark which suggestions have been added, mark which
article forum suggestions apply to, view suggestions that don't have
an article associated and view all suggestions per article.
Majken Connor
Well I think we can avoid that by what i suggested before:
"We might also just want to let people leave an email address and when
we see a support comment send them a email saying "thanks for your
feedback, but it looks like you still have a problem you need to
solve. Please ask a question in the forums or live chat so we can help
you." Maybe we can make a way to duplicate the comment contents in the
new thread."
So we send a reply to the comment, and in theory we could include a
link to automatically create a forum thread using the contents of the
comment. So we're not turning the comments into another support forum,
we're giving ourselves another way to get those people into the
support forum.
Or maybe we can even hijack this further. Ask for an email, leave a
checkbox saying "I'd like to be contacted about my feedback" and if
they check it, it makes a forum thread instead.
David Tenser
I filed https://bugzilla.mozilla.org/show_bug.cgi?id=482467 to make the
text actually readable and make the message clearer.
<h3>Submit article feedback</h3>
<p>Note that we will not respond to your feedback about this article.
However, it will be visible to contributors to this site and may be used
to improve this article.</p>
<p>If you need help with solving a Firefox problem, <a
href="/%locale%/kb/Ask+a+question">ask a support question</a> instead.</p>
Suggestions on wording welcome.
Majken Connor
> _______________________________________________
> support-planning mailing list
> support-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/support-planning
>
I got to thinking about this some more lately, esp in terms of my
suggestions below to allowing users to leave an email address when they
leave article comments. I think we've been missing a key place where we can
shine where other documentation centers fail.
I think we should be allowing users to leave their email address, and then
we should be contact them when we make the change suggested or related to
their comment. We can then extend a personal invitation to people who took
the time to give us feedback to review the change. Hopefully in the process
we'll be introducing them to how the article system works and give them some
1 to 1 introduction on helping out.
This does seem like extra work at first, but this is similar to the training
process that our Live Chat contributors accept as part of being involved,
and part of helping grow the community.
By putting these "basic contributors" in the loop, I think we get the
following benefits:
- they actually get to see how they made a difference (I think this is the
#1 motivator for any Mozilla contributor)
- they feel invited to participate, makes SUMO feel open
- those who weren't sure where to start get a personalized hand in
contributing
- we increase the chances that they'll make the change themselves next time
- we get confirmation that our change really is better for that person or
what they intended
All of the above *should* amount to multiplying our KB contributors.
It would however require a bit of work:
- We'd want to create an auto responder for users who provide an email
address on article feedback pointing them to ask a question as well as how
to contribute
- KB contributors who update articles would need to take more time to
contact the feedback authors
-- Would require some mechanism to make this easier rather than having the
contributor manually click on email links and write messages
- After contacting feedback authors, KB contributors might end up in a
discussion with the author, again, requiring more time than simply updating
the article
-- We should have some method for passing these discussions on, like how
Room Monitors are available in Live Chat to coach new helpers