Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
MDN's Open Content Contribution Pathways
6 views
Skip to first unread message
Justin Crawford
Jan 21, 2015, 2:32:14 PM1/21/15
to mdn-drivers
Over the past month the MDN team has triaged hundreds of articles submitted
by spammers containing spam content. The most active triagers now
collectively spend days per week on this effort.
We have dozens of ideas for *technical solutions* to spam articles. This
thread is not about those. We have open bugs for those that we're working
on. If you want to talk about those, please start a new thread (or open a
bug).
This thread is about a more fundamental question. MDN currently allows
anyone, anytime, to quickly create an account and immediately edit content
that is posted immediately after it is edited. Should MDN constrict one or
more of these openings to reduce the risk and impact of spam articles on
the site?
Ali is working on understanding some numbers around this question, such
as...
* how many new long-term contributors does MDN attract each month?
* what percentage of good revisions (in any language) are first-time
revisions?
How essential are these completely open pathways to making MDN what it is?
Justin Crawford
Product Manager, MDN
hoos...@mozilla.com
by spammers containing spam content. The most active triagers now
collectively spend days per week on this effort.
We have dozens of ideas for *technical solutions* to spam articles. This
thread is not about those. We have open bugs for those that we're working
on. If you want to talk about those, please start a new thread (or open a
bug).
This thread is about a more fundamental question. MDN currently allows
anyone, anytime, to quickly create an account and immediately edit content
that is posted immediately after it is edited. Should MDN constrict one or
more of these openings to reduce the risk and impact of spam articles on
the site?
Ali is working on understanding some numbers around this question, such
as...
* how many new long-term contributors does MDN attract each month?
* what percentage of good revisions (in any language) are first-time
revisions?
How essential are these completely open pathways to making MDN what it is?
Justin Crawford
Product Manager, MDN
hoos...@mozilla.com
Jean-Yves Perrier
Jan 21, 2015, 3:39:32 PM1/21/15
to mdn-d...@lists.mozilla.org
As somebody who is checking daily editions for the last 3 years, and was
doing it regularly at least 1 year before, it is clear that this will
not remove work, this will add more work.
As we are doing a lot of overtime now (and not much anymore because of
spam, but just to keep the pace of what is expected as doc), it means
that this won't be a practical solution unless we hire people. And we'll
need to have somebody on duty 24h/24 365h/365 or we will prevent people
to do a second edition (and only valuable people) and they will be lost
as contributors.
Most first editions are positive and don't lead to any further action
(just look at the e-mails you get now and remove the latest spam, as you
are designing a plan against it). More of the few editions that need
more actions, almost none is malevolent or need urgent action (A daily
clean is most of the time just enough; and we do it much more often than
this, including during week-ends and holidays).
Engaging on the first edit is a complete loss of time as most editors
don't do a second edit (and it is perfectly normal, they are fixing
typos or obvious errors); I abandoned doing it after 2 weeks last year
when I took over l10n community management for a couple of quarters.
My opinion: don't lose any time on it. It is the best way to kill the MDN.
I'd rather find technical solutions to detect spam and to block only
these editions (Like semi-protection of pages, quarantining edition from
untrusted author adding links, …). Just annoying legitimate users
because it is technically easier and giving the tree sheriffs more work
won't help.
--
Jean-Yves
> _______________________________________________
> Mdn-drivers mailing list
> Mdn-d...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/mdn-drivers
>
--
Jean-Yves Perrier
Senior Technical Writer / Mozilla Developer Network
doing it regularly at least 1 year before, it is clear that this will
not remove work, this will add more work.
As we are doing a lot of overtime now (and not much anymore because of
spam, but just to keep the pace of what is expected as doc), it means
that this won't be a practical solution unless we hire people. And we'll
need to have somebody on duty 24h/24 365h/365 or we will prevent people
to do a second edition (and only valuable people) and they will be lost
as contributors.
Most first editions are positive and don't lead to any further action
(just look at the e-mails you get now and remove the latest spam, as you
are designing a plan against it). More of the few editions that need
more actions, almost none is malevolent or need urgent action (A daily
clean is most of the time just enough; and we do it much more often than
this, including during week-ends and holidays).
Engaging on the first edit is a complete loss of time as most editors
don't do a second edit (and it is perfectly normal, they are fixing
typos or obvious errors); I abandoned doing it after 2 weeks last year
when I took over l10n community management for a couple of quarters.
My opinion: don't lose any time on it. It is the best way to kill the MDN.
I'd rather find technical solutions to detect spam and to block only
these editions (Like semi-protection of pages, quarantining edition from
untrusted author adding links, …). Just annoying legitimate users
because it is technically easier and giving the tree sheriffs more work
won't help.
--
Jean-Yves
> Mdn-drivers mailing list
> Mdn-d...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/mdn-drivers
>
--
Jean-Yves Perrier
Senior Technical Writer / Mozilla Developer Network
Chris Mills
Jan 21, 2015, 3:49:37 PM1/21/15
to Justin Crawford, mdn-drivers
> On 21 Jan 2015, at 19:32, Justin Crawford <hoos...@mozilla.com> wrote:
>
>
> How essential are these completely open pathways to making MDN what it is?
If we impose restrictions on MDN’s openness to contributions, MDN will not be quite the same; it *will* be affected. How much will depend on what those restrictions are.
If we made those pathways closed except to trusted core contributors, it would be a disaster. But I think it is possible to achieve a happy medium.
We obviously need better metrics to device this, but how about:
* Anyone can create an account, but their first edits will be kept in a queue and they are not allowed to make any more until verified by a trusted member of staff. If it is spam, their account is deleted. If it is not spam, they are allowed to keep on contributing. Surely if we have to look at every account’s first edit now anyway, this would not be much more work.
* We’d probably have to limit the frequency with which accounts can be created, otherwise they’ll start trying to DDoS that.
* This would surely not kill MDN’s community, as long as we have a decent global moderator team in place, so the queue is emptied rapidly.
Also:
* Certain other pages should be blocked from edits, such as Zone landing pages.
* We should have checks in place to try to spot spammy patterns, and flag them up if they are seen.
Janet Swisher
Jan 21, 2015, 4:29:28 PM1/21/15
to mdn-d...@lists.mozilla.org
I'm in favor of applying technical solutions as much as possible before
we change our human processes.
SUMO has an edit>review>publish model for their knowledgebase, but they
have a much smaller number of articles than MDN, and I'd expect they
have a much smaller number of edits per day.
I agree with Jean-Yves that simply putting a review step in front of
first contributions will create more work than it will avoid. I'd rather
apply a machine-intelligence filter to flag questionable contributions
(first or otherwise) than use human intelligence as the first filter.
I'm not convinced that engaging on the first edit is a waste of time. I
don't expect the percentage of return edits to be high, but I believe we
can move the needle. However, it would help to have tools to make the
workflow for it easier.
Janet Swisher <mailto:jREMOVE...@mozilla.com>
Mozilla Developer Network <https://developer.mozilla.org>
MDN Community Manager
we change our human processes.
SUMO has an edit>review>publish model for their knowledgebase, but they
have a much smaller number of articles than MDN, and I'd expect they
have a much smaller number of edits per day.
I agree with Jean-Yves that simply putting a review step in front of
first contributions will create more work than it will avoid. I'd rather
apply a machine-intelligence filter to flag questionable contributions
(first or otherwise) than use human intelligence as the first filter.
I'm not convinced that engaging on the first edit is a waste of time. I
don't expect the percentage of return edits to be high, but I believe we
can move the needle. However, it would help to have tools to make the
workflow for it easier.
Mozilla Developer Network <https://developer.mozilla.org>
MDN Community Manager
Jeremie Patonnier
Jan 22, 2015, 5:33:46 AM1/22/15
to Janet Swisher, mdn-drivers
Hi :)
I really think we should not change the way edition can be made so easy on
MDN.
Review of first contribution (as Chris suggest) could be okay if it does
not give more work to the paye staff team. Moderation could be done by
users them self. For example, users that have been their first edit
accepted could gain trust and become themselves moderator for other first
edit -> This would be a way to empowered recurring contributors by starting
delegating the sheriffing of MDN.
However, as Janet and Jean-Yves stated, increasing human work is not a good
idea in itself, we are too much under limited ressources to afford that.
Focusing on mechanical solution will always be more beneficial to us (for
example by forbidden the change or addition of any links the x first
edit... I will open a bug for that suggestion)
Best,
Jeremie
.............................
Web : http://jeremie.patonnier.net
Twitter : @JeremiePat <http://twitter.com/JeremiePat>
I really think we should not change the way edition can be made so easy on
MDN.
Review of first contribution (as Chris suggest) could be okay if it does
not give more work to the paye staff team. Moderation could be done by
users them self. For example, users that have been their first edit
accepted could gain trust and become themselves moderator for other first
edit -> This would be a way to empowered recurring contributors by starting
delegating the sheriffing of MDN.
However, as Janet and Jean-Yves stated, increasing human work is not a good
idea in itself, we are too much under limited ressources to afford that.
Focusing on mechanical solution will always be more beneficial to us (for
example by forbidden the change or addition of any links the x first
edit... I will open a bug for that suggestion)
Best,
Jeremie
> _______________________________________________
> Mdn-drivers mailing list
> Mdn-d...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/mdn-drivers
>
--
Jeremie
> Mdn-drivers mailing list
> Mdn-d...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/mdn-drivers
>
--
.............................
Web : http://jeremie.patonnier.net
Twitter : @JeremiePat <http://twitter.com/JeremiePat>
Sebastian Zartner
Jan 22, 2015, 8:02:26 AM1/22/15
to Jeremie Patonnier, mdn-drivers, Janet Swisher
Restrictions to new contributors should be limited to a minimum.
I like Jeremie's idea of sharing the reviewing work with the community. At
least I'd help if there was a simple UI.
On the other side I'm wondering if time is not better invested in
implementing automatic solutions than in creating a manual review system.
Two important questions to answer this are:
- How big is the number of spam edits?
- How high is the percentage of those that can be detected automatically?
Sebastian
On 22 January 2015 at 11:32, Jeremie Patonnier <jeremie....@gmail.com>
wrote:
I like Jeremie's idea of sharing the reviewing work with the community. At
least I'd help if there was a simple UI.
On the other side I'm wondering if time is not better invested in
implementing automatic solutions than in creating a manual review system.
Two important questions to answer this are:
- How big is the number of spam edits?
- How high is the percentage of those that can be detected automatically?
Sebastian
On 22 January 2015 at 11:32, Jeremie Patonnier <jeremie....@gmail.com>
wrote:
Stormy Peters
Jan 22, 2015, 11:56:04 AM1/22/15
to Sebastian Zartner, mdn-drivers, Jeremie Patonnier, Janet Swisher
What about a way for a (large) trusted set of users to flag a first edit as
spam that would then lock that account?
Stormy
spam that would then lock that account?
Stormy
柯拓
Jan 22, 2015, 2:12:46 PM1/22/15
to mdn-d...@lists.mozilla.org
I am the zh-CN localization leader for MDN and I strongly disagree with this proposal. New contributors are valuable to us.
Dan Callahan
Jan 22, 2015, 2:43:23 PM1/22/15
to mdn-d...@lists.mozilla.org
Just a heads up, 柯拓's message in opposition to new contributor restrictions
got caught by spam filters on my end.
On Thu, Jan 22, 2015 at 3:22 AM, 柯拓 <xiaow...@taobao.com> wrote:
> I am the zh-CN localization leader for MDN and I strongly disagree with
> this proposal. New contributors are valuable to us.
got caught by spam filters on my end.
On Thu, Jan 22, 2015 at 3:22 AM, 柯拓 <xiaow...@taobao.com> wrote:
> I am the zh-CN localization leader for MDN and I strongly disagree with
> this proposal. New contributors are valuable to us.
Justin Crawford
Jan 22, 2015, 7:12:26 PM1/22/15
to mdn-drivers
I'm hearing two categories of concern:
1) Closing contribution pathways will create new work for moderators -- the
work they're now doing to triage spam will be spent reviewing potential
contributors or potential contributions.
2) Closing contribution pathways will change what MDN is.
I promised some data. In 2014...
* 46% of contributors made only 1 revision. They collectively made 3% of
all revisions in 2014.
* 181 contributors made more than 25 revisions in any single month. They
collectively made 75% of all revisions in 2014. These people have joined
MDN at an average rate of 4.5 per month since 2009.
These numbers don't speak to the quality of contributions. We currently
have no metrics around quality. But they do illustrate that our most active
contributors make many more contributions than first-time editors, and
they're rare. I would love to know whether the open contribution path was
an important factor in recruiting these people, and I have some ideas for
finding out.
To answer questions posed earlier...
> - How big is the number of spam edits?
There are different kinds of spam on MDN. The kind we're triaging most
right now is coming in at a rate of 20+ per week, with hundreds in the
worst weeks.
> - How high is the percentage of those that can be detected automatically?
We can't answer this now. Of the current malicious wave of spam, I
personally think the percentage is quite high.
On Thu, Jan 22, 2015 at 12:42 PM, Dan Callahan <dcal...@mozilla.com>
wrote:
1) Closing contribution pathways will create new work for moderators -- the
work they're now doing to triage spam will be spent reviewing potential
contributors or potential contributions.
2) Closing contribution pathways will change what MDN is.
I promised some data. In 2014...
* 46% of contributors made only 1 revision. They collectively made 3% of
all revisions in 2014.
* 181 contributors made more than 25 revisions in any single month. They
collectively made 75% of all revisions in 2014. These people have joined
MDN at an average rate of 4.5 per month since 2009.
These numbers don't speak to the quality of contributions. We currently
have no metrics around quality. But they do illustrate that our most active
contributors make many more contributions than first-time editors, and
they're rare. I would love to know whether the open contribution path was
an important factor in recruiting these people, and I have some ideas for
finding out.
To answer questions posed earlier...
> - How big is the number of spam edits?
right now is coming in at a rate of 20+ per week, with hundreds in the
worst weeks.
> - How high is the percentage of those that can be detected automatically?
personally think the percentage is quite high.
On Thu, Jan 22, 2015 at 12:42 PM, Dan Callahan <dcal...@mozilla.com>
wrote:
0 new messages