Commit Access: Proposal re the "Not Your Employer" Rule

73 views
Skip to first unread message

Mitchell Baker

unread,
May 21, 2009, 8:54:28 PM5/21/09
to
I'll get to the punchline first, and then go through the rationale. I
believe the "not your employer" rule is harming us more than it's
helping us now. I also think the principles governing commit access
can be protected without this particular rule. I've put the rationale
below.

As a result I'm proposing we retract the "one person must not be
employed by the same person as the candidate" rule for all candidates
for commit access.

That leaves us with the following requirements:

-- two vouchers (module owners or peers) working in modules in which
the candidate is working; and
-- one super-reviewer from a different module

Looking for feedback.

+++


Detailed Rationale for Removing the "Not Your Employer" requirement
(long-ish)

Here are the principles I think we are trying to implement:

Principles:

-- Commit access is earned by each individual based on his or her own work.
-- Employment status -- with any organization -- is irrelevant to commit
access.
-- Commit access is awarded by a decision of respected peers, based on
experience with an individual.
-- Commit access is awarded when an individual has demonstrated (a)
understanding and respect for operational rules (testing, checking in,
watching for problems, resolving problems with check-ins, etc) and (b)
acceptable quality code.*


*(Mike Connor has suggested that (b) -- acceptable quality code -- is
less important than it once was. In some sense this is true. In the
early days of Mozilla we had to fight long and hard to improve the
quality of code in our tree to something we were proud of. (Anyone who
remembers Netscape 6 will understand why improving the overall code
quality was so fundamental.) Commit access was one line of defense
here, where we knowingly slowed down granting of access in order to get
a handle on who was making our code better and how they were doing so. )


Current Policy

Many years ago we codified these principles into a process for obtaining
CVS access. We identified the "respected peers" that must support a
request for commit access as:

-- two module owners or peers from modules in which the candidate has
been submitting patches (the two "vouchers"). These are the people who
presumably know the candidate best. They have been reviewing the
candidates patches. They may well be checking in the candidates patches
and have experience with how the candidate participates in the checkin
and follow up activities.

--one super-reviewer from outside the modules in which you have been
working. The goal here was to make sure we didn't have unintentional
"group-think" or that a couple of module owners/peers being overly
optimistic as a result of being friendly with the candidate.

--We also said that one of the three must NOT have the same employer as
the candidate. This aspect had a few goals. One was to explicitly
break the link in the minds of many organizations that employment =
source access. This may seem obvious now, but in the early days of
Mozilla it was a very, very contentious issue. Many organizations
simply could not understand how they could hire someone to work on
Mozilla and then not be able to provide source code commit access. A
second goal was to protect people with commit access from being
pressured by their employers. (Did I mention how contentious this issue
was?) A third reason was because the code review process was also
contentious and under great strain. Sometime poor code would be
approved by reviewers because they had already spent a lot of effort
reviewing, the code was still poor quality, the author of the code was
employed by someone to work on mozilla and so would continue to produce
poor quality code forever, and the reviewer would finally decide s/he
couldn't improve the code through the review process. It's harder to
understand this today but it was a live issue in 1999. A final aspect
was a concern that similarities of an employment setting would cause
"group-think" to outweigh independent technical judgment.

Today

I think most of the reasons for the "not your employer" rule are far
less valid today than when we adopted the current policy. The reality
that our invaluable source code asset is best protected by independent
technical judgment is ingrained in a way that simply was not true a
decade ago.

One remaining potential value that I see is possibly protecting against
group-think. One never know what a different perspective will bring to
the discussion. Employees may share some things in common simply by
being employees. So the "not your employer" rule theoretically brings
some different perspective to the evaluation of whether a person is
ready for commit access.

Against this we need to weigh the very real harm we're causing ourselves
today, where the "not your employer" rule prevents respected hackers
from getting commit access because there aren't enough module owners/
peers not employed by Mozilla to review their code and make this
evaluation.

This harms the project in a number of ways.

-- Quantities of high quality code from known and respected peers must
be processed (checked in) by others.
-- The burden on module owners / peers who are not employed by Mozilla
to do the level of review required for commit access is high
-- The backlog of code that someone has to check-in means that new
contributors are in a much longer line to have their code checked in,
and face a lot of competition


We could approach this problem by consciously NOT hiring module owners.
I think this is a bad approach. It penalizes people for earning and
accepting the responsibilities of module ownership. Becoming a module
owner then makes someone ineligible to work for Mozilla. This seems
fundamentally backwards to me. And it creates pressure to make module
owners due to their (non) employment status, whether or not the module
makes sense on its own or we have an owner with the technical background
we want. To me, the bad incentives here are worse that relying on the
vouchers and super-reviewers to make independent judgments. We do need
to explore the development of new areas, new modules and new module
owners. But we should explore this for its own sake, not because we
have a rule about commit access.

For the determination of readiness for commit access we can rely on the
core understanding that source code access must *always* be earned and
that one's own reputation will suffer for poor decisions about commit
access. If we find some individuals don't make good decisions without
the "not your employer" rule in place then we can either deal with those
individuals or reinstate the rule or find another, more targeted way of
addressing this.

Message has been deleted

Mike Connor

unread,
May 22, 2009, 5:43:14 AM5/22/09
to Simon Paquet, gover...@lists.mozilla.org

On 21-May-09, at 9:42 PM, Simon Paquet wrote:

> I agree that the "not your employer" rule should be abolished. But I
> also
> think that we should put additional safeguards in place to make sure
> that
> organizational group-think does really not occur.
>
> How about requiring that someone employed by an organization, who is
> requesting commit access, must have been employed by this organization
> for at least two (or three) months before he is granted commit
> rights for
> mozilla-central or comm-central. That way we would make sure that the
> person has had enough time to acquaint himself with the code he's
> working
> on and has had time to engage with the community.
>
> The scenario I'm trying to prevent with such a rule would be commit
> requests like "X has started at Y on Monday and needs commit access
> now
> that he's here" with people from company Y feeling obliged to vouch
> for X
> because they are working in the same company.

While I recognize what you're trying to accomplish here, I think the
groupthink problem isn't solved by a waiting period. Nor does this
rule necessarily make sense as-is (consider someone who's been a great
community contributor, to the extent that someone hires them to
continue their work before they get commit access).

I think that if someone gets vouched for by a superreviewer and two
module owner/peers, but haven't built up a viable track record, we
have a serious leadership issue, and who gets commit access is one of
the least of my worries. I'm inclined to go with Mitchell's proposal,
which is to not have any restrictions like this and see how it works.
If we hit a failure state, we'll have to examine how to prevent it
again, but I am against creating rules unless we believe that without
those rules we will have problems. Right now, I think we need to try
to trust our module owners/peers/superreviewers to act according to
their conscience and in the best interests of the project.

-- Mike

Mike Shaver

unread,
May 22, 2009, 8:54:10 AM5/22/09
to Mitchell Baker, gover...@lists.mozilla.org
On Thu, May 21, 2009 at 8:54 PM, Mitchell Baker <mitc...@mozilla.com> wrote:
> -- Commit access is awarded when an individual has demonstrated (a)
> understanding and respect for operational rules (testing, checking in,
> watching for problems, resolving problems with check-ins, etc) and (b)
> acceptable quality code.*
>
> *(Mike Connor has suggested that (b) -- acceptable quality code -- is less
> important than it once was.  In some sense this is true.  In the early days
> of Mozilla we had to fight long and hard to improve the quality of code in
> our tree to something we were proud of.  (Anyone who remembers Netscape 6
> will understand why improving the overall code quality was so fundamental.)
>  Commit access was one line of defense here, where we knowingly slowed down
> granting of access in order to get a handle on who was making our code
> better and how they were doing so. )

Also, it used to be that having commit access was enough to grant r+
on any patch anywhere in the tree, which made restricting commit
access a requisite for restricting who could review code.

Thanks for the great post, Mitchell; explained this much better than I
could have!

Mike

Mike Shaver

unread,
May 22, 2009, 9:03:54 AM5/22/09
to Simon Paquet, gover...@lists.mozilla.org
On Thu, May 21, 2009 at 9:42 PM, Simon Paquet <si...@gmx.de> wrote:
> I agree that the "not your employer" rule should be abolished. But I also
> think that we should put additional safeguards in place to make sure that
> organizational group-think does really not occur.
>
> How about requiring that someone employed by an organization, who is
> requesting commit access, must have been employed by this organization
> for at least two (or three) months before he is granted commit rights for
> mozilla-central or comm-central. That way we would make sure that the
> person has had enough time to acquaint himself with the code he's working
> on and has had time to engage with the community.

I think our general standards of evaluation here should suffice, and
different people are going to have very different rates of acclimation
to our processes and standards; I wouldn't want to guess at a time
that is sufficient or necessary. We have examples-of-work thresholds
that are much more reliable, IMO.

> The scenario I'm trying to prevent with such a rule would be commit
> requests like "X has started at Y on Monday and needs commit access now
> that he's here" with people from company Y feeling obliged to vouch for X
> because they are working in the same company.

What if the person has experience with the project beforehand? If we
hired an experienced Mozilla contributor who qualified for, but hadn't
yet got, commit access, I would definitely want them to get commit
access promptly: I don't want them spending 8 hours a day generating
more checkin-needed competition for other contributors, and I want
them to be able to help process that queue as well.

Speaking only for MoCo Engineering, if you believe that you've seen
(or see in the future) a case of our developers inappropriately
short-circuiting the process for new hires, please bring it
immediately to my attention. People's work here is evaluated
explicitly in the context of being good members of the broader
project, and "you won't have commit access" is part of pretty much
every new hire's orientation since it's a good example of how the
project is primary. I would take a short-circuit of that process for
the sake of convenience extremely seriously, as would I take anyone
using employment pressure (co-worker or reporting relationship) to
distort someone's commit access process.

(We also need to "defend" against such distortions from "I am
so-and-so's friend" or "I worked with so-and-so on another open source
project" or "OMG! It's Simon Paquet of Paquetsoft! How *dare* you ask
him to prove himself?!?", of course, and I think we do so quite
effectively so far. Neither of the two top Valgrind hackers have
commit access to our tree, in spite of both being employed or
contracted by MoCo now, as an example.)

Mike

Peter Weilbacher

unread,
May 22, 2009, 12:38:34 PM5/22/09
to
On 05/22/09 02:54, Mitchell Baker wrote:
> Against this we need to weigh the very real harm we're causing ourselves
> today, where the "not your employer" rule prevents respected hackers
> from getting commit access because there aren't enough module owners/
> peers not employed by Mozilla to review their code and make this
> evaluation.

Instead of completely abolishing the employer rule, perhaps you could
just change it a bit, in the sense that one voucher needs to be employed
at a different _place_ than the candidate. That way people from Mozilla
Japan or Mozilla Toronto could vouch for people at Mozilla Hq, so you
should get enough judges for every candidate.

This would make sure that the candidate is not only selected by his
immediate superiors (and people who know that he exists which I guess
is important, too) but has gained enough visibility in the project so
that others outside his office can judge his capabilities, too.

Peter.

Mike Connor

unread,
May 22, 2009, 12:54:37 PM5/22/09
to moz...@weilbacher.org, gover...@lists.mozilla.org

In many cases, immediate supervisors aren't in the same location. In
reality, this would mostly target the Mountain View office, nowhere
else is there a self-contained team that could vouch like that (given
the SR requirement especially). Again, we should only create rules
where we believe they are necessary, IMO.

-- Mike

Mitchell Baker

unread,
May 22, 2009, 2:38:31 PM5/22/09
to Mike Connor, moz...@weilbacher.org, gover...@lists.mozilla.org
First, no question that decisions about commit access should not be
based on criteria other than the readiness of the candidate.

The time and place ideas are interesting, and thoughtful. Right now I
share mconnor's view that they aren't likely to be very meaningful. We
could make someone wait 2 months for example. But that doesn't solve
the problem that even at 2 months, or X months, we need honest
evaluation. The project is unhealthy if some employer hires people
saying "you need to wait 2 months and then you get access." The places
idea is interesting, but as mconnor points out a lot of employee groups
are widely distributed.


Other ideas very welcome. It's really helpful to brainstorm ways to
protect the integrity of the project that reinforce the integrity of our
participants. (If individuals lose that integrity policy will only slow
our decline; individuals are key. But we can set up policies that
reinforce individual commitment.)

I'd like to leave this discussion open for another week or so and see if
new ideas emerge. If we find something that seems like it would
effectively strengthen the commit access evaluation process, great.
Otherwise I'd like to make remove the "not your employer" rule in early
June. There will be a formal announcement before any policy change.


mitchell

Mitchell Baker

unread,
May 22, 2009, 2:38:31 PM5/22/09
to Mike Connor, gover...@lists.mozilla.org, moz...@weilbacher.org
First, no question that decisions about commit access should not be
based on criteria other than the readiness of the candidate.

The time and place ideas are interesting, and thoughtful. Right now I
share mconnor's view that they aren't likely to be very meaningful. We
could make someone wait 2 months for example. But that doesn't solve
the problem that even at 2 months, or X months, we need honest
evaluation. The project is unhealthy if some employer hires people
saying "you need to wait 2 months and then you get access." The places
idea is interesting, but as mconnor points out a lot of employee groups
are widely distributed.


Other ideas very welcome. It's really helpful to brainstorm ways to
protect the integrity of the project that reinforce the integrity of our
participants. (If individuals lose that integrity policy will only slow
our decline; individuals are key. But we can set up policies that
reinforce individual commitment.)

I'd like to leave this discussion open for another week or so and see if
new ideas emerge. If we find something that seems like it would
effectively strengthen the commit access evaluation process, great.
Otherwise I'd like to make remove the "not your employer" rule in early
June. There will be a formal announcement before any policy change.


mitchell

Robert Kaiser

unread,
May 22, 2009, 4:12:22 PM5/22/09
to
Mitchell Baker wrote:
> --one super-reviewer from outside the modules in which you have been
> working. The goal here was to make sure we didn't have unintentional
> "group-think" or that a couple of module owners/peers being overly
> optimistic as a result of being friendly with the candidate.

Not sure if it belongs into this thread but as this is the only
"anti-group-think" clause left in the proposal, I think it needs to be
said that this is problematic by itself as people with the
super-reviewer hat on are supposed to *not* considers themselves as
belonging to any specific module but as someone overlooking the
interaction and integration of the whole or large parts of the codebase.
Somehow this rule contradicts itself and I heard super-reviewers being
unsure of who to take this rule, actually.

Robert Kaiser

Daniel E

unread,
May 22, 2009, 10:07:32 PM5/22/09
to
I understand the importance of not having rules that are not obviously
necessary. However, I would like to make the following suggestion for
a rule I believe could be necessary:

I don't see any codification of a process for evaluating the removal
of commit access. I think a rule to this effect would allow the
removal of the restriction that is currently harming the project while
still providing a control mechanism to the community such that there
is a documented way of correcting the grant of commit access to people
who are deemed by the community to be not yet ready for it.

Such a rule might state that anyone with sufficient merit could flag
(anonymously?) a user with commit access. If the user receives enough
flags (maybe one from anyone, maybe some greater number if the pool of
people who can flag is very broad), a voucher from someone not
employed by the same company must be made or the person would lose
commit access.

-Daniel

Mike Connor

unread,
May 22, 2009, 10:44:39 PM5/22/09
to Daniel E, gover...@lists.mozilla.org

I think I would want to tread very carefully in creating a rule like
this. Is this a probationary thing, or would people be able to call
anyone out? The former makes sense, and the latter makes me uneasy.
I think this would only make sense where there was abuse, though if
people were being prematurely granted access that feedback should be
directed to the policy owner for commit access to intervene and/or
tighten policy.

To me, whoever owns the commit policy would be the person responsible
for arbitrating any request to strip commit access from a user. That
individual may delegate discussion (i.e. to superreviewers@ etc) but
ultimately someone has to make a final call. I think the policy owner
should be the person to make that call.

-- Mike

Axel Hecht

unread,
May 23, 2009, 5:39:13 AM5/23/09
to

... almost agree, I'd like to have an explicit group in charge, though.

I guess in the old staff@ days, the revocation policy was "staff will
resolve it peacefully, or otherwise", i.e., trying to educate a
committer about "bad" behaviour, which most likely would be violation of
tree rules or committer form. Revocation would only happen if the
education piece would fail in flames. I'd like that process to stick,
and I guess it won't need to be more formal.

IIRC, we replace staff@ with .governance in most places, but in this
case, going with the policy module would be good. I don't think that it
should be "the owner and ...", I'd prefer to have an explicit group of
peers of the policy module to lead the discussion with the committer in
question. Just a known small group so that having them on an email
thread is fine while still being able to alleviate mismatches in
communication between just two individuals.

Axel

Message has been deleted

Robert Kaiser

unread,
May 23, 2009, 11:43:31 AM5/23/09
to
Axel Hecht wrote:
> IIRC, we replace staff@ with .governance in most places, but in this
> case, going with the policy module would be good. I don't think that it
> should be "the owner and ...", I'd prefer to have an explicit group of
> peers of the policy module to lead the discussion with the committer in
> question. Just a known small group so that having them on an email
> thread is fine while still being able to alleviate mismatches in
> communication between just two individuals.

What about the module ownership group? Not sure if it's the right fit,
it intention is to oversee the list of modules and their owners, but
this might be related to that topic, actually.

Robert Kaiser

Mike Connor

unread,
May 23, 2009, 6:09:44 AM5/23/09
to Simon Paquet, gover...@lists.mozilla.org

On 23-May-09, at 5:51 AM, Simon Paquet wrote:

> Mike Connor wrote:
>
>>> How about requiring that someone employed by an organization, who is
>>> requesting commit access, must have been employed by this
>>> organization
>>> for at least two (or three) months before he is granted commit
>>> rights
>>> for mozilla-central or comm-central. That way we would make sure
>>> that
>>> the person has had enough time to acquaint himself with the code
>>> he's
>>> working on and has had time to engage with the community.
>>>
>>> The scenario I'm trying to prevent with such a rule would be commit
>>> requests like "X has started at Y on Monday and needs commit access
>>> now that he's here" with people from company Y feeling obliged to
>>> vouch for X because they are working in the same company.
>>
>> While I recognize what you're trying to accomplish here, I think the
>> groupthink problem isn't solved by a waiting period. Nor does this
>> rule necessarily make sense as-is (consider someone who's been a
>> great
>> community contributor, to the extent that someone hires them to
>> continue their work before they get commit access).
>

> I see this risk, I'm just not sure how applicable to real-life it
> really
> is. Personally I know nobody, who was an accepted long-time community
> contributor, who only got his commit access once he was hired by MoCo.

Depends on your definition of long-time, the most recent example that
comes to mind is Dao Gottwald.

-- Mike

Mitchell Baker

unread,
May 24, 2009, 1:20:12 AM5/24/09
to


It probably does make sense to have the commit access policy explicitly
that commit access can be revoked, and a general description of how that
would work. Thanks for pointing this out.

Mitchell

Mitchell Baker

unread,
May 24, 2009, 1:22:49 AM5/24/09
to Axel Hecht
This thread reminds me that I proposed a Policies module a while back,
with sub-modules, one of which is the Commit Access Policy. From the
thread it looks like I didn't get that finalized. Will try to do so asap.

mitchell

Mitchell Baker

unread,
May 24, 2009, 1:26:11 AM5/24/09
to Robert Kaiser


Yes, the Module Ownership module is close, since the peers there are the
people who will address issues with module owners if and when necessary.
I hesitate a bit to make another module with the same people. I
suppose we could note that the module owner of the commit access policy
is likely to look to the module ownership peers if an issue comes up.

This also reminds me that some policies have a clear distinction between
the creation and the implementation of the policy. For example, Frank
is the owner of the policy for how we handle security bugs, but he's not
the implementor. We need to figure out how to note when this is the
case. I'll try to propose something when I return to finalizing the
Policies modules.

mitchell

Mike Connor

unread,
May 24, 2009, 8:09:15 PM5/24/09
to Mitchell Baker, gover...@lists.mozilla.org

On 24-May-09, at 1:26 AM, Mitchell Baker wrote:

> Robert Kaiser wrote:
>> Axel Hecht wrote:
>>> IIRC, we replace staff@ with .governance in most places, but in this
>>> case, going with the policy module would be good. I don't think
>>> that it
>>> should be "the owner and ...", I'd prefer to have an explicit
>>> group of
>>> peers of the policy module to lead the discussion with the
>>> committer in
>>> question. Just a known small group so that having them on an email
>>> thread is fine while still being able to alleviate mismatches in
>>> communication between just two individuals.
>> What about the module ownership group? Not sure if it's the right
>> fit, it intention is to oversee the list of modules and their
>> owners, but this might be related to that topic, actually.
>> Robert Kaiser
>
>
> Yes, the Module Ownership module is close, since the peers there are
> the people who will address issues with module owners if and when
> necessary. I hesitate a bit to make another module with the same
> people. I suppose we could note that the module owner of the commit
> access policy is likely to look to the module ownership peers if an
> issue comes up.

I've been going back and forth on the idea of "the buck stops here"
staff@ vs. explicit groups for each area. Aside from the legitimate
concerns around centralization of power, I think the model where we
have a clear group of leaders tasked with making the project run
smoothly is generally best/easiest/most transparent. I am a little
concerned that we'll end up with a meaningless balkanization of areas,
especially since the people who care most about oversight are likely
to be part of multiple groups.

The model I've been pondering privately for a while is a staff-like
group responsible for resolving issues under policy, and ensuring that
the project runs smoothly, combined with policies with explicit
owners. In this model, individuals (who may or may not be part of the
staff-like group) would be responsible for creation/evolution of
necessary policies within the open and transparent mozilla.governance
framework, which I believe is the right way to create policy in this
organization. This group (call it project-oversight or staff or what
have you) would exist to enforce these policies and mediate in
disputes, preferably in private discussions, to minimize divisive and
damaging conflicts wherever possible. Obviously they would delegate
responsibility for various tasks (i.e. to drivers for various product
releases, security-group for handling security issues) where viable.

I absolutely believe the idea of having a trusted group to handle
issues in private is critical. After recently watching a friend go
through a public challenge process in another major project, I
strongly believe that disputes must be handled in the most humane and
least divisive way possible. They were effectively forced to resign
and walk away rather than continue to deal with the divisive and
abusive mess that was the dispute process. Disputes and accusations
need not be dealt with in public to be resolved to everyone's
satisfaction, as I think we've shown over the years (most people
aren't even aware of many of these disputes, which is the best outcome
for the project).

Ideally, this group would be largely passive, and intervene only when
asked or when the situation is not going to resolve itself on its
own. But I think that it's preferable to trying to write policy that
enumerates every possible situation.

-- Mike

Axel Hecht

unread,
May 25, 2009, 6:18:00 AM5/25/09
to

/me likes.

Axel

Daniel E

unread,
May 26, 2009, 10:20:26 AM5/26/09
to
I don't want to drag out my suggestion unnecessarily. I am, at best,
only peripherally involved with any of the actual code maintenance of
public Mozilla projects. The suggestion regarding a dispute
resolution group sounds like a potentially beneficial one, but I just
wanted to highlight one important way it is different from my original
suggestion.

The driving force behind my suggestion to codify a way for the
community to be able to flag a committer for needing an external
voucher was to preserve the spirit of the system of checks and balance
the current policy has on invisible corporate influence while still
relieving the pressure that is currently causing difficulty.

Right now, the development community can be assured that no one can
receive commit access to mozilla-central based solely on what company
they work for. Even if they work for MoCo, they are still going to
have to get that external review. Mitchell and others have already
discussed several ways in which that check might not be so important
anymore, and I am inclined to agree, but at the same time, that
doesn't change the fact that the check some comfort to developers that
their work can't be easily polluted by invisible corporate motives.

If the restriction is taken out of the policy and nothing is put in
its place, then members no longer have that insurance, and they must
just trust that the spirit of the community will keep the tree pure
and that the spirit of the community *will not gradually change over
time*.

The suggestion of formalizing the revocation policy and naming a body
to be in charge of enforcing that policy is actually a critical piece
of the puzzle. If the community could object to someone's commit
access but there was no party able to actually do something about the
problem then there is little benefit. There is a separate issue of
whether that body is constructed such that it can be provably immune
to invisible corporate influence.

My suggestion revolves around relieving the pressure of needing an
external voucher for every commit access grant while still preserving
the check and balance on invisible corporate influence. If the
community can allow most cases of commit access to be fast tracked but
they know that the policy states that if they feel it is needed, they
can still formally request an external voucher, then they should have
that same comfort from the checks and balance.
I specifically structured my suggestion such that it is a passive
rule. It won't interfere with daily work in the normal case.

My biggest concern is that there is significant difference between
requesting that someone get an external voucher, and requesting that
some policy body revoke commit access from a person.
If the policy body required membership of some number of non-MoCo
employees, and one of those members had the qualifications to be able
to vouch for a commit access request in the same way as the current
policy, and the body explicitly stated that people could request
external vouching in addition to requesting revocation of access, then
I guess it would work as a replacement for my suggestion, but looking
back at all those qualifications, it seems a bit more burdensome, and
the whole purpose of this discussion is to relieve burden, not shift
it elsewhere.

Gervase Markham

unread,
May 26, 2009, 4:25:05 PM5/26/09
to
On 25/05/09 01:09, Mike Connor wrote:
> The model I've been pondering privately for a while is a staff-like
> group responsible for resolving issues under policy, and ensuring that
> the project runs smoothly, combined with policies with explicit owners.
> In this model, individuals (who may or may not be part of the staff-like
> group) would be responsible for creation/evolution of necessary policies
> within the open and transparent mozilla.governance framework, which I
> believe is the right way to create policy in this organization. This
> group (call it project-oversight or staff or what have you) would exist
> to enforce these policies and mediate in disputes, preferably in private
> discussions, to minimize divisive and damaging conflicts wherever
> possible. Obviously they would delegate responsibility for various tasks
> (i.e. to drivers for various product releases, security-group for
> handling security issues) where viable.

That model does indeed sound very familiar :-) I think you are right
that a trusted group who is able to handle contentious and/or personnel
issues with discretion is important.

> Disputes and accusations need not be dealt with in
> public to be resolved to everyone's satisfaction, as I think we've shown
> over the years (most people aren't even aware of many of these disputes,
> which is the best outcome for the project).

Quite so. A dispute process which runs in public is necessary only when
there is no trust in those in authority. In other cases, it hinders, for
the reasons you give.

Gerv

Gervase Markham

unread,
May 26, 2009, 4:37:34 PM5/26/09
to Daniel E
On 26/05/09 15:20, Daniel E wrote:
> Right now, the development community can be assured that no one can
> receive commit access to mozilla-central based solely on what company
> they work for. Even if they work for MoCo, they are still going to
> have to get that external review. Mitchell and others have already
> discussed several ways in which that check might not be so important
> anymore, and I am inclined to agree, but at the same time, that
> doesn't change the fact that the check some comfort to developers that
> their work can't be easily polluted by invisible corporate motives.

That's quite a few scary words for one sentence. Given the current
nature of MoCo/MoFo (taken together) as a public benefit organization
rather than a company with shareholders to please, I think this is
far-fetched - certainly much, much more so than when Netscape was the
entity concerned.

But let's grant the possibility for a moment. So I ask: if someone
believes that this is a risk, is this check really such a comfort anyway?

Remember, the check is about getting commit access, not about a
particular bit of code. If MoCo has "invisible corporate motives", and
has employed someone to write code to implement their "polluting" plans,
then if this rule "protected" the tree by not giving that person commit
access, then evil-MoCo would just task another developer they employ to
check the evil code in on behalf of evil-developer. It's not really any
roadblock even in this highly unlikely case.

And if there is such little trust in the good motives and intentions of
MoCo, then the project is in serious trouble anyway.

> The suggestion of formalizing the revocation policy and naming a body
> to be in charge of enforcing that policy is actually a critical piece
> of the puzzle. If the community could object to someone's commit
> access but there was no party able to actually do something about the
> problem then there is little benefit.

Having a formal policy is not required for there to be someone able to
actually do something about the problem. I'm sure that, were it
necessary, brendan (as overall technical dude) would be able to
authorize removal of commit privileges. Or do you mean that the policy
would need to be run by a non-MoCo/MoFo person?

Gerv

Gervase Markham

unread,
May 26, 2009, 4:37:43 PM5/26/09
to Simon Paquet
On 22/05/09 02:42, Simon Paquet wrote:
> I agree that the "not your employer" rule should be abolished. But I also
> think that we should put additional safeguards in place to make sure that
> organizational group-think does really not occur.

In the past, I seem to remember Mitchell explaining our attitude to
formal policy as something like "policy only when proved necessary". And
I think that's a good rule.

I would suggest an "optimistic" strategy for preventing organizational
group-think. We assume that everything goes right almost all the time,
but we publicise the name of the person you are supposed to email if you
are concerned about another person's track record of poor quality code
and/or another person's regular review of that code. (I suspect the
contact would be someone like shaver or brendan, but let's leave the
specifics.)

Hopefully, that should give those still concerned about groupthink and
over-optimistic commit-access grants a clear way in which to raise their
specific concerns as and when they see problems occurring, but without
burdening us with extra process in the normal case.

I'm sure that shaver/brendan etc. are willing ("happy" seems the wrong
word) to listen to such concerns, politely expressed and backed up with
evidence, anyway. But it does no harm to call out specifically that they
have that role.

Gerv

Mike Connor

unread,
May 26, 2009, 6:24:24 PM5/26/09
to Daniel E, gover...@lists.mozilla.org
Gerv covered off most of this already, but I wanted to specifically
respond to one part:

On 26-May-09, at 7:20 AM, Daniel E wrote:

> My biggest concern is that there is significant difference between
> requesting that someone get an external voucher, and requesting that
> some policy body revoke commit access from a person.
> If the policy body required membership of some number of non-MoCo
> employees, and one of those members had the qualifications to be able
> to vouch for a commit access request in the same way as the current
> policy, and the body explicitly stated that people could request
> external vouching in addition to requesting revocation of access, then
> I guess it would work as a replacement for my suggestion, but looking
> back at all those qualifications, it seems a bit more burdensome, and
> the whole purpose of this discussion is to relieve burden, not shift
> it elsewhere.

First, I think that any sort of explicit requirement on membership
makeup is tricky. I don't think that's really necessary, if we have a
group of trusted and respected people, though the current module
ownership group is well-distributed across multiple projects/companies.

But let's back up for a moment here. Simply having a single person
from another company vouch does not really act as an effective check
and balance against "invisible corporate interests" leading to bad
things happening. If there is real cause for concern on that front,
we should be escalating and investigating the module owners and
superreviewers who are being indirectly accused of being tainted/
acting against the best interests of the project. Ultimately, my
opposition to the employer rule is that it is a far greater problem to
have "tainted" people in positions of authority and responsibility in
the project, compared to someone getting commit access too early.

-- Mike


Daniel E

unread,
May 27, 2009, 9:47:26 AM5/27/09
to

I actually used the wrong phrase in that paragraph. Elsewhere I used
the phrase "invisible corporate influence" but I accidentally said
"motives" here. I have seen only a small number of people who suspect
the motives of Mozilla Corporation at the moment, and most of those
people seem to be more interested in just complaining rather than
being involved with guiding the project.

The rule we are removing seems to me to be protecting against the
group-think that can unintentionally occur with a group of people
employed by the same company. I was attempting to find a way to still
offer the community the same protection via the same mechanism
(voucher by an outside company) without the bottle-neck problem. The
most clear phrase is probably "unintentional corporate group-think"
and I will use that wording now.

> > The suggestion of formalizing the revocation policy and naming a body
> > to be in charge of enforcing that policy is actually a critical piece
> > of the puzzle.  If the community could object to someone's commit
> > access but there was no party able to actually do something about the
> > problem then there is little benefit.
>
> Having a formal policy is not required for there to be someone able to
> actually do something about the problem. I'm sure that, were it
> necessary, brendan (as overall technical dude) would be able to
> authorize removal of commit privileges. Or do you mean that the policy
> would need to be run by a non-MoCo/MoFo person?

If the person designated to handle resolution of a dispute is employed
by the same company as the person to be evaluated then it is no longer
the same protection as what the policy provided before. That isn't to
say that it wouldn't "work" though.
My comment here was basically pointing to the fact that if someone
disputed a commit access grant and an external reviewer could not
provide a voucher, obviously, someone must still take action to
resolve the dispute. Suggestions subsequent to mine focused solely on
having a party responsible for resolving the dispute but omitted
external voucher part.

Looking at the feedback my suggestion has received, it sounds like
there are two themes of sentiment:
1. "MoCo does not currently have a problem with corporate group think
and is not likely to be susceptible to it in the future. We no longer
need to have any check in the policy to avoid it."
I actually agree with the first part of this. I just think
that once the policy does not explicitly prevent against this problem,
if it ever does crop up, the community must rely on the action of
people within the corporation that are not affected by the group-think
to correct the policy.

2. "A party responsible for resolving disputes is all we need to
protect the tree from unintentional corporate group-think. This party
can be someone employed by MoCo."
Looking at that, I guess it is just a rewording of the first
theme. I feel it is fine for as long as the responsible party is not
affected by corporate group-think.

I won't make any more effort endorsing this suggestion. I laid out my
reasoning as best as I am able and I heard the arguments against it.
These arguments are coming from people much more involved and suited
to the governance of the Mozilla policy than I am. I do have faith
that MoFo and MoCo are capable of selfless acts and will remain so for
a very long time with the current leadership. That will be good
enough for me.

-Daniel

Mitchell Baker

unread,
May 27, 2009, 10:31:22 AM5/27/09
to
Daniel

thanks for the thoughtful notes. I agree that avoiding "group-think" is
important. And broader than just a single employer, there are lots of
ways group-think develops -- even friendship! We get some different
perspectives from the competition in much of the technical areas in
which we work, but self- driven review is important as well.


So it remains on my mind in general, even though it doesn't seem
appropriate now to the apply this concern through the "not your
employer" rule for commit access.


ml

Mitchell Baker

unread,
May 27, 2009, 10:31:55 AM5/27/09
to

*** I think we're set to drop the "not your employer rule. There are
two open items before we do so.+++ Speak up if you have further input on
these. In the meantime, I'll try to redraft the policy. Am at a
conference with a lot of events, so it may take a few days.

1. How to describe the super-reviewer requirement.

mconnor suggests we say it's someone who hasn't ever reviewed a patch
from the candidate. This helps with not knowing the person. Without
the "group-think" motivation it seems a bit odd on its face. But as
long as the policy says *why* we have this requirement so we don't
forget this I think it's fine.

2. Including an explicit way for the review and possible revocation of
commit access.

There's the beginning of a good discussion on a governance and policy
group in general, something a bit broader than specfic modules. I'd
like to continue that discussion but not hold up the commit access piece
in the meantime.

My proposal is: (a) the Commit Access Policy is clear re review and
revocation; (b) we make the Policy Module I proposed long ago, but
didn't execute (see thread called Proposed "Policy" Activities Module
started by me on Jan 7 2009); and (c) we continue with the discussion of
a review group broader than a single policy.

That leaves me as the owner of the Policy Access module. That's not
ideal in some ways as I'm not in the code -- and of course have long
relationships with any of the mozilla employess -- but part of these
discussions is to develop broader groups of people interested /
knowledgable and involved in this aspect of the project. If we should
end up in some brutally contentious place I expect Brendan and I to make
a final definitive call, but we all want these settings to be as few as
possible and the number of people who can resolve problems to grow.


mitchell

Mike Connor

unread,
May 27, 2009, 1:03:13 PM5/27/09
to Mitchell Baker, gover...@lists.mozilla.org

On 27-May-09, at 7:31 AM, Mitchell Baker wrote:

>
>
> *** I think we're set to drop the "not your employer rule. There
> are two open items before we do so.+++ Speak up if you have further
> input on these. In the meantime, I'll try to redraft the policy.
> Am at a conference with a lot of events, so it may take a few days.
>
> 1. How to describe the super-reviewer requirement.
>
> mconnor suggests we say it's someone who hasn't ever reviewed a
> patch from the candidate. This helps with not knowing the person.
> Without the "group-think" motivation it seems a bit odd on its
> face. But as long as the policy says *why* we have this requirement
> so we don't forget this I think it's fine.

It's entirely meant as a check against group optimism, though,
separately from the employment piece, as the current SR requirement
was meant to do (if I'm remembering your rationale correctly from the
last go-round). The old wording was ambiguous and hard to objectively
evaluate, so I think this is still useful for avoiding group-think-
like outcomes.

-- Mike

Gervase Markham

unread,
May 27, 2009, 5:36:03 PM5/27/09
to
On 27/05/09 14:47, Daniel E wrote:
> I actually agree with the first part of this. I just think
> that once the policy does not explicitly prevent against this problem,
> if it ever does crop up, the community must rely on the action of
> people within the corporation that are not affected by the group-think
> to correct the policy.

If it's any comfort, there are plenty of us who are keen to avoid MoCo
ever turning into anything like Netscape. For a few of us, occasionally
perhaps too keen ;-)

Patient MoCo Person: "<sigh> No, Gerv, this is not like what Netscape
did because of the following good reasons: X, Y, and Z."

Gerv: "Ah yes. OK then :-)"

Gerv

Mitchell Baker

unread,
Jun 4, 2009, 2:40:13 PM6/4/09
to Mike Connor, gover...@lists.mozilla.org
Mike Connor wrote:
[snip]


The model I've been pondering privately for a while is a staff-like group responsible for resolving issues under policy, and ensuring that the project runs smoothly, combined with policies with explicit owners.  In this model, individuals (who may or may not be part of the staff-like group) would be responsible for creation/evolution of necessary policies within the open and transparent mozilla.governance framework, which I believe is the right way to create policy in this organization.  This group (call it project-oversight or staff or what have you) would exist to enforce these policies and mediate in disputes, preferably in private discussions, to minimize divisive and damaging conflicts wherever possible.  Obviously they would delegate responsibility for various tasks (i.e. to drivers for various product releases, security-group for handling security issues) where viable.

[snip]


Ideally, this group would be largely passive, and intervene only when asked or when the situation is not going to resolve itself on its own.  But I think that it's preferable to trying to write policy that enumerates every possible situation.

-- Mike
I'm still wary of creating another group that someone interacts with current groups.  I know you are very eager to have a "mozilla.org staff" - like group and I'm less eager to try and overlay another layer onto our existing complexity.

BUT we may actually be quite close to agreement!  I went and looked at the governance module and it says

responsibilities: Policies and process for how we distribute authority and govern ourselves; including:

    * Development and Implementation of new policies as appropriate for delegation of authority and responsibility
    * Management of the source tree
    * Balancing different constituencies of the Mozilla project
    * Maintaining the Mozilla identity as we take on new activities

So now I believe the each Policy should be a sub-module of Governance and the Governance module participants end up doing pretty mresponsibilities: Policies and process for how we distribute authority and govern ourselves; including:

    * Development and Implementation of new policies as appropriate for delegation of authority and responsibility
    * Management of the source tree
    * Balancing different constituencies of the Mozilla project
    * Maintaining the Mozilla identity as we take on new activities uch what you suggest.  Two questions remain:  does governance cover everythign about the project -- there may be things about keeping the project running smoothly that aren't policy or governance exactly.  And second, does "peers" convey the right thing, should the governance module have "members" ?  NSPR does, but that has a different meaning.

mitchell

Mike Connor

unread,
Jun 4, 2009, 5:29:35 PM6/4/09
to Mitchell Baker, gover...@lists.mozilla.org

On 4-Jun-09, at 11:40 AM, Mitchell Baker wrote:

> I'm still wary of creating another group that someone interacts with
> current groups. I know you are very eager to have a "mozilla.org
> staff" - like group and I'm less eager to try and overlay another
> layer onto our existing complexity.

I don't necessarily want staff-as-it-existed, but having a group that
can intervene where policy is unclear or situations need to be
resolved is the goal, so I think what you're proposing here is much
better articulated. :)

> So now I believe the each Policy should be a sub-module of
> Governance and the Governance module participants end up doing
> pretty mresponsibilities: Policies and process for how we distribute
> authority and govern ourselves; including:

This makes a lot of sense, and seems to be a better structure than
what we have now.

> Two questions remain: does governance cover everythign about the
> project -- there may be things about keeping the project running
> smoothly that aren't policy or governance exactly.

I think that's simplest. Good governance to me means making sure
things function well and operate in ways that are viable and
successful in the larger sense, not really doing all of it.

> And second, does "peers" convey the right thing, should the
> governance module have "members" ? NSPR does, but that has a
> different meaning.

I believe this is like the module ownership group discussion, and
peers is also the right context here. Peers to me are empowered to
engage with a situation or take action as needed based on their
discretion, subject to final say by the Governance owner. Members
implies part of a group, but not individually empowered in the broader
sense. The Firefox module actually has most responsibility pushed out
to a number of sub-modules, and the peers can take corrective/
oversight action as needed in my stead. I think that's right here,
though obviously contentious/non-obvious issues would be referred to
the larger group or the owner.

-- Mike

Dan Mosedale

unread,
Jun 4, 2009, 5:45:47 PM6/4/09
to gover...@lists.mozilla.org
On 6/4/09 2:29 PM, Mike Connor wrote:
>> So now I believe the each Policy should be a sub-module of Governance
>> and the Governance module participants end up doing pretty
>> mresponsibilities: Policies and process for how we distribute
>> authority and govern ourselves; including:
>
> This makes a lot of sense, and seems to be a better structure than
> what we have now.
I like this too.

>> Two questions remain: does governance cover everythign about the
>> project -- there may be things about keeping the project running
>> smoothly that aren't policy or governance exactly.
>
> I think that's simplest. Good governance to me means making sure
> things function well and operate in ways that are viable and
> successful in the larger sense, not really doing all of it.
This makes sense to me as well. It makes it easy to think about the
other modules as being sub-modules delegated from the governance module.

> I believe this is like the module ownership group discussion, and
> peers is also the right context here. Peers to me are empowered to
> engage with a situation or take action as needed based on their
> discretion, subject to final say by the Governance owner. Members
> implies part of a group, but not individually empowered in the broader
> sense. The Firefox module actually has most responsibility pushed out
> to a number of sub-modules, and the peers can take
> corrective/oversight action as needed in my stead. I think that's
> right here, though obviously contentious/non-obvious issues would be
> referred to the larger group or the owner.
Seconded.

Dan

alfonso...@gmail.com

unread,
Apr 20, 2014, 2:22:04 PM4/20/14
to mozilla-g...@lists.mozilla.org
On Thursday, May 21, 2009 4:54:28 PM UTC-8, Mitchell Baker wrote:
> I'll get to the punchline first, and then go through the rationale. I
> believe the "not your employer" rule is harming us more than it's
> helping us now. I also think the principles governing commit access
> can be protected without this particular rule. I've put the rationale
> below.
>
> As a result I'm proposing we retract the "one person must not be
> employed by the same person as the candidate" rule for all candidates
> for commit access.
>
> That leaves us with the following requirements:
>
> -- two vouchers (module owners or peers) working in modules in which
> the candidate is working; and
> -- one super-reviewer from a different module
>
> Looking for feedback.
>
> +++
>
>
> Detailed Rationale for Removing the "Not Your Employer" requirement
> (long-ish)
>
> Here are the principles I think we are trying to implement:
>
> Principles:
>
> -- Commit access is earned by each individual based on his or her own work.
> -- Employment status -- with any organization -- is irrelevant to commit
> access.
> -- Commit access is awarded by a decision of respected peers, based on
> experience with an individual.
> -- Commit access is awarded when an individual has demonstrated (a)
> understanding and respect for operational rules (testing, checking in,
> watching for problems, resolving problems with check-ins, etc) and (b)
> acceptable quality code.*
>
>
> *(Mike Connor has suggested that (b) -- acceptable quality code -- is
> less important than it once was. In some sense this is true. In the
> early days of Mozilla we had to fight long and hard to improve the
> quality of code in our tree to something we were proud of. (Anyone who
> remembers Netscape 6 will understand why improving the overall code
> quality was so fundamental.) Commit access was one line of defense
> here, where we knowingly slowed down granting of access in order to get
> a handle on who was making our code better and how they were doing so. )
>
>
> Current Policy
>
> Many years ago we codified these principles into a process for obtaining
> CVS access. We identified the "respected peers" that must support a
> request for commit access as:
>
> -- two module owners or peers from modules in which the candidate has
> been submitting patches (the two "vouchers"). These are the people who
> presumably know the candidate best. They have been reviewing the
> candidates patches. They may well be checking in the candidates patches
> and have experience with how the candidate participates in the checkin
> and follow up activities.
>
> --one super-reviewer from outside the modules in which you have been
> working. The goal here was to make sure we didn't have unintentional
> "group-think" or that a couple of module owners/peers being overly
> optimistic as a result of being friendly with the candidate.
>
> --We also said that one of the three must NOT have the same employer as
> the candidate. This aspect had a few goals. One was to explicitly
> break the link in the minds of many organizations that employment =
> source access. This may seem obvious now, but in the early days of
> Mozilla it was a very, very contentious issue. Many organizations
> simply could not understand how they could hire someone to work on
> Mozilla and then not be able to provide source code commit access. A
> second goal was to protect people with commit access from being
> pressured by their employers. (Did I mention how contentious this issue
> was?) A third reason was because the code review process was also
> contentious and under great strain. Sometime poor code would be
> approved by reviewers because they had already spent a lot of effort
> reviewing, the code was still poor quality, the author of the code was
> employed by someone to work on mozilla and so would continue to produce
> poor quality code forever, and the reviewer would finally decide s/he
> couldn't improve the code through the review process. It's harder to
> understand this today but it was a live issue in 1999. A final aspect
> was a concern that similarities of an employment setting would cause
> "group-think" to outweigh independent technical judgment.
>
> Today
>
> I think most of the reasons for the "not your employer" rule are far
> less valid today than when we adopted the current policy. The reality
> that our invaluable source code asset is best protected by independent
> technical judgment is ingrained in a way that simply was not true a
> decade ago.
>
> One remaining potential value that I see is possibly protecting against
> group-think. One never know what a different perspective will bring to
> the discussion. Employees may share some things in common simply by
> being employees. So the "not your employer" rule theoretically brings
> some different perspective to the evaluation of whether a person is
> ready for commit access.
>
> Against this we need to weigh the very real harm we're causing ourselves
> today, where the "not your employer" rule prevents respected hackers
> from getting commit access because there aren't enough module owners/
> peers not employed by Mozilla to review their code and make this
> evaluation.
>
> This harms the project in a number of ways.
>
> -- Quantities of high quality code from known and respected peers must
> be processed (checked in) by others.
> -- The burden on module owners / peers who are not employed by Mozilla
> to do the level of review required for commit access is high
> -- The backlog of code that someone has to check-in means that new
> contributors are in a much longer line to have their code checked in,
> and face a lot of competition
>
>
> We could approach this problem by consciously NOT hiring module owners.
> I think this is a bad approach. It penalizes people for earning and
> accepting the responsibilities of module ownership. Becoming a module
> owner then makes someone ineligible to work for Mozilla. This seems
> fundamentally backwards to me. And it creates pressure to make module
> owners due to their (non) employment status, whether or not the module
> makes sense on its own or we have an owner with the technical background
> we want. To me, the bad incentives here are worse that relying on the
> vouchers and super-reviewers to make independent judgments. We do need
> to explore the development of new areas, new modules and new module
> owners. But we should explore this for its own sake, not because we
> have a rule about commit access.
>
> For the determination of readiness for commit access we can rely on the
> core understanding that source code access must *always* be earned and
> that one's own reputation will suffer for poor decisions about commit
> access. If we find some individuals don't make good decisions without
> the "not your employer" rule in place then we can either deal with those
> individuals or reinstate the rule or find another, more targeted way of
> addressing this.

Reply all
Reply to author
Forward
0 new messages