Removing the Everyone team in the jenkinsci GitHub org

[Daniel Beck]

Hi everyone,

For a long time, we didn't really manage GitHub permissions for plugins: We just added new repos and contributors to the 'Everyone' team, giving everyone access to everything, and that was it. Contrary to its name, it doesn't actually include _everyone_, at least not anymore: We've moved away from adding people to the Everyone team over a year ago. But it still has hundreds of members, and provides write access to more than 1200 repositories.

While contributors are mostly well-behaved, we've occasionally seen PRs merged by people who weren't regular contributors to those plugins, resulting in conflict with actual maintainers. Since these users cannot release the plugins anyway, the benefit of being able to do this is questionable at best. Not to mention the problem of malicious or accidental data loss through people force-pushing into Git (this, accidentally, has happened with dozens -- hundreds? -- of repositories back in 2015, and caused a considerable mess).

I know that quite a few plugin maintainers have long removed the Everyone team from their plugin repositories to prevent such problems, but the default is still to add the team to a new plugin repo.

Given the size of the Jenkins project, this is untenable. I plan to remove the Everyone team from all repositories, instead granting direct access to contributors who previously got their write access from the Everyone team. This way, we should be able to retain access for those who legitimately have it, while removing those who have no relationship to any particular plugin.

Right now, I plan to grant direct write access to a plugin to users who:

1. Have write access to a repository through the Everyone team only
2. Are considered 'contributors' by GitHub (meaning they have commits associated with them in the repo)
3. Have at least 5 'contributions' (~commits in the contributors graph)

The first two should be obvious (with the caveat that badly set up GH accounts -- their commits not associated with the account -- will lose access). The third is where it gets tricky: Not everyone who contributed to a repository, and currently has write access to it via 'Everyone' team, should legitimately have write access to it in the future. Unfortunately, GitHub makes it impossible to efficiently determine who has performed various maintainer-specific actions, such as created a release tag, or merged a pull request -- and these might still miss legitimate co-maintainers who just don't have direct access.

So I will instead use a minimum number (currently 5) of contributions required for someone to retain access. Once we've drastically limited who has access to any given repo, we can always fine-tune this later. There will also be a log of what changed so this can be referenced later in case of questions related to lost permissions.

While this may be a painful change if the above rules get the needed permissions wrong, it's an important one, and I'm trying to make it as smooth as possible. Please bring up your concerns and questions in this thread and I'll do my best to address them.

I don't think this is JEPable, as this doesn't actually create a longer-running process, it's just a one-time legacy permission migration.

Daniel

[Richard Bywater]

That sounds like a reasonable approach to me.

Semi related question re "I know that quite a few plugin maintainers have long removed the Everyone team from their plugin repositories to prevent such problems, but the default is still to add the team to a new plugin repo."

Should a plugin maintainer have full admin access to the repo? Reason I ask is that I recently took over as maintainer for the HTML publisher plugin but don't have any access outside of committing etc. As the maintainer should I have slightly more access?

Thanks

Richard.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ABF8E55F-0896-4DD0-AB07-6778F39F5A10%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.

[Daniel Beck]

> On 28. Nov 2017, at 01:39, Richard Bywater <ric...@byh2o.com> wrote:
>
> Should a plugin maintainer have full admin access to the repo? Reason I ask is that I recently took over as maintainer for the HTML publisher plugin but don't have any access outside of committing etc. As the maintainer should I have slightly more access?

Sooo… fun fact: You only have write access to that repo because you're in Everyone. Good news: With 18 contributions, you'd get direct access in this process ;-)

Added you to the dedicated per-repo team, and granted Admin access. For a while I used to reduce the access level of the 'foo-plugin Developers' team manually from Admin to Write, the reason being the "Danger Zone" options usually available to repo admins (transfer/delete/… repo). Those can now be disabled for repo admins org-wide, so we routinely grant Admin access to maintainers again.

[Richard Bywater]

Great - thanks for your help Daniel!

Richard.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/11E33829-614A-4979-A037-AE5CDD899367%40beckweb.net.

[Mark Waite]

That sounds reasonable to me.

I won't remove "Everyone" from the git-client-plugin or the git-plugin until after you've made the change, since removing that group would remove write permission for key people (Stephen, Jesse, Sam, Liam, etc.).

I don't seem to be able to add people to either the git-client-plugin-developers group or to the git-plugin-developers group.  Is that expected?

Mark Waite

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAMui946Wsi6_f6H64Y9YFKO3t_H%3D50ZL-LnFzVyyuzjx8zwPNQ%40mail.gmail.com.

[Kohsuke Kawaguchi]

Thanks for driving this. 

If this change inadvertently remove somebody's commit access, presumably we expect people to follow this and mail this list?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ABF8E55F-0896-4DD0-AB07-6778F39F5A10%40beckweb.net.

For more options, visit https://groups.google.com/d/optout.

--

Kohsuke Kawaguchi

[Robert Sandell]

> 3. Have at least 5 'contributions' (~commits in the contributors graph)

A mid sized pr normally sits around 5 commits, so does one merged pr count as five contributions?

/B

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ABF8E55F-0896-4DD0-AB07-6778F39F5A10%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.

--

Kohsuke Kawaguchi

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4xLdArgFQDYw0weXhZpmzgdzGs0sDvKG%2Ba9xy%3DeTD7-9g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Robert Sandell

Software Engineer

CloudBees Inc.

[Daniel Beck]

> On 28. Nov 2017, at 03:39, Mark Waite <mark.ea...@gmail.com> wrote:
>
> I won't remove "Everyone" from the git-client-plugin or the git-plugin until after you've made the change, since removing that group would remove write permission for key people (Stephen, Jesse, Sam, Liam, etc.).

Right, 14 people for git-plugin and three (ndeloof, stephenc, jglick) in git-client-plugin would get added to the repo.

> I don't seem to be able to add people to either the git-client-plugin-developers group or to the git-plugin-developers group. Is that expected?

Usually controlled via IRC bot, but I have no problem handing out team maintainerships on request. Others just add people as 'external contributors' directly which requires just admin access to the repo, bypassing the per-repo team system.

[Daniel Beck]

> On 28. Nov 2017, at 04:58, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:
>
> If this change inadvertently remove somebody's commit access, presumably we expect people to follow this and mail this list?

Or just email this list in panic/rage without following a guide ;-)

That's why I plan to keep the logs of this around, to check whether this change (and why) resulted in access removed.

[Daniel Beck]

> On 28. Nov 2017, at 11:26, Robert Sandell <rsan...@cloudbees.com> wrote:
>
> A mid sized pr normally sits around 5 commits, so does one merged pr count as five contributions?
>

I think so (perhaps unless squashed?). Note that this does not grant additional commit access to people who don't already have it from Everyone. It's the threshold to effectively not have it revoked. I'm flexible on the threshold here, or what other (easy to determine, GH API is a mess) criteria to apply, suggestions welcome.

Once this is implemented, it'll be straightforward to provide reports who has access to which repo (beyond just clearly showing it in each repo's collaborators form), and plugin maintainers can clean this up further.

[Baptiste Mathus]

+1. A sensible move for sure. With hundreds of repos and hundreds of committers, /some/ segregation can definitely help avoid screwing up, even from non malicious users.

Thanks Daniel for that cleanup

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/4FCB0DEA-C0D1-4B3A-ADD6-F09531615009%40beckweb.net.

[Jesse Glick]

Generally +1 for sure. Minor comment:

On Mon, Nov 27, 2017 at 7:25 PM, Daniel Beck <m...@beckweb.net> wrote:
> Have at least 5 'contributions' (~commits in the contributors graph)
>

> […] GitHub makes it impossible to efficiently determine who has performed various maintainer-specific actions, such as created a release tag, or merged a pull request

I am not sure what you consider “efficient” in this context, but it
certainly seems easy enough to see who has merged pull requests in the
past, unless they use the "rebase” option which I think is pretty
unusual. Both true merges and squashes via the UI leave a standard
pattern in the commit message, and most command-line PR merges (again
true merges or squashes) would also refer to the PR number in the
commit message. Should be possible to put together some kind of simple
script to grep for such commits, then group and count by committer.

Creation of tags per se is indeed not recorded outside the reflog,
which we lack access to, but in practice most tags in most repository
are created by `maven-release-plugin`, which again leaves a standard
commit message format that we could easily search for.

To be clear, it does not seem unreasonable to (continue to) grant
write permission to people who have already had a number of
contributions accepted; I was just unclear on why you think we cannot
detect maintainer-like actions (merging PRs, cutting releases) with
decent confidence.

[R. Tyler Croy]

+1, go-go-gadget githubfixings

> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ABF8E55F-0896-4DD0-AB07-6778F39F5A10%40beckweb.net.

> For more options, visit https://groups.google.com/d/optout.

- R. Tyler Croy

------------------------------------------------------
Code: <https://github.com/rtyler>
Chatter: <https://twitter.com/agentdero>
xmpp: rty...@jabber.org

% gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

[Daniel Beck]

> On 28. Nov 2017, at 16:46, Jesse Glick <jgl...@cloudbees.com> wrote:
>
> I was just unclear on why you think we cannot
> detect maintainer-like actions (merging PRs, cutting releases) with
> decent confidence.

Note that the GH API doesn't work just based on Git data (in fact, those are somewhat separate entirely) -- we can know the users performing even a rebase PR merge.

But I've chewed through 4+ hours worth of API rate limits just with the basic approach I've used yesterday to list existing permissions. Enumerating all merged PRs for all repos, then looking at who merged them, plus checking all release commits for the associated GitHub user would take quite a long time.

And in the end, there's plenty of Gradle-based repos, maintainers doing their own thing without PR contributions, etc. where these rules fall flat, so requiring these to retain access would just serve to remove legitimate access in many cases. Both are just approximations of where we want to be, and it's not clear that spending that additional effort will be worth it.

Starting by getting rid of Everyone is a much quicker win, and further improvements can be applied once we've significantly reduced the number of users we even need to look at for every repo. I prefer approaching this step by step to not overwhelm both contributors and maintainers, and my ability to sort out any potential messes I cause.

[Jesse Glick]

On Tue, Nov 28, 2017 at 11:30 AM, Daniel Beck <m...@beckweb.net> wrote:
> I've chewed through 4+ hours worth of API rate limits just with the basic approach I've used yesterday to list existing permissions. Enumerating all merged PRs for all repos, then looking at who merged them, plus checking all release commits for the associated GitHub user would take quite a long time.

./jenkinsci/backend-merge-all-repo/clone.groovy

and then do it from the command line.

[Ullrich Hafner]

Seems that I still can push changes to my repositories but I cannot access the settings anymore (e.g., edit the description, view the collaborators). Is this doable per IRC bot? 

How should the new structure in collaborators look like? I’m not sure if I just missed that part in your mail…

Should there be only a * developers team? Or should there be a developers and admin team? Or individual users?

BTW: you also need to update https://jenkins.io/projects/infrastructure/ircbot/ 

[Robert Sandell]

I don't think I've ever been able to access the settings and edit the description of the repos I maintain :'(

/B

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ABF8E55F-0896-4DD0-AB07-6778F39F5A10%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/C5B832CF-81CD-4E41-A332-02835A4904A9%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Daniel Beck]

> On 29. Nov 2017, at 09:56, Ullrich Hafner <ullrich...@gmail.com> wrote:
>
> Seems that I still can push changes to my repositories but I cannot access the settings anymore (e.g., edit the description, view the collaborators).

AFAICT you never were. I've looked into this about a year ago already, and you stood out as _the_ maintainer getting permissions from Everyone team -- and that has long (always?) only granted Write access.

> Is this doable per IRC bot?

Yes, as that bot adds to per-repo team, which will still be around.

> How should the new structure in collaborators look like? I’m not sure if I just missed that part in your mail…
> Should there be only a * developers team? Or should there be a developers and admin team? Or individual users?

Details TBD. I currently plan to make people collaborators with write access. This will need to be cleaned up some time in the future, but speculatively granting additional permissions (most per-repo teams grant admin access) doesn't sit right with me.

> BTW: you also need to update https://jenkins.io/projects/infrastructure/ircbot/

Yep, thanks. The comment 'make NAME a committer' will need to be removed, only 'make NAME a committer of REPO' will remain. We've only used the latter for a while now anyway.

[Kanstantsin Shautsou]

Last time Kohsuke blocked this question with his veto because jenkinsci should be a bazaar model (and everybody should be able to work on any plugin). 

What happened since last time when i (and some other persons) tried to implement this and remove RW from everyone? 

Kohsuke, do you publicly approve that jenkins is not the bazaar anymore? 

PS Or JEP now removes King of project position?

Thanks.

[Daniel Beck]

> On 30. Nov 2017, at 00:17, Kanstantsin Shautsou <kanstan...@gmail.com> wrote:
>
> Kohsuke, do you publicly approve that jenkins is not the bazaar anymore?

Could we please keep this thread focused on the specific proposed initiative?

[Daniel Beck]

> On 29. Nov 2017, at 14:52, Daniel Beck <m...@beckweb.net> wrote:
>
> Details TBD. I currently plan to make people collaborators with write access. This will need to be cleaned up some time in the future, but speculatively granting additional permissions (most per-repo teams grant admin access) doesn't sit right with me.

The GitHub API currently will always send invitations when adding people as collaborators to repositories programmatically, even if they're in the org (different from the UI), and even if they already have the same access (which replacing 'Everyone' would do). I'm in contact with GitHub support about this, but no ETA for this to change.

Based on my previous scan, 182 users would get a total of ~950 invitations to repositories (98x1, 24x2, 19x3, 7x5, 34x six or more).

On the one hand, it's spammy, and the invitations don't allow me to explain what's going on, so someone not actively reading this list might be very confused or concerned.

On the other hand, basically everyone receiving 6+ invitations is either really inactive (so can probably be excluded to begin with) or active enough to have seen this, and it would allow us to remove permissions from inactive accounts by cancelling these invitations after a month or so.

I don't really see an alternative though: Creating non-admin teams for each of the affected 494 repos (401 of these with just 1-2 members) doesn't look like a good idea either.

In case someone's curious, @jglick wins, with exactly 100 invitations.

[Stephen Connolly]

Now you’ve got me all competitive... how far was I off Jesse?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/DB63E3DA-46EF-4C60-9D99-1F6C9C8D13E7%40beckweb.net.

For more options, visit https://groups.google.com/d/optout.

--

Sent from my phone

[Daniel Beck]

> On 1. Dec 2017, at 17:06, Stephen Connolly <stephen.al...@gmail.com> wrote:
>
> Now you’ve got me all competitive... how far was I off Jesse?

You can look forward to 68 emails from GitHub ;-)

[Oleg Nenashev]

It should not be a problem with Stephen's speed-reading skills :)

I'm +1 for going forward.
I spent pretty much time on cleaning up Everyone from some my repos, and just deleting it seems to be the most feasible approach.

Even if something goes wrong with automatic migration, we have enough admins of jenkinsci org who could help to fix particular repositories manually upon-request.

BR, Oleg


пятница, 1 декабря 2017 г., 19:33:10 UTC+3 пользователь Daniel Beck написал:

[Daniel Beck]

> On 1. Dec 2017, at 16:49, Daniel Beck <m...@beckweb.net> wrote:
>
> The GitHub API currently will always send invitations when adding people as collaborators to repositories programmatically, even if they're in the org (different from the UI), and even if they already have the same access (which replacing 'Everyone' would do). I'm in contact with GitHub support about this, but no ETA for this to change.
>
> Based on my previous scan, 182 users would get a total of ~950 invitations to repositories (98x1, 24x2, 19x3, 7x5, 34x six or more).
>
> On the one hand, it's spammy, and the invitations don't allow me to explain what's going on, so someone not actively reading this list might be very confused or concerned.
>
> On the other hand, basically everyone receiving 6+ invitations is either really inactive (so can probably be excluded to begin with) or active enough to have seen this, and it would allow us to remove permissions from inactive accounts by cancelling these invitations after a month or so.
>
> I don't really see an alternative though: Creating non-admin teams for each of the affected 494 repos (401 of these with just 1-2 members) doesn't look like a good idea either.
>
> In case someone's curious, @jglick wins, with exactly 100 invitations.

GitHub just informed me that they changed this behavior: existing org members just get added to the repo, without invitation. They seem to receive an email that they've been subscribed to a repo though. I assume that only happens when they haven't been subscribed yet, but have the option to automatically subscribe enabled, but don't know for sure.

I'll implement this as originally announced.

[Daniel Beck]

> On 16. Feb 2018, at 20:15, Daniel Beck <m...@beckweb.net> wrote:
>
> GitHub just informed me that they changed this behavior: existing org members just get added to the repo, without invitation. They seem to receive an email that they've been subscribed to a repo though. I assume that only happens when they haven't been subscribed yet, but have the option to automatically subscribe enabled, but don't know for sure.
>
> I'll implement this as originally announced.

Hi everyone,

The permission migration is complete.

Please file INFRA issues if you lost permissions. The GitHub API around permissions isn't much fun to work with, so I wouldn't be surprised if I missed something.

Daniel

[Oleg Nenashev]

Thanks a lot for that, Daniel!

P.S: I have created a filter/notification for new INFRA tickets, so I will help with monitoring/fixing permissions.

BR, Oleg