Proposal: Modify Plugin Adoption process to use JIRA to track adoption status

[Craig Rodrigues]

PROBLEM

• Over time, Jenkins plugin maintainers need to change as the original maintainer may need to move on to other work.
• https://wiki.jenkins.io/display/JENKINS/Adopt+a+Plugin outlines the process for adopting a plugin.  It mentions that the Jenkins developer mailing list should be e-mailed to start the process.
  
• Due to the popularity of Jenkins, the mailing lists receive a lot of messages.  For casual Jenkins users who maintain plugins, it is easy to miss these types of e-mails.

PROPOSAL

• Plugins need to change owners and there needs to be a clear adoption process which can be tracked.
• I propose that JIRA be used as the primary mechanism by which the plugin adoption process is tracked, from proposal to adopt a plugin, all the way to when the plugin is adopted.
• Email would be used as a secondary mechanism.  If someone e-mails the Jenkins mailing list, they should be directed to the Adopt+a+Plugin JIRA page, which would instruct them to file a JIRA ticket.

DETAILS

• When someone wants to adopt a plugin, they should file a JIRA ticket.
• They should set the Component field to the name of the plugin
• In addition, they must set a tag in JIRA to something like "adopt"
• The current plugin maintainer will receive an e-mail from JIRA.  If there are multiple plugin maintainers, they should be mentioned via the "@" mechanism in JIRA.
• If the plugin owner does not respond to the JIRA ticket within 3 weeks (enough time to cover reasonable vacation time, sick time, emergency), then the plugin can be adopted.
• Plugin authors are also able to file a JIRA ticket with the "adopt" tag, and mention that they want to give up maintainership of the plugin
• By doing things in JIRA, it is easier for Jenkins administrators to query the list of plugins which are up for adoption and there is a clear audit trail of when a plugin was adopted.
  

POST ADOPTION TASKS

• new maintainer needs to be given write access to the git repository with the plugin code.  I believe that this can be done by sending special commands to the bot on the IRC Freenode #jenkins channel.
• in JIRA, new maintainer should be made the default assignee for tickets filed with Component set to that plugin
• in JIRA, all Unresolved tickets should be assigned to the new maintainer
• new maintainer is responsible for updating the wiki page for the plugin, and also for updating the pom.xml metadata for the plugin indicating that they are the maintainer

This will hopefully reduce the amount of email going to the lists (by a small amount), yet give a clear trackable process by which

plugins can change maintainershp.

Thank you.

--

Craig

[Jesse Glick]

On Tue, Aug 21, 2018 at 12:54 PM Craig Rodrigues <rod...@freebsd.org> wrote:
> PROBLEM

Nicely written up. I wonder if this is “big” enough to qualify as a
“process” JEP¹?

> … the mailing lists receive a lot of messages. For casual Jenkins users who maintain plugins, it is easy to miss these types of e-mails.

Note that the current process requires the maintainer to be CC’d, so
you need not actually be subscribed to any list at all, and some
people’s mail filters/clients will highlight messages with an direct
CC.

> The current plugin maintainer will receive an e-mail from JIRA. …

> If the plugin owner does not respond to the JIRA ticket within 3 weeks (enough time to cover reasonable vacation time, sick time, emergency)

I make no claims to be representative, but for what it’s worth I am
likely to see a message CC’d to me on the dev list within a day at the
most, whereas a mention in JIRA goes to the bucket of >1000 unread
notifications. (Maybe that just means I should not be the maintainer
of anything!)

Basically: having a clear way to track the status of an adoption
request seems like a good step, and keeping detailed discussions in
JIRA is fine too, but I think we should we using all available
channels to grab the attention of an existing maintainer, even if that
results in a bit more traffic on the dev list. It is not uncommon for
third parties to take an interest in an adoption request anyway—they
may have better information on who is active on the plugin, or want to
help out themselves, or could warn the potential adopter about reasons
why the plugin might be obsolete and to not waste time on it.


¹ https://github.com/jenkinsci/jep/blob/master/jep/1/README.adoc#jep-types

[Baptiste Mathus]

I like that idea, and I agree having a clearer way to track ongoing requests, so that they're not forgotten would be great. It's especially important IMO since people requesting to become maintainers may often be somewhat newcomers that won't necessarily dare asking twice if we miss or forget the request.

I think using the HOSTING project kind of makes sense, but I'm not feeling strongly for the actual JIRA project to use for this. I think we want to get feedback from Slide (CCed) here as he's the now one clearly doing the heavy lifting on the HOSTING process, so if we're going to add /requests/ that will appear in his bucket, he should be aware before we do.

And I think that we should have a JEP for this, yes. I think we somehow could have a global JEP for the hosting process, then have a part on adoption, but that would make this a much bigger task which likely would kill the effort. IMO let's just have a smaller process JEP, and we'll use the existing process to deprecate it once/if we include it some day in a bigger JEP to describe the whole hosting/plugin handling process. (BTW, might be something interesting to do during JW/DW. IIUC Daniel started to do this during FOSDEM, as of https://jenkins.io/doc/developer/publishing/requesting-hosting/ but I think it's not finished as  I would imagine once fully done, we'd blank https://wiki.jenkins.io/display/JENKINS/Hosting+Plugins or add a link to jenkins.io).

-- Baptiste

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3DrPVdd81B73r0aMRcOWD%2B50HfTb3qh5asTnt44tD9XWNMq%3DQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Slide]

Using the hosting project would be great. If there is anything we want to do in terms of automating the flow, I'd be interested in helping there too.

[Oleg Nenashev]

Hi Craig,

Thanks a lot for the proposal! I agree with others that we should probably make it a JEP.

I would be also interested to make some changes in the adoption process, especially to simplify it for plugins which are obviously non-maintained.

I rather prefer to keep ownership requests in the mailing list so that other developers can see them and comment if needed (e.g. request ownership as well, vote for and against ownership transfer, etc.). JIRA requests can be also easily lost in the inbox if there is no pings. E.g. I cannot physically process all notifications I receive even after doing a lot of work on setting up filters.

I am currently working on a JEP for hosting 3rd-party components used in Jenkins, and I also have an adoption section there (link). If there is a JEP submitted for this proposal, I am interested to follow similar process in that components. I would be especially interested to allow requesting ownership via pull requests and JIRA issues (see the proposal).

Maybe we could also improve the process to simplify adoption requests for abandoned plugins which have no adoption label, e.g.:

• The plugin can be considered as open for adoption if all conditions below are met...
• All plugin maintainers were inactive in GitHub for 1 years or above (Web UI check)
• All plugin maintainers were inactive in JIRA for 2 years
• All plugin maintainers were inactive in Jenkins mailing lists for 2 years or above
  
• There are outstanding issues and pull requests where a feedback from maintainers was requested
• The plugin can be considered as open for adoption if the maintainer explicitly confirmed to be no longer active in the Jenkins project in a public channel
  
• IIRC bap2000, harebrain and gboissinot have replied something like that at some point
  

It's especially important IMO since people requesting to become maintainers may often be somewhat newcomers that won't necessarily dare asking twice if we miss or forget the request.

Actually I would not consider it as a huge deal. If the requesters lose interest and do not ping if the request gets stuck, probably they are not that interested in the plugin.

But it does not excuse us for missing requests, of course

BR, Oleg 

[Jesse Glick]

On Fri, Aug 24, 2018 at 11:27 AM Oleg Nenashev <o.v.ne...@gmail.com> wrote:
> I would be especially interested to allow requesting ownership via pull requests …

Embryonic thought: maybe we could consider
`repository-permissions-updater` to be the source of truth? After all,
this is what ultimately controls whether or not a given person is
physically capable of producing a release of a given plugin. For
example, I am the maintainer of `build-token-root` by dint of this
line:

https://github.com/jenkins-infra/repository-permissions-updater/blob/f412da4896b12b4f4a39063f2c991d1e5b34841b/permissions/plugin-build-token-root.yml#L7

If someone wanted to be added as a co-maintainer, or take over from
me, at some point a PR would need to be filed amending this file
section anyway and CCing @jglick. So why not treat that as the formal
request for ownership of the plugin? Interested parties could approve
or deny the PR with comments, and if and when merged the transfer
takes effect. (I still maintain that a separate heads-up to the dev
list is in order.)

An org admin also needs to update the GitHub permissions via e.g.

https://github.com/orgs/jenkinsci/teams/build-token-root-plugin-developers/members

but that could just be a (manual or scripted) consequence of the
`plugin-*.yml` patch.

We would also want to deprecate the `<developers>` POM field, which is
very inconsistently maintained anyway, and is bound to a particular
(latest) release of the plugin which is not helpful when you are in
between releases and negotiating handover. Means the `update-center2`
generator would need to consider RPU rather than this.

[Devin Nusbaum]

> Embryonic thought: maybe we could consider
> `repository-permissions-updater` to be the source of truth? After all,

+1. Since this is what actually changes the permissions, I thinks it makes
sense to handle PRs there as formal requests.

> (I still maintain that a separate heads-up to the dev list is in order.)

Could we send automatic email notifications to the list as part of the
the build given the presence of a special tag in the PR? I’m not sure how
useful it would be, since the requester would probably need to reply
anyway in an attempt to CC the maintainers, but maybe the consistency
would be helpful.

> An org admin also needs to update the GitHub permissions via e.g.
>
> https://github.com/orgs/jenkinsci/teams/build-token-root-plugin-developers/members
>
> but that could just be a (manual or scripted) consequence of the
> `plugin-*.yml` patch.

Yes, I think it would be ideal if both Artifactory and GitHub permissions
were granted at the same time and managed in the same place. If it
were automated, there would need to be a way to specify GitHub
usernames in `plugin-*.yml` for users who have different usernames
on each platform, and we’d need to be careful that automatic
modification of GitHub permissions did not remove permissions from
existing users on GitHub who were not specified in `plugin-*.yml`.

> We would also want to deprecate the `<developers>` POM field

+1

[Craig Rodrigues]

On Tue, Aug 21, 2018 at 11:01 AM Jesse Glick <jgl...@cloudbees.com> wrote:

I make no claims to be representative, but for what it’s worth I am
likely to see a message CC’d to me on the dev list within a day at the
most, whereas a mention in JIRA goes to the bucket of >1000 unread
notifications. (Maybe that just means I should not be the maintainer
of anything!)

There are several use cases which can describe use-cases similar to yours:

1.  active in core Jenkins development

2.  work at Cloudbees

3.  maintain many plugins and is marked as the default notification e-mail in JIRA

4.  maintain a key plugin which is heavily used (for example the Git Plugin) and is marked a the default notification e-mail in JIRA

For these use cases, they would get a lot of e-mail traffic from the Jenkins JIRA.

However, for many of the 1000+ Jenkins plugins, the plugin maintainers probably do not receive as much

traffic from JIRA.  They would be more likely to see e-mails generated by @mention in JIRA.

However, when it comes to e-mail, everyone has their own use cases, e-mail filters, problems with spam, etc.

--

Craig

[Craig Rodrigues]

On Fri, Aug 24, 2018 at 8:27 AM Oleg Nenashev <o.v.ne...@gmail.com> wrote:

Hi Craig,

Thanks a lot for the proposal! I agree with others that we should probably make it a JEP.

OK.  In this discussion thread, no one has raised any opposition to the spirit of

my proposal.  I'll work on making this a JEP this weekend.

--

Craig

 

[Craig Rodrigues]

Here is my JEP submission:

https://github.com/jenkinsci/jep/pull/185

--

Craig

[Craig Rodrigues]

On Fri, Aug 24, 2018 at 8:27 AM Oleg Nenashev <o.v.ne...@gmail.com> wrote:

Hi Craig,

Thanks a lot for the proposal! I agree with others that we should probably make it a JEP.

My submission is now Draft JEP-11:

https://github.com/jenkinsci/jep/tree/master/jep/11

--

Craig

[Daniel Beck]

> On 27. Aug 2018, at 21:50, Craig Rodrigues <rod...@FreeBSD.org> wrote:
>
> My submission is now Draft JEP-11:
>
> https://github.com/jenkinsci/jep/tree/master/jep/11

Reading through the current JEP, it doesn't seem robust enough in the face of some of our historical infra baggage, and potentially introduces an additional component to the takeover that may not actually be involved otherwise.

Not all plugins use Jira. Not all plugins have components in Jira. Not all maintainers are default assignees. The first is mostly maintainers prioritizing their own tool preferences over project convention (or having inherited something like that), the others are mostly due to badly maintained old Jira components. Since there's no syncing/automated cleanup of components (Tyler refused to let me continually spam the Jira API), it will likely remain that way for a while.

There's simply too much that can go wrong with these assumptions (being able to rely on them would be nice, but we simply can't). Additionally, it seems weird that the adoption request would be filed in the JENKINS project, when it's more an INFRA kind of issue.

I agree with Devin and Oleg that a PR to jenkins-infra/repository-permissions-updater should be the entry point and reference for any plugin ownership takeover. Without a change there, no ownership transfer can actually be complete, so making that the reference, and things like default assignee changes and adding commit access secondary, would be preferable. It satisfies all the conditions laid out in the motivation with fewer of the previously mentioned pitfalls (although a dev list email CC maintainers would still be useful to announce the intended change, and increase visibility of the request). Even in the case of badly set up Git user names, https://jenkins.io/doc/developer/publishing/source-code-hosting/ tells us what GitHub users have commit access, which is usually good enough to identify maintainer GitHub accounts to ping them. The history of an individual permission file already tells the maintainership history of plugins since we started with this.

Further feedback:

- The reasoning seems to ignore that maintainers are to be copied in any emails to the dev list. They would receive emails directly. This is far more reliable than Jira notifications could ever hope to be, so seems a step backwards.
- We give repo admin access (unless dealing with historical baggage), not just write access.
- Batch reassigning issues is something anyone can trivially do in Jira, doesn't need to be part of the process (presumably to be done by infra admins).
- Wiki pages don't typically list maintainers, so nothing needs updating there (except for the last release's pom.xml metadata via update-center.json, which cannot be changed).
- We should just ignore the pom.xml maintainer metadata for being mostly useless.
- This process allows bad actors to take control of popular, but badly maintained plugins (no more than today, but still). The JEP does not acknowledge this issue in the security section.
- Tags exist in Jira as soon as they're entered once, so this probably doesn't make the threshold for an infra requirement.

I think this has the potential to be an exemplary process JEP, solving an important and currently underspecified problem, but I don't think it's quite there yet.

I only read the finished JEP and earlier messages in this thread, so may have missed further out of band communication.

[Craig Rodrigues]

On Mon, Aug 27, 2018 at 3:03 PM Daniel Beck <m...@beckweb.net> wrote:

I only read the finished JEP and earlier messages in this thread, so may have missed further out of band communication.

Thanks for the feedback.  I have had no out of band communication regarding this JEP other than this e-mail thread.

This is not really a "finished JEP" as you have stated.  As per the JEP process, it has only been accepted as a Draft JEP:

https://github.com/jenkinsci/jep/tree/master/jep/1#approval-as-draft-jep

At this point in the process, the JEP can be refined:

https://github.com/jenkinsci/jep/tree/master/jep/1#refining-a-draft-jep

I'm OK with refining the JEP.

Liam and Jesse have made direct annotations to the draft JEP.

If you have permissions to make annotations directly to the draft JEP,

feel free to provide feedback that way if you want.

--

Craig

 

[Robert Sandell]

I don't think jenkins-infra/repository-permissions-updater should be the starting point. It's hard to find and harder to remember, at least for me. And once found it's hard to find stuff in it. And the couple of times that I've done changes in there it takes me a while to figure out if I have messed something up or not before filing the PR.

So if the repo is a bit intimidating for me how is it for new contributors?

And from what I've seen so far usernames seems to only be appended to the permissions, so it's not a good source of finding the actual current maintainer in there.

/B

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3DrPVcmjTx%2B21sJEL1LWMa7imOW3kdPzx5%2BOyAt2dHxoJNgBQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Robert Sandell

Software Engineer

CloudBees, Inc.

E: rsan...@cloudbees.com

Twitter: robert_sandell

[Jesse Glick]

On Tue, Aug 28, 2018 at 5:00 AM Robert Sandell <rsan...@cloudbees.com> wrote:
> It's hard to find and harder to remember

That would certainly need to be addressed, at a minimum by having the
adoption wiki page walk you through the steps to patch this.

> it takes me a while to figure out if I have messed something up or not before filing the PR.

Are there things the CI job could be checking which it is not yet?

> so far usernames seems to only be appended to the permissions

We could certainly change that, and start pruning unused releaser
permissions to reflect who we _expect_ should be publishing releases
now.

[Daniel Beck]

> On 28. Aug 2018, at 11:00, Robert Sandell <rsan...@cloudbees.com> wrote:
>
> I don't think jenkins-infra/repository-permissions-updater should be the starting point. It's hard to find and harder to remember,

It will necessarily be linked from anywhere documenting the adoption process.

If your experience is that remembering the name to get there directly via browser history is difficult -- that won't be what potential new contributors will be doing. They will click a link in docs.

> And once found it's hard to find stuff in it.

Web based? Go to the repo root page, press 't', then enter the artifactId of whatever you want to see.

> And the couple of times that I've done changes in there it takes me a while to figure out if I have messed something up or not before filing the PR.

Perhaps we can make the upload permissions definition more approachable, but this repo is inherently part of the maintainership transfer already today. It's not like this part of the process would be skipped by going through Jira. In fact, as I write, it's basically the only thing that's actually required (for example when former and future maintainer already discussed things already in private email -- it has happened before).

[Daniel Beck]

> On 28. Aug 2018, at 11:00, Robert Sandell <rsan...@cloudbees.com> wrote:
>

> And from what I've seen so far usernames seems to only be appended to the permissions, so it's not a good source of finding the actual current maintainer in there.

Perhaps this is something that needs to be clarified -- removing previous maintainers is indeed rare. As far as I'm concerned, this just means there are multiple maintainers.

I have plenty of experience doing this, to be able to assign security issues to maintainers, and consider the upload permissions the most reliable indicator of maintainership. If there are multiple users in there, I need to check further sources (For example Git repo activity, or who created most recent releases, etc.) to find the most likely one candidate. Ironically this process really only seems to fail when people adopt a plugin, are being told "congratulations, you're the maintainer now", but never bother to file a permissions PR (i.e. cannot actually release it).

There's probably potential here to enrich the metadata (for example security contact, primary maintainer), but that's well beyond the scope of this issue.

[Craig Rodrigues]

I agree with Robert that jenkins-infra/repository-permissions-updater should

not be the entry point for users to request maintainership of a plugin.  For end users,

this seems like a weird entry point.  Even though I am on maintainership of a few plugins,

I wasn't even aware of jenkins-infra/repository-permissions-updater, so I guess that

is a relatively recent addition to the plugin adoption process.

Agreeing with Robert again, it is not obvious to me as an end-user what kind of PR 

I have to do to jenkins-infra/repository-permissions-updater, as part of the plugin adoption

process.

In JEP 11 ( https://github.com/jenkinsci/jep/tree/master/jep/11 ), I proposed to

make JIRA the entry point for requesting maintainership of a plugin, and I would still like to

stick to that.  From my experience as an end-user, JIRA is a more obvious entry point

for filing a request, and then tracking the status of the request until conclusion.

In JEP 11, I have summarized the points in the existing Plugin Adoption wiki page,

and one of the points I have summarized is mentioning that a Pull Request against

jenkins-infra/repository-permissions-updater is necessary.

I'd rather leave JEP 11 at that.

--

Craig

On Tue, Aug 28, 2018 at 5:00 AM Robert Sandell <rsan...@cloudbees.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CALzHZS0_HgG%2B8_Gb6ew00wGGvf5un39icQSGhz4q5AyOU9X9Lg%40mail.gmail.com.

[Jesse Glick]

FWIW:

On Thu, Aug 30, 2018 at 5:23 PM Craig Rodrigues <rod...@freebsd.org> wrote:
> I wasn't even aware of jenkins-infra/repository-permissions-updater, so I guess that
> is a relatively recent addition to the plugin adoption process.

It is over two years old.

> it is not obvious to me as an end-user what kind of PR

> I have to do to jenkins-infra/repository-permissions-updater>, as part of the plugin adoption
> process.

https://github.com/jenkins-infra/repository-permissions-updater/blob/master/.github/PULL_REQUEST_TEMPLATE.md#when-adding-new-uploaders-this-includes-newly-created-permissions-files

describes how to add new maintainers; TBD if we would advocate
removing old ones.

[Daniel Beck]

> On 30. Aug 2018, at 23:23, Craig Rodrigues <rod...@FreeBSD.org> wrote:
>
> I'd rather leave JEP 11 at that.

The JEP as is does not address the multiple concerns regarding suitability for its stated purpose that were raised in this thread. While there's some additional process around adoption that we're basically able to freely change, we've long established a process around maintainership changes of any kind, and that is completely omitted.

jenkins-infra/repository-permissions-updater ("r-p-u") is a _necessary_ part of changing plugin maintainership. In fact, if current and future maintainers are in contact, it's the _only_ part that's required, as it is where upload permissions are defined. Someone declaring something done in Jira means absolutely nothing in terms of anyone being able to release something. So you're not actually sparing prospective maintainers from this part of the change by omitting it.

We basically have two choices here:

* Make r-p-u the entry point and reference for taking over maintainership.
* Make a Jira issue the entry point, and r-p-u the reference.

The latter appears to have the (IMO questionable) benefit of going through an issue tracker, at the cost of introducing potential discrepancies with the primary maintainership reference (for example, when a plugin changed hands).

The option you appear to prefer, "Jira without r-p-u Git repo", is not actually feasible: As I explained above, the _only_ mandatory task in transferring maintainership is changing upload permissions, and that is currently missing from both the core process and the post-adoption tasks.

Now, I've already acknowledged it's probably more difficult to get changes in there than it needs to be. I'd be happy to look into suggestions for improvements. But it's been around for several years, and several hundred people have still managed to get their changes in -- and the vast majority of problems could be tracked back to not having read the PR checklist properly.

[Jesse Glick]

On Fri, Aug 31, 2018 at 8:53 AM Daniel Beck <m...@beckweb.net> wrote:
> [RPU is] the _only_ part that's required [to change plugin maintainership], as it is where upload permissions are defined.

Well…effectively the new maintainer also needs write permission to the
source repository. While it is technically possible to locally edit
the `<version>` of a plugin and `mvn deploy` the result as a release,
we would certainly not want to encourage people to do that. Same for
cutting the release out of a non-@jenkinsci fork. (Not sure of the
exact process when using Gradle but I imagine it is similar enough.)
So I would treat updates to the GitHub access control metadata as a
required part of any adoption process.

[Daniel Beck]

Yes. This is why I mentioned:

> In fact, if current and future maintainers are in contact

… as previous maintainers could just add the new maintainer as an 'external contributor' to the GitHub repo, if they have repo admin access -- which is the default.

So the only thing they cannot accomplish on their own _somehow_, without a project infra contributor getting involved, is updating upload permissions.

[Daniel Beck]

> On 31. Aug 2018, at 17:16, Daniel Beck <m...@beckweb.net> wrote:
>
> … as previous maintainers could just add the new maintainer as an 'external contributor' to the GitHub repo, if they have repo admin access -- which is the default.
>
> So the only thing they cannot accomplish on their own _somehow_, without a project infra contributor getting involved, is updating upload permissions.

So to clarify, most of my concerns are how the revised 'Adopt a plugin' process would interact with the established maintainership definition and transfer process.

In the proposal as it is today, the latter isn't being considered at all, and that's what the majority of my concerns are about.

For example, maintainership transfers can happen without marking a plugin for adoption, or creating Jira issues, etc. -- so Jira can never be a complete reference. We have one reference that will be up to date unless we stopped a transfer halfway through, and that is the upload permissions.