Discussion - Plugins reporting issue trackers

[Gavin Mogan]

Hey Everyone,

I'm trying to upgrade/add Issue tracking support to the plugin site. We currently have a tab that lists issues from jira, but not all plugins are using jira, and it would be nice to have a direct link to issue tracker on the sidebar.

The original plan was to default to JIRA unless Github Issues were enabled on the repo. But Daniel Beck pointed out that assumption might not be safe, github issues might be enabled but they are not used.

Ideally I'd see the process being

1) Overrides file

2) Issue Managemnet section in pom.xml (https://maven.apache.org/pom/maven/issue-management.html). I was thinking we should use jira:$component-name, or github:repo-name

3) Github if enabled

4) Jira

I'm not even sure the data for #2 is exposed to poms and stuff.

So looking for feedback on options and what people prefer.

Gavin

[Richard Bywater]

Using the pom to explicitly point to where issue management is done seems reasonable to me although I note from the docs (https://maven.apache.org/pom.html#issue-management) that there's a "system" element and a "url" element so presumably rather than using some unique format we could just use jira/github for system and the actual URL for the component or Github repo issues link?

Richard.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DutC1CgDjwykf6pEbdKzDfvpR4W%2Bi%2B9GAuvLFRM_%3DQpdEg%40mail.gmail.com.

[Tim Jacomb]

I would suspect there’s not many places where issues are enabled and not in use

I’m fine with your proposed process

Thanks

Tim

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAAy0hwc4m2eL_pPXEwm%3Da_zOGOzcRPw_p-LCxms17DfveJypwQ%40mail.gmail.com.

[Gavin Mogan]

Yea I havnt dug into the format too much. My concern is the link for querying issues and writing issues would be very different.

As I'm writing this I want to add, we absolutely need to have a fallback. As much as I love explicit configs, we have 2000ish plugins now and most won't get updated with new data.

The issue with the scm tag is it isn't going to allow us to expand it in the future. Would love to have companies be able to supply a support link. Or maybe demo repo. While that easily could be in the repo, would be nice to have a machine readable system/standard for it (can we expose properties or anything?)

Gavin

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAAy0hwc4m2eL_pPXEwm%3Da_zOGOzcRPw_p-LCxms17DfveJypwQ%40mail.gmail.com.

[Daniel Beck]

> On 17. Dec 2020, at 07:53, 'Gavin Mogan' via Jenkins Developers <jenkin...@googlegroups.com> wrote:
>
> 1) Overrides file
> 2) Issue Managemnet section in pom.xml (
> https://maven.apache.org/pom/maven/issue-management.html). I was thinking
> we should use jira:$component-name, or github:repo-name
> 3) Github if enabled
> 4) Jira
>
> I'm not even sure the data for #2 is exposed to poms and stuff.
>
> So looking for feedback on options and what people prefer.
>

Option 2 won't work due to bad data quality in plugins (tons of obsolete URLs -- thanks Jira -- and of course a lot of missing data). Suddenly starting to rely on it will not be useful. While this *may* be another source of data, it too broken to use as authoritative source. And trying to backfill it will probably take forever, see also our wiki migration.

I've been working on something similar to option 1 as the foundation for exposing the data via update site metadata (to be able to link there directly from Jenkins) for a while now: Extend the metadata we track for plugins (via repository-permissions-updater) to include issue tracker locations. The benefit is that unused or obsolete trackers that exist for historical reasons will be overridable. The drawback is some additional effort in making a switch. OTOH, that switch will actually work as opposed to a helpful script identifying that Jira/GH issues are non-empty, and adding them to the list.

An open question is whether it would remain an "override" for select cases, or the sole source of data after an initialization (similar to what Jesse did for GitHub URLs). If we want to expose the data in update site metadata (which I think would be the correct choice), then it needs to be quick to determine, so we need a static source -- which can be either a complete configuration, or a list updated periodically by another process. The former would be easier. It's also unclear to me how to identify the preferred issue tracker for new issues otherwise.

[Tim Jacomb]

Agree tracking it in RPU is probably best

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/6E71437A-E2AF-4F8F-B6D0-71FDE96EA919%40beckweb.net.

[Gavin Mogan]

Okay, I'd love to hear more ideas, but I really like the explicit configs.

(it took me way too long to figure out what RPU was)

So update plugin site to use the data in github.com/jenkins-infra/repository-permissions-updater

1 time mass PR to github.com/jenkins-infra/repository-permissions-updater for each repo.

If a repo has github issues enabled, assume it uses that.

else If the plugin name has a jira component, assume it uses that.

else no issues link

issues: github:jenkinsci/configuration-as-code-plugin

issues: jira:blueocean-plugin

or to daniel's point (might be on the github draft PR), do we want to make it a hash style to support plugins that do both? Its not likely that a plugin will want two, but during a transitory period, it might be worth listing both

issues:

  github: jenkinsci/configurations-as-code

  jira: blueocean-plugin

As a side note, it would be super easy to use that data to also fix SCM, so that things like the blueocean sub modules would have the right scm links since its already part of the metadata - https://github.com/jenkins-infra/repository-permissions-updater/blob/master/permissions/plugin-antisamy-markup-formatter.yml#L3

Gavin

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAH-3BifBhzL%2B2UwBHUpyZxjz1uNWi5bA3k%3DN0OFrtXCBU1KK%2BA%40mail.gmail.com.

[Daniel Beck]

> On 17. Dec 2020, at 23:01, 'Gavin Mogan' via Jenkins Developers <jenkin...@googlegroups.com> wrote:
>
> If a repo has github issues enabled, assume it uses that.
> else If the plugin name has a jira component, assume it uses that.
> else no issues link

The beauty of the one time mass definition of issue trackers is that it can be arbitrarily sophisticated (read: convoluted and manual), since we don't need to re-run it once every 4 minutes on trusted.ci.

Component name in Jira doesn't *exactly* match the plugin ID? No problem if I can match them up in a giant table. GH issues is enabled but has 0 total issues ever reported? Probably not relevant (and could even be disabled). Same with Jira components that could be deleted if 0 issues exist. But not both though. Plus we can purge Jira components without a corresponding published plugin (I already know these exist, it's great…).

My desire to get this right, and a lack of spare time over the last 1-2 months has delayed this however :-/ Perhaps over the upcoming vacation.

If others are willing to help with this, I can set something up for us to collaborate.

> or to daniel's point (might be on the github draft PR), do we want to make it a hash style to support plugins that do both? Its not likely that a plugin will want two, but during a transitory period, it might be worth listing both

FTR I wrote about that in https://github.com/jenkins-infra/plugin-site-api/pull/97#discussion_r544167455 and https://github.com/jenkins-infra/plugin-site-api/pull/97#issuecomment-747316256 -- some plugins do this deliberately. We also need to distinguish issue tracker references for reporting new issues (one tracker is likely preferred / enough to reference) and issue tracker references for existing issues (probably everything with open issues should be included, or even closed issues, depending on how it's used).

While the vast majority of plugins will have a single tracker and that's it, GH issues seems to become more popular as well, and since that genie's out of the bottle by now, we should have proper support for folks wanting to switch over.

> As a side note, it would be super easy to use that data to also fix SCM, so that things like the blueocean sub modules would have the right scm links since its already part of the metadata - https://github.com/jenkins-infra/repository-permissions-updater/blob/master/permissions/plugin-antisamy-markup-formatter.yml#L3

That's a great point. Correct; right now, update-center2 does not use RPU metadata at all, but would need it anyway once we define the issue tracker location there, so this should be a straightforward change on top.

(Once we do this it's probably time to rename RPU to 'component-metadata' or similar, and move the permissions script out of it. IIUC we already have incrementals relying on it as well, that's what the 'github' field is for. What blocked this so far is the terrible file format that badly needs an overhaul as well to also better support the idea of a "plugin" semantically. So much to do, so little time :-( )

[slide]

I think adding this information to the permissions files would be really nice. I'd need to update the bot for the hosting process, so just let me know what the syntax will be and I can get it place.

[Tim Van Holder]

If tweaking and renaming RPU, maybe also tweak the directory structure?

For example, changing permissions/plugin-groovy-events-listener-plugin-master.yml to permissions/plugin/g/groovy-events-listener-plugin-master.yml

This would make it a lot easier to browse on GH, which now truncates to 1000 items.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/048d3dab-7389-48d5-957e-7945dbb4a7c4n%40googlegroups.com.

[Tim Jacomb]

The way I work around that is by pressing t to go to file find mode and then typing the plugin name :) 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKMi--C1Q4SHgWLC2OrUv6FzSAMS4Zd4CvS7aSw2UOdqBChgHQ%40mail.gmail.com.

[Gavin Mogan]

I love the idea but I'll put changing the directory structure to those in charge of infra it doesn't break anything 

However adding new fields won't break things so I'm dusting off some groovy skills to make a POC

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKMi--C1Q4SHgWLC2OrUv6FzSAMS4Zd4CvS7aSw2UOdqBChgHQ%40mail.gmail.com.

[Gavin Mogan]

Based on my previous cleanup of jira components, this actually wasn't that hard

https://gist.github.com/7f30c8385f28c40f99f66967ab74b42b Def not my best work but i'm happy with it for limited computer time.

Its safe to run multiple times, so once everything is agreed upon, can update the hosting bot and run this again.

All the populated data - https://github.com/jenkins-infra/repository-permissions-updater/pull/1793

Only a handful of plugins couldn't be matched up, so we'll need to create jira components (or prevent distribution).

[MISSING] aio-tests is missing issues
[MISSING] ApicaLoadtest is missing issues
[MISSING] appetize is missing issues
[MISSING] appscan is missing issues
[MISSING] bigpanda-jenkins is missing issues
[MISSING] bitbucket-scm-filter-aged-refs is missing issues
[MISSING] bitbucket-scm-trait-commit-skip is missing issues
[MISSING] blueocean-i18n is missing issues
[MISSING] build-history-metrics-plugin is missing issues
[MISSING] build-steps-from-json is missing issues
[MISSING] bumblebee is missing issues
[MISSING] ca-apm-parent is missing issues
[MISSING] carl is missing issues
[MISSING] codecommit-url-helper is missing issues
[MISSING] codedx is missing issues
[MISSING] compress-buildlog is missing issues
[MISSING] cucumber-trends-report is missing issues
[MISSING] custom-tools-plugin is missing issues
[MISSING] diskcheck is missing issues
[MISSING] environment-labels-setter is missing issues
[MISSING] environment-manager is missing issues
[MISSING] externalresource-dispatcher is missing issues
[MISSING] failedJobDeactivator is missing issues
[MISSING] github-scm-filter-jira-validator is missing issues
[MISSING] google-deployment-manager is missing issues
[MISSING] housekeeper is missing issues
[MISSING] http-post is missing issues
[MISSING] http_request is missing issues
[MISSING] ibm-application-security is missing issues
[MISSING] impersonation is missing issues
[MISSING] itms-for-jira is missing issues
[MISSING] jclouds-jenkins is missing issues
[MISSING] jenkinsci-appspider-plugin is missing issues
[MISSING] jenkins-infra-test is missing issues
[MISSING] jenkins-jira-issue-updater is missing issues
[MISSING] jobdelete-builder is missing issues
[MISSING] jquery-detached is missing issues
[MISSING] kanboard is missing issues
[MISSING] keep-slave-disconnected is missing issues
[MISSING] kubernetes-pipeline-devops-steps is missing issues
[MISSING] lsf-cloud is missing issues
[MISSING] mber is missing issues
[MISSING] multi-branch-project-plugin is missing issues
[MISSING] NegotiateSSO is missing issues
[MISSING] paaslane-estimate is missing issues
[MISSING] packagecloud is missing issues
[MISSING] perforce is missing issues
[MISSING] performance-signature-viewer is missing issues
[MISSING] piketec-tpt is missing issues
[MISSING] pipeline-classpath is missing issues
[MISSING] pipeline-editor is missing issues
[MISSING] pipeline-model-extensions is missing issues
[MISSING] portshift-scanner is missing issues
[MISSING] publish-over is missing issues
[MISSING] qftest is missing issues
[MISSING] rpi-build-status is missing issues
[MISSING] sasunit-plugin is missing issues
[MISSING] secondary-timestamper-plugin is missing issues
[MISSING] send-stacktrace-to-eclipse-plugin is missing issues
[MISSING] setenv is missing issues
[MISSING] short-workspace-path is missing issues
[MISSING] sidebar-update-notification is missing issues
[MISSING] snakeyaml-api is missing issues
[MISSING] spoonscript is missing issues
[MISSING] srcclr-installer is missing issues
[MISSING] TestComplete is missing issues
[MISSING] testcomplete-xunit is missing issues
[MISSING] testflo-for-jira-test-management-automation is missing issues
[MISSING] teststudiomobiletesting is missing issues
[MISSING] throttle-concurrents is missing issues
[MISSING] tinfoil-scan is missing issues
[MISSING] tusarnotifier is missing issues
[MISSING] ui-test-capture is missing issues
[MISSING] uno-choice is missing issues
[MISSING] windows-cloud is missing issues
[MISSING] workplace-notifier is missing issues
[MISSING] zapper is missing issues

[Slide]

Some of these are ones that don't necessarily need an issue tracker anywhere, like publish-over or snakeyaml-api (there may be more). Those are component plugins that don't really require having an issue tracker of their own. windows-cloud is a recently hosted plugin that used the new field in the HOSTING project to select github issues as their issue tracker, so no component was created. Github issues should be enabled on the repo, is it not?

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DusLMbW6fk2MUdZ0BYRRAZoUmz-SoCAH8%3Do8mHRaK10hHw%40mail.gmail.com.

--

Website: http://earl-of-code.com

[Gavin Mogan]

So thanks for pointing that out, I had a return when i meant continue (i'm pretty sure it was in a forEach, i just converted to a for and used continue)

One I noticed was (cause i was curious what it does):

https://www.jenkins.io/doc/pipeline/steps/kubernetes-pipeline-arquillian-steps/ exists

https://plugins.jenkins.io/kubernetes-pipeline-arquillian-steps/ doesn't exist

https://github.com/jenkins-infra/repository-permissions-updater/blob/master/permissions/plugin-kubernetes-pipeline-arquillian-steps.yml points to https://github.com/jenkinsci/jenkinsci/kubernetes-pipeline-plugin which doesn't exist

I'm thinking we might want to add a validation step to the yaml files, confirm all the values point to valid items,

 

new list that couldn't match up:

[MISSING] aio-tests is missing issues
[MISSING] ApicaLoadtest is missing issues
[MISSING] appetize is missing issues
[MISSING] appscan is missing issues
[MISSING] bigpanda-jenkins is missing issues

[MISSING] bitbucket-scm-filter-jira-validator is missing issues

[MISSING] build-history-metrics-plugin is missing issues
[MISSING] build-steps-from-json is missing issues
[MISSING] bumblebee is missing issues

[MISSING] ca-apm-dist is missing issues

[MISSING] carl is missing issues
[MISSING] codecommit-url-helper is missing issues
[MISSING] codedx is missing issues
[MISSING] compress-buildlog is missing issues
[MISSING] cucumber-trends-report is missing issues
[MISSING] custom-tools-plugin is missing issues

[MISSING] declarative-pipeline-migration-assistant-api is missing issues

[MISSING] diskcheck is missing issues

[MISSING] dtkit-api is missing issues

[MISSING] environment-labels-setter is missing issues
[MISSING] environment-manager is missing issues

[MISSING] essentials is missing issues

[MISSING] externalresource-dispatcher is missing issues
[MISSING] failedJobDeactivator is missing issues

[MISSING] github-scm-filter-aged-refs is missing issues
[MISSING] github-scm-trait-commit-skip is missing issues

[MISSING] google-deployment-manager is missing issues

[MISSING] handlebars is missing issues

[MISSING] housekeeper is missing issues
[MISSING] http-post is missing issues
[MISSING] http_request is missing issues
[MISSING] ibm-application-security is missing issues
[MISSING] impersonation is missing issues
[MISSING] itms-for-jira is missing issues
[MISSING] jclouds-jenkins is missing issues
[MISSING] jenkinsci-appspider-plugin is missing issues

[MISSING] jenkins-jira-issue-updater is missing issues
[MISSING] jobdelete-builder is missing issues

[MISSING] kanboard is missing issues
[MISSING] keep-slave-disconnected is missing issues

[MISSING] kubernetes-pipeline-arquillian-steps is missing issues

[MISSING] lsf-cloud is missing issues
[MISSING] mber is missing issues
[MISSING] multi-branch-project-plugin is missing issues
[MISSING] NegotiateSSO is missing issues
[MISSING] paaslane-estimate is missing issues
[MISSING] packagecloud is missing issues
[MISSING] perforce is missing issues

[MISSING] piketec-tpt is missing issues
[MISSING] pipeline-classpath is missing issues
[MISSING] pipeline-editor is missing issues

[MISSING] pipeline-rest-api is missing issues
[MISSING] pipeline-stage-tags-metadata is missing issues

[MISSING] portshift-scanner is missing issues
[MISSING] publish-over is missing issues
[MISSING] qftest is missing issues
[MISSING] rpi-build-status is missing issues
[MISSING] sasunit-plugin is missing issues
[MISSING] secondary-timestamper-plugin is missing issues
[MISSING] send-stacktrace-to-eclipse-plugin is missing issues
[MISSING] setenv is missing issues
[MISSING] short-workspace-path is missing issues
[MISSING] sidebar-update-notification is missing issues
[MISSING] snakeyaml-api is missing issues

[MISSING] splunk-devops-extend is missing issues

[MISSING] spoonscript is missing issues
[MISSING] srcclr-installer is missing issues
[MISSING] TestComplete is missing issues
[MISSING] testcomplete-xunit is missing issues
[MISSING] testflo-for-jira-test-management-automation is missing issues
[MISSING] teststudiomobiletesting is missing issues
[MISSING] throttle-concurrents is missing issues
[MISSING] tinfoil-scan is missing issues
[MISSING] tusarnotifier is missing issues
[MISSING] ui-test-capture is missing issues
[MISSING] uno-choice is missing issues

[MISSING] workplace-notifier is missing issues
[MISSING] zapper is missing issues

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPiUgVeL%2B9kVpxF4aa9DZ9AMWTNku_dMGZ3J3bD59noY_w0fcQ%40mail.gmail.com.

[Gavin Mogan]

Daniel Beck has started up some notes about the bigger issue - https://gist.github.com/daniel-beck/a1a3fe34e2e5f27d56d64fee1de11c0e

> The main issue I see is what I mention at the end there: In theory these projects can easily be split up, but once we add consumers of RPU metadata files, changing them is a lot more work.

Keeping the mailing list involved in case others want to help out / get involved