Plugins using removed Guava APIs

[Basil Crow]

I started looking into which plugins use classes or methods from Guava
11 that have been removed in Guava 30. There is plenty of low-hanging
fruit if anyone is interested in contributing by rewriting these
usages. The list below is far from exhaustive, but it's a start. If
you maintain one of these plugins, consider taking some proactive
steps to migrate away from these APIs.

com/google/common/base/Objects#firstNonNull
- blueocean-pipeline-api-impl
- blueocean-pipeline-scm-api
- ec2-fleet
- gearman-plugin
- github
- jclouds-jenkins
- jira

com/google/common/base/Objects#toStringHelper
- blueocean-rest-impl
- build-monitor-plugin
- cloudfoundry-bosh-cli
- docker-plugin
- extreme-feedback
- google-source-plugin
- gravatar
- repository
- splunk-devops-extend

com/google/common/base/Stopwatch#elapsedMillis
- build-monitor-plugin

com/google/common/base/Stopwatch#elapsedTime
- relution-publisher
- vsphere-cloud

com/google/common/collect/Ranges
- audit-trail
- elastest
- http_request
- logstash
- scm-httpclient

com/google/common/io/Files#newOutputStreamSupplier
- repository-connector

com/google/common/net/InternetDomainName#name
- scm-api

com/google/common/util/concurrent/MoreExecutors#sameThreadExecutor
- workflow-basic-steps

[Tim Jacomb]

scm-api done in 

https://github.com/jenkinsci/scm-api-plugin/pull/85 not released

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjo5UfMB70nRvn4oG-KZ1GAy8MaAxBZPsEP8V3zJyOyZsg%40mail.gmail.com.

[Olivier Lamy]

blueocean is WIP https://github.com/jenkinsci/blueocean-plugin/pull/2163 (please don't create separate PR) 

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjo5UfMB70nRvn4oG-KZ1GAy8MaAxBZPsEP8V3zJyOyZsg%40mail.gmail.com.

--

Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy

[jn...@cloudbees.com]

HI Basil,

Apologies I said I was going to send this information and didn't get fully round to it.

I have an list (attached) of all the braking APIs, and have another list of all plugins that (potentially) use one of the breaking methods either directly (in the plugins main library) or indirectly (via a library bundled with the plugin)  The identifications of plugins does include false positives (so is a worst case scenario) due to the nature of the scanning tool I used (https://github.com/jenkins-infra/usage-in-plugins/pull/20) .  This list is not filtered so also includes CloudBees proprietary plugins.

in summary the following plugins are all potentially affected (some may be only false positives): 

• 42crunch-security-audit
• active-directory
• alauda-kubernetes-support
• alauda-pipeline
• alibabacloud-ecs
• anchore-container-scanner
• antisamy-markup-formatter
• argus-notifier
• artifact-manager-s3
• artifactory
• atlassian-jira-software-cloud
• audit-trail
• aws-beanstalk-publisher-plugin
• aws-sqs
• aws-yum-parameter
• azure-acs
• azure-ad
• azure-app-service
• azure-batch-parallel
• azure-commons
• azure-container-agents
• azure-container-registry-tasks
• azure-credentials
• azure-dev-spaces
• azure-function
• azure-vm-agents
• azure-vmss
• bart
• behave-testresults-publisher
• bitbucket-pullrequest-builder
• blackduck-detect
• blueocean-bitbucket-pipeline
• blueocean-commons
• blueocean-git-pipeline
• blueocean-github-pipeline
• blueocean-jira
• blueocean-pipeline-api-impl
• blueocean-pipeline-scm-api
• blueocean-rest
• blueocean-rest-impl
• browserstack-integration
• build-failure-analyzer
• build-history-metrics-plugin
• build-monitor-plugin
• build-user-vars-plugin
• caliper-ci
• cerberus-testing
• checkmarx
• cloudbees-analytics
• cloudbees-api-client
• cloudbees-assurance
• cloudbees-github-reporting
• cloudbees-jsync-archiver
• cloudbees-platform-common
• cloudbees-rbac-auto-configurer
• cloudbees-servicenow-jenkins-plugin
• cloudbees-unified-ui
• cloudbees-workflow-template
• cloudfoundry-bosh-cli
• cloudhub-deployer
• codebeamer-coverage-publisher
• coding-webhook
• collabnet
• compuware-ispw-operations
• configuration-as-code
• confluence-publisher
• consul-kv-builder
• coverity
• cucumber-reports
• cucumber-testresult-plugin
• database
• debian-package-builder
• defensics
• delivery-pipeline-plugin
• depgraph-view
• deploydb
• deployit-plugin
• deployment-sphere
• devoptics
• discobit-autoconfig
• docker-build-step
• docker-java-api
• docker-plugin
• docker-traceability
• DotCi
• DotCi-InstallPackages
• DotCiInstallPackages
• easyqa
• ec2
• ec2-deployment-dashboard
• ec2-fleet
• elastest
• elasticbox
• electricflow
• envinject
• envinject-api
• esr-feeder
• esr-reporter
• Exclusion
• extreme-feedback
• ez-templates
• flaky-test-handler
• gating-core
• gcp-secrets-manager-credentials-provider
• gcr-scanner
• gearman-plugin
• gerrit-code-review
• gerrit-trigger
• gerrit-verify-status-reporter
• ghprb
• git-bisect
• git-changelog
• git-client
• gitee
• github
• github-autostatus
• github-oauth
• github-pullrequest
• gitlab-oauth
• gitlab-plugin
• gogs-webhook
• google-admin-sdk
• google-analytics-usage-reporter
• google-cloud-backup
• google-cloud-health-check
• google-cloudbuild
• google-compute-engine
• google-container-registry-auth
• google-deployment-manager
• google-kubernetes-engine
• google-oauth-plugin
• google-source-plugin
• google-storage-plugin
• gradle
• gravatar
• headspin
• hipchat
• http_request
• hubot-steps
• hudson-wsclean-plugin
• ibm-cloud-devops
• in-toto
• inedo-buildmaster
• inedo-proget
• influxdb-query
• instana
• ios-device-connector
• ircbot
• jacoco
• jclouds-jenkins
• jenkins-cloudfoundry-uaa
• jgiven
• jira
• jira-steps
• jira-trigger
• JiraTestResultReporter
• jms-messaging
• job-node-stalker
• jobgenerator
• jslint
• junit
• kubernetes
• kubernetes-cd
• label-linked-jobs
• lambdatest-automation
• loadimpact-plugin
• logstash
• mabl-integration
• machine-learning
• marathon
• master-provisioning-core
• master-provisioning-kubernetes
• maven-dependency-update-trigger
• maven-invoker-plugin
• maven-release-cascade
• mesos
• minio
• minio-storage
• miniorange-saml-sp
• nectar-vmware
• neoload-jenkins-plugin
• nodelabelparameter
• nodepool-agents
• octoperf
• ontrack
• openedge
• openshift-deployer
• openshift-login
• openstack-cloud
• openstack-heat
• opentelemetry
• operations-center-analytics
• operations-center-analytics-feeder
• operations-center-analytics-reporter
• operations-center-context
• operations-center-monitoring
• operations-center-server
• operations-center-sso
• oracle-cloud-infrastructure-compute
• package-parameter
• parameterized-trigger
• perfecto
• performance
• periodicbackup
• pipeline-aws
• pipeline-cloudwatch-logs
• pipeline-huaweicloud-plugin
• pipeline-model-api
• pipeline-model-definition
• project-inheritance
• radargun-reporting
• rally-plugin
• rapid7-insightvm-container-assessment
• release-helper
• reliza-integration
• relution-publisher
• repository
• repository-connector
• requests
• rocketchatnotifier
• role-strategy
• rundeck
• sahagin
• saml
• sauce-ondemand
• scm-api
• scm-httpclient
• scm-manager
• scm-sqs
• scm-sync-configuration
• screenrecorder
• sectioned-view
• seed
• selenium
• selenium-builder
• service-fabric
• service-now
• servicenow-cicd
• sitemonitor
• slave-proxy
• smilehubnotifier
• sms
• sonar
• sonar-gerrit
• split-admin
• splunk-devops
• splunk-devops-extend
• spoonscript
• spring-config
• spring-initalzr
• srcclr-installer
• ssh-slaves
• stackrox-container-image-scanner
• sysdig-secure
• telegram-notifications
• telerik-appbuilder-plugin
• testInProgress
• testsigma
• tics
• timestamper
• tuleap-api
• tuleap-git-branch-source
• uipath-automation-package
• unleash
• usemango-runner
• venafi-vcert
• violation-comments-to-stash
• vmware-vrealize-automation-plugin
• vrealize-automation-8
• vsphere-cloud
• whitesource
• wikitext
• workflow-api
• workflow-basic-steps
• workflow-cps
• workflow-support
• xframium
• xlrelease-plugin
• xvnc
• yet-another-docker-plugin
• zanata
• zephyr-for-jira-test-management
• zos-connector

I am wondering how best we should co-ordinate this work?  CloudBees has already provided some of the plugins above to entirely remove Guava (rather than just make it forward and backward compatible).  Some of the plugins on the list are also up-for adoption with no active maintainer.  (similar to the Digester work I suggest that if thiese are not popular plugins they maybe have a PR and that is as far as it goes).

Ontop of this as updating all the plugins could take a while I have another JEP that I need to file and CloudBees is investigating to isolate core/plugin libraries.  If we had that it would potentially enable us to have a detached jenkins-2.xxx-compat plugin that bundled the old Guava and bump & isolate guava in core before all plugins had been updated (Ideally we would like to remove the Guava library however it is required by Guice)

Regards

/James

[Tim Jacomb]

FTR I've removed guava in all maintained azure plugins today (except azure-ad, needs a follow up to move to caffeine cache)

Hopefully it's easy to generate your list again, is there a script?

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/44efda9d-e987-4e7e-9be4-ad43e64bfb71n%40googlegroups.com.

[Tim Jacomb]

I see you used usage in plugins :)

[jn...@cloudbees.com]

>  Hopefully it's easy to generate your list again, is there a script?

> I see you used usage in plugins :)

I have a parameterized job I can kick off internally (the main bit being the list of things to pass "usage-in-plugins" which was created from the rev-api report.) attached these if anyone want to play (and remember is uses a PR not master of usage in plugins so it will find usage in libraries too!)

/James

[Basil Crow]

Thank you for sharing this! I posted PRs for Timestamper and Lockable Resources:

https://github.com/jenkinsci/timestamper-plugin/pull/116
https://github.com/jenkinsci/lockable-resources-plugin/pull/242

> I am wondering how best we should co-ordinate this work?

I suggest writing some developer documentation with clear instructions
and then filing an epic with individual issues for each plugin, much
like what was done for Configuration Form Modernization and JCasC.
This allows plugin maintainers to pitch in and will also start to
raise awareness of the migration in the broader Jenkins community.

For each possible match on your list, we should also keep track of
whether it has been evaluated (i.e. match or false positive), the list
of PRs (if any), and the first working release. I think the
compatibility tables provided in JEP-227 and JEP-228 worked
particularly well for this purpose, so I suggest we do the same in a
corresponding JEP.

> CloudBees has already provided some of the plugins above to entirely remove Guava (rather than just make it forward and backward compatible).

Thank you to CloudBees for funding this.

> Some of the plugins on the list are also up-for adoption with no active maintainer. (similar to the Digester work I suggest that if thiese are not popular plugins they maybe have a PR and that is as far as it goes).

I concur. With so much work that needs to be done, we need to
prioritize carefully. I note that there remain a number of usages in
Pipeline plugins as well as popular plugins such as http_request;
these seem like good places to start.

[Tim Jacomb]

> remember is uses a PR not master of usage in plugins so it will find usage in libraries too!

I merged usage in plugins this morning, so unless you need it released in some way you shouldn't need that

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjq_WnM6D5HUZv8Xkb4afyRr_uoyRaf-ZVz1rVMGC%3D4FWQ%40mail.gmail.com.

[James Nord]

I merged usage in plugins this morning, so unless you need it released in some way you shouldn't need that

Many thanks Tim!
 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAH-3BicVHL6uUu8RjiinLvN5JYsF%3DeY4JvgUE%2BGmxX6n1Nzrkw%40mail.gmail.com.

[Fritz Elfert]

On 07.05.21 04:39, Basil Crow wrote:
> I started looking into which plugins use classes or methods from Guava
> 11 that have been removed in Guava 30. There is plenty of low-hanging
> fruit if anyone is interested in contributing by rewriting these
> usages. The list below is far from exhaustive, but it's a start. If
> you maintain one of these plugins, consider taking some proactive
> steps to migrate away from these APIs.
>
> com/google/common/base/Objects#firstNonNull

[...]
> - jclouds-jenkins
[...]
For the jclouds plugin:
Fixed here: https://github.com/jenkinsci/jclouds-plugin/tree/obsolete-guava. Not yet merged.

Cheers
-Fritz

[Basil Crow]

On Thu, May 6, 2021 at 7:39 PM Basil Crow <m...@basilcrow.com> wrote:
>
> com/google/common/collect/Ranges
> - audit-trail
> - elastest
> - http_request
> - logstash
> - scm-httpclient

I opened PRs for a few of these:

https://github.com/jenkinsci/http-request-plugin/pull/63
https://github.com/jenkinsci/audit-trail-plugin/pull/49
https://github.com/jenkinsci/logstash-plugin/pull/101

[Basil Crow]

On Thu, May 6, 2021 at 7:39 PM Basil Crow <m...@basilcrow.com> wrote:
>

> com/google/common/base/Objects#firstNonNull
> - blueocean-pipeline-api-impl
> - blueocean-pipeline-scm-api
> - ec2-fleet
> - gearman-plugin
> - github
> - jclouds-jenkins
> - jira

I opened PRs for a few of these:

https://github.com/jenkinsci/github-plugin/pull/249
https://github.com/jenkinsci/jira-plugin/pull/343
https://github.com/jenkinsci/ec2-fleet-plugin/pull/267
https://github.com/jenkinsci/gearman-plugin/pull/14

[Basil Crow]

I found that we can easily remove Guava usages from SSH Build Agents
and Node Label Parameter as well:

https://github.com/jenkinsci/ssh-slaves-plugin/pull/228
https://github.com/jenkinsci/nodelabelparameter-plugin/pull/28

[Cyrille Le Clerc]

Hello,

I couldn't find the source code of the cloudfoundry-bosh-cli . This plugin can be ignored as it it completely outdated, the BOSH CLI it integrates with has been replaced a long time ago by a BOSH CLI v2.

Cyrille

[Robert Sandell]

FYI I have started some POC work to start masking core libraries from plugins, starting with guava, So that library upgrades in core can be made easier in the future. (JEP incoming as well)

https://github.com/rsandell/jenkins/tree/mask-libraries 

A lot of bits was already in place in core to do this in runtime, the challenging part of the work, I think, is to make the plugin dev environment work the same. Preferably not even have the libraries in compile scope for the plugins. So tips on how to achieve that part is welcome.

[Basil Crow]

Thanks to everyone who has been contributing to this effort, a number
of plugins have been prepared for the Guava update and released:

- Active Directory
- Audit Trail
- Blue Ocean (in alpha)
- Gearman
- Git client
- HTTP Request
- JUnit
- Lockable Resources
- Node Label Parameter
- Pipeline: Supporting APIs
- Role Strategy
- SCM API
- SSH Build Agents
- Timestamper

... and many more (apologies for missing them).

There are at least 25 additional plugins with confirmed Guava
regressions that must be prepared and released. Some of them have
significant adoption. I have enumerated them in this epic:

https://issues.jenkins.io/browse/JENKINS-65988

Within the epic, I have filed Jira issues for each specific plugin
where I am aware of a confirmed Guava regression. Each issue contains
details about what has changed along with a suggested solution. I have
made a best effort to assign these issues to the appropriate plugin
maintainer.

This list of issues is not exhaustive, but it is a good way to focus
current efforts. Most of these issues are easy to fix. If you are
interested in contributing to this effort, please assign yourself an
issue and open a PR!

[Basil Crow]

Eight CloudBees proprietary plugins appear to require preparation for
the Guava upgrade, at least in the latest released versions. I have
included my notes below. It would be helpful if a CloudBees employee
could prepare these plugins for the Guava upgrade and release new
versions.

---

CloudBees RBAC Auto Configurer 1.1.9 uses
com.google.common.base.CharMatcher#INVISIBLE.

CloudBees Cloud Foundry BOSH CLI 2.2 uses
com.google.common.base.Objects#toStringHelper.

Elasticsearch Reporter Feeder 2.263.0.1 uses the two-argument
com.google.common.util.concurrent.Futures#addCallback(ListenableFuture<V>
future, FutureCallback<? super V> callback) and the two-argument
com.google.common.util.concurrent.Futures#transform(ListenableFuture<I>
input, AsyncFunction<? super I,? extends O> function).

Elasticsearch Reporter Reporter 2.263.0.1 uses the two-argument
com.google.common.util.concurrent.Futures#addCallback(ListenableFuture<V>
future, FutureCallback<? super V> callback).

Operations Center Context 2.289.0.6 uses the four-argument
com.google.common.util.concurrent.TimeLimiter#callWithTimeout(Callable<T>
callable, long timeoutDuration, TimeUnit timeoutUnit, boolean
interruptible).

Operations Center Monitoring 2.289.0.3 uses the two-argument
com.google.common.util.concurrent.Futures#transform(ListenableFuture<I>
input, AsyncFunction<? super I,? extends O> function).

Operations Center Server 2.289.0.7 uses
com.google.common.util.concurrent.MoreExecutors#sameThreadExecutor.
You might also want to take a look at its use of
com.google.common.collect.MapMaker and switch to Caffeine.

Operations Center Single Sign-On 2.289.0.3 uses the four-argument
com.google.common.util.concurrent.TimeLimiter#callWithTimeout(Callable<T>
callable, long timeoutDuration, TimeUnit timeoutUnit, boolean
interruptible).

[James Nord]

Thanks Basil,

The operations center ones are released in lock step with a new LTS version, so I wouldn't worry too much about regressions there.

We have internal tickets to track all the work (we had previously focused on the OSS part as we knew it would take longer due to not being maintainers) the internal ones should all be addressed by the time the next (not the one in progress) LTS baseline is selected.

Regards

/James

[Tim Jacomb]

Bom is now passing with the guava change

https://github.com/jenkinsci/bom/pull/423

Just one change that's using an incremental currently and needs a release, which should be done soon-ish I believe: https://github.com/jenkinsci/workflow-basic-steps-plugin/pull/147

Thanks

Tim

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5dcf490b-6035-43e5-a577-cc3e91e285ben%40googlegroups.com.

[Basil Crow]

On Sat, Aug 7, 2021 at 12:12 PM Basil Crow <m...@basilcrow.com> wrote:
>
> There are at least 25 additional plugins with confirmed Guava
> regressions that must be prepared and released. Some of them have
> significant adoption. I have enumerated them in this epic:
>
> https://issues.jenkins.io/browse/JENKINS-65988

Today I filed PRs for most of the remaining issues in the
JENKINS-65988 epic. There are still two more PRs that need to be
filed:

1. We need a PR to migrate Jira from Guava's cache to Caffeine and to
remove its usage of Objects#firstNonNull. I do not intend to open such
a PR; volunteers are welcome.

2. We need a PR to migrate Gerrit Trigger from Guava's cache to
Caffeine. I do not intend to open such a PR; volunteers are welcome.

Apart from the above two tasks, I think that all the PRs that need to
be filed have been filed. Now it is just a matter of getting those PRs
merged and released, completing the JEP, and proceeding with the core
Guava upgrade.

[Olivier Lamy]

On Sun, 22 Aug 2021 at 10:28, Basil Crow <m...@basilcrow.com> wrote:

On Sat, Aug 7, 2021 at 12:12 PM Basil Crow <m...@basilcrow.com> wrote:
>
> There are at least 25 additional plugins with confirmed Guava
> regressions that must be prepared and released. Some of them have
> significant adoption. I have enumerated them in this epic:
>
> https://issues.jenkins.io/browse/JENKINS-65988

Today I filed PRs for most of the remaining issues in the
JENKINS-65988 epic. There are still two more PRs that need to be
filed:

1. We need a PR to migrate Jira from Guava's cache to Caffeine and to
remove its usage of Objects#firstNonNull. I do not intend to open such
a PR; volunteers are welcome.

I started looking at it. PR should be ready by the end of this week.

 

2. We need a PR to migrate Gerrit Trigger from Guava's cache to
Caffeine. I do not intend to open such a PR; volunteers are welcome.

Apart from the above two tasks, I think that all the PRs that need to
be filed have been filed. Now it is just a matter of getting those PRs
merged and released, completing the JEP, and proceeding with the core
Guava upgrade.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjp%2BOEiBFv1xLsnUmvizZrgyawoQzYgK0ru6nXEHe1wKOw%40mail.gmail.com.

[Basil Crow]

Thanks to everyone who has reviewed PRs, merged PRs, and shipped
releases. As the Jira epic shows, the majority of plugins have been
prepared, with a small number of critical PRs still awaiting merge and
an even smaller number still awaiting release. These should trickle in
over the coming weeks.

> I think that all the PRs that need to
> be filed have been filed. Now it is just a matter of getting those PRs
> merged and released, completing the JEP, and proceeding with the core
> Guava upgrade.

The JEP has been filed at jenkinsci/jep#375.

[Basil Crow]

With a new LTS release around the corner, I would like to set a target
for the merge and release of the Guava upgrade (i.e.,
jenkinsci/jenkins#5707 / jenkinsci/jep#375 / JENKINS-65988).

The plugin BOM test suite is passing against the incremental from
jenkinsci/jenkins#5707 as of today. I also ran a realistic end-to-end
test with some production configurations and builds and experienced no
issues.

I would like to target integration into one of the first few weeklies
_after_ the next LTS (i.e., the LTS _after_ 2.303.x) is released.

From my perspective, the following blockers remain:

- Merge of jenkinsci/jep#375 and assignment of a JEP number (waiting
for Owen Mehegan)
- Merge and release of jenkinsci/jenkins#5773 (just posted)
- Release of jenkinsci/gitlab-plugin#1150 (adoption in progress)
- Release of SonarSource/sonar-scanner-jenkins#203 (waiting for SonarSource)
- Release of jenkinsci/gerrit-trigger-plugin#451 (waiting for Robert Sandell)
- Merge and release of jenkinsci/ircbot-plugin#52 (waiting for Jim Klimov)
- Blog post (to be written)

I hope that these blockers will be completed in the next few weeks.

If anyone has any additional blockers to add to the list, please let me know.

[Jim Klimov]

Thanks for the analysis and effort, publishing irc-bot bump now.

Jim
--
Typos courtesy of K-9 Mail on my Android