Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

[RFC] Zuul Checks API plugin

319 views
Skip to first unread message

Adrià Vilanova Martínez

unread,
Apr 7, 2024, 12:46:34 PM4/7/24
to Repo and Gerrit Discussion
Hi everyone!

Lately I've been experimenting with building a Checks API plugin for Zuul. It's a project I wanted to embark on since the Checks plugin was deprecated.

There's an existing zuul-results-summary plugin, but it obtains the build info from the change messages made by Zuul, and displays the info in a custom tab. This new plugin retrieves the data from the existing Zuul API (so it can also show scheduled and in-progress builds) and integrates with the Checks API.

If someone wants to have a look at the source code and suggest improvements on the code, the design itself, or anything else, please feel free to do so! Here's the CL for the first prototype/MVP:
I based the design on the one of the Jenkins and Buildbucket plugins.

I want to make a lot of improvements, but it's in a usable state right now (and it's very useful to me!). Here's a demo in my personal Gerrit instance: https://gerrit.avm99963.com/c/infinitegforums/+/3868

One of the things I wanted to implement next is authentication functionality, so we can load build status from private Zuul tenants, but I'm a little bit stuck on how to do it... Since the plugin is built with Bazel and I don't have much experience with it, I don't know how I would add an OIDC client NPM library to handle authentication. My first impression is it would need to be added at the Gerrit level in the package.json file? But the package might not be useful for other plugins.

Also, if you like the idea behind the plugin and you think it would be useful in general, in the future it would be great if it was hosted in the official Gerrit instance in place of my personal instance.

Cheers :-)

Nasser Grainawi

unread,
Apr 8, 2024, 1:05:02 PM4/8/24
to Adrià Vilanova Martínez, Repo and Gerrit Discussion
On Sun, Apr 7, 2024 at 10:46 AM Adrià Vilanova Martínez <m...@avm99963.com> wrote:
Hi everyone!

Lately I've been experimenting with building a Checks API plugin for Zuul. It's a project I wanted to embark on since the Checks plugin was deprecated.

Thanks for trying this out and sharing it!
 

There's an existing zuul-results-summary plugin, but it obtains the build info from the change messages made by Zuul, and displays the info in a custom tab. This new plugin retrieves the data from the existing Zuul API (so it can also show scheduled and in-progress builds) and integrates with the Checks API.

If someone wants to have a look at the source code and suggest improvements on the code, the design itself, or anything else, please feel free to do so! Here's the CL for the first prototype/MVP:
I based the design on the one of the Jenkins and Buildbucket plugins.

I want to make a lot of improvements, but it's in a usable state right now (and it's very useful to me!). Here's a demo in my personal Gerrit instance: https://gerrit.avm99963.com/c/infinitegforums/+/3868

One of the things I wanted to implement next is authentication functionality, so we can load build status from private Zuul tenants, but I'm a little bit stuck on how to do it... Since the plugin is built with Bazel and I don't have much experience with it, I don't know how I would add an OIDC client NPM library to handle authentication. My first impression is it would need to be added at the Gerrit level in the package.json file? But the package might not be useful for other plugins.

Yes, I think if you're doing an "in-tree" build, you would need to add the dependency to that plugins/package.json file. If you do a standalone build (using the bazlets repo in your WORKSPACE file) then you can have your own package.json in the plugin. Does anyone know if there's a way to do that with an in-tree build?
 

Also, if you like the idea behind the plugin and you think it would be useful in general, in the future it would be great if it was hosted in the official Gerrit instance in place of my personal instance.

I would be fine with that. Anyone have concerns?
 

Cheers :-)

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/5c207335-8d15-4f85-b52c-76a3d364407cn%40googlegroups.com.

Adrià Vilanova Martínez

unread,
Apr 3, 2025, 3:10:10 AMApr 3
to Nasser Grainawi, Repo and Gerrit Discussion, Engdahl, Lukas, Nakama, Yoko Sarah, Bigdeli, Simon
Hi again!

I wanted to provide an update on the Zuul Checks API plugin. I paused its development due to other priorities. But recently a team of interns at Volvo Cars (Lukas, Sarah and Simon) reached out to me. They were planning to develop a similar plugin as part of their internship project and are interested in incorporating authentication support and further contributing to it.

We've decided to collaborate and continue developing it as an open-source project for the benefit of the community. Ideally, we'd like for it to be hosted in Gerrit's instance on git-on-borg.

Given that there were no concerns raised in this thread (though it has been almost a year), would it be possible to proceed with hosting the plugin on Gerrit's instance?

Thanks!

Daniele Sassoli

unread,
Apr 3, 2025, 3:24:26 AMApr 3
to Repo and Gerrit Discussion
On Thursday, 3 April 2025 at 08:10:10 UTC+1 m...@avm99963.com wrote:
Hi again!

I wanted to provide an update on the Zuul Checks API plugin. I paused its development due to other priorities. But recently a team of interns at Volvo Cars (Lukas, Sarah and Simon) reached out to me. They were planning to develop a similar plugin as part of their internship project and are interested in incorporating authentication support and further contributing to it.

We've decided to collaborate and continue developing it as an open-source project for the benefit of the community. Ideally, we'd like for it to be hosted in Gerrit's instance on git-on-borg.

Given that there were no concerns raised in this thread (though it has been almost a year), would it be possible to proceed with hosting the plugin on Gerrit's instance?
 
I'm not in charge of the official Gerrit instance, although I wouldn't see a problem with adding such plugin.
However, if you're looking for a quick solution to get away from your self-hosted gerrit instance, you could consider gerrithub[1] as an option for the time being, as you can simply add your own projects.
However, you'd first need to create a github repo for it, gerrithub imports projects from there.

[1] https://review.gerrithub.io/dashboard/self

Luca Milanesio

unread,
Apr 3, 2025, 3:42:05 AMApr 3
to Repo and Gerrit Discussion, Luca Milanesio
Hi Adrià,

Thanks for sharing this update on the Zuul Checks plugin.

On 3 Apr 2025, at 08:24, Daniele Sassoli <daniele...@gmail.com> wrote:



On Thursday, 3 April 2025 at 08:10:10 UTC+1 m...@avm99963.com wrote:
Hi again!

I wanted to provide an update on the Zuul Checks API plugin. I paused its development due to other priorities. But recently a team of interns at Volvo Cars (Lukas, Sarah and Simon) reached out to me. They were planning to develop a similar plugin as part of their internship project and are interested in incorporating authentication support and further contributing to it.

We've decided to collaborate and continue developing it as an open-source project for the benefit of the community. Ideally, we'd like for it to be hosted in Gerrit's instance on git-on-borg.

Given that there were no concerns raised in this thread (though it has been almost a year), would it be possible to proceed with hosting the plugin on Gerrit's instance?

Apologies for having missed this post and this useful plugin.
We do currently run the zuul-results-summary [2] on GerritHub.io and we would be more than happy to have it converted into using the Checks API.

Bear in mind that the zuul-results-summary [2] was built by Ian Wienand (Red Hat) 5 years ago when the Checks API did not exist, therefore it used the *best of technology available* at that time.
I am sure that today the approach used at that time can be seen as obsolete and would need rethinking, so your efforts and prototype are really appreciated.

My only concern is that we already have three plugins for Zuul integration with Gerrit [3], and the question is “do we really one yet another one”?
If the zuul-results-summary was solving a different problem, then yes, but IMHO it is solving *the same problem* and just need refactoring.

@Adrià, would you be happy to work on [2]?

However, if you're looking for a quick solution to get away from your self-hosted gerrit instance, you could consider gerrithub[1] as an option for the time being, as you can simply add your own projects.
However, you'd first need to create a github repo for it, gerrithub imports projects from there.

Sure, that would also work, but I believe Adrià is looking for a more collaborative team working on it, that’s why working on [2] would make more sense rather than creating “yet-another-plugin”.

Luca.


Thanks!


On Mon, Apr 8, 2024, 19:04 Nasser Grainawi <nasser....@linaro.org> wrote:


On Sun, Apr 7, 2024 at 10:46 AM Adrià Vilanova Martínez <m...@avm99963.com> wrote:
Hi everyone!

Lately I've been experimenting with building a Checks API plugin for Zuul. It's a project I wanted to embark on since the Checks plugin was deprecated.

Thanks for trying this out and sharing it!
 

There's an existing zuul-results-summary plugin, but it obtains the build info from the change messages made by Zuul, and displays the info in a custom tab. This new plugin retrieves the data from the existing Zuul API (so it can also show scheduled and in-progress builds) and integrates with the Checks API.

If someone wants to have a look at the source code and suggest improvements on the code, the design itself, or anything else, please feel free to do so! Here's the CL for the first prototype/MVP:
I based the design on the one of the Jenkins and Buildbucket plugins.

I want to make a lot of improvements, but it's in a usable state right now (and it's very useful to me!). Here's a demo in my personal Gerrit instance: https://gerrit.avm99963.com/c/infinitegforums/+/3868

One of the things I wanted to implement next is authentication functionality, so we can load build status from private Zuul tenants, but I'm a little bit stuck on how to do it... Since the plugin is built with Bazel and I don't have much experience with it, I don't know how I would add an OIDC client NPM library to handle authentication. My first impression is it would need to be added at the Gerrit level in the package.json file? But the package might not be useful for other plugins.

Yes, I think if you're doing an "in-tree" build, you would need to add the dependency to that plugins/package.json file. If you do a standalone build (using the bazlets repo in your WORKSPACE file) then you can have your own package.json in the plugin. Does anyone know if there's a way to do that with an in-tree build?
 

Also, if you like the idea behind the plugin and you think it would be useful in general, in the future it would be great if it was hosted in the official Gerrit instance in place of my personal instance.

I would be fine with that. Anyone have concerns?

 

Cheers :-)

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/5c207335-8d15-4f85-b52c-76a3d364407cn%40googlegroups.com.

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

James E. Blair

unread,
Apr 3, 2025, 1:15:18 PMApr 3
to repo-d...@googlegroups.com
On 4/3/25 00:41, Luca Milanesio wrote:
Hi Adrià,

Thanks for sharing this update on the Zuul Checks plugin.

On 3 Apr 2025, at 08:24, Daniele Sassoli <daniele...@gmail.com> wrote:



On Thursday, 3 April 2025 at 08:10:10 UTC+1 m...@avm99963.com wrote:
Hi again!

I wanted to provide an update on the Zuul Checks API plugin. I paused its development due to other priorities. But recently a team of interns at Volvo Cars (Lukas, Sarah and Simon) reached out to me. They were planning to develop a similar plugin as part of their internship project and are interested in incorporating authentication support and further contributing to it.

We've decided to collaborate and continue developing it as an open-source project for the benefit of the community. Ideally, we'd like for it to be hosted in Gerrit's instance on git-on-borg.

Given that there were no concerns raised in this thread (though it has been almost a year), would it be possible to proceed with hosting the plugin on Gerrit's instance?

Apologies for having missed this post and this useful plugin.
We do currently run the zuul-results-summary [2] on GerritHub.io and we would be more than happy to have it converted into using the Checks API.

Bear in mind that the zuul-results-summary [2] was built by Ian Wienand (Red Hat) 5 years ago when the Checks API did not exist, therefore it used the *best of technology available* at that time.
I am sure that today the approach used at that time can be seen as obsolete and would need rethinking, so your efforts and prototype are really appreciated.

My only concern is that we already have three plugins for Zuul integration with Gerrit [3], and the question is “do we really one yet another one”?
If the zuul-results-summary was solving a different problem, then yes, but IMHO it is solving *the same problem* and just need refactoring.

Hi,

While I share your goal of replacing the existing plugin with a Checks API based plugin, there are some open questions about how that will be accomplished in various Zuul environments, and what API changes may be needed to Zuul itself to accommodate it.  Given that we have happy users of the existing plugin and we may want or need to maintain it and continue to use it while exploring the new approach, I think it would be good to keep the new plugin in a separate repo.  Aside from its similar purpose, it won't share any development history anyway, so that seems like a good fit.  When it's obviously suitable for all cases, we can retire the old one.

I think hosting it on Gerrit's instance is a fine choice.  It has been an effective collaboration space for Zuul and Gerrit devs.

-Jim


Nasser Grainawi

unread,
Apr 3, 2025, 1:47:29 PMApr 3
to James E. Blair, repo-d...@googlegroups.com
Is anyone using the zuul-status plugin? If not, that also seems like it was intended to do the same thing as the checks API does now.

I think it would also be fine to use a set of checks-specific branches in either plugin until there's convergence. That would retain the current functionality for anyone still wanting it, but provide a parallel path to using the newer functionality and eventual replacement of the old.
 

I think hosting it on Gerrit's instance is a fine choice.  It has been an effective collaboration space for Zuul and Gerrit devs.

-Jim


--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

James E. Blair

unread,
Apr 3, 2025, 2:01:55 PMApr 3
to repo-d...@googlegroups.com
On 4/3/25 10:47, 'Nasser Grainawi' via Repo and Gerrit Discussion wrote:


On Thu, Apr 3, 2025 at 11:15 AM James E. Blair <j...@acmegating.com> wrote:
On 4/3/25 00:41, Luca Milanesio wrote:
Hi Adrià,

Thanks for sharing this update on the Zuul Checks plugin.

On 3 Apr 2025, at 08:24, Daniele Sassoli <daniele...@gmail.com> wrote:



On Thursday, 3 April 2025 at 08:10:10 UTC+1 m...@avm99963.com wrote:
Hi again!

I wanted to provide an update on the Zuul Checks API plugin. I paused its development due to other priorities. But recently a team of interns at Volvo Cars (Lukas, Sarah and Simon) reached out to me. They were planning to develop a similar plugin as part of their internship project and are interested in incorporating authentication support and further contributing to it.

We've decided to collaborate and continue developing it as an open-source project for the benefit of the community. Ideally, we'd like for it to be hosted in Gerrit's instance on git-on-borg.

Given that there were no concerns raised in this thread (though it has been almost a year), would it be possible to proceed with hosting the plugin on Gerrit's instance?

Apologies for having missed this post and this useful plugin.
We do currently run the zuul-results-summary [2] on GerritHub.io and we would be more than happy to have it converted into using the Checks API.

Bear in mind that the zuul-results-summary [2] was built by Ian Wienand (Red Hat) 5 years ago when the Checks API did not exist, therefore it used the *best of technology available* at that time.
I am sure that today the approach used at that time can be seen as obsolete and would need rethinking, so your efforts and prototype are really appreciated.

My only concern is that we already have three plugins for Zuul integration with Gerrit [3], and the question is “do we really one yet another one”?
If the zuul-results-summary was solving a different problem, then yes, but IMHO it is solving *the same problem* and just need refactoring.

Hi,

While I share your goal of replacing the existing plugin with a Checks API based plugin, there are some open questions about how that will be accomplished in various Zuul environments, and what API changes may be needed to Zuul itself to accommodate it.  Given that we have happy users of the existing plugin and we may want or need to maintain it and continue to use it while exploring the new approach, I think it would be good to keep the new plugin in a separate repo.  Aside from its similar purpose, it won't share any development history anyway, so that seems like a good fit.  When it's obviously suitable for all cases, we can retire the old one.

Is anyone using the zuul-status plugin? If not, that also seems like it was intended to do the same thing as the checks API does now.

I think it would also be fine to use a set of checks-specific branches in either plugin until there's convergence. That would retain the current functionality for anyone still wanting it, but provide a parallel path to using the newer functionality and eventual replacement of the old.
 

If branches of the same repo were used for both, would it be either difficult technically, or confusing from a user perspective, to have both plugins built and available via the plugin manager simultaneously?

-Jim


Nasser Grainawi

unread,
Apr 3, 2025, 2:05:25 PMApr 3
to James E. Blair, repo-d...@googlegroups.com
It could be? There's some technical complication to ensure you install them with unique names and to ensure each plugin uses its installed name instead of something hard-coded, but you could also probably hard-code a different name for the newer one while it's in a separate branch.
 

-Jim


--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

Luca Milanesio

unread,
Apr 3, 2025, 3:58:12 PMApr 3
to repo-d...@googlegroups.com, Luca Milanesio
IMHO we should try to avoid proliferation and have a common approach, however, I do share Jim’s concerns in keeping the current one working until we have “a new one fully working” which would also mean to have a compatibility matrix with the Zuul APIs.

What about having a mono-repo approach instead, with different folders on the same branch?
Having different folders would also mean to be able to start sharing common parts of the code.

Lastly, I agree we should be able to have them both installed at the same time.

Luca.

James E. Blair

unread,
Apr 3, 2025, 6:29:49 PMApr 3
to repo-d...@googlegroups.com

As a user and maintainer-of-last-resort of the zuul-results-summary plugin (that is to say, that if it needed maintenance and no one else stepped up to do it, I would do so), I don't want anything about it to change.  It works exactly as it should, and does all that we need and have needed from it for years.  It operates under a completely different philosophy than the Checks API -- it just re-formats information already available in Gerrit via the review message.  It's the closest thing we have in Gerrit to how Zuul reports results to Github, etc.  It will work in every environment, regardless of network topology, authentication, and even to a very large extent, Zuul version.  It doesn't do anything like the plugin that is being discussed to replace it.

It is in use by Zuul's own Gerrit.  We consume it by building it with our own Gerrit image, so any changes to the structure of the repo would mean we need to change how we build it.

As a user of and contributor to this plugin, I ask: please don't use the zuul-results-summary repo for the new plugin.

I think there should be a place for people to collaborate on a Checks API approach.  If it becomes viable in the future and we consider zuul-results-summary obsolete, I will have no problem retiring the z-r-s repo to help reduce confusion.  But we don't know if or when that will happen, and in the mean time, I don't want to risk disruption.

I am not sure I understand the resistance to just giving this new piece of software its own repo.  I understand that it's confusing from a user point of view to have so many zuul-related plugins, but they all do different things, have different authors, and different levels of maintenance.

It makes a lot of sense to me that collaboration on Gerrit plugins should happen closest to the Gerrit community, centered around Gerrit's own Gerrit.  So to me, the ideal is a new repo there.

-Jim


Luca Milanesio

unread,
Apr 3, 2025, 7:11:12 PMApr 3
to repo-d...@googlegroups.com, Luca Milanesio
That is a very good point indeed: “[…] will work in every environment, regardless of network topology […]”

That is in general a fundamental issue with the overall Checks API “client-only” plugins: they need to query the CI system API *directly* which could be an issue for two reasons:
1) The user navigating the Gerrit UI may not have direct connectivity to the CI system. That’s the typical scenario in GerritHub.io as well, where the Gerrit UI is public, but the CI often is not and therefore a version of the Zuul plugin that relies purely on the Zuul API won’t work.
2) The user authentication with Gerrit could be very different to the way the gerrit authenticates with Zuul: having a 1:1 relationship and the same identity system used for Gerrit and Zuul would be a signfiicant limitation

Honestly that’s also the reason why the “deprecated” Checks plugin is still very much used in gerrit-review.googlesource.com because we don’t give public access to the Jenkins API unless you are authenticated and trusted. Also, the Gerrit-CI instance of Jenkins uses GitHub authentication whilst gerrit-review is based on Google OAuth.

  It doesn't do anything like the plugin that is being discussed to replace it.

True, and I believe this discussion was useful though to analyse the comparison between zuul-results-summary and the new proposed plugin, which I believe fails to address the valid points raised by Jim.

@Adrià have you assessed these points and how are you going to address them? Would the new Zuul Checks API plugin be able to access private CI instances or public ones having an authentication system different than Gerrit?

It is in use by Zuul's own Gerrit.  We consume it by building it with our own Gerrit image, so any changes to the structure of the repo would mean we need to change how we build it.

As a user of and contributor to this plugin, I ask: please don't use the zuul-results-summary repo for the new plugin.

Ack.

I think there should be a place for people to collaborate on a Checks API approach.  If it becomes viable in the future and we consider zuul-results-summary obsolete, I will have no problem retiring the z-r-s repo to help reduce confusion.  But we don't know if or when that will happen, and in the mean time, I don't want to risk disruption.

Understood, however, I believe that we had too many “ghost projects” (I am speaking as one of the guilty developers leaving some of them on the road) and I would be reluctant to create yet another one that may cause confusion.
Why don’t we wait for the new Zuul Checks API plugin to become “more usable” and then host it on gerrit-review afterwards? We will then know more if that’s a new problem or not.

Possibly, it could consume the messages posted by Zuul in the changes, so that it could be also usable when Zuul isn’t accessible or have a different authentication system ;-)
That would resolve the issue of the existing changes that already contains feedback from Zuul.

Imagine that we rollout tomorrow on GerritHub.io the brand-new-shiny Zuul Checks API plugin … what would happen to all the existing changes that contains legacy comments?
How would existing users be able to see the status of them?
What if the Zuul posting them was not reachable anymore?
What about traceability?

Those are all questions that need to be answered I believe.

I am not sure I understand the resistance to just giving this new piece of software its own repo.  I understand that it's confusing from a user point of view to have so many zuul-related plugins, but they all do different things, have different authors, and different levels of maintenance.

It makes a lot of sense to me that collaboration on Gerrit plugins should happen closest to the Gerrit community, centered around Gerrit's own Gerrit.  So to me, the ideal is a new repo there.

I don’t believe there is a problem in having multiple repositories for multiple plugins that do different things.
It is just worth asking “is this really a new problem we are trying to solve, or just an evolution of an existing problem that has been already solved?”

Based on @Adrià’s feedback, we’ll know which one is it :-) and it is always good to ask questions and trigger fruitful conversation.

Luca.

James E. Blair

unread,
Apr 3, 2025, 7:22:14 PMApr 3
to repo-d...@googlegroups.com

+1 to all the above -- I'll just add that if someone wanted to start an effort to update the existing zuul-results-summary plugin to consume input from the review messages (as it does now) but present that info via the Checks API -- I think that would likely make sense as an effort to do in the zuul-results-summary repo (probably it could be done as a change or change series).  It is the different approach on getting input that makes it a new, speculative effort from my POV.

-Jim


Antoine Musso

unread,
Apr 4, 2025, 12:04:40 PMApr 4
to Adrià Vilanova Martínez, Nasser Grainawi, Repo and Gerrit Discussion, Engdahl, Lukas, Nakama, Yoko Sarah, Bigdeli, Simon
Le 03/04/2025 à 08:40, Adrià Vilanova Martínez a écrit :
Hi again!

I wanted to provide an update on the Zuul Checks API plugin. I paused its development due to other priorities. But recently a team of interns at Volvo Cars (Lukas, Sarah and Simon) reached out to me. They were planning to develop a similar plugin as part of their internship project and are interested in incorporating authentication support and further contributing to it.

We've decided to collaborate and continue developing it as an open-source project for the benefit of the community. Ideally, we'd like for it to be hosted in Gerrit's instance on git-on-borg.

Given that there were no concerns raised in this thread (though it has been almost a year), would it be possible to proceed with hosting the plugin on Gerrit's instance?

Hello,

I am working for the Wikimedia Foundation and we have been using Zuul since its early days. I did wrote a Checks API plugin which reads from Zuul (v2.5) status since I eventually got fed up with the old HTML summary table and wanted to offer the developers a dynamic feedback.  I will give an historical perspective and quick presentation of the existing plugins. I will then present the two JavaScript plugins we use to integrate Zuul messages and Zuul status API as Gerrit Checks.I will conclude with a potential shortfalls in the Check API and ideas for the future.

Existing plugins

This might sounds like a rehearsal, but I like to ensure we have the same context and this list of plugins can be used as a later reference.

plugins/zuul
The first plugin, written by Khai Do and started in 2016. He was then a member of the OpenStack Infrastructure team which also wrote Zuul. The plugin processes the `Depends-On` header in commit message, a concept invented by Zuul to express a dependency between changes in different repositories. When a change has dependencies or is needed by other changes, a small widget is displayed on the right side of the commit message (below the Relation Chain when it exists). The dependencies are exposed via a REST API and our deployment tool rely on that. The plugin does NOT enforce the dependency, that is done by Zuul.  There are some old screenshots at src/main/resources/Documentation/about.md.

Wikimedia heavily relies on it, at our core is MediaWiki which is extended by several plugins (we call them extensions). Whenever a refactor or breaking change is made to MediaWiki we need to align plugins and our CI ensures everything fits together. One would set the breaking change to depends-on the patches made to other plugins and can conveniently see the progress. A recent live example is how we would remove a method from MediaWiki: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1129981 (and CI would fail until you have the depends-on for all affected repos).

plugins/depends-on
Similar to the original Zuul plugin, this one started in 2021. I don't know the plugin, but the code base looks a bit more modern, adds support for search operators (independson and hasdependson).
I am not familiar with it. The Zuul plugin does not even depend on Zuul, so it could have been improved upon.  Then sometime it makes sense to start from scratch and the plugin has potentially different use cases.  The documentation at src/main/resources/Documentation/about.md.

https://github.com/dburm/pg-test-result-plugin
From 2019, the client side Gerrit plugin parses change messages to find builds reported by a CI system and display them in HTML table beneath the commit message. This was added to our setup as we upgraded to Gerrit 3 and has been a nice improvement. The plugin was added by Paladox

plugins/zuul-status
Started in 2019 by Paladox. When the previous pg-test-result-plugin scrapped message, zuul-status queries the Zuul change status API and add live updates about the progress of the jobs and crafts an HTML table. I assume it was the same table format.  That offers a view about the ongoing progress while the pg-test-result-plugin provides the historical view. We had it installed and it provided the same view as pg-test-result-plugin, but live.
I don't think we had it deployed on our instance, I can't remember.

plugins/zuul-results-summary
Started in late 2020 by Ian Wienand from RedHat. This is the plugin in action on https://review.opendev.org/. It integrates the result table as a tab along Files and Comments and fit naturally in Gerrit UI. The plugin parses the messages reported by Zuul and thus replaces pg-rest-result-plugin.  This does not fetch the ongoing builds from the Zuul status API.

I did consider the plugin but I needed it to be customizable, to support Zuul v2.5. I wanted to offer live updates to developers and to use the Checks API to expose our non-Zuul CI systems.

Wikimedia plugins

I wrote those plugins. I went to use vanilla JavaScript typechecked against Gerrit typescript-api and tested with QUnit.

wm-checks-api
This is a rewrite of pg-test-result-plugin in vanilla "modern" JavaScript. Like zuul-results-summary by OpenDev, it parses the messages reported by CI bots and generates the data structure for the Checks API. The code processing Zuul is in the ZuulProcessor class, for some bots reporting a simple "Build fail", crafting the Check result is straightfoward.
The code has unit tests which gives an idea of how it works.
The code is licensed under the MIT license which was the license picked by Dmitry the author of the pg-test-result-plugin.
With that plugin I have integrated the Zuul reported messages as Checks results, and also added support for our non-Zuul CI systems.

wm-zuul-status
This plugin uses our Zuul 2.5 change status API and inject a live view of the progress of the jobs.  It is similar to plugins/zuul-status though I wrote it from scratch.  If there are changes being processed you can see it in action from https://integration.wikimedia.org/zuul/ , click a change,  click the Checks tab and you should see some fancy ASCII art progress bar. The colored chips below and the table are entirely generated by Gerrit from the data structure it is being fed with.

A screenshot: https://phabricator.wikimedia.org/T214068#8706674 inserted below:


The code has unit tests , should be easy to port to a later Zuul version. I think it can be a great base to build your plugin.


Shortfalls and design choice

The Checks API is designed with only two levels: Runs which are mutables and Result which are expected to be final and immutable. You can see how it is used by Google on https://chromium-review.googlesource.com/ (look at project:chromium/src).  I could not find any great example https://android-review.googlesource.com/ but I haven't looked too much.

Zuul has pipelines, builds, Ansible tasks and one could argue they'd want to exposes the test results directly in Gerrit (using JUnit/Checkstyle etc). The Checks API only offers two levels: Run and Result so you have to choose how to do the mapping.

I initially considered assigning jobs as Run then add the test results as well Results. Doing so I would have lost the pipelines information and with dozens of jobs the list of runs would have been confusing. I would also have needed to write converters from junit.xml/checkstyle.xml to CheckResults.
I really wanted to show the ongoing progress but a Run does not offers that. I went to write my own ASCII art progress bar and updating the Results summary on each fetch :]

Modern CI systems have pipelines depending on each others, they can have a pipeline of Runs (checks > test > gate > postmerge) and some even have a DAG. But that can't not be represented.

It would be nice to add an extra level beside Runs>Results, though I don't have any idea how it can be represented.  Meanwhile one has to make choices and my choice of mapping Buildset to Run and Builds to Results is a good fit.

Conclusion

We could reuse plugins/zuul given its short name and since that was the first around. The plugins/zuul-results-summary one can then be merged-in which will bring support for the result summary from messages posted to Gerrit. That is itself an improvement. That would probably need some configuration settings to be added.

For conversion to Checks API I'd look at the wm-checks-api code which was already a conversion of an older results summary table. Support for the Checks API should not be too complicated to add and maybe my code can partly be borrowed. I highly recommend to write unit tests and checking against TypeScript API, that has dramatically helped me.

To convert the Zuul status API to Check Runs/Results, it is a solved problem thanks to wm-zuul-status I did that with Zuul 2.5 and adjstements might be needed. The code is MIT, but maybe that can be relicensed to Apache 2.

I am more than happy to follow up here by email and would be happy to chat (IRC Libera.chat: #wikimedia-releng / hashar, Gerrit Code Review discord, a Video chat, live).

That was long! Happy week-end ;)

-- 
Antoine "hashar" Musso
Release Engineering
Reply all
Reply to author
Forward
0 new messages