GSoC - Plugins Health Scoring project

12 views
Skip to first unread message

Adrien Lecharpentier

unread,
Jun 1, 2022, 7:00:00 AMJun 1
to jenkin...@googlegroups.com
Hello,

as the Plugin Health Scoring System project has been selected for this year GSoC, I wanted to start the discussion here about hosting.

At this moment, we don't have a specific requirement, but for the project, we might need a database (postgresql most probably), some process running and a fork of plugins.jenkins.io.
We would like to know how we can have that hosted by the Jenkins project but without compromising any of the existing infra and without having to bother you too much when we need to deploy something.

Everything we might want to deploy could be considered as disposable (because reproducible), so I don't think we would need any backup policy. 

What I'm asking here:
 1. is it possible to host the described services on Jenkins infra?
 2. how could it be hosted so that we have a limited impact on the infra team? (having only a "factory" reset for everything is good for me)
 3. how could we deploy / control what is deployed without impacting you?

Thank you very much for your help,
-- Adrien

Gavin Mogan

unread,
Jun 1, 2022, 12:31:05 PMJun 1
to Jenkins Infrastructure
Short version:
yes

New repo that has a jenkinsfile that builds a docker file (lots of
existing tooling for that).
Infra ci that updates kubernetes configuration when docker image
updates (already exists)
The only unknow right now is CI approving the new release, and in the
past we've done code owners for certain directories, so thats doable
too.

Thats assuming a dynamic service that needs to keep running.

If its just static files being published (like plugin sites), we can
create a netlify bucket + domain name and ci can update that bucket at
will.

That being said, I don't think plugin site should be running health
score, health score should provide json/api files that cna be consumed
by plugin site. Then if you want to make changes to plugin site and
demo it, its just a matter of opening a PR, plugin site has preview
urls.

Gavin
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Infrastructure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkins-infr...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/jenkins-infra/CAKwJSvzv4nZeXVeMXvFfDVCfrdNHJv5%3DWhO%2BB%2BD72LOkVsia0w%40mail.gmail.com.

Adrien Lecharpentier

unread,
Jun 7, 2022, 9:04:12 AMJun 7
to Jenkins Infrastructure
Thank you Gavin for your feedback.

What we have in mind at the moment is to have some code that generates a file, like the update-center.json, which will be used by plugins.j.io to display information.
We will have a CRON based process which will read the update-center.json to get details on plugins and a general process which will analyze the

Should the code for this project be in the jenkins-infra or in the jenkinsci organization in GitHub? I can create the repository in jenkins-infra but not in jenkinsci, where I lack some permissions apparently.

Best regards,
-- Adrien

Gavin Mogan

unread,
Jun 8, 2022, 2:42:39 AMJun 8
to Jenkins Infrastructure
> We will have a CRON based process which will read the update-center.json to get details on plugins and a general process which will analyze the

Other than you not finishing the sentence. My suggestion is to just
deploy the static file to netlify. That being said nothing with write
access to infra runs outside the vpn, so while it would work, it would
be hard to diagnose until you get vpn access, and i'm not sure how
that works.

For now you could build and archive the json file on ci.jenkins.io
(which is how the old plugin data worked for the plugin site). plugin
site would just grab latestSuccess/archives/plugins.json.gz

Actually as I write it, using ci.jenkins.io is probably the easiest to
do and isn't a big burden on infra, at least while its only plugin
site fetching that data once on compile.

> Should the code for this project be in the jenkins-infra or in the jenkinsci organization in GitHub? I can create the repository in jenkins-infra but not in jenkinsci, where I lack some permissions apparently.

My vote is jenkins-infra, and should get a
https://github.com/jenkins-infra/helpdesk/ ticket for tracking. I
generally feel if it has to do with jenkins.io its infra, and if its
something installed locally its jenkinsci.

Just my opinions though

On Tue, Jun 7, 2022 at 6:04 AM Adrien Lecharpentier
> To view this discussion on the web, visit https://groups.google.com/d/msgid/jenkins-infra/94f6ac25-bb13-449c-978c-c174baf17df4n%40googlegroups.com.

Adrien Lecharpentier

unread,
Jun 8, 2022, 4:08:27 AMJun 8
to jenkin...@googlegroups.com
>> We will have a CRON based process which will read the update-center.json to get details on plugins and a general process which will analyze the
> Other than you not finishing the sentence.

Sorry about that.

The current architecture diagram built by Dheeraj can be found here: https://docs.google.com/document/d/1Dxyli1LPlHdFxLoE9zFtr_3bTjnwQDMZGCxcGS79Z_I/edit

We would have a process reading the update-center.json daily and populating a database with new details about plugins (GitHub repository, name, and others). From there, we would trigger an analyze of a repository if something changed since the last analyze. The analyzing part would populate details about the repository into the database. Based on those details, we would grade each plugin using some weighting and that part would generate a file which would be read by plugins.jenkins.io (at first) but which could also be read by the plugin manager of a controller at some point in the project.

Having the last part run on ci.jenkins.io could be possible, but not the rest. Having the database seems important for us in order to limit the time to generate the grades. We don't want to reanalyze all 1800 plugins everyday, it would take too much time and requests to GitHub API. Also, the architecture is split by we don't aim to go into the micro-service universe, it is just to detail things.

-- Adrien

Reply all
Reply to author
Forward
0 new messages