Hosting a Java library for JEP-309: Bill of Materials (HOSTING-1050)

[Oleg Nenashev]

Hi all,

I am working on optimizing plugin bundling in Jenkinsfile Runner (issue 389, 390). Jenkinsfile Runner supports a performance-optimized mode when Jenkins core and its plugins are packaged as JAR files using Maven Assembly Plugin. Unfortunately Jenkins still needs an exploded HPI file to read the plugin manifests.

To resolve this, I would need to apply patches in Jenkinsfile Runner plugin manager and in Maven HPI Plugin which currently generates the list of dependencies included into Jenkinsfile Runner. Later I will also need to modify the Plugin Installation Manager Tool so that it can also recognize this form of plugin bundling. So it needs an interexchange format and a shared library so that tools do not need to reimplement the model and IO logic.

We already have Jenkins JEP-309: Bill of Materials which defines a YAML file format for exchanging data between Jenkins tools. A few years ago I added support for YAML BOM in Custom WAR Packager. I would like to use this format (not ideal, but works), and I started detaching the model from CWP to a library.

• Library draft: https://github.com/oleg-nenashev/lib-bom-jep309 
  
• Hosting request: https://issues.jenkins.io/browse/HOSTING-1050

Would appreciate feedback from other contributors who might be interested to use JEP-309 BOMs in their tools

Thanks in advance,

Oleg

[Jesse Glick]

On Mon, Nov 23, 2020 at 9:00 AM Oleg Nenashev <o.v.ne...@gmail.com> wrote:
> Jenkins still needs an exploded HPI file to read the plugin manifests.

Can you elaborate? If there is some optimization in Jenkins core that
would be beneficial, why not just do it now?

[Oleg Nenashev]

> Can you elaborate? If there is some optimization in Jenkins core that would be beneficial, why not just do it now? 

 

What I am doing could be applied to the core when immutable configurations are used, e.g. for plugins and Jenkins core defined in Docker images. If we preprocess plugin manifests, it can slightly improve the startup time. Same for more efficient build-time extension indexing that I also keep in mind. 

I would not like to do these changes right away in the core because of the following reasons:

• Jenkins Docker images do not have big size overhead for HPIs, compared to Jenkinsfile runner which has different packaging flow (without some tweaks, plugins are basically included twice).
• Performance improvement would be available only for a subset of users
• Performance improvements have relatively low impact for classic Jenkins setups. If we wanted to improve startup time there, we should be looking at lazy job loading, not at plugin manifest indexing
• Last but not least, the implementation overhead and quality requirements are much higher for the Jenkins core than for JFR which is still in Beta

Based on the reasons above, I would like to have a working implementation in JFR before we discuss including bits of it into the Jenkins core.

BR, Oleg