[GSOC 2020 PROJECT IDEA] Jenkins distribution customize service

[Rick]

Hi team,

I'd like to share an idea with you. There are some original discussions from the GSoC thread here.

For many users, they download Jenkins first, then select some plugins and config them. It might take a lot of time, like hours. But if we can get a perfect Jenkins distribution which contains all we need, it can save that time for us. Yes, I propose an out of the box solution.

The service could be like this, a website offered as https://customize.jenkins.io. Plus, it should be self-host. People can select the following configurations:

• Jenkins core version
• plugins
• common configuration, user/password, update-center site e.g.
• plugin based configuration, Kubernetes, Sonarqube plugin config e.g.
• multi-form package, jenkins.war or docker image
• other things

Two projects are expected backend and frontend. I think https://github.com/jenkinsci/custom-war-packager already did a lot of works. We can reuse it in this project. The backend should provide the modern Restful API. We can start the backend project with SpringBoot, it can save a lot of time.

About the frontend project, I don't have too many experiences on it. For example, we can start it by React framework. If you're instrested in it, please help to add more ideas on it.

Any feedbacks are very appreciated.

Best regards,

Rick

--

Zhao Xiaojie (Rick)

Blog: https://github.com/LinuxSuRen

Twitter: https://twitter.com/suren69811254

[Matt Sicker]

Sounds like an interesting project idea. For inspiration on UI, would this be similar to, say, https://start.spring.io/ ?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTFs-%3DeBOnFeTthXmViVzeBdbPCkNP%3DXn3Mq2-97zoFvQQ%40mail.gmail.com.

--

Matt Sicker
Senior Software Engineer, CloudBees

[Sladyn Nunes]

Actually Spring boot is pretty weak on the UI side it wasn't meant for that, it's more of a backend platform, but if we combine its features with a Javascript UI of our choice, it should work pretty well. 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAEot4oyntgMxch8Pm_QhHFMpB34PUrG756iHG5kFb95WWokJEA%40mail.gmail.com.

[Rick]

I agree with you that Spring boot can be the framework of backend.

start.spring.io is a great example. But I afraid that Jenkins one might take a longer time. And the size might be bigger.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAC-LeqsHQL71HCuefGVNqAVYQ--xvzfVFeZs8D2jX%3DfY6RpprQ%40mail.gmail.com.

[Sladyn Nunes]

I am unsure about Jenkins being a bigger one because IMO that would depend on the infrastructure we use rather that on the limitations of the language.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTEaQPd1WXLG0SDohM-Lg7BHycxyB_yN_BSzGUh0BxuX4A%40mail.gmail.com.

[Daniel Beck]

On Wed, Jan 8, 2020 at 3:49 PM Rick <linux...@gmail.com> wrote:

For many users, they download Jenkins first, then select some plugins and config them. It might take a lot of time, like hours. But if we can get a perfect Jenkins distribution which contains all we need, it can save that time for us. Yes, I propose an out of the box solution.

Jesse brings up some potential issues with something like this in https://groups.google.com/d/msg/jenkinsci-dev/8VnvgOVeVIs/7AM6pruADAAJ -- this could require some work on a fairly badly documented and tested part of Jenkins core to make work well. 

[Chris Kilding]

The notion of a Jenkins distribution(s) sounds relevant to us.

In my company we have a central team which is trying to put together a standardised Jenkins template to deploy on AWS, which all product teams can then adopt instead of rolling their own. (This is part of a phased deduplication strategy and isn't the end game: I could perhaps see a multi-tenant central Jenkins instance for some tasks in the future.)

The work that the central team does in assembling a set of plugin versions, configuration etc, certifying that they all work together, and then repeating this certification when things change, starts to look like the work that the community around a Linux distro does. But unlike the Linux community, Jenkins tooling teams can't share the workload with teams in other places. Each organisation must duplicate the effort of maintaining their own in-house Jenkins distro, even if a lot of those distros look very similar to each other. The exception to the rule is Jenkins X for Kubernetes, but if a company is not in a position to move their CI/CD to Kubernetes then they're out of luck.

For example, I suspect that both my company and Jeff's team at GoDaddy have developed variations on a "Jenkins for AWS" distribution, but we've reinvented them - and continue to maintain them - independently. If we were instead to share the work of building a standardised Jenkins for AWS distro, it might look a bit like this:

- Start with Jenkins vanilla

- Build in a set of core AWS-specific opinions (build agents must be spawned on [AWS compute service], logs must be forwarded to [AWS logging service], credentials must be stored in [AWS secrets service] etc). These would be the defining characteristics of the distro, something like how Ubuntu uses Gnome, SystemD etc. All plugins and config required to make this work would be pre-packed in the distro.

- Still allow users to configure this as they please, or to install extra plugins etc. This is how they'll customise it to their environments. The analogy with Ubuntu is that you would expect to install say language runtimes or apps from the APT repos as those are extras, but you wouldn't expect to replace the bundled desktop environment (you'd certainly have an uphill struggle to do it cleanly).

Anything that would help Jenkins users to reduce duplicated maintenance effort sounds good in my book. Whether that's a customisation service, or something more pre-packed equivalent to a Linux distro, or something else, I don't have a clear picture of yet.

I suppose the cost/benefit comes down to:

- How much we can share

- How many teams that use Jenkins are interested in participating

- How difficult it is to build the common packaging tooling (can we do it in 1 GSoC or is it a bigger thing?)

- Time horizons (do the potential participants foresee sticking with their Jenkins deployment stack, or something similar to it, for at least N years such that it's worth putting the effort in)

Chris

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAMo7PtK6EzOF8ghN1tyE_bKp5A-X5xe%2Bjgmd-oKO7nfpg2GMdA%40mail.gmail.com.

[Oleg Nenashev]

Hi Rick et all,

Would it be possible to submit the project idea to https://jenkins.io/projects/gsoc/2020/project-ideas/ ?

We have dozens of students reaching out through different channels, and it would be great to have this idea posted so that they can review and explore the idea.

Thanks in advance,

Oleg

To unsubscribe from this group and stop receiving emails from it, send an email to jenkin...@googlegroups.com.

[Sladyn Nunes]

Hey Rick, 

If you need any help in getting this project idea draft ready let me know and I would be happy to help.

Thanks. 

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/903a4f91-d290-4d66-b22c-13ac031597b8%40googlegroups.com.

[Jeff]

I just saw this, but I would be interested in working with you on this even if it doesn't make it into GSoC. 

Rick, do you need help getting a proposal started?

Best

Jeff

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/91a4c831-c6c8-42fc-9ea6-d7d4d5c9ed14%40www.fastmail.com.

[Rick]

Hi all,

Thanks for reaching out. Please help me to review my PR once I create it. I'll do it today. My daily routine is chaos now due to the virus. More than 9k people infected.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CADVhPTrJQdynocY-S-LVxZzxovK151t%2BCG2Dm614m6EH_KfEBg%40mail.gmail.com.

[Rick]

Hi all,

Here is the Pull Request link https://github.com/jenkins-infra/jenkins.io/pull/2813.

Please help me to do the review work, of course, any supplements are welcome.

[Chris Kilding]

Hi Rick,

At the contributors summit we discussed the possibility of introducing metapackages for certain tasks, rather than a full distro in the traditional sense.

For example, if a user uses Jenkins on AWS, they are quite likely to use several of the Jenkins AWS plugins together. They may also struggle when installing them on the same system, as the plugins may sometimes depend on different (incompatible) versions of the AWS SDK.

Rather than a full distro, a Jenkins for AWS metapackage (like the Debian/APT equivalent) that aggregates all those plugins would allow one central team of maintainers to assemble a combination of AWS plugin versions that are known to work together. Jenkins users would just have to install the metapackage.

These metapackages in turn could be consumed (and featured) by the proposed Jenkins customise service.

Thoughts?

Chris

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTHB3Gt_tNf3UhzszTGrPzBMuUGPkNe8WVbkCN%3DsnG57_w%40mail.gmail.com.

[Jesse Glick]

On Thu, Feb 20, 2020 at 9:51 AM Chris Kilding
<chris+...@chriskilding.com> wrote:
> These metapackages in turn could be consumed (and featured) by the proposed Jenkins customise service.

Also

https://github.com/jenkinsci/jenkins/blob/master/core/src/main/resources/jenkins/install/platform-plugins.json

[Rick]

Hi Chris,

Thanks for providing the advice. People usually use Jenkins as one or several specific scenes. Like you mentioned an AWS metapackage. I think we can define two kinds of metapackages. One is the pre-defined which likes AWS use cases, e.g. Another kind is that show the popular metapackages which come from the users of Jenkins distribution customise service. How do think about this?

Platform-plugins.json is a good example. Thanks Jesse!

Best regards,
Rick

> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2-mR6ShssSHEHL-f8msytS4_x6K%3DFSmaYBF6kju4gj%2Bg%40mail.gmail.com.

[Chris Kilding]

Hi Rick,

For specific use cases, perhaps we are looking more at meta-plugins than metapackages…

Meta-plugins would consist (almost) exclusively of a POM. Versions of each downstream plugin that is relevant to the use case would be pinned and controlled in the POM. Each meta-plugin release marks a combination of those plugins that are certified to work together. This takes the version testing and pinning work out of the Jenkins admin’s hands.

The various cloud providers are good candidates for meta-plugins, because every integration with a provider is typically built on the following foundational components:
- An SDK (e.g. aws-java-sdk-plugin)
- A shared configuration mechanism (e.g. aws-global-configuration-plugin)

And the cloud providers update their SDKs frequently, especially with security fixes, so the rate of dependency churn is high.


On the topic of advertising popular plugins, platform-plugins.json looks like a good in-band mechanism of doing this, and if particular use case meta-plugins are popular, they too should be advertised through it.

Chris

> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/F10030E1-760D-48BD-B149-7B8595596ABC%40126.com.

[Tim Jacomb]

This used to be done for pipeline with the 'workflow-aggregator' plugin,

afaik it's not very popular though? and isn't really being maintained anymore

Any one able to provide any background? Or why this would be a good / bad idea

Thanks

Tim

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/0371C77F-E9D3-48B1-BD77-D5B0C1173F90%40chriskilding.com.

[Jesse Glick]

On Fri, Feb 21, 2020 at 9:50 AM Chris Kilding
<chris+...@chriskilding.com> wrote:
> Versions of each downstream plugin that is relevant to the use case would be pinned and controlled in the POM. Each meta-plugin release marks a combination of those plugins that are certified to work together. This takes the version testing and pinning work out of the Jenkins admin’s hands.

This would not accomplish your goal, since plugin dependencies are on
_minimum_ versions. A new release of one of the component plugins
(say, `aws-java-sdk`) would be delivered to update sites immediately,
regardless of what the current metaplugin POM says.

This is why CloudBees offers
https://docs.cloudbees.com/docs/admin-resources/latest/assurance-program/
as a value-add feature: the general update site publishes the latest
Maven release of every plugin without any cross-testing.

As for `workflow-aggregator`, I discourage its use since it just pulls
in various stuff without any clear criteria for what is relevant and
important. You might fare better with a pack which “supports AWS”, but
you would still need to think about which parts of AWS are important
for most users and which are not.

BTW for an abandoned attempt in this area which I think was not
mentioned previously:
https://github.com/jenkins-infra/evergreen/tree/140e2a12da772917529fb93a9e48c2c34daec1c0/distribution/flavors/aws-ec2-cloud

[沈忠标]

I would prefer two types of solutions:

* pre-defined packaging. For example, AWS suite

  * the provider needs to  guarantee the quality of it and maintain it.

* users defined

  * we don’t guarantee anything about the suite. For example compatible something. We only provide the packing result

How about this? Do you think it makes sense?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3YQsE50L%2BqTnKqYFMyDrDSYbRNr_FWjFEsKZ6mK1C6Uw%40mail.gmail.com.

[Chris Kilding]

Jesse - cheers for pointing out the previous AWS packaging attempt, I will take a look.

Re version pinning, if this can’t be done in the POM, are there any Jenkins tools which can do this, apart from the cloudbees assured update feed? (I’m thinking of how we could support users on Jenkins community edition.)

Re what gets included - we could insist that any plugins included in a cloud vendor metapackage meet certain criteria, so that they can play together better:
- They must be able to no-op if the user doesn’t need them (ie their default is to quietly sit on the Jenkins server and not do anything, until the user configures them).
- They must all use the same cloud provider SDK artifact (probably the Jenkins re-packaged HPI version of the SDK).
- If appropriate, they must all take their baseline configuration from the same cloud provider ‘global config’ plugin.

Re pre-defined packages - I would certainly like to see the cloud vendors get more involved with supporting their respective Jenkins plugins.

Chris

[Manuel Ramón León Jiménez]

Hi.

I've been working on some ideas pretty similar to all you are suggesting here. The ultimate goal is, sort of:

* Allow to have a Jenkins ready with all stuff in a single click or command. Or compared to IKEA: avoid having to mount the IKEA furniture manually because unlike IKEA, we have dozens of Shrönghon screw (aka plugins) and people usually don't know what to use to get their furniture mounted :-) 

* Allow people to share "their Jenkins", meaning Jenkins version + plugins + sample-running jobs + configurations (but credentials) to allow community to reuse them. Some projects have contributed to allow that without too much effort, like JCasC, custom-war-packager, ...

* Allow people to kind of vote for packages to be able to find out the most mature or relevant among all existing.

* Have a centralized and easy to maintain repository of such "Jenkins configurations". It could be as easy as a GH repo with a certain template (readme + configuration file + assets), although some features like voting may be missed.

Beyond the specific proposal I made I find there are a lot of people thinking on the same problem and slightly different solutions. So it would be great to align efforts on the same direction to extract the most from us to achieve something deliverable in a short time.

Not sure under what umbrella we could do that, GSoC, JEP, ... though.

Best regards.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/623C5041-5D06-47C9-BD2F-CA7336B7BC53%40chriskilding.com.

[Oleg Nenashev]

Hi all,

Some notes about this topic from the contributor summit in Brussels are available here: https://docs.google.com/document/d/1-OGpDPWkOdKw-e8C0v9hUGgQshCLmToodo0biwlejdk/edit#heading=h.wzuq89kuh5wy

Maybe Chris or Tim could add more notes there as the most active participants, but I also believe that this project idea and the topics there are very well aligned. Thanks a lot to everyone who contributed to this discussion

I wonder whether we could have a dedicated session for this project idea at one of the Platform SIG meetings (noon UTC on Thursdays, next one will be on 27th).

Best regards,

Oleg

P.S: We are looking for potential GSoC project mentors! If you are interested in mentoring this project idea, let the GSoC team know!

On Tuesday, February 25, 2020 at 9:40:34 AM UTC+1, Manuel Ramón León Jiménez wrote:

Hi.

I've been working on some ideas pretty similar to all you are suggesting here. The ultimate goal is, sort of:

* Allow to have a Jenkins ready with all stuff in a single click or command. Or compared to IKEA: avoid having to mount the IKEA furniture manually because unlike IKEA, we have dozens of Shrönghon screw (aka plugins) and people usually don't know what to use to get their furniture mounted :-) 

* Allow people to share "their Jenkins", meaning Jenkins version + plugins + sample-running jobs + configurations (but credentials) to allow community to reuse them. Some projects have contributed to allow that without too much effort, like JCasC, custom-war-packager, ...

* Allow people to kind of vote for packages to be able to find out the most mature or relevant among all existing.

* Have a centralized and easy to maintain repository of such "Jenkins configurations". It could be as easy as a GH repo with a certain template (readme + configuration file + assets), although some features like voting may be missed.

Beyond the specific proposal I made I find there are a lot of people thinking on the same problem and slightly different solutions. So it would be great to align efforts on the same direction to extract the most from us to achieve something deliverable in a short time.

Not sure under what umbrella we could do that, GSoC, JEP, ... though.

Best regards.

On Mon, Feb 24, 2020 at 11:55 PM Chris Kilding <ch...@chriskilding.com> wrote:

Jesse - cheers for pointing out the previous AWS packaging attempt, I will take a look.

Re version pinning, if this can’t be done in the POM, are there any Jenkins tools which can do this, apart from the cloudbees assured update feed? (I’m thinking of how we could support users on Jenkins community edition.)

Re what gets included - we could insist that any plugins included in a cloud vendor metapackage meet certain criteria, so that they can play together better:
- They must be able to no-op if the user doesn’t need them (ie their default is to quietly sit on the Jenkins server and not do anything, until the user configures them).
- They must all use the same cloud provider SDK artifact (probably the Jenkins re-packaged HPI version of the SDK).
- If appropriate, they must all take their baseline configuration from the same cloud provider ‘global config’ plugin.

Re pre-defined packages - I would certainly like to see the cloud vendors get more involved with supporting their respective Jenkins plugins.

Chris

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkin...@googlegroups.com.

[Rick]

Hi Oleg,

I wish I could be a mentor of it. But the meeting time is too later for me. I’m not sure I can be there.

Due to the network blocked issues, I cannot access the google.com about one month. I try to fix this.

Best,

Rick

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/3b8b4036-9774-40eb-ac99-42ce54a2b6a0%40googlegroups.com.

[Oleg Nenashev]

We could try to find an alternative time for the SIG meeting, I am happy to run that. We can avoid using Google by running the meeting in Zoom.

It would be great if you could start a Doodle and list the slots possible for you there.

BR, Oleg

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/TzZ5mVqinuU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/1281a09.2371.1707bc6d370.Coremail.zxjlwt%40126.com.

[Rick]

Thanks for the proposal. I'd love to talk with all your guys. Below is the Doodle link: 

https://doodle.com/poll/r748iar7gwusekrp

Best,

Rick

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLAd_s4s3BkAj%2B9VRY2cgpf8m2mVb3wO9x0MM%3DSumUoLwQ%40mail.gmail.com.

[Oleg Nenashev]

Would it be possible to limit the meeting time to 1 hour or so?

Right now the invites are for 2 or 4 hours.

BR, Oleg

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/4d7d090e.2459.1707be01a4c.Coremail.zxjlwt%40126.com.

[Rick]

Sorry for the confusion. That’s my time slot. Not the meeting time. 2 or 4 hours will kill everyone. I’ve updated it.

https://doodle.com/poll/r748iar7gwusekrp

Best,

Rick

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLCUUJ13dZ%3DNpGJZDY56DqbEAY1TgWXYegKM1q%3DwNpehwA%40mail.gmail.com.

[Sladyn Nunes]

As a potential applicant I did share one of the short drafts for the proposal in the Jenkins developers group and would love to continue this discussion on the meeting time proposed.
We could probably achieve a task list pre-GSOC if that is allowed in the rules of GSOC that would give the potential participants a nice place to start from.
Some of the tasks that we could achieve based on the discussion above that could align our interests.

a)  Have a customizable jenkins builder where users could build their own Jenkins distribution using very simple UI.

b) Have a set of ready-made distributions maintained by the respective groups such as the AWS and have a centralized and easy to maintain repository of such "Jenkins configurations".

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/19fdd0b1.2579.1707c017b99.Coremail.zxjlwt%40126.com.

[Oleg Nenashev]

Feb 28, 2PM UTC was selected as a meeting time, I will get it added to the Jenkins calendar

[Sladyn Nunes]

As discussed in the meeting yesterday the initial phase for the project would involve building a yaml generator which would generate a YAML file based on the inputs taken from the user. However my question is that while the packager-config.yml involves only 
the groupID and artifacID, and version for the plugins. Would that be enough while generating the configuration file, because as Chris mentioned a more robust configuration file would be necessary to build a AWS distro for instance. If yes is there currently any method to pull in all of the necessary configurations for a plugin which could assist in listing out the fields.

Example of a packager-config.yml

plugins:

  - groupId: "org.jenkins-ci.plugins"

    artifactId: "matrix-project"

    source:

      version: 1.9

[Chris Kilding]

Another action point from the call yesterday...

Regarding the “share your Jenkins configuration” feature, I am talking to our core engineering team at work to see if we can publish at least a snapshot of our shared Jenkins template repo (this is the template that our product development teams clone and then adapt for their purposes). This is to provide a starting point for the discussion of what the feature should look like.

Chris

On 29 Feb 2020, at 14:30, Sladyn Nunes <sladyn...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/9091fc97-dfaa-4a4d-9ea4-6534d2428fc3%40googlegroups.com.

[Rick]

Hi Chris,

Thanks for the share.

Rick

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/FCDDFA92-2EDF-4ED8-A6D7-B9BAB2C84EE1%40chriskilding.com.

[Oleg Nenashev]

Recording of the meeting on Friday: https://youtu.be/66imA0ikMpQ

On Saturday, February 29, 2020 at 4:08:25 PM UTC+1, Rick wrote:

Hi Chris,

Thanks for the share.

Rick

On 02/29/2020 23:06，Chris Kil...@chriskilding.com> wrote：

Another action point from the call yesterday...

Regarding the “share your Jenkins configuration” feature, I am talking to our core engineering team at work to see if we can publish at least a snapshot of our shared Jenkins template repo (this is the template that our product development teams clone and then adapt for their purposes). This is to provide a starting point for the discussion of what the feature should look like.

Chris

On 29 Feb 2020, at 14:30, Sladyn Nunes <sladyn...@gmail.com> wrote:



As discussed in the meeting yesterday the initial phase for the project would involve building a yaml generator which would generate a YAML file based on the inputs taken from the user. However my question is that while the packager-config.yml involves only 
the groupID and artifacID, and version for the plugins. Would that be enough while generating the configuration file, because as Chris mentioned a more robust configuration file would be necessary to build a AWS distro for instance. If yes is there currently any method to pull in all of the necessary configurations for a plugin which could assist in listing out the fields.

Example of a packager-config.yml

plugins:

  - groupId: "org.jenkins-ci.plugins"

    artifactId: "matrix-project"

    source:

      version: 1.9

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/9091fc97-dfaa-4a4d-9ea4-6534d2428fc3%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkin...@googlegroups.com.

[Rick]

Thanks for the sharing.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/6c6bbdda-9143-4906-91b3-079839ae7e4f%40googlegroups.com.

[Sladyn Nunes]

Chris any luck with the sharing of the yaml file for AWS ? If no, then +Rick and +Jeff would we just stick with the basic packager configuration file that I mentioned in my previous message, which would include only the version artifact and the plugin.
Because on putting further thoughts into it, would it not be difficult to represent all the configurations into the User Interface, because every plugin has so many options, I am not sure if there is an automatic way to grab onto the configurations of every plugin ? 

On Wednesday, January 8, 2020 at 8:19:32 PM UTC+5:30, Rick wrote:

Hi team,

I'd like to share an idea with you. There are some original discussions from the GSoC thread here.

For many users, they download Jenkins first, then select some plugins and config them. It might take a lot of time, like hours. But if we can get a perfect Jenkins distribution which contains all we need, it can save that time for us. Yes, I propose an out of the box solution.

The service could be like this, a website offered as https://customize.jenkins.io. Plus, it should be self-host. People can select the following configurations:

• Jenkins core version
• plugins
• common configuration, user/password, update-center site e.g.
• plugin based configuration, Kubernetes, Sonarqube plugin config e.g.
• multi-form package, jenkins.war or docker image
• other things

Two projects are expected backend and frontend. I think https://github.com/jenkinsci/custom-war-packager already did a lot of works. We can reuse it in this project. The backend should provide the modern Restful API. We can start the backend project with SpringBoot, it can save a lot of time.

About the frontend project, I don't have too many experiences on it. For example, we can start it by React framework. If you're instrested in it, please help to add more ideas on it.

Any feedbacks are very appreciated.

Best regards,

Rick

[Chris Kilding]

It’ll be a while before we’ll be able to publish what we want to demonstrate (this goes beyond just the Jenkins YAML file, there are surrounding components that make up our in-house ‘Jenkins on AWS’ template). But any solution would have to generate the YAML file as a starting point, so in the meantime you could get started with something that does that.

The configurations of plugins are at least somewhat discoverable already. Some random thoughts on it…

- Plugin configuration is marked with @Extension (and often extends GlobalConfiguration), which allows config scanners inside Jenkins to find it.

- Jenkins Extensions Index (https://jenkins.io/doc/developer/extensions/) is an auto-generated site that lists all API extension points, and known implementations. My guess is that the generator is somehow scraping the plugins hosted under the jenkinsci GitHub org. This might be more of interest, because you’re probably looking to get information from the plugins without actually running Jenkins.

- The configuration-as-code-plugin can both serialise and parse CasC YAML. To try this, just install configuration-as-code-plugin on your server, then navigate to /configuration-as-code, and hit the button that serialises the active configuration as YAML. (Though do note the caveat that it may not be totally accurate.)

Finally, note that some plugins have a different model for CasC-received config than the standard one they use for Web UI-received config. This allows them to present a cleaner config model to CasC users than what the Web UI would normally allow. The translation is done in a class that looks like this...

public class PluginRootConfigurator extends BaseConfigurator<T>
        implements RootElementConfigurator<T> {

  

    public Set<Attribute<T,?>> describe() {

        // All the translation happens here

    }

}

Where T is the plugin’s standard configuration class, that is exposed to the Web UI.

Chris

-- 
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/00a824b5-9dbf-4ae6-8b7d-0307cac3dd17%40googlegroups.com.

[Sladyn Nunes]

I was discussing with @FelixQueiruga on the UX Sig channel regarding reuse of the jenkins core to generate the forms for the UI for each plugin but I guess that option is not very viable, since it would prevent update to the forms and any such update would bring down our distribution service, we therefore currently stand with the option to  generate the fields using the json schema that JCasC uses but that is not very reliable.  I was wondering if we could scan all the DataBound Constructors and generate as many fields as we can, but I guess that is too much overhead. I would love to get some thoughts on this. Feel free to add your reply here or in the docs.
The link to this section in the proposal is here :  https://docs.google.com/document/d/1C7VQJ92Yhr0KRDcNVHYxn4ri7OL9IGZmgxY6UFON6-g/edit?disco=AAAAJMf0sg0

Thanks and Cheers

Sladyn

[Lee Meador]

How does this tool interleave with version updates?

What if the package requires plugin A, version 2.0 and plugin B, version 3.0. Plugin A, version 2.0 requires a minimum Jenkins version of 2.x but plugin B, version 3.0 does not work with Jenkins version 2.x?

This has a couple of solutions I can think of:

1) The team supporting a package has to release a new version of the package for new versions of Jenkins itself. I suppose it could be required for every LTS version or the package would be marked unsupported for the Jenkins version.

2) The package could supply minimum and maximum plugin versions that all work together no matter which combination of versions are loaded. Or, similarly, there could be a list of acceptable versions for each plugin.

This would be easier to think about if I knew more about the way Jenkins decides which versions of plugins are offered for loading in Manage Jenkins > Manage Plugins. Perhaps it could cooperate with the 

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/34c9dbcc-086d-45f5-b372-89367114ce6c%40googlegroups.com.

--

-- Lee Meador
Sent from gmail. My real email address is lee AT leemeador.com

[Rick]

I've created a doodle meeting proposal for the Jenkins custom build service. Please feel free to join us.

https://doodle.com/poll/seqp4kiegc6zbgyz

Best,

Rick

[Sladyn Nunes]

Hello everyone,

This project has now been selected for Google summer of code and along with an amazing team of mentors I would be seeing this project through. This thread has a lot of useful information regarding the direction various stakeholders would like to see the project go, and I appreciate the feedback. I would love to get some more information regarding the features desired and use-cases . I would like to post some of the questions here.

a) Would you like to store common configurations in a Jenkins Public repository which can be accessed by all or private repositories.?

b) Would you like to have a configuration file generated or a jenkins formula or a Dockerfile ?

c) Would you like to see some  common distributions like aws etc provided out of the box ? 

Also if you have any more use-cases please feel free to add it here or join the gitter channel here.

Meetings:

Office hours are scheduled every Tuesday at 13:00 UTC, and every Thursday at 12:00 UTC The meeting notes available for anyone to read.

Feel Free to join this meeting. Everyone is welcome.

[Sladyn Nunes]

Hi everyone,

We have approached the end of the community bonding period for GSoC 2020 and I have spent most of my time preparing a prototype of the service that I will be demoing via an online meetup on Monday (12:00 PM to 1:00 PM GMT+1). Details for the meetup are here. https://www.meetup.com/Jenkins-online-meetup/events/270974407/

Feel free to attend and participate.
Thanks and Cheers.

[Sladyn Nunes]

Hello everyone

I would like to post this update about the custom distribution service, we have completed two very successful phases during Google Summer of Code and we are very close to releasing the initial version. The service currently supports
a) Addition of plugins
b) Generation of packager.yml
c) Generation of jenkins.war
d) Community Configuration Support
We would like to get some more feedback on the service so we can improve its functionality and user experience. You can try out the service here https://github.com/jenkinsci/custom-distribution-service
You can find the demo for the end of phase two here: https://www.youtube.com/watch?v=oSBvJwSXOVQ
Thanks and Cheers.