Develocity open source sponsorship for Jenkins' CI instances

[Oleg Nenashev]

Hi all,

Last week I had a chat with the Gradle folks about expanding our list of sponsored Develocity instances for open source projects, especially large projects running on the Maven. Develocity offers a distributed build cache that can greatly improve build speed and reduce CI infrastructure costs, and to also improve local builds. In our case, we provide publicly visible instances so that people can explore performance of their builds, and of course share feedback with us. We want to have much better integrations for Maven, so it would be a win win IMHO. Here is the list of the projects we currently host - https://github.com/gradle/develocity-oss-projects

Of course, I had Jenkins in mind too when discussing a list of whom me we could invite. As I discovered on the call, Gradle already had the conversation with Jenkins, namely Mark Waite and maybe a few other folks, but the discussion went stale at some point. I was unable to find references to this discussion in public so I'm not sure who was involved. 

What I'm happy to say is that Gradle is willing to continue this discussion should there be interest from the Jenkins folks in continuing this discussion.

I do not have access to the conversation histories and key takeaways neither from Gradle nor from Jenkins side, and I would appreciate if someone could share notes. 

Best regards,

Oleg

[Mark Waite]

On Mon, Mar 4, 2024 at 2:48 PM Oleg Nenashev <o.v.ne...@gmail.com> wrote:

Hi all,

Last week I had a chat with the Gradle folks about expanding our list of sponsored Develocity instances for open source projects, especially large projects running on the Maven. Develocity offers a distributed build cache that can greatly improve build speed and reduce CI infrastructure costs, and to also improve local builds. In our case, we provide publicly visible instances so that people can explore performance of their builds, and of course share feedback with us. We want to have much better integrations for Maven, so it would be a win win IMHO. Here is the list of the projects we currently host - https://github.com/gradle/develocity-oss-projects

Of course, I had Jenkins in mind too when discussing a list of whom me we could invite. As I discovered on the call, Gradle already had the conversation with Jenkins, namely Mark Waite and maybe a few other folks, but the discussion went stale at some point. I was unable to find references to this discussion in public so I'm not sure who was involved. 

I've briefly discussed Gradle caching with one or two different people from Gradle at conferences that I was attending.  I don't have any discussion notes to share.  I thought that Gradle caching sounded interesting, but I don't expect that the Jenkins infrastructure team will have time to do anything with Gradle Develocity in the next 6 months.

We have received a $60k donation from AWS and need to make the necessary infrastructure changes to use that $60k donation.  We need to continue our cost improvement efforts with Azure.  We need to continue our work with DigitalOcean, with mirror providers, and with many others.

My initial interest was in finding ways to reduce the amount of testing that we perform without reducing the usefulness of the testing we perform.  Gradle build caching did not seem to be a direct solution to that need.  We're using Launchable with Jenkins core already and have seen good results from its test reduction techniques.  The initial discussions seemed to indicate that Gradle's test avoidance techniques might not work very well with the Maven development environment used by Jenkins. I did not involve them in a detailed technical evaluation, just a brief conversation at one or more conferences.

I'm not aware of any public discussions either.  

 

What I'm happy to say is that Gradle is willing to continue this discussion should there be interest from the Jenkins folks in continuing this discussion.

I'm happy to have the discussion, but am not optimistic that the infrastructure team will be able to take on a large project like using Gradle Develocity.

 

I do not have access to the conversation histories and key takeaways neither from Gradle nor from Jenkins side, and I would appreciate if someone could share notes. 

I have no notes to share.  My search in my personal email did not show any hits for conversations with people from Gradle.  I'm on personal travel all this week and won't return to the office until next week.  My availability should be considered very limited.

 

Best regards,

Oleg

[Oleg Nenashev]

Hi Mark,

Thanks for the context. Nothing urgent at all, it can easily wait until you're back. If there is just an initial discussion that took place, indeed there's quite a lot to sort out

P.S: I do also share the concern that Develocity is not a great fit for test automation on the case of how Jenkins tests are organized, and rewiring them would be troublesome. Build speed is not our biggest problem, and the test selection should be indeed solved by Launchable 

I would guess PCT runs could be a good subject for Develocity though. But it is something we can review later

Best regards,

Oleg 

[Oleg Nenashev]

Also CCing Basi, IIUC he's driving the Launchable part. I saw the document on the adoption here: https://docs.google.com/document/d/12LdoAFA566P4LgHhu1L-fuOQuhipxk77tUDXQdNmhss/edit?usp=sharing , but I am not 100% sure it is the current status. 

Given the performance trends for the pull request builder, I assume it is already rolled out, right?