Caching or Quickly comparing generated DSL

15 views
Skip to first unread message

Drew Pierce

unread,
Dec 7, 2023, 6:09:48 PM12/7/23
to job-dsl-plugin
I have a single seed groovy script that generates the DSL for about 800+ pipeline jobs based on JSON files.  This script runs every time we add a job.  Currently, the DSL script is using up a lot of CPU, about 6 cores, and causing the Jenkins UI to timeout behind our ingress when generating all the Job DSL.   So far, the solution has been to put a sleep  between each DSL Job generation, which takes about 3-15 seconds per job, making it a 2-hour job.  I would like to add `ignoreExisting : true`, but if the underlying JSON file changes, I would like to update the job..

      steps{
        jobDsl removedJobAction: 'DELETE', removedViewAction: 'DELETE', targets: 'addJobs.groovy', unstableOnDeprecation: true

      }

addJobs.groovy just generates lots of `multibranchPipelineJob("$jobName") {...}` DSL with sleeps between each call.

I tried changing the groovy  script to only run on the modified json files, but since it doesn't Generate all the Jobs DSL the build is marked as aborted. 
We also want to support deleting jobs if the underlying JSON file is deleted, so it seems like a caching or quick DSL / xml hash compare solution is needed.


Reply all
Reply to author
Forward
0 new messages