[JIRA] (JENKINS-58898) Stager mass-parallel builds

[voronaam@gmail.com (JIRA)]

Aleksey Vorona created an issue

Jenkins / JENKINS-58898 Stager mass-parallel builds Issue Type: Improvement Assignee: Unassigned Components: workflow-cps-plugin Created: 2019-08-12 17:13 Priority: Minor Reporter: Aleksey Vorona When running a mass-parallel build (~300 branches of pipeline in parallel) there is a noticeable slow down of the Jenkins UI.   We have enough k8s nodes to support this, but master still gets slow in the beginning when the worker containers are starting up. This is due to each worker occupying a thread on master and hundreds of busy threads is not ideal.   Would it be possible to do something like this: parallel parallelNodes, stager: '1s' This way the master would spin up threads on master slower and we can tune the stager parameter to not visibly slow Jenkins UI at the time of starting mass parallel job. Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[voronaam@gmail.com (JIRA)]

Aleksey Vorona updated an issue

Jenkins / JENKINS-58898 Stager mass-parallel builds

Change By: Aleksey Vorona

When running a mass-parallel build (~300 branches of pipeline in parallel) there is a noticeable slow down of the Jenkins UI.   We have enough k8s nodes to support this, but master still gets slow in the beginning when the worker containers are starting up. This is due to each worker occupying a thread on master and hundreds of busy threads is not ideal.   Would it be possible to do something like this:

{code:java} parallel parallelNodes, stager: '1s'{code} This would tell the main job's thread to wait for 1s after each parallel job start before spawning a new one. This way the master would spin up threads on master slower and we can tune the {{stager}} parameter to not visibly slow Jenkins UI at the time of starting mass parallel job.

Add Comment