Is it required to return all jobs in Jenkins on pipeline view API call?

[Marcin Zajączkowski]

Hi,

We are facing an issue with quite long the pipeline view refresh time. Usually it is ~2-4 seconds using strong Amazon instance, but from time to time it is over 10 seconds or even timeout occurs (20 seconds). Looking at the network stats the crucial is an api call which retrieves json file with the pipeline view details to render. In addition to data from DeliveryPipelineView there is included also the "jobs" element with a list of all jobs available in Jenkins (via getItems() method). In our case its over 4000 elements (we have a lot of automatically generated pipelines for various projects/products/countries) and the output json has a few megabytes.  The changes in response time are probably related to number of people having those views open in the browser (each one refresh in 5 seconds).

Looking at the profiler over 80% of time in that API call is consumed by Stapler - probably to handle large number of returned elements. Looking at pipe.js those elements seems to be not used for the pipeline plugin itself. As getItems() method is defined in the View class I assume it is useful for "normal" views to return contained elements, but I don't know Jenkins code good enough to confirm it.

Question. Would it not be possible to return there just the elements related to given view (pipeline) or even an empty list if it in fact is not needed? That would improve the plugin performance for Jenkins instances with high number of jobs in general.

Marcin

--
http://blog.solidsoft.info/ - Solid Soft - Working code is not enough

[Marcin Zajączkowski]

On Friday, May 6, 2016 at 4:02:10 PM UTC+2, Marcin Zajączkowski wrote:

Hi,

We are facing an issue with quite long the pipeline view refresh time. Usually it is ~2-4 seconds using strong Amazon instance, but from time to time it is over 10 seconds or even timeout occurs (20 seconds). Looking at the network stats the crucial is an api call which retrieves json file with the pipeline view details to render. In addition to data from DeliveryPipelineView there is included also the "jobs" element with a list of all jobs available in Jenkins (via getItems() method). In our case its over 4000 elements (we have a lot of automatically generated pipelines for various projects/products/countries) and the output json has a few megabytes.  The changes in response time are probably related to number of people having those views open in the browser (each one refresh in 5 seconds).

Looking at the profiler over 80% of time in that API call is consumed by Stapler - probably to handle large number of returned elements. Looking at pipe.js those elements seems to be not used for the pipeline plugin itself. As getItems() method is defined in the View class I assume it is useful for "normal" views to return contained elements, but I don't know Jenkins code good enough to confirm it.

Just small clarification. Profiling has been performed using local Jenkins instance with similar number of generated jobs, but with local network connection. When taking network transfer into consideration Stapler negative impact is not so high. Nevertheless a change in the plugin to return empty list instead of 4K elements provides significant reduction of the whole time to get json response. The view itself seems to work properly after that change, however, I don't know if there are any general side effects (outside the plugin) I didn't spotted.

The original question is still valid - do you see any cons to adjust getItems() (and connected methods) to return only jobs displayed in the particular view?

I can provide a PR if you are interested.

Marcin


 

[Olga Maciaszek-Sharma]

Hello,

I will be preparing a PR with this change. Can you think of any issues that such change could cause?

Regards,

Olga