[JIRA] (JENKINS-60907) Poor performance with jenkins.model.StandardArtifactManager.disableTrafficCompression=true

14 views
Skip to first unread message

tom.gl@free.fr (JIRA)

unread,
Jan 29, 2020, 12:13:02 PM1/29/20
to jenkinsc...@googlegroups.com
Thomas de Grenier de Latour created an issue
 
Jenkins / Bug JENKINS-60907
Poor performance with jenkins.model.StandardArtifactManager.disableTrafficCompression=true
Issue Type: Bug Bug
Assignee: Unassigned
Components: core
Created: 2020-01-29 17:12
Environment: Jenkins 2.204.1
Linux (Ubuntu 18.04)
OpenJDK 8u242
Priority: Minor Minor
Reporter: Thomas de Grenier de Latour

With the new jenkins.model.StandardArtifactManager.disableTrafficCompression property (JENKINS-26008 / PR #4205), I was expecting improved performances when archiving artifacts from slaves to master. Especially when there is no room for further compression (when the artifact is already a compressed archive).
But my first tests with 2.204.1 and this option are disappointing. On my test setup:

  • with traffic compression (default setting), archiving a ~200 MB .tar.gz file consistently takes ~50 seconds (which is really not great already)
  • without traffic compression (the new option), it now takes ~240 seconds!

I suspect this is because FilePath.TarCompression.NONE lacks a layer of buffering input/output streams, which is there in FilePath.TarCompression.GZIP:
FilePath.java#L758

  • GZIPInputStream is a buffering InputStream (here with buffer size set to 8K)
  • GZIPOutputStream is a buffering OutputStream (with a default buffer size of 512), and here there is also an addition BufferedOutputStream involved (with its default buffer size of 8K)

I will submit a PR for FilePath.TarCompression.NONE to add the missing buffering layer, such that the difference with GZIP will really only be about compression.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

tom.gl@free.fr (JIRA)

unread,
Jan 29, 2020, 4:02:03 PM1/29/20
to jenkinsc...@googlegroups.com
Thomas de Grenier de Latour updated an issue
Change By:
With the new {{jenkins.model.StandardArtifactManager.disableTrafficCompression}} property (JENKINS-26008 / [PR #4205|https://github.com/jenkinsci/jenkins/pull/4205]), I was expecting improved performances when archiving artifacts from slaves to master. Especially when there is no room for further compression (when the artifact is already a compressed archive).   

But my first tests with 2.204.1 and this option are disappointing.  On my test setup:
- with traffic compression (default setting), archiving a ~200 MB .tar.gz file consistently takes ~50 seconds (which is really not great already)
- without traffic compression (the new option), it now takes ~240 seconds!


I suspect this is because {{FilePath.TarCompression.NONE}} lacks a layer of buffering input/output streams, which is there in {{FilePath.TarCompression.GZIP}}:
[FilePath.java#L758|https://github.com/jenkinsci/jenkins/blob/jenkins-2.204.1/core/src/main/java/hudson/FilePath.java#L758]
- {{GZIPInputStream}} is a buffering {{InputStream}} (here with buffer size set to 8K)
- {{GZIPOutputStream}} is a buffering {{OutputStream}} (with a default buffer size of 512), and
here there is also an addition additionnal {{BufferedOutputStream}} involved (with its default buffer size of 8K)

I will submit a PR for {{FilePath.TarCompression.NONE}} to add the missing buffering layer,
such so that the difference with {{GZIP}} will really only be only about compression.

tom.gl@free.fr (JIRA)

unread,
Jan 29, 2020, 4:04:02 PM1/29/20
to jenkinsc...@googlegroups.com

tom.gl@free.fr (JIRA)

unread,
Jan 29, 2020, 4:07:02 PM1/29/20
to jenkinsc...@googlegroups.com
Thomas de Grenier de Latour commented on Bug JENKINS-60907
 
Re: Poor performance with jenkins.model.StandardArtifactManager.disableTrafficCompression=true

With buffered streams (PR #4459), my test case with traffic compression disabled now completes in ~30 seconds (vs. ~50 seconds with compression).

tom.gl@free.fr (JIRA)

unread,
Feb 6, 2020, 7:57:03 AM2/6/20
to jenkinsc...@googlegroups.com

tom.gl@free.fr (JIRA)

unread,
Feb 6, 2020, 8:02:04 AM2/6/20
to jenkinsc...@googlegroups.com
Thomas de Grenier de Latour updated an issue

Tagging as a LTS candidate (for 2.204.3), to be considered after it is released in the next weekly. It's not a critical fix, but on the other hand the change is pretty simple, can hardly have any undesirable side effects, and is useful in some use cases.

Labels: lts-candidate performance

tom.gl@free.fr (JIRA)

unread,
Feb 10, 2020, 3:28:03 AM2/10/20
to jenkinsc...@googlegroups.com

o.v.nenashev@gmail.com (JIRA)

unread,
Feb 10, 2020, 4:26:06 AM2/10/20
to jenkinsc...@googlegroups.com

tom.gl@free.fr (JIRA)

unread,
Feb 18, 2020, 5:22:02 PM2/18/20
to jenkinsc...@googlegroups.com

dbeck@cloudbees.com (JIRA)

unread,
Feb 27, 2020, 5:18:02 PM2/27/20
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: lts-candidate performance
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages