[JIRA] (JENKINS-56249) userContent *zip* (all files in zip) stopped working at 2.164

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert created an issue

Jenkins / JENKINS-56249 userContent *zip* (all files in zip) stopped working at 2.164 Issue Type: Bug Assignee: Unassigned Components: core Created: 2019-02-22 16:17 Environment: Windows 2008 R2 Enterprise, Jenkins 2.164+ Labels: zip core http Priority: Major Reporter: Glenn Herbert When using the (all files in zip) functionality starting at 2.164, either from the UI or URL (https:/myjenkins/userContent/zip/image.zip), the resulting image.zip has no content.  This has silently broken our builds, which utilize this functionality to retrieve certain content. Downgrading jenkins.war to 2.163 restores the functionality. Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert updated an issue

Jenkins / JENKINS-56249 userContent *zip* (all files in zip) stopped working at 2.164

Change By: Glenn Herbert When using the (all files in zip) functionality starting at 2.164, either from the UI or URL (https:/myjenkins/userContent/* * zip* * /image.zip), the resulting image.zip has no content.

This has silently broken our builds, which utilize this functionality to retrieve certain content. Downgrading jenkins.war to 2.163 restores the functionality.

Add Comment

[dbeck@cloudbees.com (JIRA)]

Daniel Beck commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164 Wadeck Follonier FYI

Add Comment

[wfollonier@cloudbees.com (JIRA)]

Wadeck Follonier commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Glenn Herbert Thank you for the report. I will not have the time to investigate this case further this week. Could you try using the WAR provided in JENKINS-56114 ? As they were also using Windows Server, that could the same root cause. In addition, did you try with the escape hatch: java.exe -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true -jar jenkins.war ? (in both your current version and in the WAR provided in the other ticket) Then could you give me more information about your configuration, like are you using Docker inside your Windows Server, or just running java -jar ..., or using the service? If you can provide also additional information from the log of the build, that could be useful. Also, what is the content of the userContent in your instance (in terms of type, not the real files, but to know if there are symlinks, what are their targets, or just image files?) Thank you in advance for your next answer that will help me helping you

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert updated an issue

Jenkins / JENKINS-56249

userContent *zip* (all files in zip) stopped working at 2.164

Change By: Glenn Herbert Attachment: image-2019-02-28-16-52-19-863.png

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert updated an issue

Jenkins / JENKINS-56249 userContent *zip* (all files in zip) stopped working at 2.164

Change By: Glenn Herbert Attachment: image-2019-02-28-16-53-48-739.png

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert updated an issue

Jenkins / JENKINS-56249 userContent *zip* (all files in zip) stopped working at 2.164

Change By: Glenn Herbert Attachment: image-2019-02-28-16-54-36-484.png

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert updated an issue

Jenkins / JENKINS-56249 userContent *zip* (all files in zip) stopped working at 2.164

Change By: Glenn Herbert Attachment: image-2019-02-28-16-55-40-767.png

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164 Wadeck Follonier: I cannot try using the WAR from JENKINS-56114  as this is a production system.  I will try to bring up a new jenkins and try it. To answer your other questions: we are using the escape hatch -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true we are NOT using docker, just running the jenkins windows service userContent has a mix of files, folders, and directory junctions (symlinks) The build logs have no real information in them as it was a silent failure, because the zip was empty.  The gradle code used was def f = new File("$buildOutput/staging.zip") if (!f.exists()) {      new URL('http://myJenkins:8080/userContent/archive/components/Staging/zip/staging.zip').withInputStream{ i -> f.withOutputStream{ it << i }} } copy Unknown macro: {     from zipTree(f.getAbsolutePath())      into("$buildOutput/config/") } f.delete() From the UI, this is the Staging folder (this is not the actual folder used but shows the problem): If i then click on the (all files in zip), the Staging.zip file gets generated but its only 1KB And if its opened with 7zip the file is empty If I dump the file using od -c i see no content Hope this additional information helps!

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert edited a comment on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Wadeck Follonier: I cannot try using the WAR from JENKINS-56114  as this is a production system.  I will try to bring up a new jenkins and try it. To answer your other questions:

* we are using the escape hatch -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true * we are NOT using docker, just running the jenkins windows service * userContent has a mix of files, folders, and directory junctions (symlinks)

The build logs have no real information in them as it was a silent failure, because the zip was empty.  The gradle code used was

{quote} def f = new File("$buildOutput/staging.zip") if (!f.exists()) {      new URL('http://myJenkins:8080/userContent/archive/components/Staging/* * zip* * /staging.zip').withInputStream\{ i -> f.withOutputStream{ it << i }} } copy Unknown macro: \ {

from zipTree(f.getAbsolutePath())      into("$buildOutput/config/")

} f.delete() {quote}

From the UI, this is the Staging folder (this is not the actual folder used but shows the problem):

!image-2019-02-28-16-52-19-863.png!

If i then click on the (all files in zip), the Staging.zip file gets generated but its only 1KB

!image-2019-02-28-16-53-48-739.png!

And if its opened with 7zip the file is empty

!image-2019-02-28-16-54-36-484.png|width=249,height=158!

If I dump the file using od -c i see no content

!image-2019-02-28-16-55-40-767.png!

Hope this additional information helps!

Add Comment

[dbeck@cloudbees.com (JIRA)]

Daniel Beck commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Is this obsolete after recent changes in weekly releases?

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

The problem continues to persist even after upgrading all plugins and to the latest Jenkins version 2.168. The workaround we are using is to pre-zip the fetched content, to bypass the non-functional Jenkins zip functionality, so this is not as high a priority as it was.  However, it's still lost functionality that should be fixed.

Add Comment

[boards@gmail.com (JIRA)]

Matt Sicker commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Here is what I can reproduce which sound related to your problem: I run Jenkins with the -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true flag set. (Or I modify it in the script console) In my userContent directory, I have one symbolic link to a file or directory outside of userContent such as ../../jobs/TestJob/lastSuccess/archive/ I download the zip generated by the directory browser and find that it did not include those files despite using the flag. When I use Jenkins 2.163, the zip contains the data that I can also see in the browser. When I use Jenkins 2.164 or the latest release, the zip does not contain the files, though I can view the files in the browser still.

Add Comment

[boards@gmail.com (JIRA)]

Matt Sicker commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Oh, and don't be confused by the double symbolic link thing going on there. This is still reproducible by simply linking to ../../config.xml

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Thanks Matt Sicker for verification.

Add Comment

[boards@gmail.com (JIRA)]

Matt Sicker commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Me or one of my teammates will be following up on this ticket to fix the problem. I believe I've been able to isolate the issue down to a difference in behavior when using the escape hatch where you can view files that are links from outside userContent or are themselves descendants of a link, but the same files are skipped in the zip file output regardless of the escape hatch.

Add Comment

[arjo.poldervaart@funatic.nl (JIRA)]

Arjo Poldervaart commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Same issue here, we use symlinks a lot and are not able to upgrade at the moment to the latest LTS because of this issue. Is there a temporary workaround that can be made on a local Jenkins install?

Add Comment

[jthompson@cloudbees.com (JIRA)]

Jeff Thompson commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Arjo Poldervaart, could you clarify some of your use cases for symlinks? How do you use them? Where do they link from and to? Do they link from inside a workspace to somewhere else on the system? Anyone else, please respond also, about your need for symlink support. I'm especially interested in how your symlink use works with zip files, as featured in the OP. Zip files generated by Jenkins do not contain symlinks but may resolve the symlink to the target and include all of that. One scenario, described in JENKINS-56114, involves a master or agent which resides in a path containing a symlink at the root or somewhere else higher in the path.

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[jthompson@cloudbees.com (JIRA)]

Jeff Thompson commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

And which of those use cases involve userContent, as described in the OP?

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Our use of (Windows) symlinks (directory junctions) is purely for directories. We store build artifacts on a corporate (remote) NAS drive (multi-TB), but use the jenkins userContent and its built-in file server as a way to 1) download artifacts via jenkins URL (ie curl http:/myJenkins/userContent/symlink/path/to/artifact), or 2) navigate to specific artifacts.  We don't ever request a zip on the actual symlinked folder but some content that resides within that symlinked folder tree.   I would not expect a zip file to contain a symlink but rather its resolved target.

Add Comment

[arjo.poldervaart@funatic.nl (JIRA)]

Arjo Poldervaart commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Our use is similar to that of Glenn. To provide some context: If we want to release an artifact to production, we tag it in our SCM and let Jenkins build it, Jenkins then copies the artifact to a different location so we can download it at a later time (even if the workspace is cleared). The copy is done via a symlink created in the workspace by our build script (soft link on Linux) to a folder outside the workspace on disk. When we want to deploy the artifact we can simply download it via een link like: /view/assets/job/assets-BootstrapPortlet/ws/RELEASES/1.0.2/BootstrapPortletEAR.ear The _RELEASES_ folder __ is a symlink. So we basically use Jenkins as a file server just like Glenn.

Add Comment

[arjo.poldervaart@funatic.nl (JIRA)]

Arjo Poldervaart edited a comment on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Our use is similar to that of Glenn. To provide some context: If we want to release an artifact to production, we tag it in our SCM and let Jenkins build it, Jenkins then copies the artifact to a different location so we can download it at a later time (even if the workspace is cleared). The copy is done via a symlink created in the workspace by our build script (soft link on Linux) to a folder outside the workspace on disk.

When we want to deploy the artifact we can simply download it via een link like: /view/assets/job/assets-BootstrapPortlet/ws/_RELEASES_/1.0.2/BootstrapPortletEAR.ear The __RELEASES__ folder __ is a symlink. So we basically use Jenkins as a file server just like Glenn.   We do not use symlinks in the userContent folder, but we do use it in the job workspace.

Add Comment

[jthompson@cloudbees.com (JIRA)]

Jeff Thompson commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Glenn Herbert and Arjo Poldervaart, thanks for your responses on describing your use models. That helps clarify some things. Have you read the security advisory from last year at https://jenkins.io/security/advisory/2018-12-05/ ? Have you tried disabling that fix via the system property described there? It sounds like your use cases are some of the rare situations where that might make sense.

Add Comment

[arjo.poldervaart@funatic.nl (JIRA)]

Arjo Poldervaart commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Hi Jeff, We have been running with "-Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" for quite a while already. The setting does not resolve the current issue.

Add Comment

[glenn.herbert@hcl.com (JIRA)]

Glenn Herbert commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

Same here, been running with SymlinkEscape=true for quite a while.

Add Comment

[jthompson@cloudbees.com (JIRA)]

Jeff Thompson commented on JENKINS-56249

Re: userContent *zip* (all files in zip) stopped working at 2.164

I'm able to reproduce Matt's simple test case on Unix and Windows. Hopefully it's a reasonably good representation of the other problems people are experiencing. Looks like the `allowSymlinkEscape` property isn't working the way it was intended. I'll see if I can figure something out to make it work better. It may take a while to get that change put together.

Add Comment