[JIRA] [core] (JENKINS-26700) Download build artifacts as zip generates a corrupted file

279 views
Skip to first unread message

adriancaro@gmail.com (JIRA)

unread,
Mar 3, 2016, 10:01:15 AM3/3/16
to jenkinsc...@googlegroups.com
Adrian Rios commented on Bug JENKINS-26700
 
Re: Download build artifacts as zip generates a corrupted file

Hi all,
I am experiencing this same issue with Jenkins 1.643 running on Linux Debian 8 and the same symptoms as specified by Stefan, the jenkins link "all files in zip" provides a corrupted zip file, our corporate network could be somewhat slow but I don't think that could be related.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

mjdetullio@gmail.com (JIRA)

unread,
Mar 28, 2016, 6:15:01 PM3/28/16
to jenkinsc...@googlegroups.com

Steps to reproduce:

  1. Create a new freestyle job "test-job", Linux or OS X should be ok
  2. Add a shell script step with the following content 
    for i in {1..1000}
do
    dd if=/dev/zero of="ABC1MB-$(uuidgen).bin" bs=1024 count=0 seek=1024
done

mkdir -p a/b/c/
echo "text" > a/b/c/1
pushd a
ln -s b/c/1 2
popd

mkdir -p d/e/f
pushd d
ln -s e/f/1 2
popd
  3. Run a build to generate workspace
  4. Download workspace using "(all files in zip)" link in workspace browser
  5. Try to unzip

Expected results:

  • Unzipping will succeed
  • Unzipped directory mimics actual workspace, including symlinks

Actual result:

  • Invalid archive with error: 
    # Note: unzip on OS X
$ unzip test-job.zip
Archive:  test-job.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of test-job.zip or
        test-job.zip.zip, and cannot find test-job.zip.ZIP, period.
# Note: ditto on OS X
$ ditto -x -k test-job.zip .
ditto: test-job/d/2: No such file or directory
ditto: Couldn't read pkzip signature.

Problems with current state:

  • "Broken" symlinks, or symlinks to files that do not exist, apparently cause termination of zip file creation, thereby generating an invalid zip
  • "Working" symlinks, or symlinks to files that do exist, add duplicate copies of files
  • Recursive symlinks probably cause problems too

mjdetullio@gmail.com (JIRA)

unread,
Mar 28, 2016, 6:18:01 PM3/28/16
to jenkinsc...@googlegroups.com
Matthew DeTullio updated an issue
 
Jenkins / Bug JENKINS-26700
Change By: Matthew DeTullio
Labels: archive artifact artifact-download regression  symlink  zip zipfile

mjdetullio@gmail.com (JIRA)

unread,
Mar 28, 2016, 6:23:02 PM3/28/16
to jenkinsc...@googlegroups.com
Matthew DeTullio edited a comment on Bug JENKINS-26700
 
Re: Download build artifacts as zip generates a corrupted file
Steps to reproduce:

# Create a new freestyle job "test-job", Linux or OS X should be ok
# Add a shell script step with the following content
{code}

for i in {1..1000}
do
    dd if=/dev/zero of="ABC1MB-$ (uuidgen) {i} .bin" bs=1024 count=0 seek=1024

done

mkdir -p a/b/c/
echo "text" > a/b/c/1
pushd a
ln -s b/c/1 2
popd

mkdir -p d/e/f
pushd d
ln -s e/f/1 2
popd
{code}
# Run a build to generate workspace
# Download workspace using "(all files in zip)" link in workspace browser
# Try to unzip

Expected results:
* Unzipping will succeed
* Unzipped directory mimics actual workspace, including symlinks

Actual result:
* Invalid archive with error:
{code}

# Note: unzip on OS X
$ unzip test-job.zip
Archive:  test-job.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of test-job.zip or
        test-job.zip.zip, and cannot find test-job.zip.ZIP, period.
# Note: ditto on OS X
$ ditto -x -k test-job.zip .
ditto: test-job/d/2: No such file or directory
ditto: Couldn't read pkzip signature.
{code}

Problems with current state:
* "Broken" symlinks, or symlinks to files that do not exist, apparently cause termination of zip file creation, thereby generating an invalid zip
* "Working" symlinks, or symlinks to files that do exist, add duplicate copies of files
* Recursive symlinks probably cause problems too

luca.moscato@gmail.com (JIRA)

unread,
Sep 15, 2016, 5:28:03 AM9/15/16
to jenkinsc...@googlegroups.com

Hi all, sadly I'm facing the same issue, here is my environment
jenkins 2.7.4
jdk 1.8.40 (too old?)
centos 6.8 (package updated every night).

It happens that artifacts download (both .zip and other formats) fails often when, in office, the network appears slow. For fail I intend that from browser the download process seems ends correctly ok, but the file downloaded has smaller size and, in case of .zip, can't be opened.
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

luca.moscato@gmail.com (JIRA)

unread,
Sep 15, 2016, 9:51:01 AM9/15/16
to jenkinsc...@googlegroups.com

Found this issue in apache's log running on jenkins installation
[Thu Sep 15 13:46:42 2016] [error] [client XXX.XXX.XXX.XXX] (70008)Partial results are valid but processing is incomplete: proxy: error reading response, referer: https://myjenkinsserver/job/mybuild/

srinivasan.venkataraman@efi.com (JIRA)

unread,
Oct 31, 2016, 4:12:03 PM10/31/16
to jenkinsc...@googlegroups.com

I am facing the same issue. We are on Jenkins ver. 2.19.1

We have a powershell script downloading a build artifact zip. The zip is much smaller when the download is complete.

Invoke-WebRequest -URI $jenkinsLink/$majorVersion.$minorVersion/ws/$fileToDownload -ErrorAction:Stop -TimeoutSec 300 -OutFile $fileToDownload

Downloaded zip is unusable, not a valid archive. This is a critical issue and breaks the chain in our CI/Dev Ops.

srinivasan.venkataraman@efi.com (JIRA)

unread,
Oct 31, 2016, 4:16:02 PM10/31/16
to jenkinsc...@googlegroups.com
I am facing the same issue. We are on Jenkins ver. 2.19.1

We have a powershell script downloading a build artifact zip. The zip is much smaller when the download is complete.

Invoke-WebRequest -URI $jenkinsLink/$majorVersion.$minorVersion/ws/$fileToDownload -ErrorAction:Stop -TimeoutSec 300 -OutFile $fileToDownload

Downloaded zip is unusable, not a valid archive. This is a critical issue and breaks the chain in our CI/Dev Ops.
This is the exception we have in the logs
Caught exception evaluating: request.getSession() in /job/2016.2/ws/PrintStreamUpdate_2016.2.818.0.zip. Reason: java.lang.IllegalStateException: Response is committed
java.lang.IllegalStateException: Response is committed
at org.eclipse.jetty.server.Request.getSession(Request.java:1400)
at org.eclipse.jetty.server.Request.getSession(Request.java:1378)
at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:279)
at sun.reflect.GeneratedMethodAccessor243.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
at hudson.init.impl.InstallUncaughtExceptionHandler$1.reportException(InstallUncaughtExceptionHandler.java:30)
at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:52)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

chris.mh3@gmx.at (JIRA)

unread,
Nov 17, 2016, 4:49:04 AM11/17/16
to jenkinsc...@googlegroups.com

I have the same issue with Jenkins 2.19.2 (Windows) while downloading normal artifacts. In this case about 60MB jars. The issue occurs random but quite frequent.

Nov 17, 2016 10:28:16 AM hudson.ExpressionFactory2$JexlExpression evaluate
WARNING: Caught exception evaluating: request.getSession() in /job/<jobname>/artifact/<pathtoartifact>.jar. Reason: java.lang.IllegalStateException: Response is committed


java.lang.IllegalStateException: Response is committed
at org.eclipse.jetty.server.Request.getSession(Request.java:1400)
at org.eclipse.jetty.server.Request.getSession(Request.java:1378)
at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:279)

at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

chris.mh3@gmx.at (JIRA)

unread,
Nov 17, 2016, 5:24:01 AM11/17/16
to jenkinsc...@googlegroups.com
chris_mh3 edited a comment on Bug JENKINS-26700


On the other end - a Maven Tycho build the problem appears as:
[ERROR] Internal error: org.eclipse.tycho.repository.local.MirroringArtifactProvider$MirroringFailedException: Could not mirror artifact <artifactname> into the local Maven repository.See log output for details. Premature end of Content-Length delimited message body (expected: 77668135; received: 3510470 -> [Help 1]

fritz@fritz-elfert.de (JIRA)

unread,
Nov 17, 2016, 8:01:04 AM11/17/16
to jenkinsc...@googlegroups.com

Guys I bet this is just another broken symlink to be archived. There's a VERY SIMPLE workaround:
Just remove symlinks (If the point outside the archived tree, they would be useless anyway)

I use the following one-liner in my builds and never had ANY problems with zipped artifacts anymore:
find /dir/to/artifacts -type l | xargs rm -f

fritz@fritz-elfert.de (JIRA)

unread,
Nov 18, 2016, 4:32:02 AM11/18/16
to jenkinsc...@googlegroups.com

The following PR should fix this:
https://github.com/jenkinsci/jenkins/pull/2639

chris.mh3@gmx.at (JIRA)

unread,
Nov 21, 2016, 4:24:01 AM11/21/16
to jenkinsc...@googlegroups.com

I would be very surprised if this would fix Srinivasan Venkataraman's and my problem. In my case it's not with zip files at all.
Should I open a new issue for this?

fritz@fritz-elfert.de (JIRA)

unread,
Nov 21, 2016, 6:03:02 AM11/21/16
to jenkinsc...@googlegroups.com

Well thats a little confusing. After all, the title specifically names zip artifacts. So: yes, I'd suggest a separate issue if it is not about zipped artifacts. Of course thats MY personal opinion only.

chris.mh3@gmx.at (JIRA)

unread,
Nov 21, 2016, 6:55:02 AM11/21/16
to jenkinsc...@googlegroups.com
chris_mh3 edited a comment on Bug JENKINS-26700
I would be very surprised if this would fix Srinivasan Venkataraman's and my problem. In my case it's not with zip files at all.
Should I open a new issue for this?


JENKINS-39909
Reply all
Reply to author
Forward
0 new messages