[JIRA] [core] (JENKINS-28013) Archiving artifiacts fails with java.io.IOException: Truncated TAR archive

392 views
Skip to first unread message

steffan@karger.me (JIRA)

unread,
Apr 20, 2015, 8:19:05 AM4/20/15
to jenkinsc...@googlegroups.com
Issue Type: Bug Bug
Assignee: Unassigned
Components: core
Created: 20/Apr/15 12:18 PM
Description:

The 'archive artifacts' step of some build jobs have recently started failing with the following trace:

Archiving artifacts
ERROR: Failed to archive artifacts: **/*.deb, **/*.changes, **/*.rpm, **/*.egg
java.io.IOException: Failed to extract /srv/user_name/.jenkins/workspace/SomeProject/distro/sles11_64/servertype/build/transfer of 4 files
	at hudson.FilePath.readFromTar(FilePath.java:2299)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2208)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:219)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:74)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:761)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:721)
	at hudson.model.Build$BuildExecution.post2(Build.java:183)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:670)
	at hudson.model.Run.execute(Run.java:1766)
	at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:374)
Caused by: java.io.IOException: Truncated TAR archive
	at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:614)
	at java.io.InputStream.read(InputStream.java:101)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
	at hudson.util.IOUtils.copy(IOUtils.java:40)
	at hudson.FilePath.readFromTar(FilePath.java:2289)
	... 13 more
Build step 'Archive the artifacts' changed build result to FAILURE

It seems that the jobs on the slaves succeeded, but the master fails:

Apr 20, 2015 11:15:14 AM hudson.model.Run execute
INFO: SomeProject/distro=sles11_64,servertype=build #12159 main build action completed: SUCCESS
[... more slave SUCCESS reports ...]
Apr 20, 2015 11:16:16 AM hudson.model.Run execute
INFO: SomeProject #12159 main build action completed: FAILURE

The master and slave logs do not show me any other errors.

This failure occurs just for some of the slave jobs in this matrix build job. Within a specific job, this fails consistently for the same slaves, but across jobs the slaves that fail vary.

There are no obvious resource issues; both disk space and memory usage seem to be fine.
Environment: Jenkins 1.610 (currently most recent), Ubuntu LTS 14.04 64-bit master, SLES11 64-bit slave
Project: Jenkins
Priority: Major Major
Reporter: Steffan Karger
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

troymills@gmail.com (JIRA)

unread,
Apr 20, 2015, 9:40:05 AM4/20/15
to jenkinsc...@googlegroups.com
Troy Mills commented on Bug JENKINS-28013

encountered the same issue, same stack. i/we had to rollback the jenkins.war on the master. environment wise, for us its CentOS6 64bit master and slaves.

laurent@tourreau.fr (JIRA)

unread,
Apr 20, 2015, 10:10:04 AM4/20/15
to jenkinsc...@googlegroups.com

Hi I have upgraded from 1.607 to 1.610. I face the same issue.
Regards

steffan@karger.me (JIRA)

unread,
Apr 20, 2015, 10:32:05 AM4/20/15
to jenkinsc...@googlegroups.com

For the record, I'm currently using Copy Artifact Plugin 1.35.

Downgrading to 1.609 made the issue disappear again for me.

daniel@beckweb.net (JIRA)

unread,
Apr 20, 2015, 11:03:08 AM4/20/15
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-28013

Probably a regression from JENKINS-10629.

steffan@karger.me (JIRA)

unread,
Apr 20, 2015, 11:19:04 AM4/20/15
to jenkinsc...@googlegroups.com

@Daniel it is very unlikely that my artifacts would be > 8GB; the artifacts generated by 1.609 are ~ 1MB.

daniel@beckweb.net (JIRA)

unread,
Apr 20, 2015, 11:25:05 AM4/20/15
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-28013

That's why this can be considered a regression, and not just an incomplete fix. https://github.com/jenkinsci/jenkins/pull/1647 is a likely culprit.

steffan@karger.me (JIRA)

unread,
Apr 20, 2015, 11:27:05 AM4/20/15
to jenkinsc...@googlegroups.com

Makes sense. Sorry for the noise.

o.v.nenashev@gmail.com (JIRA)

unread,
Apr 20, 2015, 11:37:04 AM4/20/15
to jenkinsc...@googlegroups.com

Yes, it seems to be a regression.
In JENKINS-10629 we ported the Tars handling implementation to apache-compress from a legacy snapshot of the library.

It would be great to have an example of artefacts you're trying to publish.

mwebber@java.net (JIRA)

unread,
Apr 20, 2015, 11:44:05 AM4/20/15
to jenkinsc...@googlegroups.com
mwebber commented on Bug JENKINS-28013

I also started seeing this after upgrading from 1.609 to 1.610.
Downgrading fixed the problem.

o.v.nenashev@gmail.com (JIRA)

unread,
Apr 20, 2015, 11:46:05 AM4/20/15
to jenkinsc...@googlegroups.com
Change By: Oleg Nenashev (20/Apr/15 3:45 PM)
Assignee: Oleg Nenashev

mwebber@java.net (JIRA)

unread,
Apr 20, 2015, 11:50:06 AM4/20/15
to jenkinsc...@googlegroups.com
mwebber commented on Bug JENKINS-28013

It would be great to have an example of artefacts you're trying to publish

Ours are a bit too large to attach here, but below is the directory structure:

/exports/jenkins_home/jobs/DawnDiamond.master-create.product/builds/3053/archive]$ ls -la
drwxrwsr-x. 2 dlshudson dls_dasc 4096 Apr 20 16:38 artifacts_to_archive
drwxrwsr-x. 3 dlshudson dls_dasc 4096 Apr 20 16:38 create.product.root

$ ls -la artifacts_to_archive/
-rw-rw-r--. 1 dlshudson dls_dasc 6152 Apr 20 16:29 materialized_head_commits.txt
-rw-rw-r--. 1 dlshudson dls_dasc  345 Apr 20 16:29 materialized_targetplatform_contents.txt

$ ls -la create.product.root/
drwxrwsr-x. 2 dlshudson dls_dasc 4096 Apr 20 16:40 products

$ ls -la create.product.root/products/
-rw-rw-r--. 1 dlshudson dls_dasc 482936973 Apr 20 16:25 DawnDiamond-1.9.0.v20150420-1451-linux32.zip
-rw-rw-r--. 1 dlshudson dls_dasc 479510056 Apr 20 16:25 DawnDiamond-1.9.0.v20150420-1451-linux64.zip
-rw-rw-r--. 1 dlshudson dls_dasc 328621190 Apr 20 16:25 DawnDiamond-1.9.0.v20150420-1451-mac64.zip
-rw-rw-r--. 1 dlshudson dls_dasc 488357250 Apr 20 16:25 DawnDiamond-1.9.0.v20150420-1451-windows32.zip
-rw-rw-r--. 1 dlshudson dls_dasc 494944440 Apr 20 16:25 DawnDiamond-1.9.0.v20150420-1451-windows64.zip
-rw-rw-r--. 1 dlshudson dls_dasc        21 Apr 20 16:37 product_version_number.txt
-rw-rw-r--. 1 dlshudson dls_dasc 104848904 Apr 20 16:37 squish_tests.zip
-rw-rw-r--. 1 dlshudson dls_dasc 459423471 Apr 20 16:25 uk.ac.diamond.dawn.site_1.7.1.v20150420-1451.zip

mwebber@java.net (JIRA)

unread,
Apr 20, 2015, 11:56:05 AM4/20/15
to jenkinsc...@googlegroups.com
mwebber commented on Bug JENKINS-28013

In fact, I just noticed something strange. If I go to $JENKINS_HOME/jobs on the Jenkins master, and look in <myjob>/builds/<build-number>/archive/ for the job that failed, I see all the archived files that I expect.

It's worth noting that the failing job was running on a slave, but the slave was defined as the same machine as the master.

mwebber@java.net (JIRA)

unread,
Apr 20, 2015, 12:18:05 PM4/20/15
to jenkinsc...@googlegroups.com
mwebber commented on Bug JENKINS-28013

If you want to see the artefacts that I attempted to save, you can get a copy from http://www.opengda.org/downloads/special/archive.zip (be warned it's 2.1GB). Unzip that, and you will have what I was trying to archive.

Note: job specified "files to archive" as:
artifacts_to_archive/, create.product.root/products/.txt, create.product.root/products/*.zip

meng.qingyou@gmail.com (JIRA)

unread,
Apr 21, 2015, 2:31:05 AM4/21/15
to jenkinsc...@googlegroups.com

Same problem on a slave node.
Downgrade to 1.609 fixed the problem.
size of jar files are about 1~2M
size of a tar.gz file is about 37M

==== build log =====

Notifying upstream projects of job completion
Join notifier requires a CauseAction
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] web-common ........................................ SUCCESS [0.587s]
[INFO] web-common-open ................................... SUCCESS [3.574s]
[INFO] web-common-frontend ............................... SUCCESS [1.031s]
[INFO] web-common-backend ................................ SUCCESS [0.808s]
[INFO] web-common-business ............................... SUCCESS [10.231s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19.959s
[INFO] Finished at: Tue Apr 21 14:14:53 CST 2015
[INFO] Final Memory: 30M/165M
[INFO] ------------------------------------------------------------------------
[JENKINS] Archiving /home/jenkinsslave/workspace/workspace/web_dept/web_dept__devprj/考勤v2.0/web-common/web-common-frontend/pom.xml to com.jingoal.web-common/web-common-frontend/1.0.0/web-common-frontend-1.0.0.pom
[JENKINS] Archiving /home/jenkinsslave/workspace/workspace/web_dept/web_dept__devprj/考勤v2.0/web-common/web-common-frontend/target/web-common-frontend-1.0.0.jar to com.jingoal.web-common/web-common-frontend/1.0.0/web-common-frontend-1.0.0.jar
[JENKINS] Archiving /home/jenkinsslave/workspace/workspace/web_dept/web_dept__devprj/考勤v2.0/web-common/web-common-frontend/target/web-common-frontend-1.0.0-sources.jar to com.jingoal.web-common/web-common-frontend/1.0.0/web-common-frontend-1.0.0-sources.jar
ERROR: Failed to parse POMs
java.io.IOException: Failed to extract /home/jenkinsslave/workspace/workspace/web_dept/web_dept__devprj/考勤v2.0/web-common/web-common-frontend/transfer of 3 files


at hudson.FilePath.readFromTar(FilePath.java:2299)
at hudson.FilePath.copyRecursiveTo(FilePath.java:2208)

channel stopped
at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
at hudson.maven.MavenBuild$ProxyImpl.performArchiving(MavenBuild.java:483)
at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:851)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536)
at hudson.model.Run.execute(Run.java:1741)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:531)


at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)
Caused by: java.io.IOException: Truncated TAR archive
at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:614)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
at hudson.util.IOUtils.copy(IOUtils.java:40)
at hudson.FilePath.readFromTar(FilePath.java:2289)

... 9 more
Skipping sonar analysis due to bad build status FAILURE

omar.kohl@lsexperts.de (JIRA)

unread,
Apr 21, 2015, 5:16:05 AM4/21/15
to jenkinsc...@googlegroups.com
Omar Kohl commented on Bug JENKINS-28013

Same issue here after upgrade to 1.610. Jenkins master running on Debian Wheezy. The artifact size is a few MB at most.

mlandman99@gmail.com (JIRA)

unread,
Apr 21, 2015, 9:37:08 AM4/21/15
to jenkinsc...@googlegroups.com
boris ivan commented on Bug JENKINS-28013

Same issue here where build slave is Windows 2012. Artifact only a couple of MB.

o.v.nenashev@gmail.com (JIRA)

unread,
Apr 21, 2015, 10:56:09 AM4/21/15
to jenkinsc...@googlegroups.com

The issue seems to be platform-independent.
BTW, I still cannot reproduce it in unit tests

o.v.nenashev@gmail.com (JIRA)

unread,
Apr 21, 2015, 10:56:13 AM4/21/15
to jenkinsc...@googlegroups.com
Change By: Oleg Nenashev (21/Apr/15 2:54 PM)
Priority: Major Blocker

pollentierk@yahoo.com (JIRA)

unread,
Apr 21, 2015, 12:04:05 PM4/21/15
to jenkinsc...@googlegroups.com
K P commented on Bug JENKINS-28013

Same issue, identical stack trace.
Seems to be platform independent indeed: platform is Windows server 2008. Jobs runs on slave jenkins, which runs on the same same machine (i.e. in the same Windows instance). Artifacts are rather small (.zip of 1.7 MB, .jar of merely 30 KB).
Reverting to 1.609 restores the functionality.

mevans@parchment.com (JIRA)

unread,
Apr 21, 2015, 6:12:08 PM4/21/15
to jenkinsc...@googlegroups.com
Matt Evans commented on Bug JENKINS-28013

Same issue, same stack trace, reverted to 1.609 o fix issue.

marcel@mcbeister.de (JIRA)

unread,
Apr 21, 2015, 6:23:05 PM4/21/15
to jenkinsc...@googlegroups.com

Same issue, same stack trace after update from 1.608 to 1.609.
All nodes run on Windows 7 and artifacts are approximately 10 to 50 MB in size.

marcel@mcbeister.de (JIRA)

unread,
Apr 21, 2015, 6:23:08 PM4/21/15
to jenkinsc...@googlegroups.com
 
Marcel Beister edited a comment on Bug JENKINS-28013

Same issue, same stack trace after update from 1.608 to 1.610.


All nodes run on Windows 7 and artifacts are approximately 10 to 50 MB in size.
This message is automatically generated by JIRA.

drohne1673@gmail.com (JIRA)

unread,
Apr 22, 2015, 3:19:05 AM4/22/15
to jenkinsc...@googlegroups.com

The problem is not coming up on every build. I started building 5 artifacts. 3 of them failed. Have now rolled back to 1.609 as they had the exact same configuration

ogre@muppfarmen.se (JIRA)

unread,
Apr 22, 2015, 7:19:05 AM4/22/15
to jenkinsc...@googlegroups.com

Same problem. Reverted to 1.609 solved it.

o.v.nenashev@gmail.com (JIRA)

unread,
Apr 22, 2015, 2:56:07 PM4/22/15
to jenkinsc...@googlegroups.com

I was unable to reproduce the issue on Java 6/7 and remote Windows/Ubuntu/Mac slaves. Massive load experiments didn't help as well.

Does anybody see the issue on these Java versions? Probably, it could be something specific to Java8 or to the remoting behavior.

baba.ayako@synergy101.jp (JIRA)

unread,
Apr 22, 2015, 8:34:05 PM4/22/15
to jenkinsc...@googlegroups.com
Ayako Baba commented on Bug JENKINS-28013

Same problem.

java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)

a.regentov@ftc.ru (JIRA)

unread,
Apr 22, 2015, 11:31:06 PM4/22/15
to jenkinsc...@googlegroups.com

Oleg, positive. SLES 11, java 1.7.0_45-b18 - the problem appears. Not on every job though. But if the job is "cursed" then "Truncated TAR archive" on every its build.

o.v.nenashev@gmail.com (JIRA)

unread,
Apr 23, 2015, 2:13:05 AM4/23/15
to jenkinsc...@googlegroups.com

The issue has been caused by a binary stream corruption in Tar archiving procedures.
https://github.com/jenkinsci/jenkins/pull/1667 has been created to revert the change

mwebber@java.net (JIRA)

unread,
Apr 23, 2015, 4:11:06 AM4/23/15
to jenkinsc...@googlegroups.com
mwebber commented on Bug JENKINS-28013

I am running Java 8 on both master and slave (both Linux), and get the problem.

From the comments on https://github.com/jenkinsci/jenkins/pull/1667, it looks the maintainers can now reproduce the problem.

scm_issue_link@java.net (JIRA)

unread,
Apr 23, 2015, 12:09:18 PM4/23/15
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Oleg Nenashev
Path:
core/pom.xml
core/src/main/java/hudson/FilePath.java
core/src/main/java/hudson/org/apache/tools/tar/TarInputStream.java
core/src/main/java/hudson/org/apache/tools/tar/TarOutputStream.java
core/src/main/java/hudson/util/io/TarArchiver.java
http://jenkins-ci.org/commit/jenkins/ca21c65bf2538982126bd465eb70840071a61e4e
Log:
Merge pull request #1667 from oleg-nenashev/JENKINS-10629-FIX

[JENKINS-28013,JENKINS-28012] - Revert changes for JENKINS-10629

Compare: https://github.com/jenkinsci/jenkins/compare/8c94920973c4...ca21c65bf253

daniel@beckweb.net (JIRA)

unread,
Apr 23, 2015, 2:22:06 PM4/23/15
to jenkinsc...@googlegroups.com
Daniel Beck resolved Bug JENKINS-28013 as Fixed

Resolved by reverting the fix for JENKINS-10629.
Change By: Daniel Beck (23/Apr/15 6:21 PM)
Status: Open Resolved
Resolution: Fixed
Reply all
Reply to author
Forward
0 new messages