[JIRA] (JENKINS-52404) Job fails randomly when using copyArtifact

Issue Type:	Bug
Assignee:	Unassigned
Components:	copyartifact-plugin
Created:	2018-07-06 06:50
Environment:	Jenkins ver. 2.107.3 Win10x64 copy-artifact-plugin 1.39.1
Priority:	Minor
Reporter:	Marco Steffan

Jobs randomly fail when copying artifacts

 
                                                                [Pipeline] [DSWZ10001] }
[Pipeline] [DSWZ10022] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Archive)
[Pipeline] copyArtifacts
Copied 5 artifacts from "AutoDevSuite_on_node" build number 65
[Pipeline] copyArtifacts
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Also:   java.nio.file.AccessDeniedException: C:\appl\Jenkins\fingerprints\af\90\atomic5419828718498066796tmp -> C:\appl\Jenkins\fingerprints\af\90\f9276921737b82db27ef4b8b8710.xml
		at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
		at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
		at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
		at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
		at java.nio.file.Files.move(Unknown Source)
		at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:191)
java.nio.file.FileAlreadyExistsException: C:\appl\Jenkins\fingerprints\af\90\atomic5419828718498066796tmp -> C:\appl\Jenkins\fingerprints\af\90\f9276921737b82db27ef4b8b8710.xml
	at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
	at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
	at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
	at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
	at java.nio.file.Files.move(Unknown Source)
	at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:206)
	at hudson.model.Fingerprint.save(Fingerprint.java:1295)
	at hudson.model.Fingerprint.save(Fingerprint.java:1245)
	at hudson.model.Fingerprint.add(Fingerprint.java:1031)
	at hudson.model.Fingerprint.addFor(Fingerprint.java:1023)
	at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:113)
Caused: hudson.util.IOException2: Failed to copy C:\appl\Jenkins\jobs\AutoDevSuite_on_node\builds\66\archive\LibraryInstaller_4022_1.1.2.1_unversioned_20180706_084642.zip to C:\appl\Jenkins\workspace\AutoDevSuite\LibraryInstaller_4022_1.1.2.1_unversioned_20180706_084642.zip
	at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:118)
	at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyAll(FingerprintingCopyMethod.java:67)
	at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:527)
	at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:461)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1$1.call(SynchronousNonBlockingStepExecution.java:49)
	at hudson.security.ACL.impersonate(ACL.java:290)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1.run(SynchronousNonBlockingStepExecution.java:46)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Finished: FAILURE 
                                                            

This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

marco.steffan@gmx.at (JIRA)

unread,

Jul 6, 2018, 3:03:03 AM7/6/18

to jenkinsc...@googlegroups.com

Marco Steffan updated an issue

Jenkins /

Change By:	Marco Steffan
Environment:	Jenkins ver. 2.107.3 Win10x64 Windows 10x64 on master and slave-nodes copy-artifact-plugin 1.39.1

marco.steffan@gmx.at (JIRA)

unread,

Jul 6, 2018, 3:03:04 AM7/6/18

to jenkinsc...@googlegroups.com

Marco Steffan updated an issue

Jenkins /

Change By:	Marco Steffan

Jobs As the title states, the job fails randomly fail when copying artifacts . Thus, I could not find out any more precise scenarios where the error occurs.

{noformat}

Finished: FAILURE{noformat}

mathew.helmet+jenkinsb63@gmail.com (JIRA)

unread,

Oct 19, 2018, 5:40:02 AM10/19/18

to jenkinsc...@googlegroups.com

Mathias Helminger commented on

Re: Job fails randomly when using copyArtifact

In our case, these exceptions appear to relate to kaspersky antivirus running on the jenkins server. The underlying problem is that the antivirus file scans are integrated in windows in an asynchronous way (for performance/design reasons) Therefore, the scans can temporarily lock files or keep them alive for a short time after a delete operation appears to have succeeded.

For us the fingerprint exception above has become the most frequent cause for random build failures (p~1/200) and we are wondering if this code properly uses the retry infrastructure built elsewhere in jenkins. (Turning kaspersky off temporarily helped but we do want to keep it running)

Looking at hudson.model.Fingerprint.save() it appears there is no retry implemented, but doing so on top of AtomicFileWriter.commit() sounds like potential trouble...

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

mathew.helmet+jenkinsb63@gmail.com (JIRA)

unread,

Oct 19, 2018, 5:43:02 AM10/19/18

to jenkinsc...@googlegroups.com

Mathias Helminger edited a comment on

Re: Job fails randomly when using copyArtifact

In our case, these exceptions appear to relate to kaspersky antivirus running on the jenkins server. The underlying problem is that the antivirus file scans are integrated in windows in an asynchronous way (for performance/design reasons) Therefore, the scans can temporarily lock files or keep them alive for a short time after a delete operation appears to have succeeded.

For us the fingerprint exception above has become the most frequent cause for random build failures (p~1/200) and we are wondering if this code properly uses the retry infrastructure built elsewhere in jenkins. ( JENKINS-15331) ( Turning kaspersky off temporarily helped but we do want to keep it running)

*Looking at hudson.model.Fingerprint.save() it appears there is no retry implemented, but doing so on top of AtomicFileWriter.commit() sounds like potential trouble...*

marco.steffan@gmx.at (JIRA)

unread,

Oct 19, 2018, 7:28:02 AM10/19/18

to jenkinsc...@googlegroups.com

Marco Steffan commented on

Re: Job fails randomly when using copyArtifact

Mathias Helminger We have already excluded these Directories from the AV-scanner which has lowered the number of exceptions by roughly 50%. We are now at a point where 1/15 builds fail due to such errors which is better but still far from good...

https://issues.jenkins-ci.org/browse/JENKINS-53187?focusedCommentId=347311&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-347311 might be of interest for you.

devld@ikedam.jp (JIRA)

unread,

May 10, 2019, 11:58:03 PM5/10/19

to jenkinsc...@googlegroups.com

ikedam updated

There’s no fundamental resolution for this issue as it comes from the OS level issue.
And I don’t plan to add retry feature to copyartifact as it cause codes complicated and hard to maintain.

Please consider workarounds like excluding scan directory, remove files in the target directory before copying, and so on.

Jenkins /

Change By:	ikedam
Status:	Open Fixed but Unreleased
Assignee:	Marco Steffan
Resolution:	Won't Fix