[JIRA] (JENKINS-51913) Post-build action "Archive the artifacts" prints exception to console when no artifacts found

1,520 views
Skip to first unread message

pholden@greenhead.ac.uk (JIRA)

unread,
Jun 13, 2018, 7:20:04 AM6/13/18
to jenkinsc...@googlegroups.com
Paul Holden created an issue
 
Jenkins / Bug JENKINS-51913
Post-build action "Archive the artifacts" prints exception to console when no artifacts found
Issue Type: Bug Bug
Assignee: Unassigned
Components: core
Created: 2018-06-13 11:19
Environment: Jenkins version 2.121.1
Labels: ArtifactArchiver
Priority: Minor Minor
Reporter: Paul Holden

In my job configuration, I have Do not fail build if archiving returns nothing checked under Post-build Actions > Archive the artifacts so that build still completes even if no artifacts are created (this is intentional). The help icon describes this field as such:

Normally, a build fails if archiving returns zero artifacts. This option allows the archiving process to return nothing without failing the build. Instead, the build will simply throw a warning.

In Jenkins 2.107.3 this causes the following warning to be printed to the build console:

15:30:26 Archiving artifacts
15:30:27 WARN: No artifacts found that match the file pattern "**/data/behat-failure/**/*". Configuration error?
15:30:28 WARN: java.lang.InterruptedException: no matches found within 10000

However, in Jenkins 2.121.1 the following big ugly stack trace is now printed to the build console:

12:28:36 Archiving artifacts
12:28:36 java.lang.InterruptedException: no matches found within 10000
12:28:36 	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
12:28:36 	at hudson.FilePath$34.invoke(FilePath.java:2557)
12:28:36 	at hudson.FilePath$34.invoke(FilePath.java:2547)
12:28:36 	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
12:28:36 Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to [redacted]
12:28:36 		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
12:28:36 		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
12:28:36 		at hudson.remoting.Channel.call(Channel.java:955)
12:28:36 		at hudson.FilePath.act(FilePath.java:1036)
12:28:36 		at hudson.FilePath.act(FilePath.java:1025)
12:28:36 		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
12:28:36 		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
12:28:36 		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
12:28:36 		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
12:28:36 		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
12:28:36 		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
12:28:36 		at hudson.model.Build$BuildExecution.post2(Build.java:186)
12:28:36 		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
12:28:36 		at hudson.model.Run.execute(Run.java:1819)
12:28:36 		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
12:28:36 		at hudson.model.ResourceController.execute(ResourceController.java:97)
12:28:36 		at hudson.model.Executor.run(Executor.java:429)
12:28:36 Caused: hudson.FilePath$TunneledInterruptedException
12:28:36 	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
12:28:36 	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
12:28:36 	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
12:28:36 	at hudson.remoting.Request$2.run(Request.java:369)
12:28:36 	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
12:28:36 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
12:28:36 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
12:28:36 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
12:28:36 	at java.lang.Thread.run(Thread.java:748)
12:28:36 Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
12:28:36 	at hudson.FilePath.act(FilePath.java:1038)
12:28:36 	at hudson.FilePath.act(FilePath.java:1025)
12:28:36 	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
12:28:36 	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
12:28:36 	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
12:28:36 	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
12:28:36 	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
12:28:36 	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
12:28:36 	at hudson.model.Build$BuildExecution.post2(Build.java:186)
12:28:36 	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
12:28:36 	at hudson.model.Run.execute(Run.java:1819)
12:28:36 	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
12:28:36 	at hudson.model.ResourceController.execute(ResourceController.java:97)
12:28:36 	at hudson.model.Executor.run(Executor.java:429)
12:28:36 No artifacts found that match the file pattern "**/data/behat-failure/**/*". Configuration error?
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

pholden@greenhead.ac.uk (JIRA)

unread,
Jun 26, 2018, 11:53:04 AM6/26/18
to jenkinsc...@googlegroups.com
Paul Holden commented on Bug JENKINS-51913
 
Re: Post-build action "Archive the artifacts" prints exception to console when no artifacts found

This is the commit from JENKINS-47142 that appears to have led to this change, and caused the full stacktrace to be printed to the build console
This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

jglick@cloudbees.com (JIRA)

unread,
Jul 19, 2018, 5:40:01 PM7/19/18
to jenkinsc...@googlegroups.com

Seems to be a poor interaction with an old fix of JENKINS-7214. Jenkins is dutifully trying to look up artifacts matching the pattern you provided, but it gave up after examining ten thousand (!) files. Try a more specific file pattern.

pholden@greenhead.ac.uk (JIRA)

unread,
Jul 23, 2018, 5:23:01 AM7/23/18
to jenkinsc...@googlegroups.com

Hi Jesse Glick, I'm not expecting it to find any artifacts (they would only exist if a Behat step failed); the problem is that the stacktrace I included in this issue description is now printed to the build console when no artifacts exist, which seems to be a regression caused by 2.121.1, introduced in JENKINS-47142.

I have Do not fail build if archiving returns nothing checked for the job.

paul.thevenot@atos.net (JIRA)

unread,
Nov 26, 2018, 5:01:02 AM11/26/18
to jenkinsc...@googlegroups.com

Same issue here. Do not fail build if archiving returns nothing is checked so I don't expect to have a stacktrace if no artifact has been found.
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

pholden@greenhead.ac.uk (JIRA)

unread,
Jan 21, 2019, 12:29:02 PM1/21/19
to jenkinsc...@googlegroups.com

Just a follow up to state that the suggestion by Jesse Glick to:

Try a more specific file pattern.

Has resolved this for us: we no longer get the exception in job logs when no artifacts are found; and artifacts are successfully archived when they do exist

vincent@massol.net (JIRA)

unread,
Mar 14, 2019, 5:57:02 AM3/14/19
to jenkinsc...@googlegroups.com

I'm having the same issue and I don't see how I could use a more specific pattern. This is what I'm using:

        // Save videos generated by Docker-based tests, if any
        archiveArtifacts artifacts: '**/target/*/*.flv', allowEmptyArchive: true

        // Save images generated by functional tests, if any
        archiveArtifacts artifacts: '**/target/screenshots/*.png', allowEmptyArchive: true

And when executing on a small project (https://ci.xwiki.org/view/All%20Failed%20Builds/job/XWiki%20Contrib/job/latex/job/master/34/console), I get:

[Pipeline] archiveArtifacts
Archiving artifacts
[Pipeline] archiveArtifacts
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2847)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2726)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2707)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3086)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to Jenkins SSH Slave-0015psb3jxva1
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1072)
		at hudson.FilePath.act(FilePath.java:1061)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80)
		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67)
		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3088)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1074)
	at hudson.FilePath.act(FilePath.java:1061)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
No artifacts found that match the file pattern "**/target/screenshots/*.png". Configuration error?

Interestingly the first archive didn't throw any stack trace but it's less specific than the second one!

Any idea is most welcome. Thx

vincent@massol.net (JIRA)

unread,
Mar 14, 2019, 5:59:04 AM3/14/19
to jenkinsc...@googlegroups.com
Vincent Massol edited a comment on Bug JENKINS-51913
I'm having the same issue and I don't see how I could use a more specific pattern. This is what I'm using:

{noformat}

        // Save videos generated by Docker-based tests, if any
        archiveArtifacts artifacts: '**/target/*/*.flv', allowEmptyArchive: true

        // Save images generated by functional tests, if any
        archiveArtifacts artifacts: '**/target/screenshots/*.png', allowEmptyArchive: true
{noformat}
{noformat}
{noformat}


Interestingly the first archive didn't throw any stack trace but it's less specific than the second one!

Any idea is most welcome. Thx

EDIT: I think the reason we don't get a stack trace for the first archive is because there was one match in this case. So it seems the stack trace is only displayed when there's no match which really goes against the "allowEmptyArchive" parameter...

vincent@massol.net (JIRA)

unread,
Mar 14, 2019, 5:59:05 AM3/14/19
to jenkinsc...@googlegroups.com

jglick@cloudbees.com (JIRA)

unread,
Mar 18, 2019, 3:22:03 PM3/18/19
to jenkinsc...@googlegroups.com

Not sure offhand of details. Workaround would generally be to use shell scripts running (e.g.) find to collect relevant files into a single directory before using a simpler Jenkins directive to archive them.

pholden@greenhead.ac.uk (JIRA)

unread,
Mar 21, 2019, 6:25:02 AM3/21/19
to jenkinsc...@googlegroups.com

Hi Vincent Massol,

The resolution in our case was to change the Files archive path from **/data/behat-failure/**/* to data/behat-failure/**/*, as the documentation states that the path is relative to the workspace root.

 

vincent@massol.net (JIRA)

unread,
Mar 21, 2019, 6:34:02 AM3/21/19
to jenkinsc...@googlegroups.com

Paul Holden Very interesting. I'll try that. Does it mean that if you use a leading "/" it searches in all workspaces and not in the current one? Thx

vincent@massol.net (JIRA)

unread,
Mar 21, 2019, 6:35:05 AM3/21/19
to jenkinsc...@googlegroups.com
Vincent Massol edited a comment on Bug JENKINS-51913
[~pholden] Very interesting. I'll try that. Does it mean that if you use a leading "/" it searches in all workspaces and not in the current one? Thx

EDIT: In our cases we don't have a leading "/"... just " {{ **/target/*/*.flv " }} for ex

vincent@massol.net (JIRA)

unread,
Mar 21, 2019, 6:35:06 AM3/21/19
to jenkinsc...@googlegroups.com

vincent@massol.net (JIRA)

unread,
Mar 21, 2019, 6:36:02 AM3/21/19
to jenkinsc...@googlegroups.com
Vincent Massol edited a comment on Bug JENKINS-51913
[~pholden] Very interesting. I'll try that. Does it mean that if you use a leading "/" it searches in all workspaces and not in the current one? Thx

EDIT: In our cases we don't have a leading "/"... just { { code} **/target/*/*.flv {code } } for ex

vincent@massol.net (JIRA)

unread,
Mar 21, 2019, 6:37:04 AM3/21/19
to jenkinsc...@googlegroups.com
Vincent Massol edited a comment on Bug JENKINS-51913
[~pholden] Very interesting. I'll try that. Does it mean that if you use a leading "/" it searches in all workspaces and not in the current one? Thx

EDIT: In our cases we don't have a leading "/"... just {code}**/target/*/*.flv{code}for ex
EDIT2:

bq. Does it mean that if you use a leading "/" it searches in all workspaces and not in the current one?

I checked the doc and it does say that the root is the workspace, so I don't see the difference it would have. A bug?

vincent@massol.net (JIRA)

unread,
Mar 21, 2019, 6:37:04 AM3/21/19
to jenkinsc...@googlegroups.com
Vincent Massol edited a comment on Bug JENKINS-51913
[~pholden] Very interesting. I'll try that. Does it mean that if you use a leading "/" it searches in all workspaces and not in the current one? Thx

EDIT: In our cases we don't have a leading "/"... just {code}**/target/*/*.flv{code}for ex

bq. Does it mean that if you use a leading "/" it searches in all workspaces and not in the current one?

I checked the doc and it does say that the root is the workspace, so I don't see the difference it would have. A bug?

vincent@massol.net (JIRA)

unread,
Mar 23, 2019, 10:25:04 AM3/23/19
to jenkinsc...@googlegroups.com

Jesse Glick Thanks for the workaround idea. That's the only one that worked for me so far. For those interested, this is what I ended up with:

        // Save videos generated by Docker-based tests, if any
        // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        echoXWiki "Looking for test videos in ${pwd()}"
        def findCommand = "find . -path '*/target/*' -type f -name '*.flv' -exec cp {} tmp-jenkins-flv/ \\;"
        sh "mkdir -p tmp-jenkins-flv; ${findCommand}"
        archiveArtifacts artifacts: "tmp-jenkins-flv/*.flv", allowEmptyArchive: true

        // Save images generated by functional tests, if any
        // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        echoXWiki "Looking for test failure images in ${pwd()}"
        findCommand = "find . -path '*/target/screenshots' -type f -name '*.png' -exec cp {} tmp-jenkins-png/ \\;"
        sh "mkdir -p tmp-jenkins-png; ${findCommand}"
        archiveArtifacts artifacts: "tmp-jenkins-png/*.png", allowEmptyArchive: true

vincent@massol.net (JIRA)

unread,
Mar 29, 2019, 4:22:05 AM3/29/19
to jenkinsc...@googlegroups.com
Vincent Massol edited a comment on Bug JENKINS-51913
I've improve improved a bit my script:

{code}

        // Save videos generated by Docker-based tests, if any
        // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        echoXWiki "Looking for test videos in ${pwd()}"
        def findPath = "-path '*/target/*' -not \\( -path '*/tmp-jenkins-flv/*' -prune \\)"
        def rsyncCommand = "rsync -R {} tmp-jenkins-flv/"
        sh "find . ${findPath} -type f -name '*.flv' -exec ${rsyncCommand} \\;"
        archiveArtifacts artifacts: "tmp-jenkins-flv/**/*.flv", allowEmptyArchive: true


        // Save images generated by functional tests, if any
        // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        // Note: we look for screenshots only in the screenshots directory to avoid false positives such as PNG images
        // that would be located in a XWiki distribution located in target/.

        echoXWiki "Looking for test failure images in ${pwd()}"
        findPath = "-path '*/target/*/screenshots' -not \\( -path '*/tmp-jenkins-png/*' -prune \\)"
        rsyncCommand = "rsync -R {} tmp-jenkins-png/"
        sh "find . ${findPath} -type f -name '*.png' -exec ${rsyncCommand} \\;"
        archiveArtifacts artifacts: "tmp-jenkins-png/**/*.png", allowEmptyArchive: true
{code}

[~jglick] However, this morning, I noticed that it's still failing with:

{noformat}
➡ Looking for test videos in /home/hudsonagent/jenkins_root/workspace/ki_xwiki-platform_stable-10.11.x
[Pipeline] sh
+ find . -path */target/* -not ( -path */tmp-jenkins-flv/* -prune ) -type f -name *.flv -exec rsync -R {} tmp-jenkins-flv/ ;

[Pipeline] archiveArtifacts
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2802)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2711)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2662)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3041)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to agent-1-3
  at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
  at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
  at hudson.remoting.Channel.call(Channel.java:957)
  at hudson.FilePath.act(FilePath.java:1068)
  at hudson.FilePath.act(FilePath.java:1057)
  at hudson.FilePath.validateAntFileMask(FilePath.java:2660)

  at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
  at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80)
  at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67)
  at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:748)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3043)

at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1070)
at hudson.FilePath.act(FilePath.java:1057)
at hudson.FilePath.validateAntFileMask(FilePath.java:2660)

at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80)
at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
No artifacts found that match the file pattern "tmp-jenkins-flv/**/*.flv". Configuration error?
{noformat}

So it doesn't seem that the workaround to use a temporary directory is working.

Note: In this case, the directory exists and is empty.

vincent@massol.net (JIRA)

unread,
Mar 29, 2019, 4:22:05 AM3/29/19
to jenkinsc...@googlegroups.com

I've improve a bit my script:

        // Save videos generated by Docker-based tests, if any
        // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        echoXWiki "Looking for test videos in ${pwd()}"
        def findPath = "-path '*/target/*' -not \\( -path '*/tmp-jenkins-flv/*' -prune \\)"
        def rsyncCommand = "rsync -R {} tmp-jenkins-flv/"
        sh "find . ${findPath} -type f -name '*.flv' -exec ${rsyncCommand} \\;"
        archiveArtifacts artifacts: "tmp-jenkins-flv/**/*.flv", allowEmptyArchive: true

        // Save images generated by functional tests, if any
        // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        // Note: we look for screenshots only in the screenshots directory to avoid false positives such as PNG images
        // that would be located in a XWiki distribution located in target/.
        echoXWiki "Looking for test failure images in ${pwd()}"
        findPath = "-path '*/target/*/screenshots' -not \\( -path '*/tmp-jenkins-png/*' -prune \\)"
        rsyncCommand = "rsync -R {} tmp-jenkins-png/"
        sh "find . ${findPath} -type f -name '*.png' -exec ${rsyncCommand} \\;"
        archiveArtifacts artifacts: "tmp-jenkins-png/**/*.png", allowEmptyArchive: true

Jesse Glick However, this morning, I noticed that it's still failing with:

So it doesn't seem that the workaround to use a temporary directory is working.

Note: In this case, the directory exists and is empty.

vincent@massol.net (JIRA)

unread,
Mar 29, 2019, 4:26:02 AM3/29/19
to jenkinsc...@googlegroups.com

Side note: I'm using a "rsync -R" command because I wanted to keep the path of the artifacts because I have several artifacts with the same name at different locations and I was hoping that "archiveArtifacts" would support this and show the artifacts in a tree for example or with a full path. At some point, for some job, I saw a tree but I can't find it again.

jglick@cloudbees.com (JIRA)

unread,
Mar 29, 2019, 10:37:04 AM3/29/19
to jenkinsc...@googlegroups.com

Without having dug into the details of whether validateAntFileMask preserves Ant’s own optimizations that allow fileset scans to be more efficient when the pattern is “rooted” in a literal directory prefix, I would suggest checking whether this helps:

dir('tmp-jenkins-png') {
  archiveArtifacts artifacts: "**/*.png", allowEmptyArchive: true
}

which would guarantee that the validator is not peeking outside this directory.

Of course if your copy command has selected only PNG files to begin with, you could just use artifacts: '**' as the pattern.

vincent@massol.net (JIRA)

unread,
Mar 29, 2019, 12:27:02 PM3/29/19
to jenkinsc...@googlegroups.com

Jesse Glick Thanks for the idea, I've implemented it. Let's see if it works better. Will report back here.

vincent@massol.net (JIRA)

unread,
Apr 5, 2019, 5:43:05 AM4/5/19
to jenkinsc...@googlegroups.com

Jesse Glick So I've just realized that the "find" command is not working after all and I can't figure it out. It seems to be a potential bug of the "sh" step. Would be great if someone could confirm and whether I should open a jira issue for this (I'm still considering that it could be me making a mistake somewhere but I don't see it).

This is what I do:

        // Save videos generated by Docker-based tests, if any
        // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        echoXWiki "Looking for test videos in ${pwd()}"
        sh "find . -path '*/target/*' -not \\( -path '*/tmp-jenkins-flv/*' -prune \\)\
            -type f -name '*.flv' -exec rsync -R {} 'tmp-jenkins-flv/' \\;"
        dir('tmp-jenkins-flv') {
            archiveArtifacts artifacts: '**', allowEmptyArchive: true
        }

        // Save images generated by functional tests, if any
        // Note: We use the "find" shell command since there's some limitation currently with archiveArtifacts,
        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        // Note: we look for screenshots only in the screenshots directory to avoid false positives such as PNG images
        // that would be located in a XWiki distribution located in target/.
        echoXWiki "Looking for test failure images in ${pwd()}"
        sh "find . -path '*/target/*/screenshots' -not \\( -path '*/tmp-jenkins-png/*' -prune \\)\
            -type f -name '*.png' -exec rsync -R {} 'tmp-jenkins-png/' \\;"
        dir('tmp-jenkins-png') {
            archiveArtifacts artifacts: '**', allowEmptyArchive: true
        }

 
                                                            

                                                        
 
And when it executes on our agent in the log it says either one of these:
 
When executed using our Jenkins Docker Agent image:
 
                                                        

                                                            
 
                                                                
[Pipeline] sh
+ find . -path '*/target/*/screenshots' -not '(' -path '*/tmp-jenkins-png/*' -prune ')' -type f -name '*.png' -exec rsync -R '{}' tmp-jenkins-png/ ';'

 
                                                            

                                                        
 
When executed using the "standard" non-docker SSH agent:
 
                                                        

                                                            
 
                                                                
[Pipeline] sh
+ find . -path */target/*/screenshots -not ( -path */tmp-jenkins-png/* -prune ) -type f -name *.png -exec rsync -R {} tmp-jenkins-png/ ;

 
                                                            

                                                        
 
Both are invalid and I can't figure out why.
 
Side note: When going through this intermediary directory"tmp-jenkins-*", the directories are not cleaned between each job execution and thus the artifacts accumulate at each run... So if you use this strategy, you need to take care of cleaning the directory (which is complex in our setup).
 
FTM I've had to revert to the previous direct call of archiveArtifacts:
 
                                                        

                                                            
 
                                                                
        // Save videos generated by Docker-based tests, if any

        // This can generate some not nice stack trace in the logs,
        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        echoXWiki "Looking for test failure videos in ${pwd()}"
        archiveArtifacts artifacts: '*/target/*', allowEmptyArchive: true

        // Save images generated by functional tests, if any

        // This can generate some not nice stack trace in the logs,

        // see https://issues.jenkins-ci.org/browse/JENKINS-51913
        // Note: we look for screenshots only in the screenshots directory to avoid false positives such as PNG images
        // that would be located in a XWiki distribution located in target/.
        echoXWiki "Looking for test failure images in ${pwd()}"
        archiveArtifacts artifacts: '*/target/*/screenshots', allowEmptyArchive: true

 
                                                            

                                                        
 
https://github.com/xwiki/xwiki-jenkins-pipeline/commit/411cf4d1a2b7f206c11d01b2914fccaf403be41d
 
Thanks
 		 
                                                
 
                                            
                                        
 		 
                                
jglick@cloudbees.com (JIRA)
unread,
Apr 5, 2019, 8:37:02 AM4/5/19
to jenkinsc...@googlegroups.com


     
         
         
         
         
     
    
 
                                     
                                        
 
                                            
                                                
 
                                                     
I suspect that the two executions are differing in shell: bash vs. traditional sh / dash. At any rate, it is hard to follow your string escapes there. I strongly recommend using single quotes for sh steps, not only to improve legibility but to ensure you do not accidentally use GString interpolation (which is rarely what you want—see inline help for withCredentials for the gory details). If the command itself has single quotes, use a Groovy multiline string:
 
                                                        

                                                            
 
                                                                
sh '''
    find . -path '*/target/*/screenshots' …
'''

 
                                                            

                                                        
 
which will be more readable.
 
Or better yet, just move all this stuff into your regular build script so it produces the desired artifacts in a tidy directory ready for Jenkins to publish, which is something you can easily test locally. There is no reason to clutter Jenkinsfile with complex embedded scripts.
 
Not sure why cleaning a directory would be complex in any setup. rm -rf tmp-jenkins-png && mkdir tmp-jenkins-png does not suffice?
 		 
                                                
 
                                            
                                        
 		 
                                
vincent@massol.net (JIRA)
unread,
Apr 5, 2019, 8:53:02 AM4/5/19
to jenkinsc...@googlegroups.com


     
         
         
         
         
     
    
 
                                     
                                        
 
                                            
                                                
 
                                                     
Jesse Glick I agree with you about about single quote vs double quotes. I usually use single quotes to the max and this is what I was doing but I need GString interpolation before when I had written it as https://github.com/xwiki/xwiki-jenkins-pipeline/blob/f8c4f536dcbb0524f9348c989003857a540bc70d/vars/xwikiBuild.groovy#L190
 
Then I forgot to replace the double quotes by single quotes after I refactored it...
 
That said using single quotes shouldn't change anything to the problem. There's no GString interpolation in my string AFAICS (no dollar sign). Thus the problem will remain IMO.
 
                                                        

                                                            
Or better yet, just move all this stuff into your regular build script so it produces the desired artifacts in a tidy directory ready for Jenkins to publish, which is something you can easily test locally. There is no reason to clutter Jenkinsfile with complex embedded scripts.

                                                        
 
This is what our regular Maven build script does. It generates these results in the `target/` directory (i.e. in a clean location). The issue here is the maven reactor and multimodules. Our build script is fine IMO and gathering of artifacts is not a build responsibility. All we're asking Jenkins is to archive some of the artifacts in `target/` because we're using Docker Cloud and the docker agent container is removed after each run and thus we need to save some of the artifacts in the job results.
 
BTW all this is done in our pipeline library and not inside the Jenkinsfile which remains clean and generic.
 
                                                        

                                                            
Not sure why cleaning a directory would be complex in any setup. rm -rf tmp-jenkins-png && mkdir tmp-jenkins-png does not suffice?

                                                        
 
Yes, I think I was wrong here. We should be able to do this just before we do the checkout in our pipeline library.
 
Side note: I thought that the `clearWorkspace` parameter in the `checkout` step (https://jenkins.io/doc/pipeline/steps/workflow-scm-step/) would clean the workspace but apparently it does not (it probably does something else). I tried to find the source code that uses this parameter to see what it does but couldn't track it (I ended up in SCM.java but I see no use of it there either). I checked it by creating a file in the workspace and then calling the pipeline and the file remained.
 
Thanks
 		 
                                                
 
                                            
                                        
 		 
                                
Reply all
Reply to author
Forward
0 new messages