[JIRA] (JENKINS-41805) Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

[hiten.prajapati566@gmail.com (JIRA)]

Hiten Prajapati created an issue

Jenkins / JENKINS-41805 Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace. Issue Type: Bug Assignee: Matthew DeTullio Components: multi-branch-project-plugin Created: 2017/Feb/07 1:34 PM Priority: Major Reporter: Hiten Prajapati Add Comment

This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)

[ryan.campbell@gmail.com (JIRA)]

recampbell closed an issue as Incomplete

Please reopen with step by step instructions to reproduce the problem and clear description of what you expect to happen.

Jenkins / JENKINS-41805 Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Change By: recampbell Status: Open Closed Resolution: Incomplete

Add Comment

[andrew.bayer@gmail.com (JIRA)]

Andrew Bayer updated an issue

Jenkins / JENKINS-41805 Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Change By: Andrew Bayer Component/s: workflow-basic-steps-plugin Component/s: pipeline

Add Comment

This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)

[hiten.prajapati566@gmail.com (JIRA)]

Hiten Prajapati commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace. please Help me this is still problem me cos we have many jobs with many branches in project that's make my ssd full.

Add Comment

[henry.yei@gmail.com (JIRA)]

Henry Yei commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

I run a simple cleanup bash script inside a job every once in awhile on my nodes. #!/usr/bin/env bash get to main job directory cd .. ls -al cd .. ls -al delete all @tmp files find . | grep @tmp$ | xargs -n1 rm -fr Of course you need to make sure you don't have any artifacts that you might match.

Add Comment

[ericdnelson@gmail.com (JIRA)]

Eric Nelson commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

The standard cleanWs() leaves at least the tmp directories behind on the agents. This is a real big problem for people using secret files , as it leaves them behind and they could be discoverable from another job!  I'm going to try the workaround posted by Niels van Aken.  For what its worth Alexander Trauzzi I'm not seeing the orphaned @scipt directories on my jenkins master as of version 2.108 for my multibranch pipeline jobs.

Add Comment

[mdealer@gmail.com (JIRA)]

Edgars Batna commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Not even sure why these directories would even be required. There are system-wide temporary directories that should be used for this. The temporary directories, no matter what implementation detail, should not concern us users.

Add Comment

[mdealer@gmail.com (JIRA)]

Edgars Batna edited a comment on JENKINS-41805

[ing.comp.ibarra@gmail.com (JIRA)]

Cesar Ibarra commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Im having the same issue, but also a folder named WORKSPACE_cleanup is created.

Add Comment

[steve_magness@hotmail.com (JIRA)]

Steve Magness commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Has anyone solved this for the 'master - slave' setup? Our master is rapidly running out of disk space due to mercurial performing a full checkout into @script just to get the jenkinsfile. I'd like a solution that can be implemented within a scripted jenkinsfile eg: node ('slave') {     // do useful build things first     cleanWs() // clean up workspace on slave } cleanWs // clean workspace(s) on master (eg @script @libs directories)

Add Comment

[steve_magness@hotmail.com (JIRA)]

Steve Magness edited a comment on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Has anyone solved this for the 'master - slave' setup? Our master is rapidly running out of disk space due to mercurial performing a full checkout into @script   just to get the jenkinsfile (see https://issues . jenkins-ci.org/browse/JENKINS-50490). I'd like a solution that can be implemented within a scripted jenkinsfile eg:

node ('slave') \{     // do useful build things first     cleanWs() // clean up workspace on slave } cleanWs // clean workspace(s) on master (eg @script @libs directories)

Add Comment

[steve_magness@hotmail.com (JIRA)]

Steve Magness commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

To answer my own question, similar to Niels van Aken solution but running on 'master'

node ('slave') {     // do useful build things first     cleanWs() // clean up workspace on slave }

node ('master') {     dir("${env.WORKSPACE}@libs") {         deleteDir()     }     dir("${env.WORKSPACE}@script") {        deleteDir()     } } Although this only cleans the directories when the stages on the slave succeed. You can use try..catch to catch exceptions from the slave stages and perform the cleanup in a finally block if required.

Add Comment

[sergey.holod@gmail.com (JIRA)]

Sergii Kholod commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Together with "external workspace" plugin: node('master'){ def extWs = exwsAllocate diskPoolId: 'p2000' exws(extWs) Unknown macro: { sh "env | sort >test.txt" sleep time} cleanWs cleanWhenNotBuilt: false, notFailBuild: true } the result is even worse: $ls -la /p2000/test-extws/ total 0 drwxr-xr-x 7 jenkins jenkins 62 Jun 25 13:42 . drwxr-xr-x 3 jenkins jenkins 23 Jun 25 11:38 .. drwxr-xr-x 2 jenkins jenkins 21 Jun 25 11:38 3 drwxr-xr-x 2 jenkins jenkins 6 Jun 25 11:38 3@tmp drwxr-xr-x 2 jenkins jenkins 6 Jun 25 11:45 4@tmp drwxr-xr-x 2 jenkins jenkins 6 Jun 25 11:53 5@tmp drwxr-xr-x 2 jenkins jenkins 6 Jun 25 13:37 6@tmp Temp directories are flooding the workspace parent folder.

Add Comment

This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

[sergey.holod@gmail.com (JIRA)]

Sergii Kholod edited a comment on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Together with "external workspace" plugin:

{quote} {{ node('master'){ }}

{{ def extWs = exwsAllocate diskPoolId: 'p2000' }}

{{ exws(extWs) { }} {{  sh "env | sort >test.txt" }} {{  sleep time: 5, unit: 'MINUTES' }} {{  cleanWs cleanWhenNotBuilt: false, notFailBuild: true }} {{ } }} {{ cleanWs cleanWhenNotBuilt: false, notFailBuild: true } } { {}}}{ quote}

the result is even worse:

{quote}$ls -la /p2000/test-extws/

total 0 drwxr-xr-x 7 jenkins jenkins 62 Jun 25 13:42 . drwxr-xr-x 3 jenkins jenkins 23 Jun 25 11:38 .. drwxr-xr-x 2 jenkins jenkins 21 Jun 25 11:38 3

drwxr-xr-x 2 jenkins jenkins 6 Jun 25 11:38 +3@tmp+ drwxr-xr-x 2 jenkins jenkins 6 Jun 25 11:45 +4@tmp+ drwxr-xr-x 2 jenkins jenkins 6 Jun 25 11:53 +5@tmp+ drwxr-xr-x 2 jenkins jenkins 6 Jun 25 13:37 +6@tmp+ {quote}

Temp directories are flooding the workspace parent folder.

Add Comment

[sergey.holod@gmail.com (JIRA)]

[worldstream@protonmail.com (JIRA)]

J S commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Hello Guys, i have the following Jenkinsfile :   pipeline { options { disableConcurrentBuilds() } agent { label 'vhost01' } stages { [..AllStages..] } post { always { cleanWs() } } } I use multibranch pipeline and have the problem that the folder under my node "vhost01" are not deleted. Can anyone help ?

Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[worldstream@protonmail.com (JIRA)]

J S edited a comment on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Hello Guys, i have the following Jenkinsfile :

{code:java}

pipeline {     options { disableConcurrentBuilds() }     agent { label 'vhost01' }

stages { [..AllStages..]     }    post {         always {             cleanWs()         }     } }

{code}

I use multibranch pipeline and have the problem that the folder under my node "vhost01" are not deleted. Can anyone help ?

Add Comment

[iadar@il.ibm.com (JIRA)]

Idan Adar commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

+1 Our agents remain unclean because of this.

Add Comment

[andreimuresianu1@gmail.com (JIRA)]

Andrei Muresianu commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

+1 still a problem. The tmp directories do not add much value.

Add Comment

[karl.dean.parry@gmail.com (JIRA)]

Karl Parry commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

+1 we now have about 200-300 @tmp/@script folders being created every day across several slave servers. We are using a CRONTAB to delete rogue folders which seems to be working for now.

Add Comment

[karl.dean.parry@gmail.com (JIRA)]

Karl Parry edited a comment on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

+1 we now have about 200-300 @tmp/@script folders being created every day across several slave servers. We are using a CRONTAB to delete rogue folders which seems to be working

--EDIITED-- Will add the script snippet provided above for current jobs for now .

Add Comment

[karl.dean.parry@gmail.com (JIRA)]

Karl Parry edited a comment on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

+1 we now have about 200-300 @tmp/@script folders being created every day across several slave servers.

--EDIITED-- Will add the script snippet provided above for current jobs for now [#comment-326465]

Add Comment

[karl.dean.parry@gmail.com (JIRA)]

Karl Parry edited a comment on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

+1 we now have about 200-300 @tmp/@script folders being created every day across several slave servers. --EDIITED--

Will add the  [ script snippet provided above for current jobs for now [ |https://issues.jenkins-ci.org/browse/JENKINS-41805?focusedCommentId=326465&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel #comment-326465]

Add Comment

[jielpe-cblv38@protonmail.com (JIRA)]

jlpinardon commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

+1 I have added in a post always block a set of folderDelete operation... As far as I have only one slave, it is sustainable, but it will become ugly when using a label referencing several slaves.

Add Comment

[aheritier@apache.org (JIRA)]

Arnaud Héritier assigned an issue to Unassigned

Jenkins / JENKINS-41805

Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Change By: Arnaud Héritier Assignee: Ryan Campbell

Add Comment

[alexander.samoylov@gmail.com (JIRA)]

Alexander Samoylov commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace. Baptiste Mathus wrote: "Not a bug by the way, more an improvement." I strongly disagree. Jenkins creates the @tmp automatically and stores there temporary files. Therefore it should be removed also automatically by Jenkins. Each tool that produces temporary data should be responsible for its removal. It is easy as ABC. Following your logic, memory leaks are also "not a bugs"... +1 for the fix (which must be trivial)

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[alexander.samoylov@gmail.com (JIRA)]

Alexander Samoylov edited a comment on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

[~batmat] wrote: "Not a bug by the way, more an improvement."

I strongly disagree. Jenkins creates the @tmp automatically and stores there temporary files.  Therefore it should be removed also automatically by Jenkins. Each tool that produces temporary data should be responsible for its removal. It is easy as ABC. Following your logic, memory leaks are also "not a bugs"... +1 for the fix (which must be trivial)

Update: I confirm that the workaround dir(<dir>) { deleteDir() } is working. Luckily it does not create the nested @tmp@tmp. Thank you, [~vanaken].

Add Comment

[alexander.samoylov@gmail.com (JIRA)]

Alexander Samoylov edited a comment on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

[~batmat] wrote: "Not a bug by the way, more an improvement." I strongly disagree. Jenkins creates the @tmp automatically and stores there temporary files.  Therefore it should be removed also automatically by Jenkins. Each tool that produces temporary data should be responsible for its removal. It is easy as ABC. Following your logic, memory leaks are also "not a bugs"... +1 for the fix (which must be trivial)

Update: I confirm that the workaround dir(<dir> + '@tmp' ) { deleteDir() } is working. Luckily it does not create the nested @tmp@tmp. Thank you, [~vanaken].

Add Comment

[ogondza@gmail.com (JIRA)]

Oliver Gondža commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Seeing the workarounds, I am wondering how wise it is to delete the pipeline helper folder(s) while the pipeline is still running.

Add Comment

[szaskalski@gmail.com (JIRA)]

Szczepan Zaskalski commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Why my jenkins create tmp directories in the master workspace for Jenkins Shared library instead of using agent workspace?

Add Comment

[steven.christenson@CISCO.COM (JIRA)]

Steven Christenson commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Szczepan Zaskalski: That directory is on the MASTER to allow you to do Replay operations.  It contains the content of any library that was loaded at run time. Without a copy, Replay can't work reliably. Oliver Gondža: Indeed, deleting scripts that may well be executing is unsafe. However in general deleteDir(), IMHO should remove any copy on a slave (provided the slave isn't actually the master)

Add Comment

[karing.martin@gmail.com (JIRA)]

Martin Karing commented on JENKINS-41805

Re: Pipeline Job-- deletedir() delete only current directory but @script and @tmp dir still there in workspace.

Steven Christenson: I don't think the directory is required for the replay operation. If I delete the @libs workspace on the master by hand, replay still work fine. There is a copy of the required files of the library in the directory of all builds. I am guessing this one is utilized for what ever the replay requires it for, or the required commit is just checked out from the repository again. No matter, there is still a @libs directory in the directory for the master workspaces, that just sits there forever. It contains the in my case implicitly loaded shared libraries for each job. And there are a lot of them due to heavy use of feature branches and pull requests. It can't be deleted by the pipeline and it's not automatically deleted if the associated job in Jenkins is deleted. Also I do not have any build processors enabled on the master node, so that workaround with switching to the problematic directory directly does not fly as well, because I can't get into the master workspaces with the pipelines as far as I know. The one way to get rid of those directories I was able to come up with is to have the server run a nightly cron job that clears our the workspace directory of the master, so the issue does not get out of hand. That solution works, but I'd rather solve that issue inside of Jenkins.

Add Comment

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)