Do not delete a completed job workspace under all conditions

[Robert Beddow]

Hi,

I'm trying to find a way to keep a workspace and prevent it from being deleted by Jenkins.

If I have a project called My_Jenkins_Job, and I run it multiple times concurrently, I end up with directories in the form:

./jenkins/workspace/My_Jenkins_Job

./jenkins/workspace/My_Jenkins_Job@2

./jenkins/workspace/My_Jenkins_Job@3

./jenkins/workspace/My_Jenkins_Job@4

If I later run My_Jenkins_Job again, and some of the above are finished, the lowest value workspace is removed and the new job runs under the same directory path.

e.g.

Running: ./jenkins/workspace/My_Jenkins_Job

Finished: ./jenkins/workspace/My_Jenkins_Job@2

Finished: ./jenkins/workspace/My_Jenkins_Job@3

Running: ./jenkins/workspace/My_Jenkins_Job@4

I start a new My_Jenkins_Job, ./jenkins/workspace/My_Jenkins_Job@2 is deleted, then the new job will run under a new ./jenkins/workspace/My_Jenkins_Job@2.

My request is to find out how to occasionally force jenkins to skip a finished directory because I want to keep it.

So:

Running: ./jenkins/workspace/My_Jenkins_Job

Finished & keep: ./jenkins/workspace/My_Jenkins_Job@2

Finished: ./jenkins/workspace/My_Jenkins_Job@3

Running: ./jenkins/workspace/My_Jenkins_Job@4

and I start a new My_Jenkins_Job, ./jenkins/workspace/My_Jenkins_Job@2 is skipped, ./jenkins/workspace/My_Jenkins_Job@3 is deleted, then the new job will run under a new ./jenkins/workspace/My_Jenkins_Job@3.

The options I've seen/thought of are:
Custom Workspace - set a custom workspace for each run. This isn't what I need, as normally I want standard behaviour, i.e. cycle through the workspaces replacing them as they finish. Also, I may decide to keep a workspace after the build has started.

Archive workspace - some of the paths in the job are absolute. I believe that archiving the workspace will move it to another parent directory, which would break all the full paths. Also, this could only be enabled up front

Force the job to keep "building" even when it is finished. This doesn't work if jenkins is restarted, and again it can only be enabled up front.

The only solution that I can think of is to "touch" a file within the workspace e.g.:

./jenkins/workspace/My_Jenkins_Job@2/.jenkins_keepme

And jenkins treats that workspace directory as if it is still building. This way I can manually choose to add the file at any time. Also it works if jenkins is restarted.

Is there anything like this out there? Or does anyone have any suggestions of how else I could get the functionality I'm looking for?

Thanks,

Robert

[Baptiste Mathus]

You should treat the workspace filesystem as an internal thing for Jenkins IMO.

Archiving artefact is one feature to express the things you wanna keep.

If you want to keep a directory structure for later reuse, maybe you could either zip it and archive it. Or push it somewhere else at the end of your build?

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/4ac7d92f-feb3-48b2-bb7b-b88314748b16%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Martin d'Anjou]

On Thursday, August 25, 2016 at 11:15:43 AM UTC-4, Robert Beddow wrote:

Is there anything like this out there? Or does anyone have any suggestions of how else I could get the functionality I'm looking for?

 
Yes, you can try the External Workspace Manager Plugin.

[Michael Lasevich]

Workspaces are meant to be ephemeral and can be deleted/removed at any time.  Counting on same workspace being there for every build is a bad idea for many reasons - not the least of which is that the point of using a build system like Jenkins is to have a reliable and repeatable build - if you depend on some old data from another build and cannot function without it - you are creating very unstable builds.

You can use workspace as a place to cache things  (git repo, for one) so that you do not have to download them each time, but you have to verify they are what you expect and re-fetch them if they are not.

Otherwise, you need to create artifacts from builds if you want to keep the data.

-M

[Robert Beddow]

Hi,

Thanks for the replies.

I managed to come up with a solution using the pre-scm plugin. It checks for a file before proceeding.

Note, the reason I need this is that my jenkins jobs kick off synthesis jobs which can take >24hrs to complete. These jobs continue even if Jenkins falls over for whatever reason. If Jenkins then restarts we want to make sure that those jobs continue as we can still manually rescue the results without losing a day.

Also, sometimes a build fails and we want to debug it and run further tests. If the workspace is archived, all the fixed paths generated by the tools we use break, and it's a big project to make it work under a new parent directory.