[JIRA] (JENKINS-60938) Solve problem of too many .groovy files in job directories

2 views
Skip to first unread message

jerrywiltse@gmail.com (JIRA)

unread,
Jan 31, 2020, 8:44:03 AM1/31/20
to jenkinsc...@googlegroups.com
jerry wiltse created an issue
 
Jenkins / New Feature JENKINS-60938
Solve problem of too many .groovy files in job directories
Issue Type: New Feature New Feature
Assignee: Unassigned
Components: workflow-cps-global-lib-plugin
Created: 2020-01-31 13:43
Labels: plugin pipeline jenkins
Priority: Minor Minor
Reporter: jerry wiltse

When using Jenkins shared libraries (JSLs), by default, all the .groovy files are stored in the job directory along with the rest of the logs and data. The files are small, usually smaller than logs. However, recently, we've received warnings that the volume with the job directories is low on free INODES, which means "too many files". So, file count actually is significant at scale.

We want to keep our job logs as long as possible, but care much less about keeping the .groovy files. Deleting only the .groovy files would enable us to keep roughly 100 times the number of logs. Currently, buildDiscarder property gives us a per-job cleanup policy which is good. However, currently it provides no granularity over the files in the directory which is understandable,

This feature request is to suggest looking at the many options available for solving such a problem, and hopefully finding one which is easy to implement.

For example:

  • enhance build buildDiscarder property with special handling for groovy files
  • create similar-but-different property called "buildPostProcessor" (generic)
  • create similar-but-different property called "discardGroovyFiles" (specific)
  • provide a granular / generic cleanup step which allows access to the "current job log" dir and arbitrary groovy code for cleanup.
  • add a JSL option to auto-zip or auto-jar the groovy files in the job directory

I believe I saw a plugin which allows JSL's to be loaded from JAR files already, however I can't seem to find it now.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

jerrywiltse@gmail.com (JIRA)

unread,
Jan 31, 2020, 12:17:03 PM1/31/20
to jenkinsc...@googlegroups.com
jerry wiltse commented on New Feature JENKINS-60938
 
Re: Solve problem of too many .groovy files in job directories

Just did a recursive walk of our jobs directory.  There are 5,597,016 groovy files.

jerrywiltse@gmail.com (JIRA)

unread,
Jan 31, 2020, 12:19:02 PM1/31/20
to jenkinsc...@googlegroups.com
jerry wiltse edited a comment on New Feature JENKINS-60938
Just did a recursive walk of our jobs directory.  There are 5,597,016 groovy files.    Our JSL has about 60 groovy files. 
Reply all
Reply to author
Forward
0 new messages