How to remove pipeline console log files 7 days old

[Nayan Makwana]

Hello Folks,

I read all the GoCD configuration document i did not find the way to delete console output of every pipeline.

I want to delete console output history which is more than 30 days old I am using 22.3.0 Go-server 

Please help me out for this scenario.


Thank you.
Nayan Makwana 

[Chad Wilson]

There's nothing built-in to do this.

There are a few separate tools around such as 'gocd-janitor' to try and implement configurable artifact deletion (and console logs are a special 'artifact' in GoCD) if you also want to delete the other artifacts and GoCDs simple artifact cleanup (due to nearly full disk) isn't sufficient or desirable.

If you want to target just the console logs, you'd have to write something to do this, perhaps a shell 'find' command that looks for all the 'cruise-output' directories inside the artifacts storage directory where the console logs are stored and deletes them if they were last modified more than x days ago. Be careful :)

-Chad

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/4627a960-1176-4848-bb24-0d408c2da830n%40googlegroups.com.

[Chris Gillatt]

FWIW this is how I manage logs and artefacts for my company's use-case.  We like to keep our console logs for a year, but remove other artefacts after 7 days.

It's just in bash and we trigger it via cron on Rockylinux 8 (RHEL8 compatible).  These are destructive so use with caution; I'd strongly advise thoroughly testing these individually minus the destructive rm commands before using them in anger based on your configured filesystem in a sandbox. You should be able to take these as a reference and write something to closer match your own use-case. The shouts are just so when the script runs, it nicely outputs what's happening in ELK.

```

#!/bin/bash
SCRIPTNAME=$(basename "$0")

shout () {
echo "$SCRIPTNAME: $1"
}

shout "GoCD Artefacts cleardown script. I remove old artefacts."

# Remove artefacts older than 7 days but keep 'cruise-output' dir
shout "Removing Artefacts over 7 days old. Keeping GoCD Console logs.."
/usr/bin/find /godata/artifacts/pipelines -mindepth 6 -mtime -7 -prune -o -type d -name 'cruise-output' -prune -o -print0 | xargs -r0 rm -vrf

# Remove cruise-output by removing the artefact directory for pipeline runs older than 1 year
shout "Removing all artefacts older than one year old.."
/usr/bin/find /godata/artifacts/pipelines -mindepth 2 -maxdepth 2 -type d -mtime +365 | xargs -r rm -vrf

# Remove empty directories
shout "Removing empty directories.."
/usr/bin/find /godata/artifacts/pipelines -maxdepth 1 -type d -empty -exec rmdir -v {} \;

# Remove pipeline artefact cache dirs older than 7 days
shout "Removing pipeline cache older than 7 days old.."
/usr/bin/find /godata/artifacts/cache/artifacts/pipelines -maxdepth 2 -mindepth 2 -type d -mtime +7 | xargs -r rm -vrf

# Remove logs
shout "Removing any logs over 7 days old"

/usr/bin/find /godata/logs -type f -mtime +7 \( -name "*.log" -o -name "*.gz" \) -exec rm -vf {} \;

```