Purge puppet's reports

[Hugo Deprez]

Dear community,

I configure puppet dashboard on my server, in order to make it work I had to activate reports from clients.
The thing is that my folder /var/lib/puppet/reports/ is now 1.6 Go large.

Is there any mechanism available to auto delete old files ?
Or should I configured my own script to clean up old reports ?

Thanks in advance,

Regards,

Hugo

[vagn scott]

On 07/21/2011 03:14 AM, Hugo Deprez wrote:
>
> Is there any mechanism available to auto delete old files ?
> Or should I configured my own script to clean up old reports ?

I'm doing this:

class puppet::clean_reports {

cron { "puppet clean reports":
command => 'cd /var/lib/puppet/reports && find . -type f
-name \*.yaml -mtime +7 -print0 | xargs -0 -n50 /bin/rm -f',
user => root,
hour => 21,
minute => 22,
weekday => 0,
}
}

If there's a better way I'd love to hear it.

--
vagn

[vagn scott]

Thinking about this some more, if you need the reports for dashboard to
work,
then any cleanup script should leave the latest report from any server,
so that, even if the server has not checked in for a while it won't
disappear.

Thanks for the question. Clearly I need to revisit this.

--
vagn

[vagn scott]

On 07/21/2011 08:52 AM, vagn scott wrote:
> Thinking about this some more, if you need the reports for dashboard
> to work,
> then any cleanup script should leave the latest report from any server,
> so that, even if the server has not checked in for a while it won't
> disappear.
>
> Thanks for the question. Clearly I need to revisit this.
>

So, here is a script that ought to do the job. Not heavily tested yet.
Please let me know if you have problems with it. --vagn

#! /bin/sh
# puppet-reports-stalker
# vagn scott, 21-jul-2011

days="+7" # more than 7 days old

for d in `find /var/lib/puppet/reports -mindepth 1 -maxdepth 1 -type d`
do
find $d -type f -name \*.yaml -mtime $days |
sort -r |
tail -n +2 |
xargs -n50 /bin/rm -f
done

exit 0

And the updated class.

class puppet::clean_reports {

file { '/usr/local/bin/puppet-reports-stalker':
source => 'puppet:///modules/puppet/puppet-reports-stalker',
before => Cron[ 'puppet clean reports' ],
mode => 755,
owner => root,
group => root,
}

cron { 'puppet clean reports':
command => '/usr/local/bin/puppet-reports-stalker',

[Jo Rhett]

> On Thursday, July 21, 2011 8:50:32 AM UTC-5, vagn wrote:
> find $d -type f -name \*.yaml -mtime $days |
> sort -r |
> tail -n +2 |
> xargs -n50 /bin/rm -f

All this is really better than…?

find $d -type f -name \*.yaml -mtime $days -exec rm -f {} \;

--
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.

[Dan White]

How about logrotate ?

On Jan 30, 2013, at 11:53 AM, Andrew Sinclair wrote:

Great solution. Simple, effective. Thanks.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To post to this group, send email to puppet...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Dan White]

I take back my suggestion.

LogRotate is not built to handle many little log files with different names.  I found out the hard way - I tried to  make it work :(

The script by vagn will do the trick.

Another suggestion: put a variable in the line "tail -n +2” in place of the “2” for how-many-files-to-keep+1 -- “2” only keeps one file.  Other folks may want to keep more.

[Ken Barber]

tmpwatch is also a good approach:
http://linux.about.com/library/cmd/blcmdl8_tmpwatch.htm. Probably
requires less scripting, and its already on most distros, probably
cleaning your /tmp directories already.

[Amos Shapira]

tmpwatch/tmpreaper (for Ubuntu) is first thing I though about too - but the vagn has a good point about better keeping the latest report from each server, which justifies using his script.

[Jakov Sosic]

On 02/02/2013 06:17 PM, Ken Barber wrote:
> tmpwatch is also a good approach:
> http://linux.about.com/library/cmd/blcmdl8_tmpwatch.htm. Probably
> requires less scripting, and its already on most distros, probably
> cleaning your /tmp directories already.

What about 'tidy' resource? We use that, with 1 day retention and puppet
runs on master every 30 min. So basically in worst case scenario we have
24h29min of reports.


--
Jakov Sosic
www.srce.unizg.hr

[Andrew]

Using tidy to clean up logs, this is pretty self-explanatory, so I wont bother explaining :)

case $hostname {
    /^puppet$/: {
        tidy { 'puppet::reports':
             path => '/var/lib/puppet/reports',
             matches => '*',
             age => '14d',
             backup => false,
             recurse => true,
             rmdirs => true,
             type => 'ctime',
         }
         # notify { "debug: tidy command should run now": }
    }
}

Ciao,
Andrew.

[Dan White]

OK, but the reports live on the puppetmaster.  How do you get a list of hostnames to apply to this resource definition ?

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To post to this group, send email to puppet...@googlegroups.com.

Visit this group at http://groups.google.com/group/puppet-users.

[Andrew G]

hi dan,

it applies only to any server with a shortform dnsname == puppet.

the facter variable $hostname matches the puppet master server name.

alternatively, remove the case statement, put it in it's own class and apply that class specifically to your puppetmaster server.

Andrew

--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/q8vWDr3bn4Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.

[Dan White]

That makes sense.

Thanks.

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

---

From: "Andrew G" <andrewg...@gmail.com>
To: puppet...@googlegroups.com
Sent: Thursday, September 12, 2013 1:28:26 AM
Subject: Re: [Puppet Users] Purge puppet's reports

[Trevor Vaughan]

Be aware that this will create a file resource/checksum for EVERY file in that directory and may cause a heavy load on your system if you have a lot of reports.

Trevor

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To post to this group, send email to puppet...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

[Joshua Sinfield]

One thing I found after manually deleting gb's of reports was that they still exists in dashboard but failed to load when you click on them.

Don't forget to run the command suggested in http://docs.puppetlabs.com/dashboard/manual/1.2/maintaining.html

Josh

[Tiago Delboni]

Hi, I'm revisiting this topic.

Wasn't garbage collector supposed to clean up the old reports and leave the last "report-ttl" on disk? Or this is valid only for the reports stored in the PuppetDB's database?

Regarding "the script way" of removing old reports, we have this line in cron:

/usr/bin/find /opt/puppetlabs/server/data/puppetserver/reports -type f -mtime +30 -exec /bin/rm {} ";"

[Thomas Müller]

Am Donnerstag, 21. Januar 2016 12:45:59 UTC+1 schrieb Tiago Delboni:

Hi, I'm revisiting this topic.

Wasn't garbage collector supposed to clean up the old reports and leave the last "report-ttl" on disk? Or this is valid only for the reports stored in the PuppetDB's database?

I would exptect puppetdb settings only relate to the puppetdb data. 

check your reports setting on the puppet master:

puppet config print --section master reports

default is "store". maybe yours is "store, puppetdb". if you don't need the report files at all you could remove store.

- Thomas

https://docs.puppetlabs.com/guides/reporting.html#setting-up-reporting