Concerto 2 Log file and Cache Filling Disk Space

[Brian Yulke]

Hi Concerto Community-

My production server came to a screeching halt today because it ran out of disk space.  Upon investigation, I found that production.log is consuming 8.3GB and the cache is 2.2GB.  I managed to clean up enough space to bring the server back to life, but I still need to address these two things as they are clearly out of control.  I do not see any settings or controls that would allow me to set limits, auto-clear thresholds, or to even manually purge either the log or cache.  What can/should I do?

My environment:
Ubuntu 12.04 LTS (as a VM in an ESX environment)
Concerto 2.2.7
/ is 15GB, of which 11GB is being eaten up by the log and cache.

I'm aware that both OS and Concerto are out-of-date.  However, given the current circumstances, I cannot upgrade either (though I was planning to do both in the coming weeks).  I'm also not going to just upgrade as a fix - I need to be sure I'm addressing the core problem.

Many thanks for any help I can get.

Best regards,
Brian Yulke
 
Associate Director, Client Services
NYU School of Law, ITS
40 Washington Sq. So. Rm 211
New York, NY 10012
brian...@nyu.edu
(212) 998-6283

[August]

Hi Brian,

As I see it, you have three good options here:

1-Use your system log rotation mechanism to rotate/archive/delete these logs

2-Set the log level to something less verbose - done by adding config.log_level = :error to your config/environment/production.rb (other options are :debug,:info,:warn, :fatal, and :unknown)

3-Send the logs to the bit-bucket: config.logger = Logger.new('/dev/null’) in config/environment/production.rb

Hope that helps!

-- 
August

--
You received this message because you are subscribed to the Google Groups "Concerto Digital Signage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concerto-digital-s...@googlegroups.com.
To post to this group, send email to concerto-dig...@googlegroups.com.
Visit this group at https://groups.google.com/group/concerto-digital-signage.
For more options, visit https://groups.google.com/d/optout.

[Rob Glover]

https://gorails.com/guides/rotating-rails-production-logs-with-logrotate

We implemented this and it seems to be working well after having the same issue

[chris.g...@gmail.com]

Rob's gorails logrotate is the better way to do this but here is the quick and dirty way I'm doing it that merely zeroes the log each month.  I also reported this last year.

Thanks! - Chris

Add to Chrontab:

#concerto: zero log data 12:05am the first of every month
5 0 1 * * root rm /usr/share/concerto/log/production.log; touch /usr/share/concerto/log/production.log; chown www-data:www-data /usr/share/concerto/log/production.log; chmod 666 /usr/share/concerto/log/production.log

[Brian Yulke]

Thanks everyone!

We ended up implementing our standard Linux log rotation.  One of our dev ops guys has a script to compress and rotate logs monthly.  We decided to dump logs after 3 months.  In addition to that, all logs are now set to forward to our instance of Splunk. 

We also decided it was a good idea to setup monitoring so we will get alerts if disk space runs low or the log file grows too big.