0.9.1 db CronJob log ballooned over 120 GB

70 views
Skip to first unread message

David Golding

unread,
May 19, 2014, 5:06:30 PM5/19/14
to habar...@googlegroups.com
Hi - any suggestions on how to fix/enforce/deactivate the cronjob update_check !?
I've ran into a problem with a couple of sites on vanilla Habari 0.9.1 and database sqlite, the db file size on disk on one site has hit 128GB, the other is currently at 12GB

The habari log reports: 
CronJob update_check has failed 130293 times and is being deactivated!


SYSTEM INFORMATION
PHP Version: 5.4.19
Server Software: Apache/2.2.22 (@RELEASE@)
Database: sqlite - 3.6.20
PHP Extensions: Core, date, ereg, libxml, openssl, pcre, zlib, bz2, calendar, ctype, hash, filter, ftp, gettext, gmp, SPL, iconv, Reflection, session, standard, shmop, SimpleXML, sockets, mbstring, tokenizer, xml, apache2handler, bcmath, curl, dba, dom, fileinfo, gd, igbinary, imagick, imap, intl, json, ldap, logpipe, exif, mcrypt, memcached, mongo, mysqlnd, mysql, mysqli, odbc, PDO, pdo_mysql, PDO_ODBC, pdo_sqlite, Phar, posix, soap, SQLite, sqlite3, sysvmsg, sysvsem, sysvshm, tidy, wddx, xmlreader, xmlrpc, xmlwriter, xsl, zip, mhash
PHP Configuration Settings: safe_mode: Not set<br>open_basedir: /cluster/default/php/:/cluster/004/vclusters/und4pq/<br>display_errors: 0<br>session.gc_probability: 1<br>session.gc_maxlifetime: 1440<br>error_reporting: 32767<br>memory_limit: 256M
PCRE Version: 8.32 2012-11-30
Browser: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/537.36

SITE INFORMATION
Habari Version: 0.9.1
Habari API Version: 4958
Habari DB Version: 5104
Active Theme: html5tiles
System Locale: en-us
Cache Class: FileCache

Bryce Campbell

unread,
May 19, 2014, 5:09:12 PM5/19/14
to habar...@googlegroups.com

The addresses used are just broken. There is nothing to worry about there. You could just find out how to clear the table though, in order to decrease the size.

--
--
To post to this group, send email to habar...@googlegroups.com
To unsubscribe from this group, send email to habari-dev-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/habari-dev
---
You received this message because you are subscribed to the Google Groups "habari-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to habari-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Chris Meller

unread,
May 19, 2014, 5:11:37 PM5/19/14
to Habari Dev
What addresses are broken? https://beacon.habariproject.org works for me…

Bryce Campbell

unread,
May 19, 2014, 9:55:25 PM5/19/14
to habar...@googlegroups.com
I had the same problem kind of and the cron job stopped. When I asken on IRC, I was told that the address specified by the job is not correct.

Owen Winkler

unread,
May 20, 2014, 8:49:52 AM5/20/14
to habar...@googlegroups.com
On 5/19/2014 9:55 PM, Bryce Campbell wrote:
I had the same problem kind of and the cron job stopped. When I asken on IRC, I was told that the address specified by the job is not correct.

There may have been a brief period where the server that was meant to respond to some requests wasn't available.  I'm not sure that this window would allow a log to balloon to over 120GB.

A setting that could cause the log to expand quickly is the one that stores backtraces with each log entry for debugging purposes.  If that's on, turn it off.  It's 'log_backtraces' in the options table.  I'm not sure if the setting is accessible via the standard admin.  Ironically, the data this provides (in just one of the instances that the error occurred) could be helpful in sorting out the problem for anyone that has it.

If the update_check cron failed 130293 times, was it reporting that it was being disabled that many times, or did it report that once after many failure notices and then shut it off? 

Here's a related issue, which seems to indicate we're at least somewhat aware of what's going on:
https://github.com/habari/habari/issues/593

Owen




Chris Meller

unread,
May 20, 2014, 11:49:23 AM5/20/14
to Habari Dev
The option for backtraces is at the bottom of the Options page, along with the level of entries to log.

The crons not being deactivated (or not having their failure count reset, something) is a known issue that I don’t think we were able to nail down the source of (I wasn’t able to recreate it, they always failed properly for me).


David Golding

unread,
May 21, 2014, 8:57:44 AM5/21/14
to habar...@googlegroups.com
Hi - thanks for all the replies 

Log Backtraces isn't on, and "update_check cron failed 130293 times" represents 130293 log entries of the same error hence the ballooning.

The problem starts at line 119 cronjob.php after 10 failures $this->active = false should kick in. Appears it doesn't because the active column in crontab table is neither 1 or 0 its just blank (see attached adminer screenshot). I've manually entered 0 which has deactived the cron and stopped the ballooning. The same too with trim_log failed to deactivate which logged 11093 failures. The db at some point would have become too big to work on.





crontab_table_screenshot.png

Matt Read

unread,
May 21, 2014, 8:58:05 AM5/21/14
to habar...@googlegroups.com
One of the reasons we can't find anything (or people can't see anything) may be that curl is throwing exceptions that aren't logged in the error log. And since it's an async call, nothing is displayed.

This happened to skippy once with https calls failing cause curl wouldn't accept his certificate. there was nothing in the error logs. only manually calling the cron trigger, or changing to synchronous calls with display_errors on would show anything.
--

ΚΩΣΤΑΣ ΤΡΙΚΑΛΗΣ

unread,
Oct 8, 2014, 1:20:16 PM10/8/14
to habar...@googlegroups.com
I had same problems... log file went 9 GB, banned from previous host
I am now in a new one host and disabled error logs from habari settings...
I think the error log file started to grow when I was developing new template themes or after a migration from localhost.
I went back to 8 version. I there a clear solution for cron jobs?

David Golding

unread,
Oct 10, 2014, 12:13:11 PM10/10/14
to habar...@googlegroups.com
Hi just noticed its been wonderfully fixed in release 0.9.2 http://wiki.habariproject.org/en/Release_0.9.2 :-)
Reply all
Reply to author
Forward
0 new messages