0.9.1 db CronJob log ballooned over 120 GB

[David Golding]

Hi - any suggestions on how to fix/enforce/deactivate the cronjob update_check !?

I've ran into a problem with a couple of sites on vanilla Habari 0.9.1 and database sqlite, the db file size on disk on one site has hit 128GB, the other is currently at 12GB

The habari log reports: 

CronJob update_check has failed 130293 times and is being deactivated!

SYSTEM INFORMATION

PHP Version: 5.4.19

Server Software: Apache/2.2.22 (@RELEASE@)

Database: sqlite - 3.6.20

PHP Extensions: Core, date, ereg, libxml, openssl, pcre, zlib, bz2, calendar, ctype, hash, filter, ftp, gettext, gmp, SPL, iconv, Reflection, session, standard, shmop, SimpleXML, sockets, mbstring, tokenizer, xml, apache2handler, bcmath, curl, dba, dom, fileinfo, gd, igbinary, imagick, imap, intl, json, ldap, logpipe, exif, mcrypt, memcached, mongo, mysqlnd, mysql, mysqli, odbc, PDO, pdo_mysql, PDO_ODBC, pdo_sqlite, Phar, posix, soap, SQLite, sqlite3, sysvmsg, sysvsem, sysvshm, tidy, wddx, xmlreader, xmlrpc, xmlwriter, xsl, zip, mhash

PHP Configuration Settings: safe_mode: Not set<br>open_basedir: /cluster/default/php/:/cluster/004/vclusters/und4pq/<br>display_errors: 0<br>session.gc_probability: 1<br>session.gc_maxlifetime: 1440<br>error_reporting: 32767<br>memory_limit: 256M

PCRE Version: 8.32 2012-11-30

Browser: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/537.36

SITE INFORMATION

Habari Version: 0.9.1

Habari API Version: 4958

Habari DB Version: 5104

Active Theme: html5tiles

System Locale: en-us

Cache Class: FileCache

[Bryce Campbell]

The addresses used are just broken. There is nothing to worry about there. You could just find out how to clear the table though, in order to decrease the size.

--
--
To post to this group, send email to habar...@googlegroups.com
To unsubscribe from this group, send email to habari-dev-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/habari-dev
---
You received this message because you are subscribed to the Google Groups "habari-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to habari-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Meller]

What addresses are broken? https://beacon.habariproject.org works for me…

[Bryce Campbell]

I had the same problem kind of and the cron job stopped. When I asken on IRC, I was told that the address specified by the job is not correct.

[Owen Winkler]

On 5/19/2014 9:55 PM, Bryce Campbell wrote:

I had the same problem kind of and the cron job stopped. When I asken on IRC, I was told that the address specified by the job is not correct.

There may have been a brief period where the server that was meant to respond to some requests wasn't available.  I'm not sure that this window would allow a log to balloon to over 120GB.

A setting that could cause the log to expand quickly is the one that stores backtraces with each log entry for debugging purposes.  If that's on, turn it off.  It's 'log_backtraces' in the options table.  I'm not sure if the setting is accessible via the standard admin.  Ironically, the data this provides (in just one of the instances that the error occurred) could be helpful in sorting out the problem for anyone that has it.

If the update_check cron failed 130293 times, was it reporting that it was being disabled that many times, or did it report that once after many failure notices and then shut it off? 

Here's a related issue, which seems to indicate we're at least somewhat aware of what's going on:
https://github.com/habari/habari/issues/593

Owen

[Chris Meller]

The option for backtraces is at the bottom of the Options page, along with the level of entries to log.

The crons not being deactivated (or not having their failure count reset, something) is a known issue that I don’t think we were able to nail down the source of (I wasn’t able to recreate it, they always failed properly for me).

[David Golding]

Hi - thanks for all the replies 

Log Backtraces isn't on, and "update_check cron failed 130293 times" represents 130293 log entries of the same error hence the ballooning.

The problem starts at line 119 cronjob.php after 10 failures $this->active = false should kick in. Appears it doesn't because the active column in crontab table is neither 1 or 0 its just blank (see attached adminer screenshot). I've manually entered 0 which has deactived the cron and stopped the ballooning. The same too with trim_log failed to deactivate which logged 11093 failures. The db at some point would have become too big to work on.

[Matt Read]

One of the reasons we can't find anything (or people can't see anything) may be that curl is throwing exceptions that aren't logged in the error log. And since it's an async call, nothing is displayed.

This happened to skippy once with https calls failing cause curl wouldn't accept his certificate. there was nothing in the error logs. only manually calling the cron trigger, or changing to synchronous calls with display_errors on would show anything.

--

Matt Read
http://mattread.com

[ΚΩΣΤΑΣ ΤΡΙΚΑΛΗΣ]

I had same problems... log file went 9 GB, banned from previous host
I am now in a new one host and disabled error logs from habari settings...
I think the error log file started to grow when I was developing new template themes or after a migration from localhost.
I went back to 8 version. I there a clear solution for cron jobs?

[David Golding]

Hi just noticed its been wonderfully fixed in release 0.9.2 http://wiki.habariproject.org/en/Release_0.9.2 :-)