Error Logging

[gwfran]

Something to consider...

I use logs like mad.  It'd be cool if there was a clever way to access logs from ICEcoder.  I was thinking something along the lines of configurable tabs along the bottom edge of the editor pane (where the nesting is).  Or maybe upside down tabs (like stalactites) above the normal file tabs.  You could generate a new tab and configure it (right-click?) to link it to a log file (/var/logs/apache2, /var/www/icecoder/error-log.txt, etc.).  Any time I click on the tab, the log is "tail-ed" for the last X lines (also configurable) to the screen.

Thoughts?  

[Matt Pass]

Some good ideas there, tho I don't want to clutter the editor, especially as I'm trying to simplify the look of it.

How about an 'Errors' icon to the far right of the file tabs? It makes use of a new setting which defines the domains error logs dir plus any extras, such as maybe your own error-log.txt if you wish.

It could check for changes in these log files every 1 sec or so and if it finds changes, the Errors icon highlights, you can click it and it shows the new errors? (You'd need to create a virtual dir to domains error log dir of course to allow PHP to scan for changes there).

Kind of recent/live error reporting?

Matt

Matt

[Matt Pass]

So, elaborating on the idea...

There's a grey bug icon (perhaps a Beetle) to the far right of the file tabs. Hover your mouse over it and it displays 'Bug reporting not active' and when clicked on 'Setup bug reporting in settings'.

So you go to settings and under a new setting 'Bug reporting paths' can add a comma seperated list of error log file paths but here we'll just set up one: '/var/domain/public/errors/error.log'. When you click Update you'll go back to the editor view and the bug icon is flashing yellow. Hover over it and it displays 'Issue with bug reporting path'. Click this and it displays 'Path issue with /var/domain/public/errors/error.log - unavailable'.

This is because we're half way through our setup. We now setup a virtual dir from /var/domain/errors to /var/domain/public/errors on the server and the bug icon stops flashing yellow as we've mapped the errors dir to within the public dir and the path we setup in settings is now accessble & OK. The bug icon turns green to indicate it's live & working.

Now it's found the file and also established the last error line in that file. So we start coding and viewing our work in a browser window, but it generates errors. ICEcoders bug tracking system notices (via say a 5 sec interval check) that there's 3 new errors in the error.log file. The bug icon turns red, we click it and it shows the 3 errors. The bug turns green again as it's informed you of these latest bugs and reset itself, making a note of the last line again (so the reporting continues again from that point now).

What do you think?

Matt

[gwfran]

I'm liking this plan a lot.  The only part I'd alter would be the last paragraph (and only to minimize traffic and load).

I wouldn't worry about turning the bug icon red.  This removes the need to pulse the server for updates and it also allows us to monitor non-error logs (maybe I want to tie the apache access.log to see if my traffic is getting through).

Additionally, we may be able to use your comma-delimited list of log file paths in a different way.  Instead of referring to each log file on its own, maybe the list of log files is used to fire up a background process which just cat's the tail of the log files into icecoder/cat-tail.log.  When you click the bug icon, all it does is pull that document up (or maybe it just tails it for the last X-configurable rows).  This puts ALL of the logs into one place and makes the task of coding this solution many degrees easier.

Thoughts?

Gary

[Matt Pass]

I'd personally like it to turn red, as I'd like it to show when I've made an error/got a warning and get quick access to see what it is. (Guess I make a lot of mistakes!) ;-P

Could maybe add a 2nd option to 'Check for bugs every [10] seconds'. Where you can fill in the number to determine how often to check. Simply leave it blank to not check?

Yeah, I like your last suggestion, makes things cleaner. So when it finds new errors in say, 3 log files, it writes those changes into that cat-tail.log file. As you say, the bug icon simply displays that file, so much more efficient. Having this written ready in a file brings the extra benefit of it being something we can use for other purposes too beyond just displaying.

To open things up, I'm thinking users may want to take different actions on bugs being found and so an extra file could be added to the processes dir: processes/on-bug-found.php. That file (by default) will just trigger the bug icon turning red, but of course you can change/add to the action that takes place when a bug is found. Perhaps it fires off an email, logs the info in a DB, or whatever other custom code someone wants to add in.

This is all most definitely going to be version 4. Now I've just got to find the time to do it. :)

Cheers

Matt

[gwfran]

How did I miss this....?  Sorry for the silence.   :-)

I'm cool with turning it red for changes.  The ability to change (or blank) the frequency for checking for bugs works perfectly for those that don't want to ping the server.

The processes/on-bug-found.php idea is GOLD!  This starts to get into the world of CEP which I know my potential user base is really keen on.

If I get some time, I'll dink around with some of the back-end stuff (e.g. a tailing process that'll work across platforms - or are you keeping this Linux focused for now?)

[Matt Pass]

Yeah, it struck me as a good idea, so users can trigger whatever event they like. By default it changes the icon to indicate new error/warning log item(s).

Would be great if you could help out with a tailing process. Linux is the main focus, but if we just use PHP to do the tailing, it can continue to be cross platform and so work via XAMPP, MAMP etc.

[Matt Pass]

OK, have made a start on the bug reporting feature (working on UI and settings items, ie - things aside from the tailing process).

So we can meet in the middle and all works together nicely, it's going to call /lib/bug-files-check.php?files=|var|domain|log.txt,|var|domain|another|example.txt in an iFrame every X seconds.

If you can write the new file lib/bug-files-check.php which will contain your tailing process using PHP. As a summary I guess it needs to pick up the querystring value passed in 'files', explode by comma and then turn pipes into slashes, you'll then have your array of files to check on. When you find changes since the last check, obviously do your tailing bit and save the tail info into a new file (probably best to store it in ICEcoders 'tmp' folder, maybe a file called tmp/bugs-found.log ?). Finally, having found bugs, it can also include /processes/on-bugs-found.php

That's where that file (I'll write) will turn the bug icon red and set a new onclick event for the bug icon, so when the user clicks it, it opens tmp/bugs-found.log in a display.

Hope that all sounds good?

Cheers

Matt

[gwfran]

What I had in mind for the background process was a little different.

Configuration Screen:

1) Add text box for comma-delimited list of log files.

2) Add text box for frequency.

Tail Process:

1) Anytime UPDATE is clicked on the Configuration Screen, I'll fire off:

    pkill --full "tail -f"

2) Then, if UPDATE is clicked and there are log files listed:

    tail -f file1.log file2.log file3.log ... >>{ICECODER_HOME}/tmp/icecoder.log 2>>{ICECODER_HOME}/tmp/icecoder.log &

Pulse Service:

1)  Your iFrame (or AJAX, if you're okay with avoiding the iFrame) then calls log-files-check.php every X seconds.

2)  On the server side, log-files-check.php looks in the configuration file to see if there is a log file size (in bytes) and compares it to the size of:

    wc -c < {ICECODER_HOME}/tmp/icecoder.log

3)  If the files are the same, you get a 0 back; if they're different, you get a 1 back.  This minimizes the size of the call to log-files-check.php (no arguments) and the size of the return message.  If you see a 1, the configuration file is updated to report the new log file size (in bytes) and you fire that red bug up.  Lastly, if you see a 1, processes/on-log-change.php gets included.

Other considerations:

1)  Perhaps a button in the Configuration Screen that allows the user to delete the icecoder.log file in case it gets too large. Set the file size in the configuration file when done.

2)  When the list of log files is empty, pressing the UPDATE button kills the tail process.  Adding a file to the list kills and then fires up the tail process. This is effectively how you can turn the tail process on/off.

3)  When you open tmp/icecoder.log, perhaps we should just grab the last 1000 lines (configurable?) or so.  If someone doesn't clear their cache, they'll end up downloading increasingly larger and larger files as they monitor their logs.

Simple, lightweight...  Thoughts?

[Matt Pass]

It's a technically far better solution than my low-tech suggestion, that's for sure.

I see where you were coming from now, re asking about Linux vs multi-platform. However, this proposal does bring a few big problems:

It means it doesn't run on PC or Mac (locally, via setups such as XAMPP, MAMP etc). It also doesn't run for Linux users on hosting platforms where Linux access isn't available (eg shared hosting), or if there are strange security restrictions imposed etc. Having this run mainly via Linux I think will only open a can of worms for me, whereas being PHP-run it's the common, usable platform for everyone. I get enough problems from users having issues with the Terminal plugin! ;-)

So, certainly brilliant for the those with decent server setups, but I know a large portion of users won't have that and is only going to bring headaches for me in the long run.

My basic PHP solution is cheesy/clunky, but works for all and maintains the thinking that 'all you need to run ICEcoder is PHP'.

I think the 'last X lines' idea is very good. Maybe there's a 3rd config setting then to 'show me last [100  ] lines'. Users can change the value as they'd like. I think 100 is fairly low impact.

Oh, on the AJAX point, I'm going to be moving all file handling to AJAX/XHR soon to get away from the 'HTTP in an iFrame' calls and speed everything up. Only reason I suggested an iFrame is that's how everything else works (open, save, delete, perms etc). At some point soon everything will switch over to XHR, operations get a lot faster/less horrible. :)

[gwfran]

Hmmmm...  I can definitely make it work for Linux and Windows without the user having to install anything extra, but Mac throws a wrench in the works.  If users added Cygwin to their installation (on Win or Mac), then it'd work fine - but that's an extra installation which may not be possible on a hosted system (are there shared Mac and PC hosts out there?).

What if I provide the means for it to work on Linux and Windows (without installing anything extra), and someone familiar with Mac figure out how to make it work there?  Is there a Mac-savvy guy out there that can take the tail, wc, and kill executables from Cygwin for Mac and put them in a (for example) bin folder and modify the PHP to check if you are on Linux, Win, or Mac and then call the respective version of the file? 

Great news on the plan to migrate to XHR.  iFrame works great and I'm not TOO worried about it on a development system, but with the ability to do nasty things in older browsers and the new "sandbox" attribute in HTML5 browsers adding cross site configuration complexities, iFrames frighten me.  :-)

[Matt Pass]

Well, I don't really want to go down the route of features working for some users and not for others, plus needing extra installations, separate versions, etc.

I ventured down this route some time ago, it only caused headaches, that were solved in the end by just using PHP to handle server side bits.

If we were setting this up for just 1 person, knowing exactly what their setup was, we could/should of course go for your suggestion. But with so many different unknown setups, the only common server side tech for everyone is PHP.

Unless you can think of another solid, hassle-free route, we'll have to continue with the PHP.

Yeah, for sure. The HTTP & iFrame route was something I set up quickly (pre ICEcoder 1 I think??), that continued. Needs a big overhaul to use XHR throughout as a more modern approach, much more efficient and secure.

Anyway, re the bug reporting thing, today I'll be pushing changes to re the UI & settings, think we're going to have to go the PHP route for the server work.

[Matt Pass]

OK, I've pushed some commits to GitHub which cover the saving of 3 settings (bug file paths, timer and max lines) and those are used to check for bugs via XHR.

(I started using http and an iFrame and of course realised that the browser was loading every X secs, so we've gone the subtle & more efficient XHR route already).

It calls the lib/bug-check-files.php file with 3 querystring items (files list, the modified datetime of each and the max lines we should return).

Also includes a process should we want to do something else also.

The XHR receives back some responseText from the call and handles the status change, to change the bug icon colour.

The code will likely need some real tidying up, but seems to work well. You can change the $result value to 'off', 'error', 'ok' or 'bugs' and see the icon change.

Thoughts on the initial setup welcomed :)

[Matt Pass]

Re using PHP and minimising server load, I did some quick searching and found this guy who'd really examined the issue with proper tests:

http://stackoverflow.com/questions/15025875/what-is-the-best-way-in-php-to-read-last-lines-from-a-file

Seems it falls into 3 main categories - file() that can use up a lot of memory, tail() which isn't cross platform and fseek() which is the ideal solution.

Was good to see that a 10k and 10mb file get handled by fseek in < 1ms just the same. Hopefully it'll be the same no matter the size (100mb, 1gb, etc).

Sounds like the way to go?

Matt

[gwfran]

Yup - I've got a mock up (using fseek()) running for a single file and am looking to tie in your mocks to mine.  I'm trying to figure out how to pull your recent changes into my Git fork so you don't run into a merge conflict when I put in a pull request.  

[gwfran]

Pencils down!

Okay, I have an 80% solution ready to rock and roll.  The only thing I need to do is pull all of your updates (still haven't figured out GitHub) and "officially" grab the settings (right now I override them with hard-coded values).  If you can let me know if I'd be screwing up your merge by manually copying/pasting your new files into my fork (or if you doing a pull request from your master to my fork works at all), then I can finish up.   Easiest big-bang v4 addition yet!  :-)

[gwfran]

Okay, I removed the attribution banner from the files, moved from OOP to procedural (global variables and function collisions frighten me), removed the white space, and used the for iterator instead of foreach(array) at your request.  The only thing left to do is to fill out the function saveFileSizes().  How do you want me to update the bugFileSizes setting in the config file?

So, based on the construct, the user fills out your bugFileName setting:  /var/log/apache2/error.log,/var/log/php5/cgi/error.log

When they hit Update, you can call services/bug-files-check.php and ignore the true that you'll get back.  This will populate the bugFileSizes (when we finish the saveFileSizes() stub) and fill out the tmp/log.log tail file.

All you have to do is wait X seconds and call services/bug-files-check.php.  If you get a false back, no change.  If you get a true back, turn the bug icon red.

If the user wants to see the last X entries in the log file, call services/bug-files-load.php and I'll send you X rows back as HTML.  You can push it to a new tab or throw it into a div.

Easy peasy, lemon squeezy.  We've minimized load on the server and minimized traffic across the pipe.  It's slightly different than what you have in your mockup, but this way you don't have to push php variables into javascript (e.g. the filenames) and it minimizes the javascript size.

[Matt Pass]

Thanks, but this still isn't what I was expecting/looking for.

I wrote some bits this morning and it was all working pretty well, just needs some polishing. Should finish it tomorrow morning.

Will show you when it's done.

Thanks,

Matt

[Matt Pass]

OK, check out the latest version at https://github.com/mattpass/ICEcoder

This is all working, though with just 1 bug report file for now. You can set the 3 x settings from the settings screen and it'll start running. Icon will turn orange if there's an error finding file, green if there's no new bugs yet and red when there are new bugs.

Needs a huge tidy up and finishing, but think it's mostly there.

Feedback welcomed!

[Matt Pass]

OK, all done here now re the bug reporting system. Ironed out the last few bugs (so to speak!), plus it now works with multiple bug log files etc.

All feedback re the bug reporting system welcomed. Seems to be pretty sweet to me tho and a superb suggestion for v4.0!

Many thanks.

[gwfran]

Hmmm... I can't seem to get the Bug Reporting System to work.  I have the following values in the Settings:

check in files:  /var/www/icecoder/error-log.txt, /var/www/dev/php-errors.log, /var/log/apache2/error.log, /var/log/apache2/access.log

every: 10 secs

last: 100 lines

When I force a php error, apache error, or any apache request, I wait for the system to pulse the server (as evidenced by a network tracker).  Then I look in the tmp directory and nothing is there.  Naturally, the bug light doesn't light up.

Any ideas? 

[Matt Pass]

Ah, the paths need to be relative. Probably shows that it should be accepting absolute paths also. Should look to tweak that to allow either.

If should turn the bug icon orange though to show that it can't find the files? Did it not do that?

You do also have write permissions on the tmp dir?

[Matt Pass]

(relative from the doc root that is, so /icecoder/error-log.txt and /dev/php-errors.log).

Won't handle the other 2 paths just yet.

[gwfran]

Ah - makes sense.  I've updated the paths to be relative.  The bug icon did not turn orange.  Clicking on the bug icon said that it had found no new bugs.  Yes, 777 on icecoder folder on down with user www-data:

drwxrwxrwx 2 www-data www-data  4096 Mar 19 01:08 tmp

When I hover over the icon, it says, "Bug reporting not active."

[Matt Pass]

That's odd, sounds like it's not running on the interval. You did also set an interval and max lines?

There are 3 commented out 'console.log' lines in the startBugChecking() function in lib/ice-coder.js (see lines 1957, 1959 and 1975). You could try putting ICEcoder into devMode, uncommenting those and look in your console to see what URL is being called every X seconds and what response it gets back.

If there's an incorrect path, the bug icon should turn orange. Green when it's running and there's no bugs and red when it finds some.

Maybe also a caching issue that could be sorted with a CTRL+F5 hard refresh.

[gwfran]

10 seconds and 100 max lines.  Here's the URL being called:

http://xxxxxxxxxxxxxxxxxx.com/icecoder/lib/bug-files-check.php?files=icecoder|error-log.txt,dev|php-errors.log&filesSizesSeen=null,null&maxLines=100

The error returned is:

Parse error: syntax error, unexpected '[' in /var/www/icecoder/lib/bug-files-check.php on line 26

[Matt Pass]

Couple of things then - the paths you set need to be slash prefixed (see the [?] icon on the settings screen next to setting - maybe needs to be more obvious?)

However, it sounds like it won't do both the explode and getting array value on: explode(",",$_GET['filesSizesSeen'])[$i] and needs breaking into 2 parts.

Maybe instead of:

if (explode(",",$_GET['filesSizesSeen'])[$i]!="null" && explode(",",$_GET['filesSizesSeen'])[$i] != $filesSizesSeen[$i]) {

We instead need to break it down to:

$fileSizesSeenArray = explode(",",$_GET['filesSizesSeen']);

if ($fileSizesSeenArray[$i]!="null" && $fileSizesSeenArray[$i] != $filesSizesSeen[$i]) {

The current line 31 also needs to become this for the same reason:

$chars = ($filesSizesSeen[$i]-$fileSizesSeenArray[$i]);

I'm using PHP 5.4.7, but I suspect you have an older version and it won't do both, needs breaking down? Can you confirm if this works and I'll push this a patch?

[gwfran]

PHP 5.3.10 here.  Made the changes and the bug is green!  :-)

The [?] icon is perfect as is.  It looks like the PHP error was causing a JS error as well (now resolved).  That could be why I wasn't getting the help popup.  

I was able to throw an error and turn the bug red.  Working great now!