Incorrect naming and logic in Agents section

[Alexander Obuhovich]

There is a functionality in In-Portal, that allows several events in the system to be scheduled to be executed in a specific time. Looks a lot like CRON in Linux.

This functionality never had an interface, that could be used to track how a particular event execution is going and so on.

Part 1

Then, since In-Portal 4.3.9 we've introduced new Agents section, that allowed administrator to review each agent progress. While using this new section I've realized that we had some problems with scheduled event concept:

1. scheduled events are not treated as PHP CLI processes, however they are executed as such (from Linux cron)
2. scheduled events modify data in session, but they don't have session at all in PHP CLI environment
3. when not executed from PHP CLI, then they delay page rendering process (in case if event execution overlaps with page rendering)

Part 2

Then I don't like Agent term, since it doesn't associate with the job that they do. That's why I'm proposing to call them as Job, like all other software products do. This way admin will create a Job and assign an event to perform it.

Part 3

When agents are not executed from PHP CLI (as they should), then they are executed before and after page rendering process, which of course delays it.

I'm proposing to:

1. get rid of AgentType (Before/After) field, since there no such idea in PHP CLI and make all before agents as hooks to adm:OnAfterStartup event
2. after page was displayed to user execute code below, after which:
1. web browser "Loading ..." indicator should show 100%
2. connection to webserver will be terminated
3. and reset of the script will continue to execute on the server

ignore_user_abort(true);

while ( ob_get_level() ) {

ob_end_clean();

}

header('Connection: close');

// header('Content-Length: 0'); // only uncomment when such header isn't sent already

flush();

Part 4

Also "Run Interval" field doesn't seem to be pretty handy, since it doesn't allow to specify exact time when a job should be executed. I'm proposing to use native crontab format: http://en.wikipedia.org/wiki/Cron , where it has 5 numbers:

• min (0 - 59)
• hour (0 - 23)
• day of month (1 - 31)
• month (1 - 12)
• day of week (0 - 7) (Sunday=0 or 7)

For example "0 * * * *" will mean, that job should be executed every hour. And "*/5 * * * *" will mean to run this job every 5 minutes. This is way better, then specifying seconds on every corner. Maybe we can create fancy control with 5 input boxes or so to ease input process.

Showing expected execution count per year would also be a useful information.

Note:

This mass renaming would be a real hard task, since in 5.1.x bugfix branch some code uses "RegularEvent" term and some code uses "Agent" term. While in 5.2.x feature branch this code is heavily refactored already (moved a lot across the files) and all naming was changed to "Agent" completely.

--
Best Regards,

http://www.in-portal.com
http://www.alex-time.com

[Phil -- wbtc.fr --]

Hi Alex,

I like the whole idea, because as per my experience, Agents are really important to keep heatlhy DB and running functions.

I'd just recommend to keep an user friendly GUI to enter agents timers, i.e. having ability to enter delay in human langage, like you said using input boxes or better, drop down, to avoid any error when entering values.

p

2011/6/11 Alexander Obuhovich <aik....@gmail.com>

[Alexander Obuhovich]

Any ideas on interface, since at the end there will be "N N N N N" in database anyway. On the other hand I want to keep full functionality of original crontab file, where each of 5 number can indicate:

• one time execution per interval (e.g. "5" in minutes number will mean "run on 5 minute one time")
• execution in regular intervals (e.g. "*/5" in minutes will mean "run every 5 minutes")
• always execute (e.g. "*" in minutes will mean "run every minute")
• range (e.g. "MON-FRI" in day of week will mean "run only from Monday through Friday")

Maybe we need to keep current input (in seconds), but convert it to minutes (for ease of the users) and display nice hint, like "this agent will be executed each N minutes" and show "advanced time settings" checkbox. This checkbox will open extra row with settings, that I've described in my previous post and that input box will be disabled. User can uncheck advanced checkbox anytime he wants (with all settings lost of course).

Here is how it will look like:

Job Timer (simple): run this job each [100__] minutes [x] Advanced Job Timer

Job Timer (advanced): [100_] minutes [*__] hours [*__] day of month [*__] month [*__] day of week

Also I've noticed that our agents doesn't count time from 00:00 of today, but rather from the time when they were first executed.

That also should be fixed.

[Dmitry A.]

Hi Alex,

I am reviewing all you have put here together - lots of interesting ideas.

One of my questions is - haven't we agreed to rename Agents to Scheduled Jobs? We even have task which will change DB name too...

DA

[Phil -- wbtc.fr --]

I was thinking about this kind:

http://macdevcenter.com/mac/2004/07/27/graphics/CronManagerInterface.jpg

we have dropdown and command line for expert users who don't want to clic in menus, but finally both are able to setup the same tasks (no missing choices in drop down).

What's the way of working of "one time execution per interval" ? Would it be a one time forever task? Otherwise what's the difference between others ones?

2011/6/13 Dmitry A. <dand...@gmail.com>

[Phil -- wbtc.fr --]

I'd also add this shot from Cpanel I'm using.

There's a "common settings menu" at the top, + "common settings for each value.

Command line could be also a dropdown with actual setup agents, don't you think so?

2011/6/13 Phil -- wbtc.fr -- <ph...@wbtc.fr>

[Alexander Obuhovich]

To Phil:

Wow, that CPanel interface is awesome. Could you please take screenshorts of all other expanded dropdowns.

Maybe there is a demo somewhere, when I can click and see what really happens.

To Dmitry:

Interesting, can't seem to find that discussion, that has gone into oblivion and hasn't been realized by now.

[Phil -- wbtc.fr --]

Alex, here you are:

official doc

http://docs.cpanel.net/twiki/bin/view/AllDocumentation/CpanelDocs/CronJobs

Common settingsin dropdowns:

	<option value="--">-- Common Settings --</option>
	<option value="* * * * *">Every minute (* * * * *)</option>
	<option value="*/5 * * * *">Every 5 minutes (*/5 * * * *)</option>
	<option value="0,30 * * * *">Twice an hour (0,30 * * * *)</option>
	<option value="0 * * * *">Once an hour (0 * * * *)</option>
	<option value="0 0,12 * * *">Twice a day (0 0,12 * * *)</option>
	<option value="0 0 * * *">Once a day (0 0 * * *)</option>
	<option value="0 0 * * 0">Once a week (0 0 * * 0)</option>
	<option value="0 0 1,15 * *">1st and 15th (0 0 1,15 * *)</option>
	<option value="0 0 1 * *">Once a month (0 0 1 * *)</option>
	<option value="0 0 1 1 *">Once a year (0 0 1 1 *)</option>
	</select>
	<select style="width: 175px" id="minute_options" onchange="select_single_option('minute')">
	<option value="--">-- Common Settings --</option>
	<option value="*">Every minute (*)</option>
	<option value="*/2">Every other minute (*/2)</option>
	<option value="*/5">Every 5 minutes (*/5)</option>
	<option value="*/10">Every 10 minutes (*/10)</option>
	<option value="*/15">Every 15 minutes (*/15)</option>
	<option value="0,30">Every 30 minutes (0,30)</option>
	<option value="--">-- Minutes --</option>
	<option value="0">:00 top of the hour (0)</option>
	<option value="1">:01 (1)</option>
	<option value="2">:02 (2)</option>
	<option value="3">:03 (3)</option>
	<option value="4">:04 (4)</option>
	<option value="5">:05 (5)</option>
	<option value="6">:06 (6)</option>
	<option value="7">:07 (7)</option>
	<option value="8">:08 (8)</option>
	<option value="9">:09 (9)</option>
	<option value="10">:10 (10)</option>
	<option value="11">:11 (11)</option>
	<option value="12">:12 (12)</option>
	<option value="13">:13 (13)</option>
	<option value="14">:14 (14)</option>
	<option value="15">:15 quarter past (15)</option>
	<option value="16">:16 (16)</option>
	<option value="17">:17 (17)</option>
	<option value="18">:18 (18)</option>
	<option value="19">:19 (19)</option>
	<option value="20">:20 (20)</option>
	<option value="21">:21 (21)</option>
	<option value="22">:22 (22)</option>
	<option value="23">:23 (23)</option>
	<option value="24">:24 (24)</option>
	<option value="25">:25 (25)</option>
	<option value="26">:26 (26)</option>
	<option value="27">:27 (27)</option>
	<option value="28">:28 (28)</option>
	<option value="29">:29 (29)</option>
	<option value="30">:30 half past (30)</option>
	<option value="31">:31 (31)</option>
	<option value="32">:32 (32)</option>
	<option value="33">:33 (33)</option>
	<option value="34">:34 (34)</option>
	<option value="35">:35 (35)</option>
	<option value="36">:36 (36)</option>
	<option value="37">:37 (37)</option>
	<option value="38">:38 (38)</option>
	<option value="39">:39 (39)</option>
	<option value="40">:40 (40)</option>
	<option value="41">:41 (41)</option>
	<option value="42">:42 (42)</option>
	<option value="43">:43 (43)</option>
	<option value="44">:44 (44)</option>
	<option value="45">:45 quarter til (45)</option>
	<option value="46">:46 (46)</option>
	<option value="47">:47 (47)</option>
	<option value="48">:48 (48)</option>
	<option value="49">:49 (49)</option>
	<option value="50">:50 (50)</option>
	<option value="51">:51 (51)</option>
	<option value="52">:52 (52)</option>
	<option value="53">:53 (53)</option>
	<option value="54">:54 (54)</option>
	<option value="55">:55 (55)</option>
	<option value="56">:56 (56)</option>
	<option value="57">:57 (57)</option>
	<option value="58">:58 (58)</option>
	<option value="59">:59 (59)</option>
	</select>
	<select style="width: 175px" id="hour_options" onchange="select_single_option('hour')">
	<option value="--">-- Common Settings --</option>
	<option value="*">Every hour (*)</option>
	<option value="*/2">Every other hour (*/2)</option>
	<option value="*/3">Every 3 hours (*/3)</option>
	<option value="*/4">Every 4 hours (*/4)</option>
	<option value="*/6">Every 6 hours (*/6)</option>
	<option value="0,12">Every 12 hours (0,12)</option>
	<option value="--">-- Hours --</option>
	<option value="0">12:00 a.m. midnight (0)</option>
	<option value="1">1:00 a.m. (1)</option>
	<option value="2">2:00 a.m. (2)</option>
	<option value="3">3:00 a.m. (3)</option>
	<option value="4">4:00 a.m. (4)</option>
	<option value="5">5:00 a.m. (5)</option>
	<option value="6">6:00 a.m. (6)</option>
	<option value="7">7:00 a.m. (7)</option>
	<option value="8">8:00 a.m. (8)</option>
	<option value="9">9:00 a.m. (9)</option>
	<option value="10">10:00 a.m. (10)</option>
	<option value="11">11:00 a.m. (11)</option>
	<option value="12">12:00 p.m. noon (12)</option>
	<option value="13">1:00 p.m. (13)</option>
	<option value="14">2:00 p.m. (14)</option>
	<option value="15">3:00 p.m. (15)</option>
	<option value="16">4:00 p.m. (16)</option>
	<option value="17">5:00 p.m. (17)</option>
	<option value="18">6:00 p.m. (18)</option>
	<option value="19">7:00 p.m. (19)</option>
	<option value="20">8:00 p.m. (20)</option>
	<option value="21">9:00 p.m. (21)</option>
	<option value="22">10:00 p.m. (22)</option>
	<option value="23">11:00 p.m. (23)</option>
	</select>
	<select style="width: 175px" id="day_options" onchange="select_single_option('day')">
	<option value="--">-- Common Settings --</option>
	<option value="*">Every day (*)</option>
	<option value="*/2">Every other day (*/2)</option>
	<option value="1,15">1st and 15th (1,15)</option>
	<option value="--">-- Days --</option>
	<option value="1">1st (1)</option>
	<option value="2">2nd (2)</option>
	<option value="3">3rd (3)</option>
	<option value="4">4th (4)</option>
	<option value="5">5th (5)</option>
	<option value="6">6th (6)</option>
	<option value="7">7th (7)</option>
	<option value="8">8th (8)</option>
	<option value="9">9th (9)</option>
	<option value="10">10th (10)</option>
	<option value="11">11th (11)</option>
	<option value="12">12th (12)</option>
	<option value="13">13th (13)</option>
	<option value="14">14th (14)</option>
	<option value="15">15th (15)</option>
	<option value="16">16th (16)</option>
	<option value="17">17th (17)</option>
	<option value="18">18th (18)</option>
	<option value="19">19th (19)</option>
	<option value="20">20th (20)</option>
	<option value="21">21st (21)</option>
	<option value="22">22nd (22)</option>
	<option value="23">23rd (23)</option>
	<option value="24">24th (24)</option>
	<option value="25">25th (25)</option>
	<option value="26">26th (26)</option>
	<option value="27">27th (27)</option>
	<option value="28">28th (28)</option>
	<option value="29">29th (29)</option>
	<option value="30">30th (30)</option>
	<option value="31">31st (31)</option>
	</select>
	<select style="width: 175px" id="month_options" onchange="select_single_option('month')">
	<option value="--">-- Common Settings --</option>
	<option value="*">Every month (*)</option>
	<option value="*/2">Every other month (*/2)</option>
	<option value="*/4">Every 3 months (*/4)</option>
	<option value="1,7">Every 6 months (1,7)</option>
	<option value="--">-- Months --</option>
	<option value="1">January (1)</option>
	<option value="2">February (2)</option>
	<option value="3">March (3)</option>
	<option value="4">April (4)</option>
	<option value="5">May (5)</option>
	<option value="6">June (6)</option>
	<option value="7">July (7)</option>
	<option value="8">August (8)</option>
	<option value="9">September (9)</option>
	<option value="10">October (10)</option>
	<option value="11">November (11)</option>
	<option value="12">December (12)</option>
	</select>
	<select style="width: 175px" id="weekday_options" onchange="select_single_option('weekday')">
	<option value="--">-- Common Settings --</option>
	<option value="*">Every weekday (*)</option>
	<option value="1-5">Mon thru Fri (1-5)</option>
	<option value="0,6">Sat and Sun (6,0)</option>
	<option value="1,3,5">Mon, Wed, Fri (1,3,5)</option>
	<option value="2,4">Tues, Thurs (2,4)</option>
	<option value="--">-- Weekdays --</option>
	<option value="0">Sunday (0)</option>
	<option value="1">Monday (1)</option>
	<option value="2">Tuesday (2)</option>
	<option value="3">Wednesday (3)</option>
	<option value="4">Thursday (4)</option>
	<option value="5">Friday (5)</option>
	<option value="6">Saturday (6)</option>
	</select>

And of course each input in text boxes are checked immediately against errors via JS. (Did I already said that we miss a lot of JS in admin? ^^)

2011/6/13 Alexander Obuhovich <aik....@gmail.com>

[Alexander Obuhovich]

Now, that I know your control panel name I can use it's demo to see how it works:

http://x3demob.cpx3demo.com:2082/frontend/x3/cron/index.html (user: x3demob, password: x3demob)

[Phil -- wbtc.fr --]

you already knew the name, you even asked about more info on Cpanel 2 post before :)

2011/6/14 Alexander Obuhovich <aik....@gmail.com>

[Alexander Obuhovich]

No, cPanel is short name of "control panel" :) I didn't suspected, that somebody will name real product like that.

[Phil -- wbtc.fr --]

haha ok :)

[Dmitry A.]

Hi Alex,

Do you think it's enough info here to finalize and create a task for this?

DA

[Alexander Obuhovich]

Yes, I think so.

However I recommend to create 4 tasks for each of parts I've described, since they are pretty large and unconnected.

This way we can faster create some visible progress on this discussion.

[Alexander Obuhovich]

get rid of AgentType (Before/After) field, since there no such idea in PHP CLI and make all before agents as hooks to adm:OnAfterStartup event 

Done in http://tracker.in-portal.org/view.php?id=1191 task (5.2.0-B1).

scheduled events modify data in session, but they don't have session at all in PHP CLI environment

Task created: http://tracker.in-portal.org/view.php?id=1340 (5.2.0).

Part 2

Then I don't like Agent term, since it doesn't associate with the job that they do. That's why I'm proposing to call them as Job, like all other software products do. This way admin will create a Job and assign an event to perform it.

Done in http://tracker.in-portal.org/view.php?id=981 task (5.2.0-B1).

Part 3

When agents are not executed from PHP CLI (as they should), then they are executed before and after page rendering process, which of course delays it.

I'm proposing to:

1. get rid of AgentType (Before/After) field, since there no such idea in PHP CLI and make all before agents as hooks to adm:OnAfterStartup event
2. after page was displayed to user execute code below, after which:
1. web browser "Loading ..." indicator should show 100%
2. connection to webserver will be terminated
3. and reset of the script will continue to execute on the server

ignore_user_abort(true);

while ( ob_get_level() ) {

ob_end_clean();

}

header('Connection: close');

// header('Content-Length: 0'); // only uncomment when such header isn't sent already

flush();

Task created: http://tracker.in-portal.org/view.php?id=1339 (5.2.0)

Part 4

Also "Run Interval" field doesn't seem to be pretty handy, since it doesn't allow to specify exact time when a job should be executed. I'm proposing to use native crontab format: http://en.wikipedia.org/wiki/Cron , where it has 5 numbers:

• min (0 - 59)
• hour (0 - 23)
• day of month (1 - 31)
• month (1 - 12)
• day of week (0 - 7) (Sunday=0 or 7)

For example "0 * * * *" will mean, that job should be executed every hour. And "*/5 * * * *" will mean to run this job every 5 minutes. This is way better, then specifying seconds on every corner. Maybe we can create fancy control with 5 input boxes or so to ease input process.

Showing expected execution count per year would also be a useful information.

Task created: http://tracker.in-portal.org/view.php?id=1341 (5.2.0).

[Alexander Obuhovich]

Here are patches to http://tracker.in-portal.org/view.php?id=1341 task.

[Alexander Obuhovich]

Here are patch to http://tracker.in-portal.org/view.php?id=1340 task.

[Alexander Obuhovich]

Here are patch to http://tracker.in-portal.org/view.php?id=1339 task.