Lessons Learned: Doing idle/periodic processing with "after"
Dean Brettle
# Hi all,
# I just wanted to share with you some lessons learned concerning
# doing continuous idle or periodic processing with Tcl. This message
# is a discussion of various techniques I've tried. It is also a
# runnable wish script which can be used to experiment with the
# different techniques. Anyway, here is my story...
# I wanted to do some processing whenever Tcl was idle. Reading the
# man page for the "after" command, I decided that calling the
# following proc might work:
proc bad_idle_proc1 {} {
puts -nonewline "."
flush stdout
after idle bad_idle_proc1
}
# Unfortunately, I discovered that GUI events are never processed. It
# seems that once in idle processing starts, all pending idle tasks
# must complete before processing any more GUI events. Since the
# above code ensures that there is always a pending idle task, GUI
# events are never processed. So I decided to add an "update" within
# the idle proc to process the GUI events. Like so:
proc bad_idle_proc2 {} {
puts -nonewline "."
flush stdout
update
after idle bad_idle_proc2
}
# This solved the above problem, but introduced a new one: "update"
# blocks until all pending events are processed. If any of those
# events result in a call to something like "vwait" or "tkwait", then
# events and idle tasks are processed until the wait condition is met
# (e.g. variable is set). Unfortunately, my idle proc has not been
# rescheduled. As a result, the idle proc is stopped until "update"
# returns. This problem is most commonly seen with modal dialog
# boxes. Here is a simple button which illustrates the point.
pack [button .b -text "Call vwait" -command {vwait done}]
# OK, how about doing the "update" after registering the idle proc?
proc bad_idle_proc3 {} {
puts -nonewline "."
flush stdout
after idle bad_idle_proc3
update
}
# Nope. That causes an infinite recursion because "update" will also
# do any idle tasks including the one just registered. Somewhat
# confused, I decide to search the FAQ and dejanews for suggestions.
# Some articles suggested using "after <ms>" instead of "after idle" to
# reschedule the handler, like this:
proc bad_idle_proc4 {} {
puts -nonewline "."
flush stdout
after 1 bad_idle_proc4
after 10; # sleep for 10 ms to simulate a heavily loaded machine
}
# Without the "after 10" line, this _usually_ works. However, if the
# machine I am using is slow or heavily loaded, the timer always
# expires before any GUI events are processed. The result is the same
# as bad_idle_proc1, but the bug would likely go undiscovered for a
# long time. Note that this is a potential problem with any handler
# that periodically reschedules itself this way.
# So after reading through the Tcl source code (glad I decided to use
# a freeware system), I came up with the following
# solution/workaround:
proc good_idle_proc {} {
puts -nonewline "."
flush stdout
after idle {after 0 good_idle_proc}; # "Immediately" after idle
after 10; # Sleep for 10 ms to simulate a heavily loaded machine
}
# This seems to be the only reliable way to do continuous idle
# processing. Here is how it works. After doing the processing, I
# register an idle handler. Once all pending events are processed
# (i.e. we are idle), I set a timer to expire immediately. After
# processing any other idle handlers, Tcl will go back to processing
# events and find that my timer has expired. Then the whole process
# will be repeated.
# To see this in action, just uncomment one of the following commands:
# bad_idle_proc1; # Button never appears
# bad_idle_proc2; # Button appears, but pressing it stops processing
# bad_idle_proc3; # Tcl complains about infinite recursion
# bad_idle_proc4; # Button never appears
good_idle_proc; # Works!
# One final note. This technique can also be used for periodic
# processing by changing "after 0 good idleproc" to "after
# <ms> good idleproc".
# I hope this helps someone. Comments, corrections, etc. are welcome.
# --Dean
Kjell Kolsaker
Hey Dean Brettle,
Thank you for sharing your experiences. Since I started using
TCL-TK last summer, i have been struggling with these questions,
without getting a full overview.
The issue is further complicated when working on different platforms.
Related problems solved on unix seem not to work under NT and
vice versa. One example is balloon help, which I have had to
fix by blind experimentation to make it work under NT. The problem
now is that my NT solution behaves odd under unix..., so more
work remains yet.
One very urgent related problem is catching output from a running
program. Here is a snippet:
#======= BEGIN SNIPPET ======
# Flag variable
global RUNNING ;# In case this is done inside a proc
set RUNNING 1 ;# Flag that controls loop
# Just indicating that a text widget and a button exists (must be
# packed to be visible, but not shown here)
toplevel .app
text .app.text
button .app.button -text {Stop} \
-command {global RUNNING;set RUNNING 0}
# "$command" invokes a simulation program may run for a long time
set channel [open |$command]
while {![eof $channel]} {
# s catches standard output from the executing program
set s [gets $channel]
# Output to a textwidget $text
.app.text insert end $s\n
.app.text see end
# On unix, this works, buttun is active.
update idletasks
# On NT, this must be done to make button active:
update
}
# Close the channel afterwards
catch {close $channel}
# This closing does not stop the execution, but it is "disconnected"
# from the tcl-tk session.
#======= END SNIPPET ======
We are facing several problems/oddities, and our current
solution is a kind of compromize that works, but not perfectly.
- "update" seems to do a lot of things, and it seems that the
process is slowed down. This is however the only option under
NT/Win95
- Under NT (on a 233 MHz processor), the output is received in
chuncs with intervals of seconds. This is very annoying.
This also happens under UNIX (alpha platform), even
when "update idletasks" is used.
Under Win95, however, output is smooth and fine (why?) :-)
- I have tried fileevent instead of the loop, but it doesn't
seem to work under NT/Win95
- Since our program must run under Windows 95, i don't know
if the program can be stopped from within TCL-TK. What we do
now is to produce a file "stop.me" from TCL that the running
program tests for repeatedly
We don't have time to experiment too much, so we therefore stick to
our workarounds without trying to understand the reasons for the
oddities. If, however, other people struggle the same kind of
problems, we could share our solutions.
Except from these "incompatibilities", TCL-TK is great, and I
really hope that further releases can target these questions. The
concept of platform independence is almost 100% working for practical
use, and the idea of pure TCL-TK as user interface towards pure
standard C/FORTRAN/PASCAL simulation programs with console io makes
program maintenence and development very simple.
Finally, after trying your "lesson.tcl" script on NT, I found
another update problem: When starting the script from inside
wish, the button
never occurred. If I instead started the script with
"wish lesson.tcl", the button appeared, but now of course with no
console...
Regards, Kjell Kolsaker
Chris Nelson
Kjell Kolsaker wrote:
> <snip>
I think you want to look into the fileeven command. Consider something
like this (untested, but close):
proc HandleOutput { fid text } {
if { [eof $fid] } {
# Handle child done
catch {close $channel} status
$text insert end "Child exit status $status"
} else {
$text insert end [gets $fid]
}
$text see end
}
set channel [open |$command]
fileevent $channel readable [list HandleOutput $channel .app.text]
...do other app stuff...
Hope this helps.
Chris
--
As MIT is not "Massachusetts" neither is RPI "Rensselaer"
William R. Athanasiou
Wouldn't this have done what you wanted?
-----------------
proc idle_proc {} {
puts -nonewline "."
flush stdout
update idletasks
after 10 idle_proc
}
pack [button .b -text "Call vwait" -command {vwait done}]
idle_proc
----------
Works if 10 is replaced with idle as well.
Will.
--
---------------------------------------------------------------------
William Athanasiou Inet: w...@unx.dec.com
Digital Equipment Corporation
200 Route 9 North Tel: (908) 577-6055
Manalapan, New Jersey 07726 Fax: (908) 577-6003
---------------------------------------------------------------------
Dennis R. LaBelle
Kjell Kolsaker <kjell.k...@kkt.ntnu.no> wrote:
>One very urgent related problem is catching output from a running
>program.
.
.
>We are facing several problems/oddities, and our current
>solution is a kind of compromize that works, but not perfectly.
>
> - Under NT (on a 233 MHz processor), the output is received in
> chuncs with intervals of seconds. This is very annoying.
> This also happens under UNIX (alpha platform), even
> when "update idletasks" is used.
> Under Win95, however, output is smooth and fine (why?) :-)
>
This sounds like your standard output (stdout) is buffered and your
simulation program is not flushing the stdout stream at convenient
times. Instead you are waiting for automatic flushing of the stdout
buffer when it is full. Place a "fflush(stdout)" command after each
write to stdout in the simulation program.
I believe the TCL FAQ covers this topic.
> - I have tried fileevent instead of the loop, but it doesn't
> seem to work under NT/Win95
This is an admitted bug in TCL for Windows. It is stated in the README
file in the "win" directory of TCL's C source code.
> - Since our program must run under Windows 95, i don't know
> if the program can be stopped from within TCL-TK. What we do
> now is to produce a file "stop.me" from TCL that the running
> program tests for repeatedly
I too use TCL/TK as a graphical front end for a simulation process.
However, I "open" the process for both reading and writing. When I
want to terminate the simulation, I use "puts" to send my TERMINATE
command to the simulation. The simulation program reads its standard
input, sees the command to quit and exits.
Dennis LaBelle
Dean Brettle
I wrote:
>
> # Hi all,
>
> # I just wanted to share with you some lessons learned concerning
> # doing continuous idle or periodic processing with Tcl. This message
> # is a discussion of various techniques I've tried. It is also a
> # runnable wish script which can be used to experiment with the
> # different techniques. Anyway, here is my story...
[ Discussion, code and proposed solution snipped ]
Will Athanasiou responded with an alternative solution which, much to my
surprise, also _seemed_ to work. After playing with variations on
Will's solution, it became clear to me that I had misunderstood (and
misrepresented) how Tcl's event processing works. As a result, PLEASE
DISREGARD ALL EXPLANATIONS IN MY PREVIOUS MESSAGE! :(
In an attempt to correct my ignorance I studied the Tcl/Tk source some
more, and found another race condition which can cause _both_ my
previous solution and Will's solution to fail on a heavily loaded or
slow machine. The easiest way to see this is to run the following
script (based on Will's solution for simplicity):
---------------------------------
set reschedule_time 10; # The "rate" at which the idle_proc is run
set sleep_time 20; # The time to sleep to simulate load on the machine
proc idle_proc {} {
global reschedule_time sleep_time
puts -nonewline "."
flush stdout
update idletasks
after $reschedule_time idle_proc
after $sleep_time; # sleep for 20 ms to simulate a heavily loaded
machine
}
idle_proc
puts "Calling update"
flush stdout
update
# The next line will not be reached if sleep_time > reschedule_time
puts "Update returned"
flush stdout
-----------------------------------------------------
Note that "Update returned" is never printed. The problem is that
"update" will not return until there are no more events or idle handlers
to process. Unfortunately, "update" always finds a timer event for our
idle_proc, and never returns. In the above code this condition is
forced using "after $sleep_time", but the same thing could (and does)
happen due to load. Of course, you can avoid the problem by making
reschedule_time big enough. But how big is big enough? Moreover, the
bigger you make it, the slower your idle processing. This may be
unacceptable. In my case it definitely is.
As I noted above, my "good_idle_proc solution" suffers from the same
problem. This probably explains why Kjell Kolsaker said of my solution:
> When starting the script from inside wish, the button never occurred. If I
> instead started the script with "wish lesson.tcl", the button appeared, but
> now of course with no console...
I'm guessing that the code which implements the wish console must call
"update" at some point and hang before the button is created.
I don't think that there is an easy way around this race condition in
general. The only potential workaround I have thought of is based on
creating a new "update" command which cancels the idle_proc before
updating and restarts it afterward. It would also require creating new
"vwait" and "tkwait" commands which restart the idle_proc before waiting
and stop it afterward if within an "update". When I get around to
implementing this, I will post it. Of course, if anyone has a better
way, I'd love to hear about it.
I apologize for the inaccuracies in the previous posting. Hopefully I
have it right this time. Thanks to Will and Kjell for their feedback.
--Dean
Donal K. Fellows
In article <34e5a192...@news.albany.net>,
Dennis R. LaBelle <drla...@albany.net> wrote:
> Kjell Kolsaker <kjell.k...@kkt.ntnu.no> wrote:
>> - I have tried fileevent instead of the loop, but it doesn't
>> seem to work under NT/Win95
> This is an admitted bug in TCL for Windows. It is stated in the README
> file in the "win" directory of TCL's C source code.
I believe this is fixed in 8.1a1
Donal.
--
Donal K. Fellows http://r8h.cs.man.ac.uk:8000/ fell...@cs.man.ac.uk
Department of Computer Science, University of Manchester, U.K. +44-161-275-6137
--
Never underestimate the power of the penguin...