Tcl event loop and threading
Kevin Walzer
community's "best practices" approach toward using threading my
programs--i.e, that the Tcl event loop makes threading unnecessary in
most instances. None of my Tcl programs use threads, but instead use
pipes, open, fileevent, and after to keep a GUI responsive during a long
operation. This approach seems to work quite well.
As I delve into other languages, such as Python, I'm struck by how much
threading is assumed to be an integral part of any GUI program. All the
code samples/"best practices" discussions I've seen emphasize using
threading to keep a GUI responsive. Now, Python does have good support
for threading built into the language, and it's easy to add, but threads
are still an extra layer of complexity that makes debugging harder. I've
sometimes run into mysterious issues in my Python/Tk programs that seem
to crop up when threads are used, and they are hard to solve.
So, my question is this: Is Tcl's event loop that good, a "crown jewel"
if you will? Is threading something to be avoided in Tcl, or is
threading simply superfluous when used in Tcl? Why don't other languages
have an event loop as good as Tcl's?
----
Kevin Walzer
Code by Kevin
http://www.codebykevin.com
Robert Heller
'Threading' is one of those 'buzzword' things (like Object Oriented
Programming), and probably started when 'Java' became one those
'buzzword' things (Java is probably one the first languages to have
threading 'built-in', so many languages have since opted for the
buzzword solution (threading) when there were in fact other (Better?
Simplier?) solutions (eg a plain old(-fasioned) event-loop). Also some
languages, never had an event-loop as part of the basic language
structure and rather than design one in ('Event-Loops are so
old-school'), threading was implemented instead ('because threading is
so very *cool*'). Not because threading was really 'better' in any
sense, but because it was the 'fashionable' solution. Like people
buying new bluejeans that are not only pre-washed (faded) but also
pre-ripped.
Tcl/Tk predates the whole Threading craze and was (originally) written
by people used to writing X11 event-loops (in C). So, an X11
event-loop was incorporated into Tk from day one. Eventually, the Tk
event loop was extended and moved into Tcl itself as a logical
progression in the evolution of Tcl/Tk, when it became clear that
event-loops were good for other things besides X11 GUIs (like Tcp/Ip
servers and such like).
>
> ----
> Kevin Walzer
> Code by Kevin
> http://www.codebykevin.com
>
--
Robert Heller -- 978-544-6933
Deepwoods Software -- Linux Installation and Administration
http://www.deepsoft.com/ -- Web Hosting, with CGI and Database
hel...@deepsoft.com -- Contract Programming: C/C++, Tcl/Tk
Neil Madden
...
>
> So, my question is this: Is Tcl's event loop that good, a "crown jewel"
> if you will?
Yes.
Is threading something to be avoided in Tcl, or is
> threading simply superfluous when used in Tcl?
Multi-threaded apps aren't necessarily bad. The problem is controlling
access to resources shared among those threads, which is typically
solved with explicit use of locks. Manual use of locks is complicated
(remembering to release them, avoiding deadlock, acquiring locks in
correct order, etc, etc).
In Tcl, you have a lot of options before you need to use shared-state
multi-threading: you can use the event loop, you can use multiple
processes (as you do), and you can use Tcl's default thread model in
which threads share nothing and are really more similar to lightweight
processes, communicating via message passing. If you really really
(*really*) need to fine-tune performance then you can even do explicit
locking in Tcl.
Why don't other languages
> have an event loop as good as Tcl's?
Good question. Tcl isn't entirely unique in having a good approach to
concurrency. Erlang and Oz are two languages worth a look for ideas.
-- Neil
Don Porter
> 'Threading' is one of those 'buzzword' things (like Object Oriented
> Programming), and probably started when 'Java' became one those
> 'buzzword' things (Java is probably one the first languages to have
> Tcl/Tk predates the whole Threading craze ...
Someone who's a better tech. historian than you or me will have to
fill in how Ada fits into the picture. It was certainly offering
"built-in" threading at the time Tcl was being birthed.
--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|
Darren New
> access to resources shared among those threads, which is typically
> solved with explicit use of locks.
Only in lame languages. Languages where threads aren't a
slap-on-the-side addition control it via the type system or by having
lockless primitives built into the language.
--
Darren New / San Diego, CA, USA (PST)
"Let the wine breathe" does not mean to blow bubbles.
Trust me: your wine does not need CPR.
Robert Heller
>
> Robert Heller wrote:
> > 'Threading' is one of those 'buzzword' things (like Object Oriented
> > Programming), and probably started when 'Java' became one those
> > 'buzzword' things (Java is probably one the first languages to have
> > threading 'built-in', ...
>
> > Tcl/Tk predates the whole Threading craze ...
>
> Someone who's a better tech. historian than you or me will have to
> fill in how Ada fits into the picture. It was certainly offering
> "built-in" threading at the time Tcl was being birthed.
Ada never achived 'buzzword' status, at least not as widely as Java did.
Christian Gollwitzer
> Because Tcl is my primary programming language, I've absorbed the Tcl
> community's "best practices" approach toward using threading my
> programs--i.e, that the Tcl event loop makes threading unnecessary in
> most instances. None of my Tcl programs use threads, but instead use
> pipes, open, fileevent, and after to keep a GUI responsive during a long
> operation. This approach seems to work quite well.
Some problems cannot really solved without (native) threads. The event
model is fine, as long as the individual event handlers run only for a
short time, which is very typical for GUI applications or network I/O.
But lets assume you are building the GUI for a PDE solver in TCL, you
have an interface like
solveproblem <list of equations>
which calls into a C function that runs 1 hour each call. Without native
multitasking it is not possible to have a responsive GUI, e.g. to
abolish the computation. The multitasking needs not be realized by
threads, however, you can run the solver in a separate process. But also
in pure Tcl; something like
while {
for {
for {
for {
}
}
}
}
doing a long computation inside the event handler of a button is a
killer for threads. Of course, one can spread "update" all over the
code, but this isn't very elegant either...
Christian
Gerald W. Lester
> Robert Heller wrote:
>> 'Threading' is one of those 'buzzword' things (like Object Oriented
>> Programming), and probably started when 'Java' became one those
>> 'buzzword' things (Java is probably one the first languages to have
>> threading 'built-in', ...
>
>> Tcl/Tk predates the whole Threading craze ...
>
> Someone who's a better tech. historian than you or me will have to
> fill in how Ada fits into the picture. It was certainly offering
> "built-in" threading at the time Tcl was being birthed.
Yes, but it became clear very early on that Ada was not going to be the
silver bullet it promised to be -- so a lot of language designed threw any
usable pieces of the baby out with the bath water.
--
+--------------------------------+---------------------------------------+
| Gerald W. Lester |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+
Gerald W. Lester
The key phrase in the original post was "in *most* instances" (emphasis
added by me).
The case you site is one of those instances -- and for most people a rather
rare instance.
Neil Madden
> Neil Madden wrote:
>> access to resources shared among those threads, which is typically
>> solved with explicit use of locks.
>
> Only in lame languages. Languages where threads aren't a
> slap-on-the-side addition control it via the type system or by having
> lockless primitives built into the language.
Care to be more explicit about the non-lame languages you are talking
about? I know of things like Software Transactional Memory in Haskell
and other languages, and have heard of type systems that can prove
absence of deadlock etc, but as far as I'm aware these are (sadly) all a
long way from being "typical" solutions to shared-state multi-threading.
Am I missing something?
-- Neil
Michael Schlenker
> Kevin Walzer wrote:
>> Because Tcl is my primary programming language, I've absorbed the Tcl
>> community's "best practices" approach toward using threading my
>> programs--i.e, that the Tcl event loop makes threading unnecessary in
>> most instances. None of my Tcl programs use threads, but instead use
>> pipes, open, fileevent, and after to keep a GUI responsive during a
>> long operation. This approach seems to work quite well.
>
> Some problems cannot really solved without (native) threads. The event
> model is fine, as long as the individual event handlers run only for a
> short time, which is very typical for GUI applications or network I/O.
> But lets assume you are building the GUI for a PDE solver in TCL, you
> have an interface like
>
> solveproblem <list of equations>
>
> which calls into a C function that runs 1 hour each call. Without native
> multitasking it is not possible to have a responsive GUI, e.g. to
> abolish the computation.
> doing a long computation inside the event handler of a button is a
> killer for threads. Of course, one can spread "update" all over the
> code, but this isn't very elegant either...
The usual way is to use something like continuation passing/trampoline
style to write something like this instead of the tight loop.
(see the examples linked on the pages from http://wiki.tcl.tk/update )
But your right that for monolithical calculations you need threads (or
seperate processes running the heavy lifting code controlled via pipes,
which has some advantages for memory management (you can free resources
by terminating the child process, which isn't always possible when using
threads).
Michael
Michael Schlenker
I currently have to write a lot of python code and therefor have taken a
close look at Twisted, which is pythons event loop programming toolkit.
Other languages like Perl have also event loops in their libraries, but
not as much part of the language as Tcl.
IMHO the Tcl event loop is much easier to use than twisted, although the
python twisted code has some really nice features (deferred values) and
a good framework for writing specific things like network servers.
Python has good support for threads, but you have the usual shared-state
threading with all the typical problems. (all the time having to kill
the hung process due to some threads not terminating during development
and so on...). Tcls threading support may be not as fast in some
situations, but its much more friendly for the typical uses.
Threading isn't superfluous in Tcl, but you can get very far without
needing it. Sometimes its really easier to use threads than to craft a
continuation passing style event loop oriented program, sometimes its
easier to use the event loop. It depends.
In Python i often use threads because it is so cumbersome (in contrast
to Tcls event loop with fileevent) to simply start a process and run it
on a pipe with redirected stdio. There are some modules for it, but not
as easy and robust as the Tcl ones.
Michael
Erik Leunissen
> doing a long computation inside the event handler of a button is a
> killer for threads. Of course, one can spread "update" all over the
> code, but this isn't very elegant either...
>
[update] would do more than keeping a GUI alive.
According to various documentaion, [update idletasks] should be
appropriate here. However, I had at least one situation where [update
idletasks] did not live up to expectations (and reverting to threads
came to mind).
Erik Leunissen
--
leunissen@ nl | Merge the left part of these two lines into one,
e. hccnet. | respecting a character's position in a line.
Neil Madden
...
> Some problems cannot really solved without (native) threads. The event
> model is fine, as long as the individual event handlers run only for a
> short time, which is very typical for GUI applications or network I/O.
> But lets assume you are building the GUI for a PDE solver in TCL, you
> have an interface like
>
> solveproblem <list of equations>
>
> which calls into a C function that runs 1 hour each call. Without native
> multitasking it is not possible to have a responsive GUI, e.g. to
> abolish the computation. The multitasking needs not be realized by
> threads, however, you can run the solver in a separate process.
As you say, this can be solved with separate processes. Where threads
become compelling is when you have some complex data structure that is
very heavily used. For instance, you might have a GUI displaying a
complex data structure, while at the same time lots of complex and
time-consuming updates are being made to that structure. With a process
model, it is difficult to pick one of the processes (GUI or updates)
that should "own" the structure. You need a finer-grained coordination
mechanism, so that processes can concurrently access separate parts of
the data structure, while ensuring consistency of the structure as a
whole. I can only think of locks or transactions that provide that level
of concurrency. I'd certainly be interested if anyone knows of designs
to accomplish this without sharing the data.
In my experience, those kinds of situations seem to be pretty rare,
thankfully.
-- Neil
Uwe Klein
with people not being able to _grok_
concurrency.
That includes a majority of programmers.
You can not realy fix that with any language construct.
What you can do for these people is reduce
having them bump into obstacles, nothing more.
uwe
Darren New
> Ada never achived 'buzzword' status, at least not as widely as Java did.
Part of that is that (for many years) you had to get the compiler
certified. After all, if the object code is targeting missiles at the
proper people, you really don't want the typical "unfit for any purpose"
warranty.
Darren New
> Care to be more explicit about the non-lame languages you are talking
> about?
Ada's type system helps tremendously with these things, as well as with
having guards and selects and such built in. Tcl's inter-thread message
passing mechanism is another example. Hermes and NIL both did quite a
lot with having tons of "threads" in the same address space while not
allowing anyone to walk on anyone else, often getting better performance
than if you actually had OS-level threads.
You can't really avoid deadlocks and still have a fully flexible system.
But you can definitely eliminate race conditions without reducing the
power of your languages.
jbra...@gmail.com
> So, my question is this: Is Tcl's event loop that good, a "crown jewel"
> if you will? Is threading something to be avoided in Tcl, or is
> threading simply superfluous when used in Tcl? Why don't other languages
> have an event loop as good as Tcl's?
At one point, Tcl and Java were both championed by Sun. For business
reasons, this is not a good idea. They realized this soon afterwards
and
dumped Tcl and put their full (marketing) power behind Java. It was a
good business decision because their Java strategy was one of the few
things that
countered their (up til then) bad business decisions with Solaris and
SPARC.
Unfortunately, it invalidated the difficult-to-prove, but still
true (in my opinion) case being made that threads are a bad idea
(for most purposes).
Here are two Computer Science PhDs from respected universities
making the case, the first a high level argument, the second with
specific examples:
John Ousterhout
http://home.pacbell.net/ouster/threads.pdf
As an interestingly linguistically related problem, it is also
difficult to prove that string theory is a bad idea for theoretical
physics.
--
Jason Brazile
suchenwi
> Edward A. Leehttp://www.computer.org/portal/site/computer/menuitem.5d61c1d591162e4...
Very interesting (and nicely written) article - thanks for sharing
that!
Two nice quotes:
"To offer another analogy, a folk definition of insanity is to do the
same thing over and over again and expect the results to be different.
By this definition, we in fact require that programmers of
multithreaded systems be insane. Were they sane, they could not
understand their programs."
"Deterministic ends should be accomplished with deterministic means.
Nondeterminism should be judiciously and carefully introduced where
needed, and it should be explicit in programs. This principle seems
obvious, yet threads do not accomplish it. They must be relegated to
the engine room of computing, to be suffered only by expert technology
providers."
Neil Madden
> On 27 Mrz., 08:12, jbraz...@gmail.com wrote:
>> Edward A. Leehttp://www.computer.org/portal/site/computer/menuitem.5d61c1d591162e4...
>
> Very interesting (and nicely written) article - thanks for sharing
> that!
I agree. A very nice article. The quote that sums up the argument for me
is this one:
"Nondeterminism should be explicitly added to programs, and only where
needed, as it is in sequential programming. Threads take the opposite
approach. They make programs absurdly nondeterministic and rely on
programming style to constrain that nondeterminism to achieve
deterministic aims."
That seems to really get to heart of the matter. This is also the
approach taken in the book "Concepts, Techniques and Models of Computer
Programming" by Van Roy/Haridi (http://wiki.tcl.tk/15160), which quite
nicely guides through a variety of programming models, from completely
deterministic up to highly non-deterministic concurrent and distributed
models.
-- Neil
Arjen Markus
bauersa...@siemens.com> wrote:
> On 27 Mrz., 08:12, jbraz...@gmail.com wrote:
>
> > Edward A. Leehttp://www.computer.org/portal/site/computer/menuitem.5d61c1d591162e4...
>
> Very interesting (and nicely written) article - thanks for sharing
> that!
>
Apart from these quotes, quite apt I must say, I was also impressed by
the
mathematical model the article presents. It inspired me to think about
this
trivial program:
Thread 1 writes zeros to the screen in an infinite loop
Thread 2 writes ones to the screen, also in an infinite loop
Interpret the pattern of ones and zeros as the binary representation
of a real (mathematically speaking) number.
Given enough patience, this program will produce _any_ real (!)
number
between 0 and 1.
In other words: the set of possible outcomes is uncountable!
Not bad for a trivial program :)
Regards,
Arjen
Alexandre Ferrieux
>
> [...] inspired me to think about this trivial program:
>
> Thread 1 writes zeros to the screen in an infinite loop
> Thread 2 writes ones to the screen, also in an infinite loop
>
> Interpret the pattern of ones and zeros as the binary representation
> of a real (mathematically speaking) number.
> Given enough patience, this program will produce _any_ real (!)
> number between 0 and 1.
Unless you have a very bad scheduler, it will have a bounded "time
quantum". Hence no sequence of (say) ones will be longer than a fixed
value k. This precludes quite a bunch of real numbers !
Moreover, the average density of ones will be 0.5 unless you have an
*unfair* scheduler (which is worse than bad !). This precludes numbers
with an imbalance like .001001001001... == 1/7.
So it's not generating the full [0,1[ interval, and it's not dense in
it either (because of the bounded-sequence property), but it's a funny
set anyway ;-)
-Alex
Antti Järvinen
> So, my question is this: Is Tcl's event loop that good, a "crown
> jewel" if you will? Is threading something to be avoided in Tcl, or
> is threading simply superfluous when used in Tcl? Why don't other
> languages have an event loop as good as Tcl's?
Well well, dunno about best practices but I've always used tcl with
threading so that all tcl stuff runs in single thread, including the event
loop. there may then be number of "workhorse" threads doing something
unuseful in the background and they're controlled from this single
thread having the interpreter and ui related stuff. those background
threads don't call tcl functions directly ; it might be safe if
a mutex was placed around every Tcl_EvalEx() etc. but haven't tried that.
--
Costello the Warrior St:18/09 Dx:14 Co:18 In:8 Wi:12 Ch:7 Neutral
Dlvl:16 $:0 HP:129(129) Pw:52(52) AC:-6 Xp:14/83896 T:19462 Satiated
Gerald W. Lester
> Kevin Walzer <k...@codebykevin.com> writes:
>> So, my question is this: Is Tcl's event loop that good, a "crown
>> jewel" if you will? Is threading something to be avoided in Tcl, or
>> is threading simply superfluous when used in Tcl? Why don't other
>> languages have an event loop as good as Tcl's?
>
> Well well, dunno about best practices but I've always used tcl with
> threading so that all tcl stuff runs in single thread, including the event
> loop. there may then be number of "workhorse" threads doing something
> unuseful in the background and they're controlled from this single
> thread having the interpreter and ui related stuff. those background
> threads don't call tcl functions directly ; it might be safe if
> a mutex was placed around every Tcl_EvalEx() etc. but haven't tried that.
I think he was talking about multiple threads each with a Tcl interp. in
them doing stuff (i.e. the Thread extension).
Arjen Markus
wrote:
If one of the threads starves, you get a rational number. Which
excludes
countably many.
I did not assume anything about the properties of the scheduler, so
you are
probably right about the exclusion of quite a few numbers ... (Now:
how to
determine which numbers will be excluded given the (im)balance of the
scheduler?)
Regards,
Arjen
bill...@alum.mit.edu
due to its origin on Unix systems. Threads are lightweight processes
and arose, I believe, on systems in which context-switches were very
expensive. One difference between Unix and many other OSes is that it
is relatively cheap to spawn new processes using fork() and exec().
That meant that there has been much less need for threads in Unix.
Bill Poser
Donal K. Fellows
> Given enough patience, this program will produce _any_ real (!)
> number between 0 and 1.
>
> In other words: the set of possible outcomes is uncountable!
No it won't. :-)
Ignoring the fact that the scheduler probably isn't going to act as a
true random source, you still won't get all real numbers. This is
because each thread will produce digits at a finite rate and hence the
total number of digits produced, even over infinite time, will be at
most countably infinite. While for some (arbitrary) number, K, every
_finite_ subsequence of length K might be in there, you will not get
anything like every infinite sequence (otherwise the Cantor
diagonalization proof is trivially wrong).
The set of reals in the interval [0,1) is uncountably infinite[*].
This means that there is no algorithm possible for enumerating them
all. It *can't* be done. Adding randomness doesn't change that.
Donal.
[* It is also believed to be equivalent to/have a bijective map to the
set of all possible sets of integers, though this is not proven. ]
Arjen Markus
wrote:
> Arjen Markus wrote:
> > Given enough patience, this program will produce _any_ real (!)
> > number between 0 and 1.
>
> > In other words: the set of possible outcomes is uncountable!
>
> No it won't. :-)
>
> Ignoring the fact that the scheduler probably isn't going to act as a
> true random source, you still won't get all real numbers. This is
> because each thread will produce digits at a finite rate and hence the
> total number of digits produced, even over infinite time, will be at
> most countably infinite. While for some (arbitrary) number, K, every
> _finite_ subsequence of length K might be in there, you will not get
> anything like every infinite sequence (otherwise the Cantor
> diagonalization proof is trivially wrong).
>
Ah, that is not quite what I meant - I probably took some shortcuts
in describing my idea.
What I mean is that the set of _possible_ outcomes is every _real_
number between 0 and 1. Of course you would need an uncountably
large set of computers and an infinite amount of time to actually
produce all these numbers, but the outcome of a single run of the
program
would be any one of them (modulo effects of the balancing act done
by the scheduler).
Regards,
Arjen
Larry W. Virden
> I wonder whether the limited use of threading in Tcl
Here's the seminal article discussing the Tcl creator's thoughts on
threads...
Jeff Hobbs
I would agree with this as an old aspect of threads vs lwp. However,
recent redesigns in linux threading have focused on making threads much
more efficient. This same is true of many big iron OSes native
threading libraries, but those were rarely used in favor of the
x-platform portability of the (chunky) posix threads (pthreads).
With CPUs moving quickly towards multi-core designs, the requirement for
true native threading (which among dynamic languages, only a few
including Tcl actually have) is becoming more useful. I would like to
see it "default-capable" for 8.5, but we need to resolve the
pthreads/atfork issues.
Jeff