threads & callbacks...
mat...@users.sourceforge.net
This is continued from the MACTCL list.
I'm writing a Tcl-C extension for the IAX client library for VoIP
solutions and are having problems with its callbacks which do not
come from the main (Tcl) thread, but want to make a callback to
script level.
Zoran Vasiljevic wrote:
>
> Am 23.01.2006 um 16:36 schrieb Mats Bengtsson:
>
> > Then I just discovered that those callbacks are on "Thread 5"
> > while the main Tcl interpreter lives on "Thread 1".
> > Since this is my first experience with this, how do I "move"
> > those callbacks to the Tcl interpreters thread, and what is actually
> > happening in a situation just described. Any ideas.
>
> Generally, you can't assign a callback to be executed
> in any other thread then your current thread. You can
> however send a script to be run in some other thread
> either synchronously or asynchronously by using the
> thread::send command of the Tcl threading extension
> (a part of the Tcl project on SF).
>
> If you want to do all this from the C-level, then
> you might want to get yourself acquainted with
> Tcl event loop processing and most specifically the
> Tcl_ThreadAlert API call.
>
I just came to realize that this is a much harder problem than I first
anticipated.
The IAX library makes all the worker threads so they are not made by
Tcl.
I don't belive I can use the Tcl event loop in that case.
One solution is to have a static global string that is "large enough",
and create the callback as a string in the worker threads callback,
copy (or strcat) it then to the global array (using mutex_locks etc.).
I then keep a Tcl_CreateTimerHandler() running at 100ms, and poll
for any nonempty content in the static string, in which case the
command is executed.
This is a very bad solution but it seems to work pretty ok in this
case.
I don't want to keep it unless this can't find a realistic solution.
But this situation must be fairly frequent for extension developers,
and there must be a "standard" way of solving it.
/Mats
Zoran Vasiljevic
> I just came to realize that this is a much harder problem than I first
> anticipated.
This is mostly true in parallel programming...
>
> The IAX library makes all the worker threads so they are not made by
> Tcl.
In which case you need to "teach" those threads to do
"something" every now and then. Remember, threads are not
preemptive, rather cooperative computing.
But, to be honest, I seem to miss the point in your
question. Do you need to:
a. pass a script and tell (some specific thread) to execute it?
b. create a script and let any of the threads execute it?
What happens to the result? Do you need the result in the
calling thread or are you not interested in the result?
I do not know the IAX library and your application hence
there is not much I can say about that. But whatever you
do, one thing is sure: there are no pre-cooked solutions
in Tcl beside the event loop and threading extension.
If you describe (in a little more detail) what you are after
perhaps somebody can give some ideas how to solve it.
Cheers
Zoran
Gerald W. Lester
See the man/help page for Tcl_AsyncCreate and Tcl_AsyncMark, I beleive they
are what you need.
mat...@users.sourceforge.net
> "something" every now and then. Remember, threads are not
> preemptive, rather cooperative computing.
>
> But, to be honest, I seem to miss the point in your
> question. Do you need to:
>
> a. pass a script and tell (some specific thread) to execute it?
> b. create a script and let any of the threads execute it?
In the Iaxclient_Init() I start up the iax stuff that in turn
create the worker threads it needs etc. This happen inside the lib
which I have no control over. I also register for
callbacks from the iax lib using iaxc_set_event_callback(IAXCCallback)
and from the Tcl script level I register a Tcl command to be executed
for specific iax events.
In IAXCCallback() I assumed that I could use Tcl_EvalObjEx() to execute
the
registered Tcl command appended with a number of event specific stuff.
But the events show up in various threads, "Thread 1" or "Thread 5"
etc.
That's the problem.
> What happens to the result? Do you need the result in the
> calling thread or are you not interested in the result?
I don't care about any results.
Zoran Vasiljevic
gets executed in *any* of the threads *within* the library, whereas
you'd
like them to be executed in your *main* thread?
Is this what you expect?
I anticipate that it is, in which case in IAXCCallback you need to
pass the script to your main thread for execution and be sure you
enter the event loop in your main thread to execute it.
This means that any of the IAX threads which are now "execting"
your callbacks should rather pass the callbacks back to your main
thread and the callback has to have enough info about the thread
who registered it so it can be bounced back. Your main thread must
be sitting in the event loop and it should be notified by any of the
IAX
lib threads that something is posted.
Normally in Tcl threading extension you can post a script from thread A
to be executed in the thread B. In this case, the thread A can be any
of
the IAX lib threads and thread B would be your main thread. The
callback
you register in the IAX library should do an equivalent of:
Tcl_EvalInThread(theradB, yourScript, flags);
The threadB being your main thread. The yourScript being the script of
the callback you want to run. The flags may be a bitmask about what
to do with the result (i.e. wait for result or ignore it or similar).
Tcl itself has no such call built in the core which would allow you to
simply evaluate a script in a given thread. This is what the Tcl
threading extension is doing. But... the extension has no C-API
which you could link against as it was not designed for such things.
It was designed to elevate Tcl threading caps to the script level and
not to introduce new ones.
I recall wanting to add such call in Tcl core as a part of a channel
relection TIP from Andreas in order to better support the Tcl VFS
extension but it turned up to be a cut/paste of about 30% of the
code within the Tcl extension hence I abandoned that.
Now this leaves you with the blues: either figure out how to do it
yourself or go get the threading extension and extract the ThreadSend()
call out of it. This one will allow you to send a script for execution
from any to any thread within the lib.
As I look at this from this perspective, a kind of:
Tcl_EvalInThread(Tcl_ThreadId threadId, const char *script, int
flags);
call should eventually get added into the core as it would ease
your life in this respect considerably.
OTOH, this all is really not that complicated that you can't do it
by yourself. You need to watch for the:
Tcl_ThreadQueueEvent
Tcl_ThreadAlert
and little glue code for locking if you need to collect results.
Cheers,
Zoran
mat...@users.sourceforge.net
> gets executed in *any* of the threads *within* the library, whereas
> you'd like them to be executed in your *main* thread?
Exactly!
> As I look at this from this perspective, a kind of:
>
> Tcl_EvalInThread(Tcl_ThreadId threadId, const char *script, int flags);
>
> call should eventually get added into the core as it would ease
> your life in this respect considerably.
> OTOH, this all is really not that complicated that you can't do it
> by yourself. You need to watch for the:
>
> Tcl_ThreadQueueEvent
> Tcl_ThreadAlert
>
> and little glue code for locking if you need to collect results.
Seems I have to reproduce these two ones from threadCmd.c:
static int ThreadSendObjCmd(dummy, interp, objc, objv)
static int ThreadSend(interp, id, send, clbk, wait)
I will give it a try. Seems difficult though, at least in my
perspective.
Perhaps I learn something...
Many Thanks, Mats
Zoran Vasiljevic
are pretty complex as they are doing *far* more than you actually
need. What you really need is just a couple of glue lines arround
Tcl_ThreadQueueEvent and Tcl_ThreadAlert.
Your callback running in any IAX lib threads will actually generate
yet another event for your main thread (Tcl_ThreadQueueEvent)
and then ping your main thread to run it when possible
(Tcl_ThreadAlert).
You just need to be sure that the callback itself contains at least
two pieces of info: the script to eval and the originating thread ID
(that is the ID of your main thread). And, of course, your main thread
should eventually visit its event loop (Tcl_DoOneEvent) by calling
a [vwait] or [update] or such.
Cheers
Zoran
mat...@users.sourceforge.net
some glue code around it, but is unshure of how much I should remove. A
few questions:
ThreadSpecificData: this seems to be used, amoung other things, to keep
track of main interp for each thread, but I'm sending to Tcl main
thread which doesn't know about the thread package. I'm really confused
here.
There is also a
Tcl_ConditionWait(&tsdPtr->doOneEvent, &threadMutex, NULL);
which I don't know what it does. Seems to check for the max size of the
event queue?
Throw or keep?
I also had to add ThreadEventProc() and strip it down. It's first line
is:
ThreadSpecificData* tsdPtr = TCL_TSD_INIT(&dataKey);
again this ThreadSpecificData; for interp and some flags.
I have put my modified source code at:
http://www.visit.se/~matben/download/threadUtil.c
which is perhaps easier for you to analyze.
Mats
Zoran Vasiljevic
I have removed quite a lot. On the
http://www.archiware.com/www/downloads/threadUtil.c
you will find stripped-down version of your file.
You will note that I added 3 XXX_ calls:
1. XXX_RegisterThread
2. XXX_UnregisterThread
3. XXX_EvalInThread
The 1. needs to be called from your master thread, i.e. the
one you would like to execute the callbacks within.
The 2. needs to be called when your master thread exits (if ever)
or when you do not want to execute any callbacks.
The 3. needs to be called from your IAX threads to post callbacks
to the master thread.
Look at this and tell me if you understand now what's happening.
Cheers
Zoran
mat...@users.sourceforge.net
I am most grateful for all your support. Just tested with the IAX lib,
and everything seems to work fine. Since my initial guess that this
must not be an uncommon situation for extension developers is still
valid, I will try to make this code snippet directly available to
others. Perhaps a wiki page.
Hope to announce the IAX VoIP package as soon it gets a liitle more
tested, and gets windows and linux builds.
Mats
mat...@users.sourceforge.net
Mats