Multithreading in X11 and Qt
Joerg Friebe
I am trying to write a multithreaded program using the
Qt toolkit.
In my application I have a working thread running in
background to do some extensive work. After completing,
it should generate an event and post it to the system
message queue so that I can respond to in in the
main thread (the one that is running the X11 event queue).
This can easily be done unter Windows 95/NT and OS/2,
since those systems are thread safe. When I'm trying to
do this with X11, I always get unexpected async return
messges opening Widgets in something else but the main
thread. When I try to generate a Qt-Event using
QApplication::postEvent from the secondary thread,
the Event seems to be handled by QObject::event in the
secondary thread as well, allthough I want it to be handled
in the first thread. What can I do? I can't believe that neither
X11 nor Qt are'n thread-safe...
Please reply to
Joerg....@Informatik.Uni-Oldenburg.DE
gmd
Joerg Friebe wrote:
At the lower levels of the X toolkit I typically use one
of the following to perform "multi-threading". These allow
the X event dispatcher to be the main dispatcher for the
process.
XtInputId XtAddInput(source, condition, proc, client_data)
int source;
XtPointer condition;
XtInputCallbackProc proc;
XtPointer client_data;
XtIntervalId XtAddTimeout(interval, proc, client_data)
unsigned long interval;
XtInputCallbackProc proc;
XtPointer client_data;
XtWorkProcId XtAddWorkProc(proc, client_data)
XtWorkProc proc;
XtPointer client_data;
If I need finer grain "multi-tasking" I typically write my
own main select based loop and do the dispatching
directly. These techniques mean you need to write
in an async event driven technique, but they give
you more control over priority/time slice scheduling.
Based on your description, I would setup an X based
loop, and add a work process to do the background
task. The work process needs to decide how much
processor time to use before returning to the
main loop(read the man page).
Kevyn B. Ford
> In my application I have a working thread running in
> background to do some extensive work. After completing,
> it should generate an event and post it to the system
> message queue so that I can respond to in in the
> main thread (the one that is running the X11 event queue).
[...]
> When I try to generate a Qt-Event using
> QApplication::postEvent from the secondary thread,
> the Event seems to be handled by QObject::event in the
> secondary thread as well, allthough I want it to be handled
> in the first thread. What can I do? I can't believe that neither
> X11 nor Qt are'n thread-safe...
I am fairly sure that Xt/X11 is *not* thread-safe. I think this was
due to both simplicity and performance issues. I would suggest that
you change your design and have only one thread (main?) handling
GUI-related tasks. You would then communicate from your secondary
threads through the use of non-GUI "event" queues (or some other type
of user-defined data structure).
This is actually the model that is recommended for use with
Java and the new Swing GUI components. For performance reasons, the
new GUI componenets are not thread-safe and they suggest relegating
all GUI tasks with only one thread.
Alan Linton
Kevyn B. Ford <kbf...@computer.org> wrote:
>Joerg Friebe wrote:
>
>> In my application I have a working thread running in
>> background to do some extensive work. After completing,
>> it should generate an event and post it to the system
>> message queue so that I can respond to in in the
>> main thread (the one that is running the X11 event queue).
>[...]
>> When I try to generate a Qt-Event using
>> QApplication::postEvent from the secondary thread,
>> the Event seems to be handled by QObject::event in the
>> secondary thread as well, allthough I want it to be handled
>> in the first thread. What can I do? I can't believe that neither
>> X11 nor Qt are'n thread-safe...
>
>I am fairly sure that Xt/X11 is *not* thread-safe.
they will not do what Joerg is trying to do.
Qt is not thread-safe. AFAIK.
>I think this was
>due to both simplicity and performance issues.
They used to be non-thread-safe because not a lot of unices had threads
10 years ago.
>I would suggest that
>you change your design and have only one thread (main?) handling
>GUI-related tasks. You would then communicate from your secondary
>threads through the use of non-GUI "event" queues (or some other type
>of user-defined data structure).
This is a good idea regardless of whether the library is ts or not.
It's also what I think Joerg was trying to do.
Joerg: Unfortunately you can't use the Qt event system. You'll have
to find some other manner of passing information between your threads
(monitors, semaphores, etc.). I personally don't use threads, I use
the functions of Xt that are mentioned in the other response to your post.
Alan
--
Alan Linton
Motif and Xt hacker looking for work.
Arnt Gulbrandsen
> I am fairly sure that Xt/X11 is *not* thread-safe.
Xlib, at least, can be built to be thread-safe. If your target
platform is sufficiently narrow and well-defined, you can even assume
that the potential users of your program will have an Xlib built to be
thread-safe.
Then again, talking about narrow target platforms always reminds me of
a quote from the O'Reilly POSIX.1 book, by Donald Lewine. It's from
the internationalization chapter, so it's not directly relevant, but I
quote it anyway.
I have lots of horror stories about people who knew their software
was only for the domestic market, only to have the boss come in
with the big deal they just closed in Saudi Arabia. Then there
was the person who discovered that Puerto Rico was part of the
United States...
> ...
(stuff I agree with deleted)
--Arnt
Joerg Friebe
Alan Linton schrieb in Nachricht <72nink$rua$1...@elessar.middle.earth>...
>>[...]
>>> in the first thread. What can I do? I can't believe that neither
>>> X11 nor Qt are'n thread-safe...
>they will not do what Joerg is trying to do.
>Qt is not thread-safe. AFAIK.
good. :-)
>>I think this was due to both simplicity and performance issues.
>They used to be non-thread-safe because not a lot of unices had threads
>10 years ago.
those were the days.... ;-)
>>threads through the use of non-GUI "event" queues (or some other type
>>of user-defined data structure).
>This is a good idea regardless of whether the library is ts or not.
>It's also what I think Joerg was trying to do.
Yes. One thread for X, and some other threads for the real work.
>Joerg: Unfortunately you can't use the Qt event system.
I see. Since threads are mentioned nowhere in the Qt docs, this is
what I thought after all. But couldn't I put an event into X11 from my
secondary threads to which the GUI-thread can react? Qt won't even
know there are other threads around. But I have to synchronize with
the first thread in some way, and I don't know how I can do it.
>You'll have to find some other manner of passing information
>between your threads (monitors, semaphores, etc.).
Yes, I've built my own message queue with semaphores. Works great,
and all my secondary threads can communicate the way I want to. But
when I once put an event into one of my secondary threads, there's no
way back, so I can't display my results. Very unsatisfactory :-)
>I personally don't use threads,
But threads are very nice!
Joerg Friebe
>of the following to perform "multi-threading". These allow
>the X event dispatcher to be the main dispatcher for the
>process.
>
> XtInputId XtAddInput(source, condition, proc, client_data)
> int source;
> XtPointer condition;
> XtInputCallbackProc proc;
> XtPointer client_data;
>
> XtIntervalId XtAddTimeout(interval, proc, client_data)
> unsigned long interval;
> XtInputCallbackProc proc;
> XtPointer client_data;
>
> XtWorkProcId XtAddWorkProc(proc, client_data)
> XtWorkProc proc;
> XtPointer client_data;
OK. Maybe I can make use of this. I have no idea of Xt internals,
so I don't know what those three functions are about. But maybe
I can use them to achieve what I want.
What does
XtAddInput (int source, XtPointer condition, XtInputCallbackProc proc,
XtPointer client_data)
do? It looks like I can use it to put events into X11 and associate a
callback procedure to
be called when the event arrives; and even specify some client data to be
used in that
callback proc. What does proc look like? And what is source? Maybe I can use
it like this:
void callback_proc (void *client_data){
do_something_with_my_event ((MyData*)client_data);
}
...
int input_queue = something();
cond_t condition = foo();
MyData *my_data = get_my_data();
XtAddInput (input_queue, condition, &callback_proc, my_data);
How do I put my Event into input_queue? What does condition look like? Is it
some
kind of semaphore/condition? Like this:
cond_t condition;
cond_init (&condition, USYNC_THREAD, 0);
and then use
cond_signal (&condition);
to signal the event to X? That yould be great. Then I could let my_data
point to
an event_queue which could be processed in callback_proc.
So I would do this /* I love C++ :-) */:
class MyEventQueue {
public:
MyEventQueue (){
cond_init (&condition, USYNC_THREAD, 0);
mutex_init (&mutex,
}
~MyEventQueue (){
cond_destroy (&condition);
mutex_destroy (&mutex);
}
void AddToX (int source){
XtAddInput (source, &condition, &MyEventQueue::callback, this);
}
void PutEvent (MyEvent *event){
mutex_lock (&mutex);
queue.Add (event);
mutex_unlock (&mutex);
cond_signal (&condition);
}
private:
cond_t condition; // a condition, see solaris man pages
mutex_t mutex; // a mutex, see solaris man pages
Queue queue; // some sort of Queue having Add() and GetEvent()
private:
static void callback (void *my_data){
MyEventQueue *q = (MyEventQueue*) my_data;
q->ProcessEvents();
}
void ProcessEvents(){
MyEvent *event;
for(;;) {
mutex_lock (&mutex);
event = queue.GetEvent();
mutex.unlock (&mutex);
if (event){
event->Process();
}else{
break;
} // if
} // for
} // ProcessEvents
}; // class MyEventQueue
MyEventQueue *event_queue = new MyEventQueue;
event_queue->AddToX (source);
Now what is source?????
Then from my secondary thread I would do:
event_queue->PutEvent (new RepaintEvent (some_data));
Which would be processed by X in the first thread. Looks good.
Might even work. ... Can somebody please tell me, if I am right?
Thank you in advance.
Wolfram Gloger
> > XtInputId XtAddInput(source, condition, proc, client_data)
> > int source;
> > XtPointer condition;
> > XtInputCallbackProc proc;
> > XtPointer client_data;
> >
> > XtIntervalId XtAddTimeout(interval, proc, client_data)
> > unsigned long interval;
> > XtInputCallbackProc proc;
> > XtPointer client_data;
> >
> > XtWorkProcId XtAddWorkProc(proc, client_data)
> > XtWorkProc proc;
> > XtPointer client_data;
>
>
> OK. Maybe I can make use of this. I have no idea of Xt internals,
> so I don't know what those three functions are about. But maybe
> I can use them to achieve what I want.
Not with Qt, though, since its authors have decided they can do so
much better than Xt and don't use libXt at all.
For a possible solution for threaded X11 apps which is based on Xt and
other really free libraries, please check out
http://www.dent.med.uni-muenchen.de/~wmglo/wxthread/
This can be made to work both with libc5 (and other systems with
non-threadsafe X11) and libc6.
Regards,
Wolfram.
Mike McDonald
"Joerg Friebe" <Joerg....@Informatik.Uni-Oldenburg.DE> writes:
> OK. Maybe I can make use of this. I have no idea of Xt internals,
> so I don't know what those three functions are about. But maybe
> I can use them to achieve what I want.
>
> What does
> XtAddInput (int source, XtPointer condition, XtInputCallbackProc proc,
> XtPointer client_data)
> do?
man XtAddInput! Basicly, it adds the file descriptor source to the list of
file descriptors that X does a select() on. If there's data waiting to be read
on source, it'll call proc with your client_data. So, if your "events" you're
trying to send back to your main thread are on a file descriptor, it'll work
fine.
>It looks like I can use it to put events into X11 and associate a
> callback procedure to
> be called when the event arrives; and even specify some client data to be
> used in that
> callback proc. What does proc look like? And what is source? Maybe I can use
> it like this:
Nope, the proc needs to be an XtWorkProc. Check the X header files for what
it's defined as.
> Now what is source?????
A file descriptor.
> Then from my secondary thread I would do:
>
> event_queue->PutEvent (new RepaintEvent (some_data));
>
> Which would be processed by X in the first thread. Looks good.
> Might even work. ... Can somebody please tell me, if I am right?
> Thank you in advance.
Most likely, it won't work. You'll be tempted to make a pipe, have one end
added to the X main loop with XtAddInput() and then write "events" on the
other end. It'll seem to work but occasionally, you'll hang your program.
(Since your program is both the producer and consumer of the pipe, you can
very easily hang your program blocking on the pipe.) So don't do that.
How do you currently tell if you have an "event" in one of your queues? What
you want the event loop to look like is something like:
while (1) {
if (XtPending()) {
XtNextEvent(&xevent);
XtDispatchEvent(&xevent);
}
if (event_queue->IsReady()) {
ev = event_queue->GetEvent();
DispathEvent(ev);
}
}
Now, the problem with this is it'll eat CPU cycles up when there are no
events pending on either source. Whether you can get around that depends on
how your queues are implemented.
Mike McDonald
mik...@mikemac.com
Alan Linton
Joerg Friebe <Joerg....@Informatik.Uni-Oldenburg.DE> wrote:
>
>Alan Linton schrieb in Nachricht <72nink$rua$1...@elessar.middle.earth>...
>>>[...]
>>>of user-defined data structure).
>>This is a good idea regardless of whether the library is ts or not.
>>It's also what I think Joerg was trying to do.
>
>Yes. One thread for X, and some other threads for the real work.
XInitThreads(3).
>>Joerg: Unfortunately you can't use the Qt event system.
>
>I see. Since threads are mentioned nowhere in the Qt docs, this is
>what I thought after all. But couldn't I put an event into X11 from my
>secondary threads to which the GUI-thread can react? Qt won't even
>know there are other threads around. But I have to synchronize with
>the first thread in some way, and I don't know how I can do it.
>
>>You'll have to find some other manner of passing information
>>between your threads (monitors, semaphores, etc.).
>
>Yes, I've built my own message queue with semaphores. Works great,
>and all my secondary threads can communicate the way I want to. But
>when I once put an event into one of my secondary threads, there's no
>way back, so I can't display my results. Very unsatisfactory :-)
I should think.
You could use a subclass of QIODevice and a unix pipe to pipe information
from one of your other threads into your display thread. That is, one
end of the pipe is in the QIODevice in your display thread and the
other end is writable by another thread. You could use this merely as
a signalling mechanism to alert your display thread that data is waiting
for it to display if you don't want to actually pass the data over the pipe.
This is actually not a perfect solution, as it may not be portable to non-unix
platforms. (Of course, I would be suprised to discover that Qt is portable
to non unix platforms.)
>>I personally don't use threads,
>But threads are very nice!
Yes but then you get into problems like you are having. It's also much
harder to debug threaded applications.
Joerg Friebe
>> so I don't know what those three functions are about. But maybe
>> I can use them to achieve what I want.
>much better than Xt and don't use libXt at all.
So I can't use Xt calls in a Qt app? bad, bad, really bad...
It looks like in fact they can do so much better than Xt that
they don't provide any possibility to build a modern program which
uses modern techniques like CORBA or threads.
If KDE wasn't built on top of Qt I definitely wouldn't use that Qt.
I don't like libraries which require a preprocessor like moc to run before
I can make use of the library. And I don't like libraries that extend the
C++ syntax with slot, signal and emit. All very nonstandard.....
But which other free C++ class library with a tree view, splitter
window, docking toolbars and docking menubars, tab controls etc
can I use? KDE is so damned good looking.... :-)
Joerg Friebe
>> XtAddInput (int source, XtPointer condition, XtInputCallbackProc proc,
>> XtPointer client_data)
>> do?
> man XtAddInput! Basicly, it adds the file descriptor source to the list
of
>file descriptors that X does a select() on. If there's data waiting to be
read
>on source, it'll call proc with your client_data. So, if your "events"
you're
>trying to send back to your main thread are on a file descriptor, it'll
work
>fine.
:-) Our man pages are a little not so well installed :-)))
But in the meantime I went to our library and took a look at one
of those famous X11 books, volume 4 and 5. Now I know what
XtAddInput is all about and that it has been replaced by XtAppAddInput.
>> callback proc. What does proc look like? And what is source? Maybe I can
use
>> it like this:
> Nope, the proc needs to be an XtWorkProc. Check the X header files for
what
>it's defined as.
Ok, found out that.
>> Now what is source?????
> A file descriptor.
yes, indeed, but very bad. So I think I'm going to build a pipe or a socket,
add
it to my program, and every time I have an event, I write one byte into that
pipe
from my secondary thread. Then, the X thread sees it in its select() and I
eat
that byte and have a look at my homemade message queue and process all
events in it. That should work, I guess. So I use the pipe as some kind of a
Semaphore. Maybe I'll open a socket..... don't know yet.
> Most likely, it won't work. You'll be tempted to make a pipe, have one
end
>added to the X main loop with XtAddInput() and then write "events" on the
:-) yes. X11 is too old to know anything about cond_t :-(.
>other end. It'll seem to work but occasionally, you'll hang your program.
Oh, no!!! Why the hell is that???
>(Since your program is both the producer and consumer of the pipe, you can
>very easily hang your program blocking on the pipe.) So don't do that.
But if one thread writes into the pipe and the other thread reads, it should
work, shouldn't it? And the reading thread always looks into the pipe using
select(), so I hope, it should work. That's what I'm going to try out.
>
> How do you currently tell if you have an "event" in one of your queues?
I don't care. X should see my event in its select() call, and then in my
callback routine, I take the events from my real message queue. The
event structures are passed using my own event queue, the pipe contains
only one byte per event to make X see that there is input.
>What you want the event loop to look like is something like:
>
> while (1) {
> if (XtPending()) {
> XtNextEvent(&xevent);
> XtDispatchEvent(&xevent);
> }
Ok, that's what X is doing in its main loop... (*looking into O'Reilly
book*)
> if (event_queue->IsReady()) {
> ev = event_queue->GetEvent();
> DispathEvent(ev);
> }
... and event_queue->IsReady() is non-blocking, right? And XtPending() is
non-blocking, too? Then this program should produce CPU load like crazy...
No, no, I can't do that. That's polling, and I hate to do that. I think I'll
take
the risk of deadlocking my program using the pipe. :-)
> }
>
> Now, the problem with this is it'll eat CPU cycles up when there are no
:-) yes!
>events pending on either source. Whether you can get around that depends on
>how your queues are implemented.
>
> Mike McDonald
Thanks very much for your help.
Joerg Friebe
>Good. If multiple threads may end up in xlib don't forget to call
>XInitThreads(3).
Aaaaaah! So maybe this is why under linux the app crashes when
the first X event arrives and I have linked with -lpthread??
Very dubious effect, though. Just linking in linux pthread and
Qt goes limbo. No problem at all under Solaris....
>>Yes, I've built my own message queue with semaphores. Works great,
>>and all my secondary threads can communicate the way I want to. But
>>when I once put an event into one of my secondary threads, there's no
>>way back, so I can't display my results. Very unsatisfactory :-)
>I should think.
:-)
>You could use a subclass of QIODevice and a unix pipe to pipe information
>from one of your other threads into your display thread. That is, one
Hey, great Idea! I guess, that's it. Then I also work around those
XtAppAddInput
calls which are unavaillable since Qt doesn't use Xt .....
>end of the pipe is in the QIODevice in your display thread and the
... so I'll use int pipe (int fildes[2]) from unistd.h ... one end in
QIODevice, one
end protected by mutexes in the other threads... sounds good.
>other end is writable by another thread. You could use this merely as
>a signalling mechanism to alert your display thread that data is waiting
:-) yeah, right! :-) No real data flowing through the pipe, just using it as
kind of an event semaphore ;-)
>for it to display if you don't want to actually pass the data over the
pipe.
>This is actually not a perfect solution, as it may not be portable to
non-unix
>platforms. (Of course, I would be suprised to discover that Qt is portable
>to non unix platforms.)
No problem. I'm building a Mechanism to run under UNIX only. For the
other platforms (Windows NT and OS/2, that is) I don't have problems like
these since their GUIs are thread safe and I can communicate my events
to window procedures without harm.
Troll says they have portet Qt to Win32, but I don't need it. The Win32
solution we have here makes full use of ActiveX controls IUnknown and
things like that, so Qt in Win32 would be of no use for us.
>>>I personally don't use threads,
>>But threads are very nice!
>Yes but then you get into problems like you are having.
They have to be solved. It's essential. Everybody building modern apps
should have these problems and it's up to the people building things like
Qt and Xt and Motif to find a standard means to communicate with threads.
Everybody who ever wrote a Win32 or OS/2 program needs it.
>It's also much harder to debug threaded applications.
Debuggers should be able to handle threads, too. But ddd using gdb does
a quite good job. And that debugger inside Microsoft Visual Studio 97 is
good, too.
So, thank you very much for your hints. Subclassing QIODevice is the clue,
along with using int pipe(int fildes[2]). :-) It should work. Next week, I'm
going
to try it out, since now I have more important work to do.... :-) we've a
deadline
to keep.
Joerg Friebe
I found out another method, which is entirely Qt based. Goes like this:
class MyEventQueue: QObject {
Q_OBJECT
public:
MyEventQueue(){
pipe (pipes);
sn = new QSocketNotifyer (pipes[0], QSocketNotifier::Read);
connect (sn, SIGNAL(activated(int)), this, SLOT(dataReceived(int)));
}
void PutEvent (MyEvent *event){
mutex.Lock();
queue->Put (event);
mutex.Unlock();
char c = 1;
write (pipes[1], &c, 1);
}
slots:
int dataReceived(int socket){
read_bytes_from (socket);
for(;;) {
MyEvent *event = queue->Get();
if (!event)
break;
event->process();
}
}
private:
MyQueue queue;
int pipes[2];
QSocketNotifyer *sn;
MyMutex mutex;
};
Hope this helps. Thanks to Martin Vogt.
Thomas Rink
AFAIK at least v. 1.41 of Qt has an interface to Xt. Check out the
documentation at www.troll.no. I don't like the way the signal/slot
mechanism is implemented, however, the idea behind it is good.
-- Thomas
--
Thomas Rink
Lehrstuhl f. Biophysik
Ruhr-Universitaet Bochum
D-44780 Bochum
*** please reply to tho...@bph.ruhr-uni-bochum.de ***
Mike McDonald
"Joerg Friebe" <Joerg....@Informatik.Uni-Oldenburg.DE> writes:
>>> Now what is source?????
>> A file descriptor.
>
> yes, indeed, but very bad. So I think I'm going to build a pipe or a socket,
pipe() and socket() return file descriptors.
> But if one thread writes into the pipe and the other thread reads, it should
> work, shouldn't it? And the reading thread always looks into the pipe using
> select(), so I hope, it should work. That's what I'm going to try out.
As long as the reader can keep up with all of the writers, it'll work. If
the reader can't keep up, when the buffer associated with the pipe fills up,
the writer will block. Having more writers than readers increases this chance.
Now, if you make sure your reader thread never, ever does a write, then you'll
probably get away with it. (Provided you have a non blocking thread
implementation.)
Mike McDonald
mik...@mikemac.com