Re: [Caml-list] Co-existing with non OCaml threads
Ted Kremenek
From my understanding of OCaml threads, they are non-preemptive user-
level threads that run within a single kernel thread. Further, OCaml
does not employ a concurrent garbage collector; essentially the whole
program stops (including all OCaml threads) when the garbage
collector runs. OCaml threads do not provide any additional actual
parallelism (they are not preemptive like kernel threads), but like
all user-level threads they provide an abstraction of parallel
threads as a programming idiom. I am not certain if in the
implementation of OCaml threads that one thread automatically yields
to another when it calls a blocking operation like an I/O operation
(like the user-level thread implementation for C described in the
Capriccio paper that appeared in SOSP a few years ago), but I doubt it.
You might be able to run the OCaml run-time (I am talking about
native code) within a program that employs multiple kernel threads as
long as ALL the OCaml code runs in one designated thread, but the run-
time was not designed to be executed within multiple kernel threads.
Of course I could be completely wrong about this, but this was my
interpretation from the documentation, previous emails on this list,
and from my brief circumspection of the code from the OCaml run-time.
Ted
On May 11, 2006, at 5:29 PM, Francois Rouaix wrote:
> I'm contemplating writing an OCaml interface for a C++ middleware
> library that my company develops and uses internally. Typically
> this middleware will start an event loop on a thread in the
> background, leaving the application responsible for its own threads
> (and potentially using none and having its own code running
> entirely on events within the eventloop thread).
> How's this likely to be compatible with OCaml use of native threads
> (this is on Linux by the way)?
> The manual section for interfacing with C isn't mentionning threads
> anywhere.
> Should Caml code be restricted to run on threads it has created? Or
> can it run on any threads?
> How can I synchronize between a thread running C++ code and a
> thread running OCaml code (i.e. both communicating on a message
> queue)?
> Thanks for any suggestions.
> --f
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Gerd Stolpmann
> I'm contemplating writing an OCaml interface for a C++ middleware
> library that my company develops and uses internally. Typically this
> middleware will start an event loop on a thread in the background,
> leaving the application responsible for its own threads (and
> potentially using none and having its own code running entirely on
> events within the eventloop thread).
> How's this likely to be compatible with OCaml use of native threads
> (this is on Linux by the way)?
> The manual section for interfacing with C isn't mentionning threads
> anywhere.
> Should Caml code be restricted to run on threads it has created? Or
> can it run on any threads?
> How can I synchronize between a thread running C++ code and a thread
> running OCaml code (i.e. both communicating on a message queue)?
> Thanks for any suggestions.
If you compile ocaml with pthread support, the O'Caml threads are real
Linux threads. When using them, be aware of:
- The memory management code is not thread-safe. That means
only one thread is allowed to run O'Caml code at any time.
To ensure this there is the so-called "master lock". You
can acquire the master lock by calling
leave_blocking_section()
and you can release it by
enter_blocking_section()
When you call a C function that may block for indefinite time
one usually releases the lock first, calls the function, and
then requires it. The stub function looks like
value foo_stub (...) {
...
enter_blocking_section();
/* From here: NO ACCESS to any O'Caml memory, not even
* CAMLparam/CAMLlocal variables!
*/
...
foo();
...
leave_blocking_section();
/* Normal rules for stub functions apply again */
...
}
For callbacks from C code you will typically first call
leave_blocking_section() and later enter_blocking_section()
before you return.
- O'Caml must never see threads not created by itself. This
means a thread created by your middleware must not run
O'Caml code. (You get immediately a segfault if you try.)
- Although O'Caml mutexes and condition variables actually
base on their pthread counterparts, there is no interface
from the C side. So you better do any synchronization
completely in C, i.e. the message queue is written in C
and uses the normal pthread primitives. The O'Caml stub
functions accessing this queue need to call
enter/leave_blocking_section as explained.
Hope this helps. Had to solve a similar problem for a customer.
Gerd
--
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany
ge...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------
Vincenzo Ciancia
> - O'Caml must never see threads not created by itself. This
> means a thread created by your middleware must not run
> O'Caml code. (You get immediately a segfault if you try.)
>
However, let's make it clearer, you can create a thread in ocaml and then
call a C function, and this C function, which will be run concurrently with
other C functions called the same way, can call back ocaml.
Vincenzo
--
Please note that I do not read the e-mail address used in the from field but
I read vincenzo_ml at yahoo dot it
Attenzione: non leggo l'indirizzo di posta usato nel campo from, ma leggo
vincenzo_ml at yahoo dot it
Markus Mottl
rote:
> However, let's make it clearer, you can create a thread in ocaml and then
> call a C function, and this C function, which will be run concurrently wi
th
> other C functions called the same way, can call back ocaml.
Yes, this is possible, and good libraries provide ways of letting user
created threads enter the library to execute callbacks.
Markus
--
Markus Mottl http://www.ocaml.info markus...@gmail.com
Ted Kremenek
some questions that would clarify a few things for me.
Because of the run-time lock, should I gather that the threads are
still essentially cooperative threads? For example, does it work
this way: if a thread holding the master lock is switched out and the
kernel schedules another thread, that other thread will start running
and try and acquire the lock. It won't be able to, so it goes back
to sleep, and another thread will wake up, try and acquire the lock,
goes back to sleep, and so on, until the original thread holding the
lock is rescheduled. Only when the thread releases the lock
(yields?) will another thread be able to run. Is this how it works?
If so, this would lend itself to extremely poor performance: if I had
100 threads, 99 of them may have to wake up and go to sleep before
the original one is scheduled. That is 99 useless context switches.
Or rather is the lock acquired and released (within the generated
native code for the OCaml part of a program, not C code) on calls to
the memory management functions and other run-time code that are not
thread-safe? This is also seems slow, since the heap is actively
manipulated all the time, so locks will constantly be acquired and
released, and you will have the same wake-up, make little or no
progress and go back to sleep problem I mentioned before.
Your last email said that only one thread is allowed to run OCaml
code at any time, so it seems to me that this mutual exclusion must
come from somewhere. I'm very curious to know how this is
implemented. I gather that most people want to use threads in OCaml
to have multiple threads running OCaml code, and not necessarily have
a bunch of threads executing called C code (allowing the master lock
to be released). I'm just trying to understand how actual
performance would ever resemble anything desirable.
Just curious. Thanks for the informative email.
Ted
On May 12, 2006, at 4:22 AM, Gerd Stolpmann wrote:
> If you compile ocaml with pthread support, the O'Caml threads are real
> Linux threads. When using them, be aware of:
>
> - The memory management code is not thread-safe. That means
> only one thread is allowed to run O'Caml code at any time.
> To ensure this there is the so-called "master lock". You
> can acquire the master lock by calling
>
> leave_blocking_section()
>
> and you can release it by
>
> enter_blocking_section()
_______________________________________________
Gerd Stolpmann
> This is very interesting. Thank you for pointing this out. I have
> some questions that would clarify a few things for me.
>
> Because of the run-time lock, should I gather that the threads are
> still essentially cooperative threads?
Essentially they are pthreads, i.e. on Linux kernel threads. However,
there is a level of cooperation on top of this.
> For example, does it work
> this way: if a thread holding the master lock is switched out and the
> kernel schedules another thread, that other thread will start running
> and try and acquire the lock.
For all threads known to O'Caml we have:
- Either the thread is running
- Or it is waiting for the master lock (the kernel knows that,
and won't schedule this thread)
- Or it is executing non-O'Caml code because it passed
enter_blocking_section.
I hope it is now clear that getting the master lock is not the problem.
The trick is how a thread releases it.
The running thread (that has the lock) is notified when it must release
the master lock. There is a special controller thread that periodically
checks whether the running thread ran too long. If so, a special
notification mechanism will cause that this thread gives the master lock
up.
Effectively, this is a second scheduler on top of the kernel scheduler.
> It won't be able to, so it goes back
> to sleep, and another thread will wake up, try and acquire the lock,
> goes back to sleep, and so on, until the original thread holding the
> lock is rescheduled.
No, this won't happen, because the kernel does not wake up threads
waiting for a lock that is not free. These are kernel-level locks!
> Only when the thread releases the lock
> (yields?) will another thread be able to run. Is this how it works?
> If so, this would lend itself to extremely poor performance: if I had
> 100 threads, 99 of them may have to wake up and go to sleep before ie..
> the original one is scheduled. That is 99 useless context switches.
> Or rather is the lock acquired and released (within the generated
> native code for the OCaml part of a program, not C code) on calls to
> the memory management functions and other run-time code that are not
> thread-safe?
No this is not done.
> This is also seems slow, since the heap is actively
> manipulated all the time, so locks will constantly be acquired and
> released, and you will have the same wake-up, make little or no
> progress and go back to sleep problem I mentioned before.
>
> Your last email said that only one thread is allowed to run OCaml
> code at any time, so it seems to me that this mutual exclusion must
> come from somewhere. I'm very curious to know how this is
> implemented. I gather that most people want to use threads in OCaml
> to have multiple threads running OCaml code, and not necessarily have
> a bunch of threads executing called C code (allowing the master lock
> to be released). I'm just trying to understand how actual
> performance would ever resemble anything desirable.
Performance is quite good, but you should keep in mind:
- O'Caml code can never run on more than one CPU
- Switches between O'Caml threads are less fine-grained than
between kernel threads. Imagine you stat a file, and it is
necessary to load some blocks from disk. This can take
0.01 s of time. During that time a switch is impossible.
(I.e. the problem is that there are system calls that are
non-blocking in general but that can nevertheless consume
lots of time in unlucky data cases.)
Gerd
>
> Just curious. Thanks for the informative email.
>
> Ted
>
> On May 12, 2006, at 4:22 AM, Gerd Stolpmann wrote:
>
> > If you compile ocaml with pthread support, the O'Caml threads are real
> > Linux threads. When using them, be aware of:
> >
> > - The memory management code is not thread-safe. That means
> > only one thread is allowed to run O'Caml code at any time.
> > To ensure this there is the so-called "master lock". You
> > can acquire the master lock by calling
> >
> > leave_blocking_section()
> >
> > and you can release it by
> >
> > enter_blocking_section()
>
>
--
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany
ge...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------
_______________________________________________
Ted Kremenek
Excellent. I assumed that there was second scheduler of some sort,
but it wasn't clear how it was done. Thank you very much for
elucidating this.
Ted
Markus Mottl
> Because of the run-time lock, should I gather that the threads are
> still essentially cooperative threads? For example, does it work
> this way: if a thread holding the master lock is switched out and the
> kernel schedules another thread, that other thread will start running
> and try and acquire the lock. It won't be able to, so it goes back
> to sleep, and another thread will wake up, try and acquire the lock,
> goes back to sleep, and so on, until the original thread holding the
> lock is rescheduled.
No. The other threads will block in the master lock, which is a
POSIX-mutex + condition variable. Releasing the master lock will
signal exactly one of the blocking threads that it may run. The
context switch, however, does not necessarily happen immediately. It
depends on the OS when this will happen, and the scheduling policy
determines which thread is going to run.
> If so, this would lend itself to extremely poor performance: if I had
> 100 threads, 99 of them may have to wake up and go to sleep before
> the original one is scheduled. That is 99 useless context switches.
That would be horrible, and that's not the way it works.
> Or rather is the lock acquired and released (within the generated
> native code for the OCaml part of a program, not C code) on calls to
> the memory management functions and other run-time code that are not
> thread-safe? This is also seems slow, since the heap is actively
> manipulated all the time, so locks will constantly be acquired and
> released, and you will have the same wake-up, make little or no
> progress and go back to sleep problem I mentioned before.
A timer signal makes sure that the current thread yields once in a
while. Thus, it is not necessary to acquire/release locks at each
allocation, which would make everything run dog slow.
> Your last email said that only one thread is allowed to run OCaml
> code at any time, so it seems to me that this mutual exclusion must
> come from somewhere. I'm very curious to know how this is
> implemented. I gather that most people want to use threads in OCaml
> to have multiple threads running OCaml code, and not necessarily have
> a bunch of threads executing called C code (allowing the master lock
> to be released). I'm just trying to understand how actual
> performance would ever resemble anything desirable.
Unless INRIA implements a GC that can handle multiple threads running
in parallel, which would have its own performance tradeoffs, you'll
essentially always share one processor only. It depends on your
application whether that's a problem. I/O-heavy applications will do
fine, because the system calls can all be performed in parallel. You
can also always call C or Fortran to run in parallel on bigarrays
(matrices), because they don't mess with the OCaml-heap.
Regards,
Markus
--
Markus Mottl http://www.ocaml.info markus...@gmail.com
_______________________________________________
Ted Kremenek
managed by the kernel (since it is managed through the pthreads API)
and that the notify would be on a single blocked thread. If the
signaling was a broadcast where all blocked threads were moved to the
ready queue (versus waking up a single thread) the problem I
mentioned would still appear. I.e., if all the blocked threads woke
up, only one would acquire the lock, but the remaining would get
scheduled and block. Fortunately a broadcast isn't needed for this
problem, since any waiting thread can be rescheduled. I should have
realized this on my own, but I appreciate the description of
"secondary" scheduler implemented by the timer mechanism.
Xavier Leroy
> [Interfacing Caml code with multithreaded C/C++ code]
> The manual section for interfacing with C isn't mentionning threads
> anywhere. Should Caml code be restricted to run on threads it has
> created? Or can it run on any threads?
It depends whether your Caml code is single-threaded or
multi-threaded.
Case 1: The Caml code is single-threaded (more precisely: it is not
linked with Caml's threading library). For instance, Caml provides a
number of functions that you will call back from the main C/C++
multithreaded program. In this case, you can call back Caml code from
any thread. All you need to make sure is that you never call back from
two threads concurrently. Just put a lock around the callbacks and
you're done.
Case 2: The Caml code is linked with Caml's threading library.
Then, you can only call back from threads created by Caml, and as Gerd
Stolpmann mentioned, there are additional subtleties to ensure mutual
exclusion. I'd recommend you avoid this scenario.
> How can I synchronize between a thread running C++ code and a thread
> running OCaml code (i.e. both communicating on a message queue)?
If your message queue is already implemented in C++, all you need to
do is wrap the "put" and "get" operations of the message queue using
the Caml foreign-function interface, and call these functions from
your Caml code.
Hope this helps,
- Xavier