memory management, allocation and collection in kernel mode.
Sussillo, David
Hello,
Est ce qu'il ya quelqun qui sait ce que j'ai besoin de changer dans le "
OCaml memory management system " pour que le "program" funcion bien dans
le "kernel" mode?
------------------------------
Does anybody have an idea what would need to change in the OCaml memory
management system in order for an OCaml program to function properly in
kernel mode? (Linux on a PII). Linux has a physically mapped kernel
space because it doesn' make much sense for a kernel to manage it's own
virtual memory, so it just ignores the whole problem by ignoring virtual
addressing. What that means is one has to deal with a physical
addressing scheme if one wants to run in kernel/system mode.
I've not looke at the OCaml sources at all, but I would assume that the
memory management routines are assuming that whatever process they are
running in is running in user mode. At this point I don't understand the
ramifications of this fact.
As far as I can tell, there are a couple of alternatives:
a . The allocation and GC routines might not even think about the
difference between user mode and system mode at all since memory is
segmented and each process is running in it's own address space. However,
if one was to run these routines in the kernel, then it would be
important to 'gel' with the way memory is allocated in a kernel, at the
very least, an offset of something like 0xffff0000.
b. The allocation and GC rountines take full advantage of running
in their own address space (and segment) and put memory all over the
place. In which case, one would be hard pressed to figure out how to
make it work the memory management machinery work in kernel mode.
So then there are a few questions:
1. Will someone point out to me where I might look in the compiler
source for all of the memory management routines, allocataion and GC? I
guess it's important to see how the compiler reasons about the memory
requirements of OCaml structures, as well as the emitted code (or
libraries?) that it using during runtime.
2. If anyone has any insight into the memory management ramifications of
running an OCaml program in kernel mode, might they elucidate? As I'm
just starting out, I'm interested in both general and specific remarks.
Thanks,
-dave
Stefan Monnier
> Est ce qu'il ya quelqun qui sait ce que j'ai besoin de changer dans le "
> OCaml memory management system " pour que le "program" funcion bien dans
> le "kernel" mode?
Problèmes:
1 - l'allocation de mémoire dans le kernel se fait généralement par page
(c'est difficile est surtout coûteux d'allouer plusieurs pages
consécutives). Donc le GC doit garder trace de tout plein de pages
un peu partout.
2 - l'adressage physique signifie qu'il n'y a pas de swapping.
Donc si tu alloues 32MB et que tu ne les utilises pas, c'est 32MB
temporairement inutilisables (plutôt que 32MB swappés sur disque
où ils n'ennuient personne). Il est donc désirable d'utiliser très peu de
mémoire. Un GC a tendance à gaspiller la mémoire un peu pour diminuer la
fréquence des collections.
3 - le kernel n'est souvent pas préemptible (?), donc il faudra faire attention
à appeler explicitement le yield() de temps à autre pour ne pas perdre trop
d'interruptions et ne pas rendre la machine unresponsive. O'Caml utilise
déjà un GC incrémental si je ne me trompe, donc c'est peut-être pas trop
difficile (famous last words).
4 - ...
-- Stefan
Mark Hayden
however, I can tell that it is possible to run Ocaml
code in the Linux kernel. I got an Ocaml program to run
as a dynamically loadable kernel module without
too many problems. As far as memory management is concerned,
the approach I took was to intercept the calls to malloc()
that Ocaml made to add new heap segments.
--Mark
"Sussillo, David" wrote:
>
> Hello,
>
> Est ce qu'il ya quelqun qui sait ce que j'ai besoin de changer dans le "
> OCaml memory management system " pour que le "program" funcion bien dans
> le "kernel" mode?
>
dol...@pa.dec.com
>From: "Sussillo, David" <suss...@ftintl.com>
>Does anybody have an idea what would need to change in the OCaml memory
>management system in order for an OCaml program to function properly in
>kernel mode?
My first answer would be nothing at all if the kernel has a malloc
function.
>1. Will someone point out to me where I might look in the compiler
>source for all of the memory management routines, allocataion and GC? I
>guess it's important to see how the compiler reasons about the memory
>requirements of OCaml structures, as well as the emitted code (or
>libraries?) that it using during runtime.
I guess you should start with the byte-code interpreter, and when
that's working, you can tackle the native code stuff. The files to
look for are all in byterun:
alloc.[ch]
compact.[ch]
freelist.[ch]
gc.h
gc_ctrl.[ch]
major_gc.[ch]
minor_gc.[ch]
mlvalues.h
roots.[ch]
stacks.[ch]
weak.[ch]
>2. If anyone has any insight into the memory management ramifications of
>running an OCaml program in kernel mode, might they elucidate? As I'm
>just starting out, I'm interested in both general and specific remarks.
The memory management should be easy if you have malloc. For the
native code system, you might have to do something for the stack.
Stefan Monnier made a very good remark:
>3 - le kernel n'est souvent pas priemptible (?), donc il faudra faire attention
> ` appeler explicitement le yield() de temps ` autre pour ne pas perdre trop
> d'interruptions et ne pas rendre la machine unresponsive. O'Caml utilise
> dij` un GC incrimental si je ne me trompe, donc c'est peut-jtre pas trop
> difficile (famous last words).
Translation: since kernel code is never preempted, you'll have to call
yield() from time to time.
Fortunately, that is exactly what the Macintosh version does for
cooperative multi-tasking. Look for "#if macintosh" in interp.c and
replace the call to rotatecursor_action() with yield(). That should
be enough. For native code, I don't have an easy answer.
-- Damien