[Caml-list] DSL for handling parsing binary memory structures?
Richard Jones
I wonder if anyone has thought about or developed a domain-specific
language in OCaml for handling the processing of binary structures?
I was thinking of writing something to make system calls into the Xen
hypervisor. The main problem is that the hypervisor takes & returns
binary structures which have very precise size / alignment
requirements, bitfields, etc. (and the requirements change with
versions of the hypervisor). It sounds like the ideal use for a DSL
to describe the binary structure of each version of each call and
automatically build accessors.
Rich.
--
Richard Jones
Red Hat
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Joel Reymont
Take a look at the Erlang bit syntax:
Could something like this be handled by camlp4?
Joel
On May 17, 2007, at 10:21 PM, Richard Jones wrote:
>
> I wonder if anyone has thought about or developed a domain-specific
> language in OCaml for handling the processing of binary structures?
Gerd Stolpmann
> I wonder if anyone has thought about or developed a domain-specific
> language in OCaml for handling the processing of binary structures?
>
> I was thinking of writing something to make system calls into the Xen
> hypervisor. The main problem is that the hypervisor takes & returns
> binary structures which have very precise size / alignment
> requirements, bitfields, etc. (and the requirements change with
> versions of the hypervisor). It sounds like the ideal use for a DSL
> to describe the binary structure of each version of each call and
> automatically build accessors.
Ocamlnet contains support for XDR, a simplified version of what you are
looking for (smallest unit are 4 bytes). But this might be a starting
point for this project.
Gerd
--
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany
ge...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------
Jon Harrop
> I was thinking of writing something to make system calls into the Xen
> hypervisor. The main problem is that the hypervisor takes & returns
> binary structures which have very precise size / alignment
> requirements, bitfields, etc. (and the requirements change with
> versions of the hypervisor). It sounds like the ideal use for a DSL
> to describe the binary structure of each version of each call and
> automatically build accessors.
I think XenSource sell a toolstack for Xen that is written in OCaml. So you
might start there.
--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
The F#.NET Journal
http://www.ffconsultancy.com/products/fsharp_journal/?e
Christian Lindig
If you are interested in the research aspect of this, Kathleen Fisher
did interesting work on parsing ad-hoc data (mostly ascii):
http://www.research.att.com/~kfisher/hancock/release.php
http://www.research.att.com/~kfisher/hancock/release.php
I remember to have seen something like this for OCaml in the recent
proceedings of ICFP or POPL but can't remember right now.
My own work in the context of calling conventions proposes an algebra
to describe structures in memory. This could be a starting point for
a DSL; it is also implemented in the Quick C-- compiler (in OCaml) in
module block.nw.
http://www.st.cs.uni-sb.de/~lindig/papers/blocks/next.pdf
http://www.cminusminus.org/rsync/qc--/src/block.nw
-- Christian
Markus Mottl
> I wonder if anyone has thought about or developed a domain-specific
> language in OCaml for handling the processing of binary structures?
>
> I was thinking of writing something to make system calls into the Xen
> hypervisor. The main problem is that the hypervisor takes & returns
> binary structures which have very precise size / alignment
> requirements, bitfields, etc. (and the requirements change with
> versions of the hypervisor). It sounds like the ideal use for a DSL
> to describe the binary structure of each version of each call and
> automatically build accessors.
I'm currently developing a preprocessor that generates readers/writers
for binary protocols from OCaml type specifications. It should be
very easy to add custom binary decoders by specifying an abstract type
for such values + custom readers/writer. Or one can (ab)use the
existing ones to get the intended result (e.g. just use the int32 type
to read a 32bit integer from some memory loccation, etc.).
The protocol is very compact (even slightly more so than the
OCaml-marshal format), and performance is extremely high, usually even
higher than the standard marshal function, though it doesn't handle
shared/cyclic datastructures. I also haven't looked into endianness
functionality yet (should be easy to add), but both 32bit and 64bit
platforms are supported.
In later stages there will also be a feature for a "protocol
handshake", which guarantees that e.g. the type specifications used by
two communicating machines are compatible.
We (Jane Street Capital) will eventually release this library as open
source so stay tuned...
Regards,
Markus
--
Markus Mottl http://www.ocaml.info markus...@gmail.com
Mike Furr
> If you are interested in the research aspect of this, Kathleen Fisher
> did interesting work on parsing ad-hoc data (mostly ascii):
> I remember to have seen something like this for OCaml in the recent
> proceedings of ICFP or POPL but can't remember right now.
I believe you're thinking of PADS/ML (http://www.padsproj.org/) which
was presented at POPL'07 by Fisher et al.
Cheers,
-mike
Richard Jones
>
> I wonder if anyone has thought about or developed a domain-specific
> language in OCaml for handling the processing of binary structures?
>
> I was thinking of writing something to make system calls into the Xen
> hypervisor. The main problem is that the hypervisor takes & returns
> binary structures which have very precise size / alignment
> requirements, bitfields, etc. (and the requirements change with
> versions of the hypervisor). It sounds like the ideal use for a DSL
> to describe the binary structure of each version of each call and
> automatically build accessors.
I had a play with some code to do this. A write-up of my experiment
is here:
http://et.redhat.com/~rjones/hvcalls/