[Caml-list] node.ocaml

137 views
Skip to first unread message

Jeffrey Barber

unread,
Aug 21, 2010, 11:27:52 PM8/21/10
to caml...@inria.fr
I've been working with libevent2 and OCaml for the past couple of weeks to
build node.ocaml. It is far from done, but it is interesting enough to
share.

node.ocaml as of now contains a web server and a terminal server that
provides asynchronous programming to OCaml to enable some of my research.
The first example server is a key value pair server that brings OCaml's
Hashtbl to http and terminal IO:

example code:
http://github.com/mathgladiator/node.ocaml/blob/master/test/kvp.ml

This is the first test program, and it works fairly well in a single
threaded environment. I was inspired by node.js to build an evented io
system, and so I begged the question "how does 'OCaml's FFI' compare with
v8's in node.js". In my virtual machine environment (ubuntu 10.04), I got
the following results doing 10,000 requests 50 at a time:

node.js with the "Hello World" script:
2100 requests/second

node.ocaml with kvp.ml:
5300 requests/second

This seems to me to be a very positive first benchmark considering I haven't
optimized anything yet nor have I hacked caml_copy_string yet.

The code is licensed under BSD and available
http://github.com/mathgladiator/node.ocaml .

Any thoughts/questions are appreciated; thank you for your time.

-J

Jake Donham

unread,
Aug 23, 2010, 5:32:08 PM8/23/10
to Jeffrey Barber, caml...@inria.fr
On Sat, Aug 21, 2010 at 8:27 PM, Jeffrey Barber <je...@mathgladiator.com> wrote:
> example code:
> http://github.com/mathgladiator/node.ocaml/blob/master/test/kvp.ml

Have you considered using Lwt (http://ocsigen.org/lwt) as a layer on
top of raw continuation-passing style? It has a lot of nice functions
for writing asynchronous code, as well as a syntax extension to make
it look more like direct-style code. It provides a select loop
already, but you don't need to use it, or you could integrate
libevent2 with Lwt's event loop.

Jake

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

Jeffrey Barber

unread,
Aug 23, 2010, 6:32:46 PM8/23/10
to Jake Donham, caml...@inria.fr
Hello,

No I haven't, thanks for the link. In looking through the docs, it seems
similiar at some level to what I am doing. I'm trying to get away from the
language of threading (In my code, I use chain and io_pumps for context
storage) since I'm thinking of node.ocaml as an event based io library. This
is probably just a marketing decision, :/, rather than a technical decision.

More context on node.ocaml:

I was recently inspired by unix, memcache, and redis and realized that all
you need as a packet format is a single line, so I'm taking that as the
packet format and am going to try to build some neat things with it for both
my own research/education.

For my research, I want to build a fault tolerant JSON data store where I
can get the performance benefits of Redis/Memcache but also have
transactional transformations over indexed sets. But not only that, I want
it to be easy to build them so it can be extended down the road. I've
noticed that from using Redis, there are a lot of design patterns emerging
that would be better migrated from the application server to a redis-like
server.

For my education, I really want to do something with Paxos that's useful.
I'm not sure what, but I've yet to master it and it sounds fun. I'm tempted
to build a chubby clone for distributed locking, but who knows.

Oh, I may also cause the universe to explode since I may enable OCaml to
call JavaScript via v8... In case it does, I'm sorry now.

Gerd Stolpmann

unread,
Aug 23, 2010, 7:30:21 PM8/23/10
to Jeffrey Barber, caml...@inria.fr
You may also look at ocamlnet
(http://projects.camlcity.org/projects/ocamlnet.html), which provides a
stack of abstractions for event-driven programming:

- The Netsys library defines an object type for sets of file
descriptors one can poll:

http://projects.camlcity.org/projects/dl/ocamlnet-3.0test5/doc/html-main/Netsys_pollset.pollset.html

There is right now only an implementation for poll(), but
no fancy things like what libevent provides (but this is
just lack of time).

- Above that, there is the equeue library with its main
interface Unixqueue:

http://projects.camlcity.org/projects/dl/ocamlnet-3.0test5/doc/html-main/Unixqueue.html

This provides a queue of events, and a mechanism to consume
events.

- Above that, there is the engines API:

http://projects.camlcity.org/projects/dl/ocamlnet-3.0test5/doc/html-main/Uq_engines.html

This is an abstraction allowing easy composition of code. An
engine represents a suspended I/O operation to which one can
attach continuations.

There are even implementations of high-level protocols using these APIs
(most hooking in at the Unixqueue level), and this includes HTTP,
Telnet, FTP (partly), and SunRPC. Companies are using this already in
products, so this is really a mature implementation.

It is interesting that in your code the callbacks are implemented on the
C side. I think this is generally more complicated than doing the same
in Ocaml which is way more a "callback-friendly" language.

Gerd

> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs


--
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
ge...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------

Reply all
Reply to author
Forward
0 new messages