[erlang-questions] Linux device driver communication with Erlang node

Motiejus Jakštys

unread,

Nov 29, 2011, 6:23:43 PM11/29/11

to Erlang

Hello,

I am thinking of a way for Linux kernel driver to communicate with
Erlang node. The purpose is send binary blobs and have some logic:
* The blob has been ack'ed by erlang node
* Some business operation has been completed with the blob (by node)

C node seems to be the perfect match. Is there anything I should know
before starting messing around? Any possible gotchas running Erlang node
in kernel space? Has anybody tried that?

What I am trying to do
======================

I am thinking of implementing NBD stack with Erlang and C. NBD server is
easy (100% userspace), the client is more tricky, since device driver
code has to be kernel space.

NBD-client would consist of two parts:
* nbd.c that implements block device interface and talks to Erlang node
* Erlang node that communicates with another Erlang node nbd-server
(which does the "business" part with the blob -- writes to disk)

RFC
===

For me distinction between "client" and "server" is unclear in the
tutorial: http://www.erlang.org/doc/tutorial/cnode.html Could somebody
explain it from the higher perspective?

I have never mixed C and Erlang before, and would appreciate some light
on C nodes generally. What is a C node from the system point of view?
Does it use sockets, pipes? What OS processes/threads does it spawn?

Many thanks,
Motiejus
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

Sanya Potapov

unread,

Nov 30, 2011, 2:38:48 AM11/30/11

to erlang-q...@erlang.org

I'm not an expert, but AFAIK Linux device driver is essentially a set of callbacks, by which it communicates with the kernel - it's not an arbitrary program that could run in the user space put to the kernel space. Usually it communicates with the user space programs by means of device files in the /dev/ folder and /proc/ fs. The later is usually used for some setting and reading some options. For passing a blob from the user space program to the driver you write it to the device file, like you would do with any regular file. And in the driver code you have a callback for reading that data.

On Nov 30, 3:23 am, Motiejus Jakštys <desired....@gmail.com> wrote:

> Hello,

>

> I am thinking of a way for Linux kernel driver to communicate with

> Erlang node. The purpose is send binary blobs and have some logic:

> * The blob has been ack'ed by erlang node

> * Some business operation has been completed with the blob (by node)

>

> C node seems to be the perfect match. Is there anything I should know

> before starting messing around? Any possible gotchas running Erlang node

> in kernel space? Has anybody tried that?

>

> What I am trying to do

> ======================

>

> I am thinking of implementing NBD stack with Erlang and C. NBD server is

> easy (100% userspace), the client is more tricky, since device driver

> code has to be kernel space.

>

> NBD-client would consist of two parts:

> * nbd.c that implements block device interface and talks to Erlang node

> * Erlang node that communicates with another Erlang node nbd-server

> (which does the "business" part with the blob -- writes to disk)

>

> RFC

> ===

>

> For me distinction between "client" and "server" is unclear in the

> tutorial:http://www.erlang.org/doc/tutorial/cnode.htmlCould somebody

> explain it from the higher perspective?

>

> I have never mixed C and Erlang before, and would appreciate some light

> on C nodes generally. What is a C node from the system point of view?

> Does it use sockets, pipes? What OS processes/threads does it spawn?

>

> Many thanks,

> Motiejus

> _______________________________________________

> erlang-questions mailing list

> erlang-questi...@erlang.orghttp://erlang.org/mailman/listinfo/erlang-questions

Daniel Goertzen

unread,

Dec 2, 2011, 11:21:51 AM12/2/11

to Motiejus Jakštys, Erlang

Sanya is correct.

You would need to write a device driver that implements the block device interface and also provides a conduit to userspace; probably a character device node which Erlang could read and write. After being spoiled by Erlang, writing linux device drivers in C will make you want to rip your hair out. :)

Incidentally, people abuse the existing NBD client driver to write their own userspace block devices. You could easily do the same without having to touch kernel code at all.

Dan.

--
Daniel Goertzen
Senior Software Engineer
--
Network Integrity Systems
We Bring Security To Light™

1937 Tate Blvd. SE
Hickory, North Carolina 28602

Phone: 828.610.4596
Fax: 828.322.5294

Network Integrity Systems’ INTERCEPTOR™ Optical Network Security System is a Smart-PDS™ that ensures superior protection and cost effectiveness of classified networks.

For more information, visit our website at: www.networkintegritysystems.com <http://www.networkintegritysystems.com>
__________________________________________
INTERCEPTOR™ Optical Network Security System is made in the USA for the USA. Although not an export controlled item, because of the role it plays in the assurance of the safety and integrity of National Security Information, Network Integrity Systems (NIS) is committed to compliance with the U.S. Export Administration Act. Accordingly, NIS will not ship INTERCEPTOR products to certain foreign government end users without U.S. government approval and will refuse transactions with individuals or entities that have been denied export privileges.

Motiejus Jakštys

unread,

Dec 2, 2011, 4:31:42 PM12/2/11

to Daniel Goertzen, Erlang

On Fri, Dec 02, 2011 at 10:21:51AM -0600, Daniel Goertzen wrote:
> Sanya is correct.
>
> You would need to write a device driver that implements the block device
> interface and also provides a conduit to userspace; probably a character
> device node which Erlang could read and write. After being spoiled by
> Erlang, writing linux device drivers in C will make you want to rip your
> hair out. :)

Yes, I get it. So it's quite clear that it will not be a C node. It
will be either home-made port driver, or wrapped in Erl_interface. I
will have a look.

> Incidentally, people abuse the existing NBD client driver to write their
> own userspace block devices. You could easily do the same without having
> to touch kernel code at all.

But as far as I see nbd-client.c sets up the socket:
262 void finish_sock(int sock, int nbd, int swap) {
263 if (ioctl(nbd, NBD_SET_SOCK, sock) < 0)

drivers/block/nbd.c fetches the socket:

602 case NBD_SET_SOCK: {

and sends/receives packets to the nbd server:

191 result = kernel_sendmsg(sock, &msg, &iov, 1, size);

195 result = kernel_recvmsg(sock, &msg, &iov, 1, size,
196 msg.msg_flags);

Transferring data from kernel to nbd-client.c would not make sense,
because it would make +2 context switches per buffer while sending data.
How it would work then:
1) userspace calls write(), and sends data to kernel-space
2) kernel driver nbd.c sends data to nbd-client.c (again to userspace)
3) nbd-client.c sends the blob to kernel space to send it to NIC

Now second step is skipped.

Mentioning NBD was misleading here... It would be better to name it
"ErlBD". Since I will implement neither nbd client, nor server. Just the
idea is similar.