A case for distributed ETS

[Maxim Kharchenko]

Erlang is certainly one of the the best languages to write distributed applications, particularly because Erlang simplifies distributed programming by making many aspects transparent for a programmer. Though, sometimes, developers have to tackle with some basic mechanisms which do not support distributed access at all. ETS could be a good example of such mechanism in Erlang.

Right now every ETS table is local to the node where it has been created. We are going to change this approach in Erlang on Xen, and offer distributed ETS tables which will be capable of using memory on other nodes in the cluster and even grow elastically by starting new nodes when available memory is running out.

Such approach promises numerous advantages like better granularity of nodes, support of a really huge tables,  running distributed queries on sharded ETS tables transparently, avoiding external Memcached-like servers, etc.

We are on very early stage of implementation of this feature. Now we test different data structures and approaches and try to predict their shortcomings. Indeed, we would like to know the opinion of Erlang community about this feature, and welcome of ideas about possible use-cases for distributed ETS.