"Is item X in our distributed hash ?" where the distributed hash exists as a gigantic hash in the RAMs of a bunch of machines ? If Erlang can answer this, how can I write Erlang programs to answer this question ? How can I load this distributed hash with my contents ? Obviously, each machine can only load a piece of the total contents... Thanks. -eric |
Do you want to answer the question with an absolute yes or no, or is a
probabalistic answer good enough?
If the latter, then you need to do some research into "bloom filters".
If your dht supports deletion then you will probably want a counted
bloom filter, and if the dht is very large or network bandwidth is
constrained you will want to examine compressed bloom filters as well.
Jim
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://www.erlang.org/mailman/listinfo/erlang-questions
And if you're looking for an implementation of a Bloom Filter in Erlang,
you can check out this one:
http://code.google.com/p/bloomerl/
--
:: atomly ::
[ ato...@atomly.com : www.atomly.com ...
[ atomiq records : new york city : +1.917.442.9450 ...
[ e-mail atomly-new...@atomly.com for atomly info and updates ...
I'm sorry... Actually, I meant not a dht, but more like a general distributed memory -- i.e. there may be duplicates in it and multiple actors (on different machines) can insert stuff into this memory. This distributed memory can be configured to be either RAM-only or RAM-and-disk, across all machines. Thanks. -eric --- On Mon, 8/11/08, Jim McCoy <jim....@gmail.com> wrote: |
http://en.wikipedia.org/wiki/Merkle_tree
I believe there is also an Erlang version somewhere on Google Code.
[Eric Ho <drwho1020...@yahoo.com>]
> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://www.erlang.org/mailman/listinfo/erlang-questions
--
:: atomly ::
[ ato...@atomly.com : www.atomly.com ...
[ atomiq records : new york city : +1.917.442.9450 ...
[ e-mail atomly-new...@atomly.com for atomly info and updates ...
> Merkle trees:
>
> http://en.wikipedia.org/wiki/Merkle_tree
>
> I believe there is also an Erlang version somewhere on Google Code.
There is indeed:
http://code.google.com/p/distributerl/
(and a minor improvement/bug-fix release will be going up this week)
-Justin
I'm wondering if there is something equivalent to HyperTable in Erlang. |
Well, what features of HyperTable are you looking for, exactly? There
are a lot of similar projects and I imagine you could even integrate
with HyperTable from Erlang.
Try checking out Kai/Dynomite (Amazon Dynamo clones in Erlang), CouchDB,
bigdata or even Mnesia.
--
:: atomly ::
[ ato...@atomly.com : www.atomly.com ...
[ atomiq records : new york city : +1.917.442.9450 ...
[ e-mail atomly-new...@atomly.com for atomly info and updates ...