When a client inserts a key/value, I believe it is up to the caller to determine how many servers it wants to write to.
If you have 3 servers, and you only write to a single server, voldemort will have to eventually write to the rest of the servers.
During a read/get for a key, if you don't check enough servers, the client may think they key doesn't exist when in fact it does it just hasn't propogated just yet.
So my question is (please correct me if my assumptions are not correct), where does voldemort store this list of keys that it has to propogate and/or keys it has to update their version (i.e. the value for that key)?
Since there is no master server, I'm guessing each node will have this list?
I want to look this up in the source code so if you can point me in the right direction with an explanation that would be great!
I would recommend you to take a look to http://www.project-voldemort.com/configuration.php mainly to the "store configuration" section. There you'll see how to configure the replication-factor and the amount of read/writes.
All the read/writes are done simultaneously, and the destination server is defined by the hash of the key you're storing, based in the routing configuration of your cluster. If you use replication-factor, your write either works or doesn't work if it can put the value in the specified amount of servers. If you use hinted-hand-off, then the replication is stored in a middle store (slop) in case the replication doesn't work and is asynchronously updated.
As you say, there is no master, and every node has a list of all the available partitions. That's the information needed to find which is the correct node to write your info (based in the key's hash).
I hope my explanations where clear enough. Anyway I really recommend you to check the project's wiki, almost all the basic information is there.
On Fri, Apr 6, 2012 at 5:25 AM, S Ahmed <sahmed1...@gmail.com> wrote: > When a client inserts a key/value, I believe it is up to the caller to > determine how many servers it wants to write to.
> If you have 3 servers, and you only write to a single server, voldemort > will have to eventually write to the rest of the servers.
> During a read/get for a key, if you don't check enough servers, the client > may think they key doesn't exist when in fact it does it just hasn't > propogated just yet.
> So my question is (please correct me if my assumptions are not correct), > where does voldemort store this list of keys that it has to propogate > and/or keys it has to update their version (i.e. the value for that key)?
> Since there is no master server, I'm guessing each node will have this > list?
> I want to look this up in the source code so if you can point me in the > right direction with an explanation that would be great!
> -- > You received this message because you are subscribed to the Google Groups > "project-voldemort" group. > To post to this group, send email to project-voldemort@googlegroups.com. > To unsubscribe from this group, send email to > project-voldemort+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/project-voldemort?hl=en.
On Fri, Apr 6, 2012 at 9:01 AM, Carlos Tasada <ctas...@gmail.com> wrote: > Hi,
> I would recommend you to take a look > to http://www.project-voldemort.com/configuration.php mainly to the "store > configuration" section. There you'll see how to configure the > replication-factor and the amount of read/writes.
> All the read/writes are done simultaneously, and the destination server is > defined by the hash of the key you're storing, based in the routing > configuration of your cluster. If you use replication-factor, your write > either works or doesn't work if it can put the value in the specified amount > of servers. If you use hinted-hand-off, then the replication is stored in a > middle store (slop) in case the replication doesn't work and is > asynchronously updated.
> As you say, there is no master, and every node has a list of all the > available partitions. That's the information needed to find which is the > correct node to write your info (based in the key's hash).
> I hope my explanations where clear enough. Anyway I really recommend you to > check the project's wiki, almost all the basic information is there.
> Regards, > Carlos.
> On Fri, Apr 6, 2012 at 5:25 AM, S Ahmed <sahmed1...@gmail.com> wrote:
>> When a client inserts a key/value, I believe it is up to the caller to >> determine how many servers it wants to write to.
>> If you have 3 servers, and you only write to a single server, voldemort >> will have to eventually write to the rest of the servers.
>> During a read/get for a key, if you don't check enough servers, the client >> may think they key doesn't exist when in fact it does it just hasn't >> propogated just yet.
>> So my question is (please correct me if my assumptions are not correct), >> where does voldemort store this list of keys that it has to propogate and/or >> keys it has to update their version (i.e. the value for that key)?
>> Since there is no master server, I'm guessing each node will have this >> list?
>> I want to look this up in the source code so if you can point me in the >> right direction with an explanation that would be great!
>> -- >> You received this message because you are subscribed to the Google Groups >> "project-voldemort" group. >> To post to this group, send email to project-voldemort@googlegroups.com. >> To unsubscribe from this group, send email to >> project-voldemort+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/project-voldemort?hl=en.
> -- > You received this message because you are subscribed to the Google Groups > "project-voldemort" group. > To post to this group, send email to project-voldemort@googlegroups.com. > To unsubscribe from this group, send email to > project-voldemort+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/project-voldemort?hl=en.
There is an easy way to understand this. Lets say the replication
factor = N (i.e. each key would be present on N nodes : 1 pseudo
master and N-1 replicas).
Lets ignore required reads/writes for now. When you're doing a put -
the client will write to all N nodes itself (client routed strategy).
If the write succeeds on N nodes, then we're done. Otherwise, a slop
would be registered for each of the failed nodes. This slop is
registered on some other Voldemort server which will written
eventually to the correct node by the Slop-pusher service. So, hinted-
handoff is used only in case of write failures (not for replication
per say).
required-writes / reads is a way of maintaining consistency - to
indicate whether the user operation was successful or not. If the
individual gets/puts did not succeed on the required # nodes, the
whole operation will fail as far as the end user is concerned (it is
independent of hinted-handoff). The purpose is that you can tune this
factor to be strongly / leniently consistent : depending on what your
app can tolerate.
On Apr 10, 5:12 am, Maarten Koopmans <maarten.koopm...@gmail.com>
wrote:
> If I understood correctly hinted handoff is used for required-writes,
> not for replication?
> Best, Maarten
> On Fri, Apr 6, 2012 at 9:01 AM, Carlos Tasada <ctas...@gmail.com> wrote:
> > Hi,
> > I would recommend you to take a look
> > to http://www.project-voldemort.com/configuration.phpmainly to the "store
> > configuration" section. There you'll see how to configure the
> > replication-factor and the amount of read/writes.
> > All the read/writes are done simultaneously, and the destination server is
> > defined by the hash of the key you're storing, based in the routing
> > configuration of your cluster. If you use replication-factor, your write
> > either works or doesn't work if it can put the value in the specified amount
> > of servers. If you use hinted-hand-off, then the replication is stored in a
> > middle store (slop) in case the replication doesn't work and is
> > asynchronously updated.
> > As you say, there is no master, and every node has a list of all the
> > available partitions. That's the information needed to find which is the
> > correct node to write your info (based in the key's hash).
> > I hope my explanations where clear enough. Anyway I really recommend you to
> > check the project's wiki, almost all the basic information is there.
> > Regards,
> > Carlos.
> > On Fri, Apr 6, 2012 at 5:25 AM, S Ahmed <sahmed1...@gmail.com> wrote:
> >> When a client inserts a key/value, I believe it is up to the caller to
> >> determine how many servers it wants to write to.
> >> If you have 3 servers, and you only write to a single server, voldemort
> >> will have to eventually write to the rest of the servers.
> >> During a read/get for a key, if you don't check enough servers, the client
> >> may think they key doesn't exist when in fact it does it just hasn't
> >> propogated just yet.
> >> So my question is (please correct me if my assumptions are not correct),
> >> where does voldemort store this list of keys that it has to propogate and/or
> >> keys it has to update their version (i.e. the value for that key)?
> >> Since there is no master server, I'm guessing each node will have this
> >> list?
> >> I want to look this up in the source code so if you can point me in the
> >> right direction with an explanation that would be great!
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "project-voldemort" group.
> >> To post to this group, send email to project-voldemort@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> project-voldemort+unsubscribe@googlegroups.com.
> >> For more options, visit this group at
> >>http://groups.google.com/group/project-voldemort?hl=en.
> > --
> > You received this message because you are subscribed to the Google Groups
> > "project-voldemort" group.
> > To post to this group, send email to project-voldemort@googlegroups.com.
> > To unsubscribe from this group, send email to
> > project-voldemort+unsubscribe@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/project-voldemort?hl=en.
