Master-Master replication
Zsombor
We previously used mongodb in a master-master replication setup, in
the following way : two separate machine, with one-one server
instances, and one-one 'application server' sitting on the same
machine, connecting to the local database. We choosed this
configuration, because we need very fast reads and writes - and we
only need eventual consistency - and we know that concurrent updates
cannot happen in our system, because all of our writes are inserts,
houndreds/sec, updates are very rare, once a day at most, when some
configuration is changed, and we have thousands of reads mostly
lookups by key. It worked pretty well for us, with some minor hiccups
here and there.
Unfortunately the new 1.9 mongodb lost this feature, so we are in a
bit trouble. The replica set feature is not quite what we like to be,
because if we deploy a replica set, the clients needs to be connected
to the 'master' server, not only to the local server. The 'slaveOk'
settings only works for the reads, not for the writes. Can you help
us, how to overcome this limitation ?
BR,
Zsombor
Scott Hernandez
more shards. I assume when you started there was no sharding support
and you went with master/master; was that the reason for this choice?
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>
Zsombor
it doesn't provide high availability, which we absolutely needs.
BR,
Zsombor
Scott Hernandez
I can see how you want something like master/slave where the slave can
be master as well; I assume that is what you used in your old setup.
It was not a common configuration and I don't know if it was ever a
supported configuration; you can do something similar by using an
external sync'r like the open source tools provided by wordnik:
https://github.com/wordnik/wordnik-oss
Eliot Horowitz
Zsombor
Yes, we need replication, but it seems that we can't get with the new
mongodb :( That master-master replication were supported,
unfortunately I'm not able to find the wiki page which described it,
only a page, which linked to it :
http://www.mongodb.org/pages/viewpage.action?pageId=16449791&navigatingVersions=true
Using wordnik seems to be a bit band-aid solution to me, and not
clear, that their IncrementalBackupUtil is able to sync two way
without confusing himself totally.
I fear, we have to find an other db which supports AP from the CAP-
theorem, and not only CP as mongo.
BR,
Zsombor
Zsombor
applications sitting in the datacenter W (near the backup servers),
have to connect to the far away primary servers in 'datacenter E' for
querying, and inserting datas, which is much slower than to access
local - backup servers.
BR,
Zsombor
Scott Hernandez
cross datacenter client access from happening. By default, clients
will not access data on secondaries (slaves) unless you use slaveOk as
an option in your code (and even if you want to do that you keep
queries from crossing the datacenter boundaries).
What you describe will not happen in that example configuration.
Maybe it is best if you read up more on sharding and replicasets:
http://www.mongodb.org/display/DOCS/Replica+Sets
http://www.mongodb.org/display/DOCS/Sharding
Zsombor
the 'master', not to the much faster and local 'slave' servers, so the
'slaveOk' settings doesn't help in our case. And sharding doesn't
solves our problem - in fact we shard our data, but on the application
server layer, not in the mongo processes. We need the way to access
and modify the database locally, and therefore we accept the fact,
that the changes propagated to the other servers slowly, only a couple
of seconds later.
So from the 'Replica Sets' description:
* Drivers can automatically detect when a replica set primary changes
and will begin sending writes to the new primary.
* Automated fail-over
- for us, this were be a problem : if we cant access the local
server, it is better the declare, that this app server is out of
order, notify the loadbalancer, etc..., instead of working slowly
* Distribute read load
- we need distributed writes too. Of course this will cause giving
up some of the consistencies guaranteed by the replica set.
So, as there isn't any solution for our requirements, as I believe,
should I open a ticket for this regression ? Do you think it is
feasible to expect that this feature is restored some later release ?
BR,
Zsombor
On ápr. 17, 16:54, Scott Hernandez <scotthernan...@gmail.com> wrote:
> The simple example is not a definitive; there are ways to keep keep
> cross datacenter client access from happening. By default, clients
> will not access data on secondaries (slaves) unless you use slaveOk as
> an option in your code (and even if you want to do that you keep
> queries from crossing the datacenter boundaries).
>
> What you describe will not happen in that example configuration.
>
Scott Hernandez
> But this is the problem, for writing the clients need to connect to
> the 'master', not to the much faster and local 'slave' servers, so the
> 'slaveOk' settings doesn't help in our case. And sharding doesn't
> solves our problem - in fact we shard our data, but on the application
Why not?
> server layer, not in the mongo processes. We need the way to access
> and modify the database locally, and therefore we accept the fact,
What does local mean? Is it the datacenter or the (local) machine?
> that the changes propagated to the other servers slowly, only a couple
> of seconds later.
> So from the 'Replica Sets' description:
> * Drivers can automatically detect when a replica set primary changes
> and will begin sending writes to the new primary.
> * Automated fail-over
> - for us, this were be a problem : if we cant access the local
> server, it is better the declare, that this app server is out of
> order, notify the loadbalancer, etc..., instead of working slowly
Are you saying that writes to the other app-servers/machines are slow
in the same datacenter? Is the latency and bandwidth on the
interconnects in your datacenter so poor that you cannot go over the
wire to write? At that point you have already come in from a
load-balancer to get to the app server, no? Wouldn't that have the
same issue?
> * Distribute read load
> - we need distributed writes too. Of course this will cause giving
> up some of the consistencies guaranteed by the replica set.
That is what sharding is for in the suggested configuration, write
distribution. Replicasets have a single primary (master) which can
write; that is by design.
> So, as there isn't any solution for our requirements, as I believe,
> should I open a ticket for this regression ? Do you think it is
> feasible to expect that this feature is restored some later release ?
You are welcome to file a jira issue for a feature request, but I'm
not sure how high a priority it will get (esp. if nobody votes for
it); it only meets a very small set of needs -- without much better
support for multi-master replication with conflict detection and such
I don't see it as very useful, or on the majority path of features.
http://jira.mongodb.org/
Zsombor
On ápr. 17, 18:40, Scott Hernandez <scotthernan...@gmail.com> wrote:
> > But this is the problem, for writing the clients need to connect to
> > the 'master', not to the much faster and local 'slave' servers, so the
> > 'slaveOk' settings doesn't help in our case. And sharding doesn't
> > solves our problem - in fact we shard our data, but on the application
>
> Why not?
>
> > server layer, not in the mongo processes. We need the way to access
> > and modify the database locally, and therefore we accept the fact,
>
> What does local mean? Is it the datacenter or the (local) machine?
>
master pair :
- server A with one mongo database and one application server
- server B with one mongo database and one application server
Accessing the database which sits on the same hardware is 3-4 ms at
most, both read and write perform pretty fast. Accessing the other
machine is at least 30-40ms, so 10 times slower. And we need very fast
response times ~50 ms in average, and because for one request, we need
4-5 read request, and 1 write, so if we have to connect to the other
server, we certainly miss the deadline for response times. Our service
is in a deep layer of a service stack, we are only trying to
guarantee, that from our servers we are responding timely. Of course
there is an obvious latency accessing our servers, going through the
load balancer, which we can't reduce further.
With the current master-master setup, we tested, that it works and
replicates properly, if one of the servers are stopped or suspended or
somehow gets inaccessible for a couple of minutes, and the other
server quickly respond without problems during the outage.
> > that the changes propagated to the other servers slowly, only a couple
> > of seconds later.
> > So from the 'Replica Sets' description:
> > * Drivers can automatically detect when a replica set primary changes
> > and will begin sending writes to the new primary.
> > * Automated fail-over
> > - for us, this were be a problem : if we cant access the local
> > server, it is better the declare, that this app server is out of
> > order, notify the loadbalancer, etc..., instead of working slowly
>
> Are you saying that writes to the other app-servers/machines are slow
> in the same datacenter? Is the latency and bandwidth on the
> interconnects in your datacenter so poor that you cannot go over the
> wire to write? At that point you have already come in from a
> load-balancer to get to the app server, no? Wouldn't that have the
> same issue?
>
127.0.0.1. I mean slow, that it has a built-in latency, which we
trying to avoid, if possible.
> > * Distribute read load
> > - we need distributed writes too. Of course this will cause giving
> > up some of the consistencies guaranteed by the replica set.
>
> That is what sharding is for in the suggested configuration, write
> distribution. Replicasets have a single primary (master) which can
> write; that is by design.
>
> > So, as there isn't any solution for our requirements, as I believe,
> > should I open a ticket for this regression ? Do you think it is
> > feasible to expect that this feature is restored some later release ?
>
> You are welcome to file a jira issue for a feature request, but I'm
> not sure how high a priority it will get (esp. if nobody votes for
> it); it only meets a very small set of needs -- without much better
> support for multi-master replication with conflict detection and such
> I don't see it as very useful, or on the majority path of features.http://jira.mongodb.org/
>
multi-master replication setup. For us, I think it would be enough if
the 'slave' machines could accept the writes too, apply it locally,
and send it to the master server to propagate to other instances.
The current situation is very unfortunate, because we choosed mongodb
because that master-master replication were supported in that time, so
we have to stay with some pre-1.8 version of mongodb or to migrate to
some other database.
BR,
Zsombor
Scott Hernandez
As I said, you can use an external sync tool. Since you don't want the
automatic failover of replicasets that is not much of a problem.
I would suggest filing a jira feature request if you want more than that.
Zsombor
On ápr. 17, 19:59, Scott Hernandez <scotthernan...@gmail.com> wrote:
Thanks,
Zsombor