Setting Up Replication on Existing Database

[Jeremiah Barndt]

Is this recommended/possible? I have an existing database that I would like to add replication to. From what I've read, it's not generally advised to add bundles to existing databases, though I'm not sure I understand why. At the moment, I'm strictly interested in replication and would like to know where this can be done easily enough and if so, how I would go about doing it since there's no option for it unless I'm adding a new database in the interface.

[Kijana Woodard]

The recommendation would be to export/import into a new database with the desired Bundles turned on. 

I'll let others answer why this is necessary.

My opinion is that the complexity of trying to enforce the bundle on a database that is in flight is too great atm.

On Tue, Jun 18, 2013 at 10:09 AM, Jeremiah Barndt <jdba...@gmail.com> wrote:

Is this recommended/possible? I have an existing database that I would like to add replication to. From what I've read, it's not generally advised to add bundles to existing databases, though I'm not sure I understand why. At the moment, I'm strictly interested in replication and would like to know where this can be done easily enough and if so, how I would go about doing it since there's no option for it unless I'm adding a new database in the interface.

--
You received this message because you are subscribed to the Google Groups "ravendb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Jeremiah Barndt]

If I go ahead and create a new database and check the "replication" bundle checkbox in the silverlight interface: what I'm wondering is does the replication server need to be setup before I even add any data to the master, or can I simply create the database with the "replication" bundle checked, add all my data even though replication isn't occurring yet, then actually add the replication server at a later time?

[Paul Hinett]

Yes this will work.

The replication bundle needs to be enabled up front which keeps track
of deleted documents etc, you can actually turn on replication to
another server at a later time then.

> it, send an email to ravendb+u...@googlegroups.com <javascript:>.

> For more options, visit
> https://groups.google.com/groups/opt_out

> <https://groups.google.com/groups/opt_out>.

[Jeremiah Barndt]

That's perfect.

A couple more questions in regards to replication setup if you don't mind:

We think initially we're going to have a two server setup. We're thinking master-slave, but will have a load balancer in front of them both so that read requests get distributed between the two evenly.

Now, if I want to add more slaves, my understanding is this is the process:

1. Setup new server with fresh Raven instance.

2. Create empty database on new instance.

3. Add destination server to replication list on master

4. Replication populates fresh slave database and gets it synced

In regards to the above:

1. Is there a way to more quickly load the data onto the slave server initially rather than letting the master send all the initial data?

2. If alternatively we add a new master instead of a slave later on, and create a fresh database on that new "master" server, can I just immediately tell the fresh master to replicate to the first one even though the new master database is empty? Will this cause any problems, or do I need to make sure the new master has all the synced data before making it a master?

Thanks!

[Kijana Woodard]

Load balancer? The raven client will handle balancing the traffic between the two servers. There is an option to allow reads from secondaries. Make sure that your write scenarios still make sense.

For number 1, restore the new slave from a recent backup and let replication catch it up.

Not sure on number 2, but I'd let it catch up before adding the current master as a replication peer.

--

[Oren Eini (Ayende Rahien)]

Please don't have a load balancer, RavenDB API already can do that for you.

You can do a backup / restore of the slave (make sure to change the db id, mind!).

Yes, you can add a master at any time. You don't need to wait for it to be fully synced

--

[Jeremiah Barndt]

Is load balancing something that happens by default?

Does it matter whether I connect to the master or slave server initially from the client in order to get this capability?

What happens if the master server fails entirely (OS crash/maintenance)? The connecting client, if a session was already open would fail over I'm assuming, but a new client connecting would fail and then not know about the slave since it didn't get a list of servers? Am I supposed to just handle this manually?

To give a little more detail on my setup: I can still get to each server individually of course, but the reason I have a load balancer in front of the two servers is because on both servers I have a Web API running on IIS that interfaces against that server's local Raven instance.

So like this:

API request goes to load balancer then gets distributed to Server 1 or Server 2.

Server 1: Request -> Web API -> RavenDB (Master)

Server 2: Request -> Web API -> RavenDB (Slave)

This is a read-only API as well. No writes occur. Writes occur from an off-site location. I open up a connection directly to the RavenDB master and do my writes/updates as needed.

[Kijana Woodard]

If read only and each server has their own instance, then they can point directly at their local.

Does fail over of the web app on the same server with the master to the secondary make sense? If the master raven server is "down", more than likely that web app is down too.

This doc describes the failover:

http://ravendb.net/docs/2.0/server/scaling-out/replication/client-integration

Normally you would connect to the master and the client would cache the list of salves. If the master was down and you had a brand new client, then yes, they wouldn't know about the secondaries and you'd have to step in. My guess though would be that all energy at that moment would be directed at getting the master back up rather than new web clients.

[Oren Eini (Ayende Rahien)]

For load balancing, you need to configure the client to ReadFromAllServers

You always need to connect to the primary server.

If the master fails, the clients will failover to the secondaries. A new client would also fail over, since it caches the topology locally.

[Jeremiah Barndt]

Was pulled away from this for about a month, just coming back to it now.

My understanding from reading the above posts is that I should always have clients connecting to the primary RavenDB server and not the slave.

However, the reason I have it setup where two different clients (that reside locally on each Raven server) are connecting to their local RavenDB instance is because I figured performance would be better if they both just connected locally. I understand I can't have the client on the slave server doing writes -- that's fine, both of the clients that reside on these servers are read ONLY anyway. I have a separate application not on any of these servers that connects ONLY to the Master RavenDB server in order to do writes/updates to data.

I then have a load balancer in front of the servers so that requests that goto each client (which is a port 80 Web API: see http://ravendb.net/docs/2.5/samples/web-api/createaspnetwebapiproject) get load-balanced evenly to each server. To be clear: I'm not using the external load balancer for Raven, just for the Web API (client), and then each Web API (client) connects to their local instance of Raven. One server happens to be a master, one is a slave, and again, all read-only.

I know this might not be a typical setup, but it causes me to run less servers and now if any one server goes down, I still have Web API and RavenDB running on one of those servers just fine.

Is there still a problem with this setup?

[Kijana Woodard]

IMO, no problem. Especially if your 'in the load balancer' scripts take into account if raven is down, remove the server from the farm. That way you don't ever have cross server traffic.

[Oren Eini (Ayende Rahien)]

No, there is no issue.

[Jeremiah Barndt]

Thanks for the confirmation.

I was also wondering, in regards to open ports and replication. Obviously the Master needs to be able to connect to the Slave to push changes. Is there any reason that a Slave server needs to be able to connect to the Master? I'm locking down some firewall settings and if it isn't necessary, I won't allow Slave -> Master communication. Thanks!

[Kijana Woodard]

Not sure about the firewall settings, but setup your db security so that the slave is read-only as well.

[Oren Eini (Ayende Rahien)]

Yes, the slave _might_ be talking to the master as well. For example, if the slave was down, it will notify the master on startup so the master can resume replication

[jason.s...@corcoran.com]

@Oren is it possible to enable replication on an existing database? Our version is 2.5.

[Oren Eini (Ayende Rahien)]

No, it is not

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.