Clustering SG & CBS

[Daniel McGreal]

Hi Couchbasers,

Suppose I had a global deployment with multiple instances of Sync Gateway working with a cluster of CB Server nodes. I also have similar setups around the world and I synchronise these clusters with XDCR.

Is there any special concerns in clustered scenarios? Any chance of data corruption using XDCR? I'm thinking of a scenario where the same document is edited in cluster A and B simultaneously, the replication will overwrite SG's meta data or..?

Best, Dan.

[Jens Alfke]

> On Apr 29, 2015, at 11:59 PM, Daniel McGreal <d.j.m...@gmail.com> wrote:
>
> Is there any special concerns in clustered scenarios? Any chance of data corruption using XDCR? I'm thinking of a scenario where the same document is edited in cluster A and B simultaneously, the replication will overwrite SG's meta data or..?

You can't use XDCR with any bucket owned by SG. XDCR uses a more simplistic replication algorithm that has no notion of revision ordering or conflicts; it just overwrites the destination document.

The best approach is to replicate directly between Sync Gateways. We have an experimental SG-to-SG replicator that Traun wrote, which may be merged into SG itself, or you can use a CouchDB instance to do the replication since the protocol is compatible.

--Jens

[Daniel McGreal]

Thanks Jens,

> CouchDB instance to do the replication since the protocol is compatible.

Could you elaborate a little on how this would be achieved (links to documentation/terminology to search for appreciated also).

> We have an experimental SG-to-SG replicator that Traun wrote

Is this publicly accessible?

Thanks again, Dan.

> --
> You received this message because you are subscribed to a topic in the Google Groups "Couchbase Mobile" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/mobile-couchbase/aFjg0NA6hmE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to mobile-couchba...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/E7297B0E-A2D4-4B2C-ACBA-5EFB8ECDF652%40couchbase.com.
> For more options, visit https://groups.google.com/d/optout.

[Traun Leyden]

On Thu, Apr 30, 2015 at 8:59 AM, Daniel McGreal <d.j.m...@gmail.com> wrote:

Thanks Jens,
> CouchDB instance to do the replication since the protocol is compatible.
Could you elaborate a little on how this would be achieved (links to documentation/terminology to search for appreciated also).
> We have an experimental SG-to-SG replicator that Traun wrote
Is this publicly accessible?

Yeah it's here:

https://github.com/couchbaselabs/sg-replicate

If you run into issues, find me on our gitter channel.

 

Thanks again, Dan.

> On 30 Apr 2015, at 16:55, Jens Alfke <je...@couchbase.com> wrote:
>
>
>> On Apr 29, 2015, at 11:59 PM, Daniel McGreal <d.j.m...@gmail.com> wrote:
>>
>> Is there any special concerns in clustered scenarios? Any chance of data corruption using XDCR? I'm thinking of a scenario where the same document is edited in cluster A and B simultaneously, the replication will overwrite SG's meta data or..?
>
> You can't use XDCR with any bucket owned by SG. XDCR uses a more simplistic replication algorithm that has no notion of revision ordering or conflicts; it just overwrites the destination document.
>
> The best approach is to replicate directly between Sync Gateways. We have an experimental SG-to-SG replicator that Traun wrote, which may be merged into SG itself, or you can use a CouchDB instance to do the replication since the protocol is compatible.
>
> --Jens
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Couchbase Mobile" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/mobile-couchbase/aFjg0NA6hmE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to mobile-couchba...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/E7297B0E-A2D4-4B2C-ACBA-5EFB8ECDF652%40couchbase.com.
> For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Couchbase Mobile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mobile-couchba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/D3633B4B-AE98-4D6C-A956-DCE4F0E535CD%40gmail.com.

[Daniel McGreal]

Thanks.

Otherwise, there should be no problems with internal Couchbase clustering without XDCR?

To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/CACSSHCHdi8a-A9erz3LMoeTMiBLSLyCrcRMO8tONjYVGQ7KBgw%40mail.gmail.com.

[Jens Alfke]

On Apr 30, 2015, at 8:59 AM, Daniel McGreal <d.j.m...@gmail.com> wrote:

Could you elaborate a little on how this would be achieved (links to documentation/terminology to search for appreciated also).

CouchDB is at http://couchdb.apache.org. It's the architectural ancestor of CBL and SG — they're implemented differently and the feature sets have diverged, but the core of the document model and replication protocol come from CouchDB, and all three will replicate with each other.

The CouchDB replicator can be pointed to two external databases and will replicate from one to the other. So you could for example POST to /_replicate with a JSON body like

{"source": "http://europe.example.com/db", "target": "http://asia.example.com/db", "continuous":true}

and it would replicate from the source SG database to the target. (Then you'd create a second replication going the other way, probably.)

For deployment you'd probably want to use persistent replications, which are described in the CouchDB docs. The setup is slightly different.

—Jens

[Daniel McGreal]

Super, thanks

--
You received this message because you are subscribed to a topic in the Google Groups "Couchbase Mobile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mobile-couchbase/aFjg0NA6hmE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mobile-couchba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/A4C6DB53-98D4-41DB-8EE6-461CB1F9AACF%40couchbase.com.