Extending Sync Gateway to allow multi-tenancy - a good idea?

[Marcus Roberts]

I love what Couchbase Lite is letting me do with my mobile apps, but when developing at the smaller end of the scale as I am, I miss the ability to run multiple websites and SQL databases on a single server.  Because of its memory requirements, it appears that essentially it's one Couchbase bucket per server.

I've modified the Sync Gateway to give multi-tenancy control.  The idea is that an account identifier is specified in the sync gateway configuration, and then only documents with this account identifier in the document meta-data can be read by the gateway, and all documents added through a gateway are stamped with the account identifier.  Because the sync gateway is light on resource usage, I figure I can then run multiple sync gateways on a server to give me the multi-tenancy I need for serving my apps.   I know this can be done with channels, but I'm also looking to extend this multi tenancy to the sync meta-data such as users, roles, etc.

This is working for me at the moment, but before I carry on and make users, roles etc work in this environment I thought I would ask - is this a really stupid idea?  Are there better ways to do this?

[Jens Alfke]

On Sep 30, 2014, at 12:04 AM, Marcus Roberts <marcus....@gmail.com> wrote:

I love what Couchbase Lite is letting me do with my mobile apps, but when developing at the smaller end of the scale as I am, I miss the ability to run multiple websites and SQL databases on a single server.  Because of its memory requirements, it appears that essentially it's one Couchbase bucket per server.

I am far from being an expert on Couchbase Server, but I believe this is improved in version 3.0 (to be released imminently.) It has much better support for buckets whose contents are larger than their RAM allocation.

Also, you might consider CBGB, which is an experimental Couchbase-compatible server designed for multi-tenancy: it doesn't scale as well to high demand, but it scales extremely well to lots of buckets. It's not an official product, but it's stable enough that we used it for the (now retired) couchbasecloud hosted system.

I've modified the Sync Gateway to give multi-tenancy control.  The idea is that an account identifier is specified in the sync gateway configuration, and then only documents with this account identifier in the document meta-data can be read by the gateway, and all documents added through a gateway are stamped with the account identifier.  

That sounds interesting, but I would imagine that it's difficult to do and requires a lot of changes to the code. Did you make sure to change the views, the change cache, etc.?

Because the sync gateway is light on resource usage, I figure I can then run multiple sync gateways on a server to give me the multi-tenancy I need for serving my apps.

You don't have to run multiple gateways; each one can already support any number of independent databases, each with its own user accounts.

—Jens

[Matt Ingenthron]

On Sep 30, 2014, at 10:25 AM, Jens Alfke <je...@couchbase.com> wrote:

On Sep 30, 2014, at 12:04 AM, Marcus Roberts <marcus....@gmail.com> wrote:

I love what Couchbase Lite is letting me do with my mobile apps, but when developing at the smaller end of the scale as I am, I miss the ability to run multiple websites and SQL databases on a single server.  Because of its memory requirements, it appears that essentially it's one Couchbase bucket per server.

I am far from being an expert on Couchbase Server, but I believe this is improved in version 3.0 (to be released imminently.) It has much better support for buckets whose contents are larger than their RAM allocation.

Indeed, 3.0 (currently in Beta) has more fine grained control of the use of memory and some better management of buckets.  I know that in the future, the goal is to allow a much larger number of buckets, but I know there’s also more to it than the memory management.  The 3.0 release moves in the right direction with more scalability around number of buckets coming in the future.

I don’t know what the team is testing to with 3.0, but the 2.5 recommendation was < 10 buckets.  

[Marcus Roberts]

I am far from being an expert on Couchbase Server, but I believe this is improved in version 3.0 (to be released imminently.) It has much better support for buckets whose contents are larger than their RAM allocation.

Also, you might consider CBGB, which is an experimental Couchbase-compatible server designed for multi-tenancy: it doesn't scale as well to high demand, but it scales extremely well to lots of buckets. It's not an official product, but it's stable enough that we used it for the (now retired) couchbasecloud hosted system.

Both those are interesting to learn about - I'll take a look, thanks

 

You don't have to run multiple gateways; each one can already support any number of independent databases, each with its own user accounts.

That's what I thought when I first read the documentation, but when I set it up and tested it I found the user accounts were the same in each database.  

My sync config on a new server is:

{

   "interface":":4984",

   "adminInterface":":4985",

   "log":["REST"],

   "databases":{

      "sync_gateway":{

         "server":"http://localhost:8091",

         "bucket":"default",

         "sync":`function(doc) {channel(["public"]);}`

      },

      "database2":{

         "server":"http://localhost:8091",

         "bucket":"default",

         "sync":`function(doc) {channel(["public2"]);}`

      }

   }

}  

I only PUT a user to sync_gateway.  I added a second database (database2) to the config file and the user seems to be visible in the second database

$ http GET localhost:4985/database2/_user/marcus

HTTP/1.1 200 OK

Content-Encoding: gzip

Content-Length: 75

Content-Type: text/plain; charset=utf-8

Date: Tue, 30 Sep 2014 18:09:00 GMT

Server: Couchbase Sync Gateway/1.00

{"name":"marcus","admin_channels":["public"],"all_channels":["public"]}

$ http GET localhost:4985/sync_gateway/_user/marcus

HTTP/1.1 200 OK

Content-Encoding: gzip

Content-Length: 75

Content-Type: text/plain; charset=utf-8

Date: Tue, 30 Sep 2014 18:09:41 GMT

Server: Couchbase Sync Gateway/1.00

{"name":"marcus","admin_channels":["public"],"all_channels":["public"]}

I hope I've missed something simple here, as multiple sync gateway "databases" in the same bucket is all I need!

[Jens Alfke]

On Sep 30, 2014, at 11:13 AM, Marcus Roberts <marcus....@gmail.com> wrote:

That's what I thought when I first read the documentation, but when I set it up and tested it I found the user accounts were the same in each database.  

You put both databases in the same bucket — that's wrong. It's going to make them basically the same database under two different names. Instead create a separate bucket for each one.

—Jens

[Marcus Roberts]

That makes sense - it's why I was looking at multi-tenancy of multiple gateways in the same bucket, because of the problems of running multiple buckets on one server.

[Jens Alfke]

On Sep 30, 2014, at 11:23 AM, Marcus Roberts <marcus....@gmail.com> wrote:

That makes sense - it's why I was looking at multi-tenancy of multiple gateways in the same bucket, because of the problems of running multiple buckets on one server.

Yes, but your answer implied that you would need to run multiple instances of the Sync Gateway as well, to host multiple databases, and I was pointing out that you don't need to.

—Jens

[Marcus Roberts]

On Tuesday, 30 September 2014 19:04:41 UTC+1, ingenthr wrote:

Indeed, 3.0 (currently in Beta) has more fine grained control of the use of memory and some better management of buckets.  I know that in the future, the goal is to allow a much larger number of buckets, but I know there’s also more to it than the memory management.  The 3.0 release moves in the right direction with more scalability around number of buckets coming in the future.

I don’t know what the team is testing to with 3.0, but the 2.5 recommendation was < 10 buckets.  

I'm probably not going to have a lot of data per "database" - a couple of hundred megabytes at most.    If I had a server with 8GB of RAM is it OK to run 8 buckets, each with a 1GB RAM allocation say?  I really want to stick to the standard sync gateway if I can, and multiple buckets provide exactly what I need.    I don't want to run 8 server for 8 apps if I can help it.

[Jens Alfke]

On Sep 30, 2014, at 11:28 AM, Marcus Roberts <marcus....@gmail.com> wrote:

I'm probably not going to have a lot of data per "database" - a couple of hundred megabytes at most.    If I had a server with 8GB of RAM is it OK to run 8 buckets, each with a 1GB RAM allocation say?  I really want to stick to the standard sync gateway if I can, and multiple buckets provide exactly what I need.    I don't want to run 8 server for 8 apps if I can help it.

You definitely don't need to allocate more RAM than the size of the bucket. And if your server isn't under heavy load, you should be able to get away with less RAM — you'll just incur some disk I/O when less recently-used documents are accessed.

—Jens

[Marcus Roberts]

On Tuesday, 30 September 2014 19:26:07 UTC+1, Jens Alfke wrote:

That makes sense - it's why I was looking at multi-tenancy of multiple gateways in the same bucket, because of the problems of running multiple buckets on one server.

Yes, but your answer implied that you would need to run multiple instances of the Sync Gateway as well, to host multiple databases, and I was pointing out that you don't need to.

Sorry, my mistake, I meant multiple sync gateway databases.    What I'm really trying to achieve is multiple apps per server, where the primary security is the channels of sync gateway, but the fallback security of an app not being able to see another app's data if I make a mistake (or through malicious action) that comes from the apps being in separate buckets, and because multiple buckets per server seems an issue (at least < 3.0) multi-tenancy in a single bucket.

Marcus

[Marcus Roberts]

You definitely don't need to allocate more RAM than the size of the bucket. And if your server isn't under heavy load, you should be able to get away with less RAM — you'll just incur some disk I/O when less recently-used documents are accessed.

That sounds like the best solution then, I think the initial Couchbase server setup led me to allocate all my RAM to my first bucket, and my brain went off track thinking I needed large amounts of RAM for every bucket.

I'm glad I asked now, you've saved me a lot of Go programming! :) I learnt a lot digging through the code though.

Thanks for the helpful advice

Marcus

[Traun Leyden]

Our demo cluster is running a small handful of Sync Gateway databases, each of which has its own corresponding bucket on a single Couchbase Server cluster.

You can see the config here:

https://github.com/couchbase/sync_gateway/blob/master/examples/democlusterconfig.json

--
You received this message because you are subscribed to the Google Groups "Couchbase Mobile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mobile-couchba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/f2b6568b-b4fc-4bc0-8564-8c4aa9dd7a99%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Marcus Roberts]

On Tuesday, 30 September 2014 19:54:36 UTC+1, Traun Leyden wrote:

Our demo cluster is running a small handful of Sync Gateway databases, each of which has its own corresponding bucket on a single Couchbase Server cluster.

You can see the config here:

https://github.com/couchbase/sync_gateway/blob/master/examples/democlusterconfig.json

It's amazing how with one wrong assumption you can lead yourself so off track!   On creating my first Couchbase server I created a bucket and assigned it all the available RAM.  So my brain got stuck on the idea - one bucket takes all the RAM.

I've gone back to basics, created multiple buckets each with a share of the memory, created multiple databases in sync gateway, one per bucket.   And now I have all the security and separation I need!    Awesome!

(Sometimes I don't see the wood for the trees! ;-) )

Marcus