Multi-tenant support?

[Joshua Gough]

Hello,

I saw mentioned during the birthday video (https://www.youtube.com/watch?v=0aVCXXGNELo) that multi-tenant support would be coming. 

We were wondering how far out that would be, and if there's a way to get early access to builds for it when available?

Our team is working on a project using EventStore (https://github.com/openAgile/CommitStream.Web) and we are starting to add in our own form of multi-tenancy.

Thanks!

Josh

[Greg Young]

What form of multi-tenancy do you need (e.g. isolation levels)

> --
> You received this message because you are subscribed to the Google Groups
> "Event Store" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to event-store...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test

[Joshua Gough]

We'd like to be able to host multiple customers within a single EventStore installation, ideally -- but with the ability to easily relocate customer data to dedicated instances if-and-when that becomes necessary for larger customers.

Our approach so far will be to create a tenants stream that we or our main app will POST to with a uuid and name, and description to represent a distinct tenant.

We will then create streams like eventName-<tenantUuid>

And use projections via fromCategory and foreachStream with partition to query for certain "states".

One thing we wanted to do is ensure that a name is unique so we had this:

----

Projection to place TenantAdded events into a tenantName stream:

var callback = function(state, event) {

  linkTo('tenantName-' + event.data.name, event);

}

fromStream('tenants')

  .when({

    'TenantAdded': callback

  });

Projection to let us query for a name being available:

fromCategory('tenantName')

.foreachStream()

.when({     

    "TenantAdded": function() { return "taken"; },

});

Example:

https://localhost:2113/projection/tenantNameAvailability/state?partition=tenantName-VersionOne

Returns:

"taken"

This seems OK mostly, but if we specify a non-existent tenant, the result is just an empty string, which is the same as if we had on accident mangled the partition name like:

https://localhost:2113/projection/tenantNameAvailability/state?partition=tenantsdfdsfdfsdsffdsName-VersionOne

--

Supposing that is sufficient, we then figured we would:

* For calls coming into our service layer at routes like /api/<tenantName>/commits, pull out the tenantName, look up the tenant's internal guid and when we push new events in we'd make sure each event gets added with metadata { tenantId: uuid }, regardless of whether tenantId is a property on the "data" object itself.

* Having that tenantId, we could push to a stream named commits-tenantId

(Currently, we have a single stream for "commits", and then we project out to substreams via linkTo based on pattern-matching within the event itself. But, when we incorporate tenancy, we want to at least ensure that the commit messages are stored in per-tenant streams, hence the "commits-<tenantId>" )

* Having the tenantId in the metadata would allow us to also have a projection that aggregate of all events happening by tenant to give us an idea of how much aggregate load a particular tenant is putting on the system

* Also, even with "commits-" as the category, we could still fromAll on the eventType determine the aggregate load for specific types of events (We discussed this briefly when you last visited VersionOne I think)

---

All that being said...we gave not yet tried this tenancy approach, but it's functionally similar to what we're doing (with an assumed single tenant) for what we call digests (a summary view of all commit events in a logical grouping) and inboxes (an HTTP endpoint to which a single instance of a source control system sends commit message notifications on post-commit hook or webhook) :

We have a root stream called "digests" (concceptually similar to "tenants" above) and linkTo a digest-<digestId> stream with this projection when a new "InboxAdded" event happens associated with a parent digestId:

var callback = function(state, event) {
  linkTo('digestInbox-' + event.data.digestId, event);
}
fromStream('digests')
  .when({
    'InboxAdded': function(state, event) {
      callback(state, event);
    }
  });

--

We then have a state-keeping projection that associates new InboxAdded events with the parent Digests like so:

fromCategory('digestInbox')
.foreachStream()
 .when({
     '$init': function (state, event) {
         return { Inboxes: {} }
     },
     'InboxAdded': function (state, event) {
         state.Inboxes[event.data.inboxId] = event.data;
     }     
 });

We can query that state like this via partition: https://localhost:2113/projection/digestInboxes/state?partition=digestInbox-4827d245-5928-4bf2-ae68-75a9aa5efcc3

Result:

{
Inboxes: 
{
d3762724-4cb9-41aa-b144-9e9066a2529f: 
{
inboxId: "d3762724-4cb9-41aa-b144-9e9066a2529f",
digestId: "4827d245-5928-4bf2-ae68-75a9aa5efcc3",
name: "Github repo"
},
f68c20fe-7cb9-11e4-b116-123b93f75cba: 
{
inboxId: "f68c20fe-7cb9-11e4-b116-123b93f75cba",
digestId: "4827d245-5928-4bf2-ae68-75a9aa5efcc3",
name: "HooBoo repo"
}
}
}

Finally, because we want to keep the internal "inbox id" private from the outside world, we do a mapping between a public facing key that we can retire and remap and the internal permanent ID with a projection as well:

First the linking projection:

//Partitioner projection
var callback = function(state, event) {
  linkTo('inboxAlias-' + event.data.aliasId, event);
}
fromStream('digests')
  .when({
    'InboxAliasAdded': callback,
    'InboxAliasObsoleted': callback
  });

And then the state-keeping projection:

fromCategory('inboxAlias')
.foreachStream()
.when({
    "$any": function(state, ev) {
        return {
            inboxId: ev.data.inboxId
        }
    }
});

We again query via partition like this, giving us the most-recent value of "iboxId". The idea being that to obsolete this mapping, we just post an InboxAliasObsoleted event with { iboxId: null }

https://localhost:2113/projection/inboxAliasMap/state?partition=inboxAlias-dba26c8a-7cbe-11e4-b116-123b93f75cba

Result:

{
inboxId: null
}

The two most recent events on said stream are:

{
  "inboxId": null,
  "aliasId": "dba26c8a-7cbe-11e4-b116-123b93f75cba"
}

---

{
  "inboxId": "f68c20fe-7cb9-11e4-b116-123b93f75cba",
  "aliasId": "dba26c8a-7cbe-11e4-b116-123b93f75cba"
}					

Thanks,

Josh

[Bob Archer]

On Tuesday, December 9, 2014 at 10:32:32 AM UTC-5, Greg Young wrote:

What form of multi-tenancy do you need (e.g. isolation levels)

Sorry for necro posting, but I've recently come to this product and had the same question/issue.

For us, operations wants each tennant to have their own database. This is simple in SQL Server, since the DB is specified in the connection string. It would be nice to be able to have one instance (cluster?) be able to serve multiple data bases. Currently it looks like since you specify the db (path) when you start the instance I would have to have a different instance per tenant. (Perhaps that's doable using containers, but I'm not sure what the operations overhead of that would be).

Certainly we could include a Tenant id in each aggregate id and create per-tennant projections, but once again operations wants physical data separation.

Thanks for any updates on this issue.