Axon and multitenancy

[Andrea Mariottini]

I have a multitenant application where each tenant has it's own database and where the application is completely stateless (no sessions).

In this context I setup the Axon infrastructure (eventstore, event bus, command bus, etc...) on each call to the API exposed.

Any idea to optimize this? For example how to have at least a single instance of the infrastructure per tenant?

Thanks

[Allard Buijze]

Hi,

you can use a single architecture for all (or multiple) tenants and use meta data on the events and commands to identify the tenant for which the command/event is meant. 

On the JpaEventStore, if that's the one you use, you can use an EntityManager resolver that provides a different EntityManager for each tenant. 

Cheers,

Allard

--
You received this message because you are subscribed to the Google Groups "Axon Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to axonframewor...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Andrea Mariottini]

Yes I use JpaEventStore and the SimpleEntityManagerProvider and a TransactionManager to provide the tenant entity manager. Is the resolver another thing or you meant "provider"?

But is not clear to me how a single CommandBus or FactoryRepository could be used by concurrent threads on different tenants.

Can you provide an example or at least some hints?

Thanks

 

2013/10/23 Allard Buijze <bui...@gmail.com>

--
You received this message because you are subscribed to a topic in the Google Groups "Axon Framework Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/axonframework/GNXJUOXyalY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to axonframewor...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Andrea Mariottini

[Allard Buijze]

Hi,

I meant provider, you're right.

A CommandBus doesn't have any state, other than the command handlers that have been registered to it. If each tenant has the same command handlers you can simply use a single command bus for all tenants. In the command meta-data, you can attach the tenant ID, so that each handler knows the scope in which the command needs to be executed. The same can be done for events.

When loading an aggregate, you could validate that the loaded aggregate is owned by the tenant that sent the command (using custom code in your command handler). If your tenants may have different command handler logic alltogether, there could be other workarounds

I don't have any code samples ready I could send, but I hope these hints provide you with some guidance.

Cheers,

Allard

[Andrea Mariottini]

My problem is each tenant has a different EntityManager (different DB).

The command bus requires a TransactionManager to get the transaction but the TransactionManager interface doesn't have parameters an implementation could use to choose among different EntityManagers.

The same problem is for JpaEventStore with the EntityManagerProvider. Here I suspect JpaEventStore have a state....

I could add the tenant-id to commands/event but I need to dynamically change the underlying EntityManager in a thread safe and concurrent way.

[Allard Buijze]

Hi Andrea,

It seems that Hiberante has multi-tenancy support. When googling for "spring multi tenancy", I also found some pages about multi tenancy configuration of hibernate. Maybe that's something you can use to simplify it?

To me, this looks like a very generic problem. Not really one that's Axon or CQRS related. Or am I mistaken?

Cheers,

Allard

[Andrea Mariottini]

Yes, I'm already using Hibernate's multi tenancy support, so I can provide the right EntityManager to each tenant.

My problem is how to tell Axon to use the right EntityManager.

When the CommandBus calls TransactionManager.startTransaction() he doesn't provide any parameter, so the TransactionManager can't know the tenant-id

and can't use the tenant's EntityManager to get the transaction.

The JpaEventStore has the same problem with the EntityManagerProvider: getEntityManager() has no way to know the tenant-id.

A possible solution is to have a CommandBus and a JpaEventStore for each tenant and route each request to the API to the tenant's CommandBus.

I wonder if is possible to use a single instance of CommandBus and a single instance of JpaEventStrore for all tenants...

[Allard Buijze]

Hi Andrea,

the only way to currently pass on the information between these components is by using ThreadLocals. You could create an EntityManagerProvider implementation that uses this ThreadLocal value to decide which EntityManager to return. A similar construct should work for the TransactionManager.

In future Axon versions, I am planning to support custom properties on the Unit of Work. That would be a place where you can then attach the tenant-id. That prevents you from needing to know exactly how the Unit of Work is handled in combination with threading.

Cheers,

Allard

[Andrea Mariottini]

Using ThreadLocals is another possible solution in case the process from command handling to denormalization is on a single Thread. (I already use ThreadLocals in combination with Hibernate multi-tenancy support).

Another question: is this safe even with JpaEventStore? If multiple threads access the same instance of JpaEventStore, is safe to provide a different EntityManager to each thread?

Of course this implies some non trivial assumptions on Axon internals…..

Thanks.

Andrea Mariottini

2013/10/28 Allard Buijze <bui...@gmail.com>

[Allard Buijze]

Hi,

I don't see a problem for each thread to use a different EntityManager instance. When command processing is not on a single thread (e.g. DisruptorCommandBus and AsyncCommandBus), you could use a (Concurrent)Map to map the UnitOfWork (CurrentUnitOfWork.get() will always give you the UoW, even when multiple threads are used) to a TenantId. Make sure to clean the mapping when the unit of work is committed.

To help you clean the entries from the map, you can register a UnitOfWorkListener with the UnitOfWork.

About the different EntityManager in the JpaEventStore. Do you have a single EntityManager instance per tenant, or do they all share the same EntityManager? Or do you need a different EntityManager instance for each request?

Cheers,

Allard

[Andrea Mariottini]

I don't use Spring or another framework so is responsibility of my application to get the EntityManager from EntityManagerFactory.

The EntityManagerFactory is a singleton for the application but on each request a new EntityManager is provided, I don't know exactly how Hibernate works internally, but I think a different instance is built each time.

Andrea

2013/10/28 Allard Buijze <bui...@gmail.com>

[Allard Buijze]

Hi,

in that case, you should be fine creating a new EntityManager each time. Make sure to properly close it as well. You probably want to do that in a UnitOfWorkListener.

Cheers,

Allard

[Andrea Mariottini]

Thanks, I'l do some spikes.

Yes I already close the entity manager in a filter of the container.

Andrea

[b.ris...@relay42.com]

Hello,

I find this approach very useful as I was working on something similar (having single architecture for multiple tenants, using Axon 2). Basically, we have tenant aware data source and using thread local we can get the appropriate data connection for the tenant. Now I have another issue: data connection should depend also from the aggregate type. Some events should not be tenant aware and always go in single database, the other ones are tenant specific. Till now, i was into the this approach:
- Having 2 data sources, one for the default db and other one for tenant aware. Also 2 persistance units and 2 entity factories and transactions managers. They are registered in the unit of work. I have entity manager provider used in a JpaEventStore to decide which entity manager to be get. Having done tests and find that this is working fine.

But I'm not sure if this is the best approach for this. I want to remove the 2 entity manager factories etc. and have only the tenant aware one but I'm not sure how and in which moment provide command type (into the tenant aware data source) as part of the decision when getting the database connection?

Or you might have some other suggestion?

Thank you very much in advance,

Boban

[Allard Buijze]

Hi Boban,

unlike Axon 2, Axon 3 has the aggregate type as part of the DomainEventMessage. As a solution in Axon 2, you may want to assign the aggregate type (as String or class) as meta data to the published event. Based on this meta-data, your tenant-aware connection providers can decide which connection to use. Perhaps you can even identify the aggregate type on the Events themselves (either using a static list, or by putting a custom annotation on your event payloads).

Hope this helps.

Cheers,

Allard

For more options, visit https://groups.google.com/d/optout.

[Boban Ristovski]

Hi Allard,

thanks for your comment.The problem that I have with this approach is how to send the aggregate type to the database provider which in my case is simple extension of the AbsractDataSource. I can for sure use a ThreadLocal here. But then will have to push the ThreadLocal on places where I need. 

I was looking is there is some easy way to actually override the connection in the event-sourcing-repository but till now without success. We can make a custom entry store for every event-sourcing-repository but the connection is already created within the unit of work. Am I missing something here or it is not possible actually?

Thanks,

Boban

[Allard Buijze]

Hi Boban,

you can use a ThreadLocal, or just the UnitOfWork (which is effectively also a ThreadLocal).

You can also wrap the ConnectionProvider (the one that looks at the current unit of work) and make it return another instance, right? So even if there is already a Connection known to the UnitOfWork, it would select another one.

Cheers,

Allard