Multiple persistence units?

[Muz]

Hi guys, been playing with DAO Fusion for the last day or so and want
to say it looks great!
Especially the HelloDAO example using smartGWT...very nice use case.

I'm pretty new to JPA so please bear with me. One thing I was
wondering, how hard would it be to be able to use multiple persistence
units for entity management, dynamically?

I'm writing a multi-tenant application, whereby there is one running
web application and multiple database instances (1 per tenant). The db
instances are all identical in structure (same tables etc), but each
one stores different data for each tenant/company. So I have one set
of JPA-annotated entities and one set of DAOs. If the client tier can
pass along what company/tenant a user belongs to (I use spring
security and a separate auth database for this), can DAO Fusion
dynamically route CRUD transactions to that tenant's specific database
instance? Would I override EntityManagerHolder for something like
this?

thanks and great work
Murray

[Vojtech Szocs]

Hi Murray,

if I understand you correctly you want to do some kind of data
partitioning (based on tenant/company) under the hood of your DAO
layer implementation. With DAO Fusion, everything depends on the
implementation of BaseHibernateDataAccessor#getHibernateEntityManager
() since this is the entry point from which Hibernate Sessions are
created. DAO classes in "daofusion-test" use EntityManagerHolder bean
that gets its HibernateEntityManager instance injected via
@PersistenceContext (this is done so that EntityManagerAwareEntityDao
and EntityManagerAwareEnumerationDao can operate on the same entity
manager).

Now you actually need a way of creating or retrieving
HibernateEntityManager based on the given tenant/company ID (which is
somehow mapped to persistence unit name).

Quick and dirty solution: declare HibernateEntityManager's for all
your persistence units like this (assuming you are using Spring's
JPA / transaction support like we do in "daofusion-test"):
@PersistenceContext(unitName="tenant1")
private HibernateEntityManager tenant1em;
...etc

However, Spring's OpenEntityManagerInViewFilter works with a _single_
EntityManagerFactory bean, binding the JPA EntityManager created from
this bean to the current thread for the entire request processing. So
you would have to write your own implementation of
OpenEntityManagerInViewFilter if you want to work with multiple entity
manager factories (something similar to filter we created in Hello DAO
for simple cases). Or, you can use aspects to create appropriate
EntityManager instance based on some thread-local variable which has
been set before when receiving the client request (tenant/company ID)
- for example, using HTTP filter.

The most complex and robust solution is to use Hibernate Shards
(however they don't support configuration via JPA yet as far as I
know):
http://docs.jboss.org/hibernate/stable/shards/reference/en/html_single/

You just started an interesting topic :)

Vojtech

[Misiak]

Hi Murray,

for the simplest scenario, you can simple take OpenEntityManagerInView
filter
implemented in hello-dao example and extend it (by the way, I have
already did
it:-) to support multiple instances of EntityManagerFactory. For each
request
the following sequence will be executed:
1. Decide which EntityManagerFactory to use. The decision is made
according specified request parameter(s)
2. Use chosen EntityManagerFactory to instantiate EntityManager.
3. Store created EntityManager into a thread local variable, from
where the right EntityManager
will be picked up by DAOFusion when executing DAO methods.
4. After the request is processed clean up thread local variable.

The appropriate web.xml configuration should look like this:
...
<filter>
<filter-name>oemivFilter</filter-name>
<filter-
class>com.anasoft.os.daofusion.sample.hellodao.server.OpenEntityManagerInViewFilter</
filter-class>
<init-param>
<param-name>persistenceUnitNames</param-name>
<param-value>tenant1,tenant2,tenant3</param-value>
</init-param>
</filter>
...

The new version of OpenEntityManagerInView is is already committed in
SVN
and available in hello-dao example so you can use it right now if you
like.

Hope it helps:-)If not, ask and we will try to help.

Cheers,

Michal

[Vojtech Szocs]

Hello guys,

Michal did a good job of extending our simple
OpenEntityManagerInViewFilter implementation to support multiple
persistence units. It's now possible to switch between persistence
units (and corresponding entity manager factories) based on request
parameters - see OpenEntityManagerInViewFilter#getEntityManagerFactory
(ServletRequest) method for details. However, our
OpenEntityManagerInViewFilter provides the most simple transaction
support - there's a single transaction for the entire request
processing (one request-scoped transaction). I believe that
declarative transaction management (e.g. Spring's @Transactional
annotation) is far more better to handle method-scoped transactions
where you can mix and match different transaction propagation modes
according to your business logic - for example, a read-only operation
that executes single SELECT query should NOT be in a transaction (or
else you risk performance and unnecessary locking issues). More info
on this topic:
http://opensource.anasoft.com/daofusion-site/technote/transaction-management.html

I think all depends on how you answer the question: what is the scope
of my entity managers? In OpenEntityManagerInViewFilter
implementations, the entity manager lives during the entire request
processing (this is why it is called "entity manager in view"). On the
other hand, entity managers can live shorter - e.g. method-scoped
entity managers on some RPC service.

Vojtech

[Muz]

Wow you guys are fast. Great work Michal, I got it from svn. Instead
of the array I'd probably use a map to store EntityManagerFactories so
they could be looked up by a tenant code.

Just to complete your helloDAO example, how would the smartgwt client
pass in the ServletRequest parameter (e.g. tenant=xyz) so the OEMIVF
filter could look up the right factory? I know this is not your focus
but I'm just envisioning how the end-to-end use case would work. I'll
play with it.

I did look at shards last year but it seemed overly complex. That
being said I was short on time so didn't give it a fair shake.
Hopefully when we dive into the next cold northern winter I'll give it
another go.

thanks and cheers
Murray

On Jun 25, 2:57 am, Vojtech Szocs <vojtech.sz...@gmail.com> wrote:
> Hello guys,
>
> Michal did a good job of extending our simple
> OpenEntityManagerInViewFilter implementation to support multiple
> persistence units. It's now possible to switch between persistence
> units (and corresponding entity manager factories) based on request
> parameters - see OpenEntityManagerInViewFilter#getEntityManagerFactory
> (ServletRequest) method for details. However, our
> OpenEntityManagerInViewFilter provides the most simple transaction
> support - there's a single transaction for the entire request
> processing (one request-scoped transaction). I believe that
> declarative transaction management (e.g. Spring's @Transactional
> annotation) is far more better to handle method-scoped transactions
> where you can mix and match different transaction propagation modes
> according to your business logic - for example, a read-only operation
> that executes single SELECT query should NOT be in a transaction (or
> else you risk performance and unnecessary locking issues). More info

> on this topic:http://opensource.anasoft.com/daofusion-site/technote/transaction-man...

[Vojtech Szocs]

Hi Murray,

> how would the smartgwt client pass in the ServletRequest parameter (e.g. tenant=xyz) so the OEMIVF filter could look up the right factory?

This depends on the chosen client/server communication mechanism and
the way you store/pass tenant ID information. Let's say you want to do
a HTTP request and include the tenant ID as its parameter. In GWT, the
default communication scheme is represented by asynchronous RPC calls:
http://code.google.com/webtoolkit/doc/1.6/DevGuideServerCommunication.html#DevGuideRemoteProcedureCalls

So GWT RPC is the way how Hello DAO SmartGWT client communicates with
the servlet backend.

If your app uses servlet sessions (HttpSession), you can associate
tenant ID with the user session and do the "tenant ID -> EMF instance"
resolution in your OEMIV filter on the backend only, after the RPC
request has been received (no need to adapt the client code). However,
using server-side sessions has (mostly negative) implications on
scalability and overall performance of your server(s). This is
actually quite important if you have a cluster of servers (you usually
end up with sticky sessions or session replication and you can kiss
your horizontal scalability goodbye :P).

If your app stores tenant ID on the client ("rich clients" manage user
state on their own), you need to include this ID into each appropriate
GWT RPC call (HTTP request) so that OEMIV filter can pick it up back
on the server side. I've done a simple project that implements client-
side sessions in GWT -- extending GWT RPC mechanism so that you can
easily specify additional data to be passed along the standard RPC
payload in a transparent way:
http://code.google.com/p/rcx/

The decision is yours..

Vojtech

[Muz]

Thanks Vojtech, makes sense.

So just to complete your HelloDAO example, which I understand uses
asynchronous GWT-RPC, where specifically would one pass in the
ServletRequest parameter to indicate the tenant ID? Would this be in
the GwtRpcDataSource class?

Murray

On Jun 26, 10:09 am, Vojtech Szocs <vojtech.sz...@gmail.com> wrote:
> Hi Murray,
>
> > how would the smartgwt client pass in the ServletRequest parameter (e.g. tenant=xyz) so the OEMIVF filter could look up the right factory?
>
> This depends on the chosen client/server communication mechanism and
> the way you store/pass tenant ID information. Let's say you want to do
> a HTTP request and include the tenant ID as its parameter. In GWT, the

> default communication scheme is represented by asynchronous RPC calls:http://code.google.com/webtoolkit/doc/1.6/DevGuideServerCommunication...

[Vojtech Szocs]

Try putting following code at the beginning of
GwtRpcDataSource#transformRequest(DSRequest) method:

// use simple HTTP request without SmartGWT-specific request encoding
request.setUseSimpleHttp(true);

// add some HTTP parameters
Map<String, String> params = new HashMap<String, String>();
params.put("tenantId", "xyz");
request.setParams(params);

And on the server side, in
OpenEntityManagerInViewFilter#getEntityManagerFactory(ServletRequest)
method:

String tenantId = request.getParameter("tenantId");

The above code ensures that each HTTP request issued by SmartGWT data
source (GwtRpcDataSource) contains parameters you need to transmit to
the server. In other words, this works for basic data source
operations bound to the data grid (fetch, add, update, remove).

Hope this helps.

Vojtech

> ...
>
> read more »