Replicated Entities

[Danny Antonetti]

Max and I have been discussing our options with respect to Replicated Entities.

Currently Shards makes the assumption that every entity will reside on exactly one shard.
This assumption does not work well for data that is small and/or slow changing data. 
Any data that needs to be persisted on every shard is problematic for Shards.

For instance Country Codes would probably typically be wanted on all shards.

For these types of Entities, the data may need to be replicated across all shards.

We believe we have come up with a solution, but we wanted ask for comments or suggestions.

There are two aspects to our solution

1) We must prevent cascades from non-replicated entities to replicated entities.
This will probably be implemented in a similar way to the CrossShardRelationshipDetectingInterceptor, or detect illegal cascades at startup.


2) For now Replicated Entities will be persisted to a designated Shard, we will default to the same shard that is used to generate the IDs.
And the user should setup database replication from that shard to all other shards, for the tables corresponding to the replicated entities.
Because of this don't need to worry about conflicts because only one shard gets the updates.
The other option was to perform the same updates to every shard, and suggest XA transactions.

With this solution, the slaves will have some level of staleness, but for this type of data staleness is probably not as problematic.
Foreign Keys from non-replicated to replicated tables will be troublesome.
This is just a first step, and we may change it in a later release.

Max did I miss anything important?

We would appreciate your thoughts or suggestions, or pitfalls with this solution.


Danny

[Tomislav Nad]

Why not go with this option?

[Danny Antonetti]

> The other option was to perform the same updates to every shard, and suggest
> XA transactions.

Why not go with this option?

I think that it should be configurable, and we can implement both solutions.

The main reason that we were going to do this first was that it was less complicated and faster to implement, and we wanted a solution for GA.
In a later release we can implement a solution using XA-transactions, but we thought that this one would take more time and need more testing.


Danny

[Tomislav Nad]

It is simpler to implement, but it isn't really replication support in
Shards, rather we would just be adding support for data that is
replicated by lower layers. As long as we don't try to claim we have
real replication support, I'm ok with this.

[Max Ross]

Nice write-up Danny.  I think the important thing is that we have a reasonable solution that we can guide devlopers towards.  If at first that solution mostly involves configuring the shards themselves to replicate certain tables, I'm fine with that.  Tomislav's point is important though - we're not adding real support for replication.  We'll make this clear in the docs.

Danny,  I think detecting illegal cascades at startup is preferable to using an interceptor.  The data we need to do this at startup is available so why wait until some random time after the app has started to let users know they've done something wrong?

[Emmanuel Bernard]

I don't think it is significantly harder to implement besides running
changes on all shards. The fact that the transaction should be XA
makes no difference. Hibernate is unaware of the (distributed) nature
of the transaction, so should Hibernate Shards.

[Danny Antonetti]

On Sun, Sep 21, 2008 at 10:34 AM, Max Ross <max....@gmail.com> wrote:

Nice write-up Danny.  I think the important thing is that we have a reasonable solution that we can guide devlopers towards.  If at first that solution mostly involves configuring the shards themselves to replicate certain tables, I'm fine with that.  Tomislav's point is important though - we're not adding real support for replication.  We'll make this clear in the docs.

Danny,  I think detecting illegal cascades at startup is preferable to using an interceptor.  The data we need to do this at startup is available so why wait until some random time after the app has started to let users know they've done something wrong?

Yes, as I was looking at it, I realized that it could be done at startup.
 

[Danny Antonetti]

OK I will look into it.

My concern, is when we have a configuration which includes cascades from replicated entities to other replicated entities.

Then as we save on the shards, the second session factories will throw an exception, because the object and all of it's cascades will be associated with the first shard's session factory.

To implement the XA transaction solution it seems like I would need to create a deep copy of the object for each shard.
I did not see another way to ensure thread safety for the ParallelShardAccessStrategy.


Danny