Shards status - EJB3, GA - and kudos

[Adrian R]

Hello Max et al,

I was wondering what the general status of Shards is - it seems like
it has been around quite a while and getting quite some positive
press. From my early tests, so far it has been stable, yet I see no
discussions about when it is going to be made GA.

I do note from the Jira statuses that there are a number of "high"
severity issues outstanding with apparently little movement (for close
to a year, March, May, Aug 2007), are these the hold-up ?

Is there a plan to upgrade Shards to EJB3 (implement
javax.persistence) ? I have just started looking into what might be
involved in doing this myself.

Finally some kudos for you guys - Shards is great from what I have
seen so far. I was dismayed by the early prototype performances I was
seeing out of SessionFactory/EntityManagerFactory initializations.
Looking at the logs however I can understand why they are taking so
long when they are doing the schema/annotation evaluations. Anyway, my
task is exactly what (one of) the Shards reasons-for-being is,
customer partitioning of data, customer A's data needs to be stored
separately from customer B.

My prototypes this week reduced the 5-10 second Hibernate core/
annotations start-ups (per database), down to less than 50 msecs for
the Shard activation (once they have been initialized of course -
which still only took the 5-10 seconds of a regular start-up).

Thanks again, we'll see how the prototyping progresses ...

Adrian

[Max Ross]

Hi Adrian, thanks for getting in touch.

I think it's no secret at this point that progress on Shards is a bit stalled, not due to lack of interest on the part of the developers but rather because of a magical combination of higher priority work (the stuff Google actually pays us to do) and Major Life Events (marriages, births, geographical displacements).  Excuses excuses, I know, but that's how these things go.

We definitely intend to support EJB3, and in fact this is what we're waiting for before making Shards GA.  Hooking the Sharded implementations up to the EntityManagerFactory and the EntityManager won't be too bad, but getting a useful implementation of ShardedQuery will be most of the work.  I've spent a bit of time looking at what will be involved and my feeling is that it will be a bit more subtle than what we did for Criteria/ShardedCriteria.  The reason is that we need to be able to decorate the queries that we dispatch to the individual shards.  For example, if we need to perform an average over the values in a column across all shards, we need to make sure we request not just the avg but also the count from each shard, otherwise we don't have enough information to return the correct aggregate value.  Decorating a Query might be tricky because we either need to write our own parser, which seems like a waste of time considering Hibernate has already done this (and anyways, ugh), we need to find a place to hook into the syntax tree that Hibernate is building, or we need to modify the existing parser to, for example, turn 'avg(col)' into 'count(col), avg(col)'.  Ideally we could find a way to do this without relying on anything below the Hibernate interface layer, but I'm not optimistic.

So anyway, that's where we stand.  I'm glad that Shards is doing well for you so far.  If you'd like to contribute in any way we'd be very pleased to have your help and we can definitely support you as you move forward.

Thanks,
Max