Question concerning Bitemporal History

[Nico R.]

Hey guys,
first off compliments to this great project. Even though the
bitemporal History Implementation seems to be just a side issue, I
find the way of implementation and API Design very thoughtfull.

So I was trying out and testing some different scenarios of bitemporal
history and I found some weird finding when trying to compare the end
of a certain interval with TimeUtils.endOfTime().

The Time retrieved by endOfTime() was 1 Milli off the other one.
I found this to be caused by these Declerations in TimeUtils:

public static final long ACTUAL_END_OF_TIME = 38742530400000L; //
14.9.3197
public static final long END_OF_TIME = ACTUAL_END_OF_TIME - 1;

The method "Interval from(DateTime start)" uses ACTUAL_END_OF_TIME,
while "endOfTime()" uses END_OF_Time. I was wondering why this -1
Milli is necessary?

Thanks in advance!

[Igor Mihalik]

Hi Nico,

the answer is quite simple and has to do with intervals. Both validity
and record intervals are defined as:
interval = <start, end) (notice the brackets, "end" is open).
Now image a scenario where you want to get all intervals (for example
information valid at "certain_time"). You'd iterate all intervals and
return only those that are TRUE for condition:
(interval.start<= "certain_time" && "certain_time" < interval.end)

BUT, and this is the catch, imagine "certain_time" to be INFINITY. You
would never get any result (due to open end) so that's why when doing
search of "what is valid/know at INFINITY" you have to use INFINITY-1
value.

Hope this explanation helps a little, let us know if not.

Cheers,
- Igor

[Nico R.]

Ahh yeah thanks,
I was aware of the left open, right closed time interval, my fault was
only to create an interval to save a new bitemp with
TimeUtils.interval(tt,TimeUtils.endOfTime()) where I should have used
TimeUtils.from(tt).
So TimeUtils.endOfTime() is only intended for querying, but not for
persiting a open end interval I guess.

I have another question concerning the possible boilerplate code in
the bitemporal classes:
It seems to me that the Bitemporal Class extending
BitemporalWrapper<T> only has code in it, that could be handled
generically.
But at a first try of mine I end up with some ClassCastExeptions and
JPA Problems.
Same thing with the "Anchors", the holder of the bitemp Trace.

So I wanted to know if you have maybe already tried to reduce the code
in the Anchors and Bitemp Classes and what barriers you ran into, so I
don't have to try and fail with the same ideas ;o)

[Igor Mihalik]

The bitemporal code has origins in the work of ervacon and with their
permission we were able to include the source and enhance it to use
JPA/Hibernate annotations instead of xml mapping files. You can find
the original code here: https://svn.ervacon.com/public/projects/bitemporal/

Regarding your question, what kind of simplifications do you have in
mind? It would definitely be a positive step forwards to simplify the
implementation and get rid of any boilerplate code. If you have any
idea feel free to create your fork of the project, currently it's also
hosted on https://github.com/igm/daofusion and send pull request if
you come with something, we'll be happy include it.

[Nico R.]

I just mean the whole "value" Attribute declaration and the copyWith()
stuff in the specific BitemporalWrapper and the "hists" Attribute
declaration and getHists() stuff in the "Anchor".
I found a way now to generalize this stuff and put it in 2 abstract
superclasses "AbstractBitemporalModelWrapper" and "AbstractAnchor" so
the specific implementaions only have to contain one codeline. I will
happily share these when I find the time.

But, as always there is another question ahead, and a important one I
think it is:

As my last step of testing DAO Fusion history I did some performance
anaylsis and found it shocking that the whole bitemporal trace is
read, as soon as one get() order gets to the holder of the trace, the
Anchor.
Okay, you can limit the amount of read data by putting the value
Attribute in the BitemporalWrapper on Lazy fetch, but still then there
are 2 Intervals read for every Bitemp.
Imagening a szenario where timelines of objects have grown to hundreds
of Validities and also hundreds of Revisions, this can be a killer to
performance and maybe even heap size.

Having some experience with a custom made history implementation, I
know that they reversed their relations between Anchor and Bitemp, so
that the Bitemp knows his Anchor, but the Anchor has no List of
Bitemps, to exactly avoid the mentioned szenario. This of course has
other pitfalls. You basically loose all JPA Entity Cache advantages
and you manage your historic gets directly on the DB with Select
queries.

I found your idea of working directly on a List of persited, cached
Bitemps nice and easy to understand, but if the cost of it is to
always have to load the entire history of an Object, I don't believe
that it can be used in real applications.

But lets see first what you think and if I maybe missed some important
point, e.g. smart loading the history trace via some spectacular
Hibernate Annotation, before dragging on too long on the topic.

On Mar 7, 5:46 pm, Igor Mihalik <igor.miha...@gmail.com> wrote:
> The bitemporal code has origins in the work of ervacon and with their
> permission we were able to include the source and enhance it to use
> JPA/Hibernate annotations instead of xml mapping files. You can find
> the original code here:https://svn.ervacon.com/public/projects/bitemporal/
>
> Regarding your question, what kind of simplifications do you have in
> mind? It would definitely be a positive step forwards to simplify the
> implementation and get rid of any boilerplate code. If you have any
> idea feel free to create your fork of the project, currently it's also

> hosted onhttps://github.com/igm/daofusionand send pull request if

[Nico R.]

Hmm long time no answer, maybe I was a little harsh in my
complaints? ;o)
But I'm not here to complain, but to improve when it's nessesary. So I
checked out the dao fusion project from googlecode and put some effort
into reducing the described boilerplate and switching the selects of
bitemps from object based (asking the trace list) to DB based.
I'm not completely done yet, but already finished the necessary
structural changes and the switch of the get() methods.
So if you like I can contribute my first results and you can have a
review of it. I used the SVN version of your repo, since I didn't like
Git too much from experience.
So what's the best way to contribute my changes via SVN?

On Mar 7, 5:46 pm, Igor Mihalik <igor.miha...@gmail.com> wrote:
> The bitemporal code has origins in the work of ervacon and with their
> permission we were able to include the source and enhance it to use
> JPA/Hibernate annotations instead of xml mapping files. You can find
> the original code here:https://svn.ervacon.com/public/projects/bitemporal/
>
> Regarding your question, what kind of simplifications do you have in
> mind? It would definitely be a positive step forwards to simplify the
> implementation and get rid of any boilerplate code. If you have any
> idea feel free to create your fork of the project, currently it's also

> hosted onhttps://github.com/igm/daofusionand send pull request if

[Igor Mihálik]

Hi Nico,

no problem, sorry for delayed response I was a little busy with other stuff last week. If git is a problem to use I advice you to create a patch and attach to either this thread or preferably create an issue ( http://code.google.com/p/daofusion/issues/entry ) and attach it over there. We'll definitely look at it and include it in the trunk. Basically the only requirement that has to be met is that the code does not depend on specific database and all the current unit tests pass to prevent regression.

Cheers,
- Igor

--
You received this message because you are subscribed to the Google Groups "DAO Fusion" group.
To post to this group, send email to dao-f...@googlegroups.com.
To unsubscribe from this group, send email to dao-fusion+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/dao-fusion?hl=en.

--
igor.m...@gmail.com

[Nico R.]

Hi Igor,
So changing all the historic get statements worked like a charm,
specially when utilising the Persistent entity criteria API DAO
provides.
But when it comes to the set() method of adding to the trace and
manipulating the history I get stuck on Hibernates stupid lazy loading
character.
I just can't seem to get it to work to just add a new Bitemp to the
trace, without loading the whole List of Bitemps in the trace. I know
that I have to manipulate the other bitemps anyways in order to end
them or create fragments, but I would do that in a different manner
afterwards.
I have extensively googled this topic and found various opposing
opinions of how it is possible to add an item to a lazy loaded
Collection without initialising it. I tried all possible options but
never with the wished result.
As I looked into the PersitentBag and PersitentList Classes now, I
seem to realize that the code only allows inverse Relation Ends to add
items to, without initializing the Collection.
Here's the code:

/**
* @see java.util.List#add(Object)
*/
public boolean add(Object object) {
if ( !isOperationQueueEnabled() ) {
write();
return list.add(object);
}
else {
queueOperation( new SimpleAdd(object) );
return true;
}
}

/**
* Is this collection in a state that would allow us to
* "queue" operations?
*/
protected boolean isOperationQueueEnabled() {
return !initialized &&
isConnectedToSession() &&
isInverseCollection();
}

Now since your project deals with lots of Hibernate issues, I wondered
if you can help me with this with some expert knowledge and explain if
I'm right that lazy adding is only allowed for inversed relations? ;o)
The only solution I found possible for now is to persist the new
Bitemp by itself and then perform a dirty native sql update to set the
Id of the anchor afterwards. I hate to think that this is the only
possible solution to this common problem....

P.S.: The link on the start of the google code page http://code.google.com/p/daofusion/
to the official Project Site is broken. It directs to
http://opensource.anasoft.sk/daofusion-site/ where it should direct to
http://opensource.anasoft.com/daofusion-site. I always run into that
annoyance since I bookmarked the google page :oD

On Mar 14, 5:24 pm, Igor Mihálik <igor.miha...@gmail.com> wrote:
> Hi Nico,
>
> no problem, sorry for delayed response I was a little busy with other stuff
> last week. If git is a problem to use I advice you to create a patch and

> attach to either this thread or preferably create an issue (http://code.google.com/p/daofusion/issues/entry) and attach it over there.

> > > hosted onhttps://github.com/igm/daofusionandsend pull request if

> igor.miha...@gmail.com

[Igor Mihalik]

Hi Nico,

I think that you got a little further than we did. We were aware of
these disadvantages of Hibernate collection loading but we did not get
into the optimization phase. We had couple of reasons for that:
1. we were developing an application that relied on bitemporal and we
did not encounter tremendous performance issues. I'm not saying that
there are not any, but we tried to avoid premature optimization. So I
suppose you experience performance issues in your application, what
are they? Do your entities keep large history?
2. the plan of entire daofusion project is to move away from hibernate
specifics towards JPA2.0 support and the more Hibernate specifics we
would use the more difficult it would be to achieve the goal

If performance is of high importance to you then I would suggest the
same as you did and use native sql updates. That would be just to use
the concepts and principles of bitemporal. There are even databases
that support bitemporal data (http://en.wikipedia.org/wiki/
Temporal_database) like Oracle, PostgreSQL,... If performance is what
you really need I'd go that direction.

-Igor

> P.S.: The link on the start of the google code pagehttp://code.google.com/p/daofusion/
> to the official Project Site is broken. It directs tohttp://opensource.anasoft.sk/daofusion-site/where it should direct tohttp://opensource.anasoft.com/daofusion-site. I always run into that

> > > > hosted onhttps://github.com/igm/daofusionandsendpull request if