I just wanted to say thank you in advance for tackling Annotation/Aggregation. IIRC, Anssi Kääriäinen was working on a large ORM refactor which put his improvements to aggregation on hold. It would be worth checking out if he has moved forward, and I'm sure he will have more than a few pointers to give you in this subject.
Off the top of my head, two tickets worth checking out are #11305 and #14030, which are somewhat related and much needed by the community (me included), based on what I saw in the comments. I've even helped bring the patch to #11305 up to date twice, but it hasn't been merged pending said refactoring and/or a solution to #14030.
Good luck and happy coding!
Cheers,
AT