That would be sweet. I hope to mess around with this plug in and possibly add features like this. Another thing I want to do is make the ability to have multiple timelines and you can choose which categories show on which timelines.
Relating to your idea though, I was thinking that it'd be nice to be able to attach an XML, XLS, or Google Spreadsheet to a post and for all of that content to appear for it's selected category. This would be nice.