development ideas

[jcothran]

Added the following ideas to the http://code.google.com/p/xenia/wiki/VMwareDevelopmentIdeas
wiki page. Sqlite has been a very interesting relational database
experiment, but I think we're running into some concurrency multi-user
issues with the development and number of processes we're trying to
run against it - most others are using Postgresql, so will probably
move back in that direction for a 'recent' database of two weeks data
in Postgres database and then possibly continue shuffling older data
off to julian weekly sqlite files. The trade-off would be more work
on archival queries in possibly querying/stitching together
resultsets, but less maintenance and upkeep on archival database
indexes,etc that are better partitioned off from dynamic changes in
recently acquired data/backfilling for the recent two week interval.

With the second point, CSV as a import/export format should be easier
for folks in general to process and target than XML or custom XML like
ObsKML.

Jeremy

June 5, 2009

* migrate focus back to postgresql from sqlite for latest/recent 2
weeks to help resolve multi-user, database/process contention issues -
may still keep archives as weekly julian data sqlite files
* migrate away from ObsKML as intermediate XML import/export format,
simple CSV files/resultsets are probably easier/simpler to produce and
comprehend