The idea of having all of these be persistent objects is very
appealing, but I'm not sure the ZODB is ideal for what I'm doing. Can
people with more experience working with data sets like this give me
some advice on what they've had luck using?
My own experience with the ZODB for large number of small objects, lot
of metadata and indexes, frequent updates, and querying the whole thing,
has been at least very disappointing.
RDBMS are possibly not the perfect solution for each and any kind of
problem datastructures, but unless you have to deal with complex
heterogenous trees or graphs, they sure are a safe bet for most data
storage, retrieval and analysis jobs.
Also, I don't have ant experience with Google's AppEngine's datastore
nor Apache CouchDB, but given the description of your project they might
be appropriate solutions.
My 2 cents.