Proposed changes in v0.5

[Max]

Here's the proposed list of changes for version 0.5:

• Use SQLAlchemy to abstract the storage layer to extend beyond SQLite to allow for all compatible RDBMS databases currently supported by SQLAlchemy
• Add column indexes which extend down into the storage layer
• Optimize document queries to use indexed columns if they exist
• Allow user to specify how data is serialized in underlying storage engine
• Currently thinking of offering two options: pickle and jsonpickle

I've recently looked into performance comparisons for jsonpickle, pickle and cPickle. Pickle is about 35% faster than jsonpickle, and cPickle is about 5x faster than jsonpickle. However, the benefit for using jsonpickle is readability if you ever wanted to look at the underlying table data.

That's the list so far, if anyone want's to add something feel free.

- Max

[vada]

Looks great.  I would add that the storage engine layer needs to at least work for one other persistence solution out of the box.  My vote is for DynamoDB on AWS.  The indexes and data can be stored at scale that way.  Thoughts?  I think the easiest way to get up and running would be to go through the boto library:  https://github.com/boto/boto

- Travell

[Max]

Yeah, that's doable. I looked at the boto library and don't see much trouble implementing Underverse on top of it. I'm planning for the main Underverse class to support all the RDBMS that sqlalchemy does then have additional classes (which will implement the same methods as Underverse) for NoSQL databases such as DynamoDB, MongoDB, etc. So basically a separate storage engine/class for each non-relational database or anything that sqlalchemy doesn't support.

Is that kind of what you are thinking? I don't have an account on AWS for Dynamo, so you may have to help in that area for testing.

[vada]

I'd be happy to assist with testing!

[vada]

I have a Amazon account and can help kick the tires.

On Monday, June 25, 2012 10:04:56 AM UTC-4, Max wrote: