Hi,
The last week I have devoted to optimising the b-trees in AlchemyDB in
terms of memory usage.
The Integer-index was heavily optimised, introducing optimisations
such as variable-key-size in the btree, and merging auto-increment
keys heavily to the left. Also, Tcmalloc can be compiled in via make
options.
The best example, is run "(cd test; lua TEST_filters.lua)", the table
"customer_profile" w/ its 5 indexes, used to need 420MB for 1million
rows, using tcmalloc and the new optimisations, this has been
decreased to 80MB. This is a 5X+ reduction in size and strangely
enough (due to simply smaller size) the insert speed is 30% quicker
(this was a surprise to me :)
I changed the project homepage to reflect these new bumbers:
http://code.google.com/p/alchemydatabase/#MEMORY_EFFICIENT:
An in-memoryDB really does have to optimise memory efficiency, so this
was a major sweeping improvement, of which there will be many more
(slab allocators, in-btree-storage, etc.... all to come)
- Jak