massive memory optimsations

2 views
Skip to first unread message

Jak Sprats

unread,
Jan 20, 2011, 5:59:57 PM1/20/11
to redisql-dev
Hi,

The last week I have devoted to optimising the b-trees in AlchemyDB in
terms of memory usage.

The Integer-index was heavily optimised, introducing optimisations
such as variable-key-size in the btree, and merging auto-increment
keys heavily to the left. Also, Tcmalloc can be compiled in via make
options.

The best example, is run "(cd test; lua TEST_filters.lua)", the table
"customer_profile" w/ its 5 indexes, used to need 420MB for 1million
rows, using tcmalloc and the new optimisations, this has been
decreased to 80MB. This is a 5X+ reduction in size and strangely
enough (due to simply smaller size) the insert speed is 30% quicker
(this was a surprise to me :)

I changed the project homepage to reflect these new bumbers:
http://code.google.com/p/alchemydatabase/#MEMORY_EFFICIENT:

An in-memoryDB really does have to optimise memory efficiency, so this
was a major sweeping improvement, of which there will be many more
(slab allocators, in-btree-storage, etc.... all to come)

- Jak
Reply all
Reply to author
Forward
0 new messages