SCION iterator, REVERSE Iterator, ORDERBY Iterator

[Jak Sprats]

Hi All,

just an update.

I have added 2 new iterators for SQL tables.

They are roughly documented here: http://code.google.com/p/alchemydatabase/wiki/HighLights

The SCION iterator, uses counts of descendants at each btree node to
efficiently do "SELECT * FROM TABLE ORDER by pk LIMIT 10 OFFSET
9999999" queries. Finding the 9999999th row is done efficiently.

The SCION Iterator is even efficent w/ the following query:
"SELECT * FROM TABLE WHERE pk BETWEEN 666666 AND 777777 ORDER by pk
LIMIT 10 OFFSET 9999999"
which is more difficult as the 666666th entry is located deep in the
Btree.

The SCION iterator works for PKs and FKs.

Reverse Iterators are now available for both PKs and FKs, and in combo
w/ the SCION iterator.

The ORDERBY index is pretty cool. Normally, when you do a "SELECT *
FROM user_actions WHERE user_id =2" [user_id is an FK] you will get
rows back sorted to PK ASC. Using a ORDERBY index, the rows can be
sorted (in the index) to a different column (for example a timestamp
column) and the rows will be now be returned in timestamp order.

The ORDERBY index is incredibly useful for timestamp like queries that
need ORDERBY timestamp ... think my most recent 10 tweets. Relying on
the natural order you INSERT such rows does not work in distributed
environments, it is best to have a specific ordering ...

ORDERBY, SCION, & Reverse all work w/ one another (and as
MultipleColumnIndexes).

These iterators have all been tested to 10M PKs & 3M FKs ... they are
fast.

Enjoy

- jak

[Jak Sprats]

Hi,

I also forgot, I have been steadily building Alchemy's AppStack:
http://code.google.com/p/alchemydatabase/wiki/AppStack

In a short time I will have a framework that can do distributed
webserving w/ geographic failover at very high velocity (like 1million
HTTP REQ/s).

Ripping the bottlenecking part of a website out and doing it in
Alchemy's AppStack should solve any Requests-per-second or high-
latency-webrequests issues.

More to come

- jak

[NailK]

Hi Jak,

This looks awesome.
Do you plan to release a stable version?

Thanks.
Nail

[Jak Sprats]

Hi Nail,

Yeah a stable release is really needed. It is a bit more complicated
as I would like as there are redis dependencies (scripting + to a
lesser degree clustering). Luckily, it only takes about a day to cut
Alchemy over to a new redis release.

I have a short list of functions that I want to do before a stable
release, which should take 3-5 weeks, after that, hopefully redis 2.6
will have an RC and I can glue them together and put out a stable
release (which I have not done in forever).

thanks for reminding me :)

- jak