Lua Benchmarking

[Jak Sprats]

Hi All,

I added Lua benchmarking today, here is a gist w/ the results
https://gist.github.com/673570

The idea was to see
1.) the overhead that calling routines in Lua has
2.) if nested Lua calls were linearly more costly

1M requests on a 3.0GHz CPU
----------------------------------------------------------
SET 89783.27 requests per second
GET 92997.39 requests per second
LUA(set) 50913.95 requests per second
LUA(get) 55218.44 requests per second
LUA(set_session) 25155.99 requests per second
LUA(get_session) 42436.16 requests per second

set_session and get_session are nested routines. set_session does 4
seperate SETs and get_session does 2 GETs (plus some logic).

Lua Overhead: 42%
Nesting costs: better than linear
(linear == 4-to-1 and 2-to-1)
(results are 2-to-1 and 1.3-to-1)

The value of embedding Lua, in this example, is in get_session,
instead of making sequential calls from a web-server to Redisql (which
takes forever in relative terms [6-8 OrderOfMagnitude longer]), the
web-server<->Redisql is a single request w/ sequential logic done
datastore side.

If you can push logic into the Datastore you can cut latency down to
nothing :)

- Jak

[David Yu]

Stored procedures :-)

Off topic question:

In traditional rdbms, its hard to do online schema change(alter table, creating an index ... typically doing them on the replica and then switch over)

With rediSQL, is this possible to do without involving a slave and still be fast to make the change?

Its a good goal to have since its basically enables rediSQL to be an rdbms with schemaless nosql agility.

 

- Jak

--
When the cat is away, the mouse is alone.
- David Yu

[Jak Sprats]

Hi David,

Good points and your ideas are totally in line w/ Redisql's goals.

Redisql builds indices pretty quickly ... 18 seconds to build an index
on a 10million row table.
This would effectively block the Database for 18 seconds, but its not
often you need a 10million row table index addition.
1 million rows took 1.5 secs. 2million rows took 3.3 secs. 5 million
rows took 8.9 secs.

Traditional databases are slow at building indexes because they have
locks, Redisql does not.

Oracle does Alter Table correctly, Mysql does not. Doing it correctly
is non-trivial.

I have not yet implemented "ALTER TABLE" but I have designed several
fields so that Alter table will set some flags and actually do the
adding or deleting of columns the next time the row is accessed by
another command (and possibly also in a server-cron when the server is
idle). This is more complex, but it avoids the terribleness that all
Mysql DBAs associate w/ the "ALTER TABLE" command.

Alter table will be implemented in December at the earliest, it is not
yet a pressing priority.

- Jak

On Nov 11, 7:33 pm, David Yu <david.yu....@gmail.com> wrote:

[David Yu]

On Fri, Nov 12, 2010 at 1:24 PM, Jak Sprats <jaks...@gmail.com> wrote:

Hi David,

Good points and your ideas are totally in line w/ Redisql's goals.

Redisql builds indices pretty quickly ... 18 seconds to build an index
on a 10million row table.
This would effectively block the Database for 18 seconds, but its not
often you need a 10million row table index addition.
1 million rows took 1.5 secs. 2million rows took 3.3 secs. 5 million
rows took 8.9 secs.

Nice.

Traditional databases are slow at building indexes because they have
locks, Redisql does not.

Oracle does Alter Table correctly, Mysql does not. Doing it correctly
is non-trivial.

I have not yet implemented "ALTER TABLE" but I have designed several
fields so that Alter table will set some flags and actually do the
adding or deleting of columns the next time the row is accessed by
another command (and possibly also in a server-cron when the server is
idle). This is more complex, but it avoids the terribleness that all
Mysql DBAs associate w/ the "ALTER TABLE" command.

If I understand it correctly, with your design the database won't block for n seconds when an alter table is made. Yes?

More like a lazy update w/c is cool.

[Jak Sprats]

Hi David,

a lazy update is the right metaphor.

I may actually do 2 different commands
1.) ALTER TABLE -> lazy
2.) SCANALTER TABLE -> do the alter table right now (blocking)

but this is all at least a month off, just brainstorming

- Jak

On Nov 11, 10:59 pm, David Yu <david.yu....@gmail.com> wrote:

[Jak Sprats]

Hi All,

I wrote a blog entry on embedding Lua in Redisql
http://jaksprats.wordpress.com/2010/11/15/datastore-side-scripting-can-prevent-bottlenecks/

I have to start calling it Alchemy Database: http://code.google.com/p/alchemydatabase/
- still getting used to the new name

- Jak