Second Meeting (30th June) notes

[Makoto]

Hello.

Here is the notes about our "Google Big Table" paper (http://
nosqlsummer.org/paper/google-bigtable) discussion. Many thanks to
@mloughran for restructuring my notes massively.

This note is just a starting point. I also added some questions, as
certain points were either not clear to us or no definitive answers. I
would appreciate everybody's comment (answering questions, raising
more questions and so on).

Availability:

- BigTable is good for scaling out, but does not solve the
availability problem (which Cassandra and Dynamo does). If BigTable
goes down, you will notice certain downtime (while Cassandra and
Dynamo do not).
- Dan mentioned that Big Table does not offer availability, but
doesn't GFS have 3 replicas? can they be used to avoid availability
problem?
- All writes are written to commit log (aka table log) before stored
into memtable, so that it can be recovered when tablet server crashes.

Unique aspects

- To benefit from locality, deciding the format of row keys is
important. In the example of the thesis, it uses inverted fqdn (eg:
com.google.app, com.google.www) so that documents in same domain tend
to be in the same tablet.
- What else?

Limitations

- BigTable only offers primary indexing sorted by key values.
- However, as Tav mentioned, Megastore provides a mechanism to create
index on very column names automatically, stored into different
location so that it can query each index table in parallel.
- We had discussion about transactions. So does big table actually
offer transaction or not (the big theory only mentioned that big table
does offer row level locking and Chubby offers read/write lock, but no
mention about transactions)?

Similarity with other systems

- Rob noted that the tablet compaction mechanisms are very similar to
how Lucene works.
- Data recovery from commit log is similar to ordinary DBMS recovery
management.

System Diagram

http://twitpic.com/21h6ml


Thanks.

Makoto