Current status of LevelDB
Vladimir Rodionov
server is a precursor of LevelDB and Current version of LevelDB is 1.2
- its quite mature version but is it used in production @Google or its
still lab experiment?
Jeff Dean
The tablet implementation for the BigTable system was developed
starting in about 2004, and is based on a different, Google internal
code base than the LevelDB code. That code base relies on a number of
Google code libraries that are not themselves open sourced, so
directly open sourcing that code would have been difficult. We wanted
to put together something like the BigTable tablet stack that had
minimal dependencies and would be suitable for open sourcing, and also
would be suitable for use in Chrome (for the IndexedDB
implementation). Thus, Sanjay Ghemawat and I wrote LevelDB starting
in early 2011, with the same general design as the BigTable tablet
stack, but not sharing any of the code.
LevelDB is used in production in the Chrome browser, among other
things, but isn't used in the BigTable system at Google.
Does that clarify things?
-Jeff
Vladimir Rodionov
One more question:
What is the largest database size you have ever tested? Is there are
any design or implementation
limitations you are aware about which can affect LevelDB scalability?
By scalability I mean predictable
behavior of LevelDB as data size grows.
-Vladimir
On Aug 7, 3:30 pm, Jeff Dean <j...@google.com> wrote:
> On Sun, Aug 7, 2011 at 12:06 PM, Vladimir Rodionov
>
Jeff Dean
<vladro...@gmail.com> wrote:
> OK, thank you for quick response, Jeff
>
> One more question:
> What is the largest database size you have ever tested?
We've successfully created databases in our testing of 1 billion
entries with 16 byte keys and 100 byte values (roughly 100 GB of raw
data, about half that after compression).
> Is there are any design or implementation
> limitations you are aware about which can affect LevelDB scalability?
> By scalability I mean predictable
> behavior of LevelDB as data size grows.
LevelDB keeps a separate file for every couple of MB of data, and
these are all in one directory. Depending on the underlying file
system, this might start causing trouble at some point.
Other than this, though, I suspect scalability is more limited by the
frequency of reads and writes that are being done, rather than the
number of bytes in the system.
-Jeff