How to implement configuration-specific storage parameters?

[Mark Rambacher]

We are looking to change the database schema for MySQL storage to make
it more efficient. We would like to do this in a manner that is
compatible with existing stores but allows new stores to take
advantage of any improvements we might make. We would also like to
future-proof this to work with future schema variants.

What is the best way to implement this change? I would like to have a
"store-specific" property, that says what schema to use for the store,
ala:
<persistence type="MySQL">
<schemaVersion>2</schemaVersion>
</persistence>

However, it does not appear that it is currently possible to put
engine-specific properties into the stores.xml file. Additionally,
this information is not readily available to the StorageEngine (or
StorageConfiguration for that matter) at the time the stores are
created.

If I put the configuration attributes in "server.properties", then the
values would apply to ALL stores, which means that I would break
compatibility with existing stores and puts me back to square one.

I would propose making the following two changes:
1. Extend the "persistence" XML attribute in the stores.xsd schema to
allow it to contain engine-specific properties.
2. Change the StorageEngine::getStore API to take in more parameters
(such as the StoreProperties and possibly some other information, like
the Cluster).

Have others encountered similar issues or is it assumed that the
database schema and layout will never change? I would assume a
similar issue would exist if one wanted to store keys in separate
files/databases on a per-partition basis (ala
http://code.google.com/p/project-voldemort/issues/detail?id=179).

- Mark

[bhupesh bansal]

Hey Mark,

I think changing the MySQL database layout is similar to adding new store, so you can add
a new store type as "MySQL-fast" or something with your  changes.

Best
Bhupesh

--

You received this message because you are subscribed to the Google Groups "project-voldemort" group.
To post to this group, send email to project-...@googlegroups.com.
To unsubscribe from this group, send email to project-voldem...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/project-voldemort?hl=en.

[Mark Rambacher]

Bhupesh,

If I do what you suggest (add a new store-type), that would mean that
I need to add a new StoreConfiguration as well. Granted, through
inheritance I could minimize the code duplication but it seems like it
would be a lot of effort to code and maintain if there is more than
one variable parameter in the mix.

For example, we are working right now on a performance improvement to
SQL by changing how the engine uses keys. This changes the schema of
a table (change #1).

Another change is to alter how data is organized -- right now, there
is one DB per node with a table per store. We have talked about
changing it to allow either a database per store and a table per
partition or a database per store per partition (this is similar to
changes being proposed for BDB).

With these two changes and three configuration options, am I now up to
having to have 6 StoreConfigurations defined? If so, it gets
unmanageable very quickly.

- Mark

[bhupesh bansal]

Mark, 

hmm , I think we can model these changes on the lines of  changes done to move one bdb env for all stores to individual bdb env per store. 

The implementation bascilly define some switches in VoldemortConfig() ( isBdbOneEnvPerStore() ..) 

and in the implementation uses these switches to change the behavior. 

Changing stores.xml xsd is okay but I dont think its needed for this case, if you think you need to change xsd lets think about how we can make it more generic for such situations by allowing a <special-store-config> tag or something which does special stuff for different store and you can pass special strings which the particular store understands. 

Best

Bhupesh 

- Mark

[Jay Kreps]

Hey Mark,

If you want to add the full support for storage engine specific
properties, I think that would be a good feature. To do this we would
need to add an optional section to the store definition xml and
propagage that to the StoreDefinition object that goes to the storage
configuration and engine. The XSD can be defined to support arbitrary
xml sections (I forget the exact xsd data type but I know xsd supports
this). The the mysql config can look for whatever options it expects.
Alternately you could make a global property in server.properties that
forces all mysql stores to be of one type or the other.

One question I would ask is do you have any data on how the new schema
is better? The MySQL is less used than the BDB backend, and if you
have concrete data on how it is better and you know it is not worse in
some other way, then maybe a better approach would be to just convert
everything to your way and we could make a simple conversion tool.

-Jay

[Mark Rambacher]

Jay,

I really do not like requiring an upgrade utility, nor do I think the
changes are all likely to be universally desired.

For example, one change I would like to make is to be able to specify
the maximum value size supported by a store. Based on this parameter,
the MySQL store would change how it creates the tables.

Another change would be on how tables and databases are used by
MySQL. Currently, all values are written to one database and each
store goes into its own table. I would like to be able to have a
configuration parameter to allow the store to be the name of the
database and the partition to be the table ID (this feature might also
be useful in BDB).

There are two ways to go about adding these properties to the
store.xml file:

1. Arbitrary properties can be added to the store: <property
name="mysql.maxDataSize">64MB</property>
- Advantages:
- Simple to implement and validate in the schema
- No versioning
- Disadvantages
- No validation of property names (missing or mis-spelled)
possible.
- Properties/Names are not specific to a store
implementation (e.g. mysql vs bdb)
2. Store-specific properties could be added to the store schema.
<properties type="MySQL">
<maxDataSize">64MB</maxDataSize>
</properties>
- Advantages:
- Properties can be specific to a store
- Schemas can be validated
- Disadvantages
- Harder to implement (requires more schemas)
- Harder to share properties (schemas would need
inheritance or common naming)

Personally, I prefer option #1 as it is much simpler to implement but
am willing to investigate #2 if that is what the community prefers.

- Mark

On Dec 18, 6:54 pm, Jay Kreps <jay.kr...@gmail.com> wrote:
> Hey Mark,
>
> If you want to add the full support for storage engine specific
> properties, I think that would be a good feature. To do this we would
> need to add an optional section to the store definition xml and
> propagage that to the StoreDefinition object that goes to the storage
> configuration and engine. The XSD can be defined to support arbitrary
> xml sections (I forget the exact xsd data type but I know xsd supports
> this). The the mysql config can look for whatever options it expects.
> Alternately you could make a global property in server.properties that
> forces all mysql stores to be of one type or the other.
>
> One question I would ask is do you have any data on how the new schema
> is better? The MySQL is less used than the BDB backend, and if you
> have concrete data on how it is better and you know it is not worse in
> some other way, then maybe a better approach would be to just convert
> everything to your way and we could make a simple conversion tool.
>
> -Jay
>

[Mark Rambacher]

Jay,

- Mark

> Hey Mark,
>
> If you want to add the full support for storage engine specific
> properties, I think that would be a good feature. To do this we would
> need to add an optional section to the store definition xml and
> propagage that to the StoreDefinition object that goes to the storage
> configuration and engine. The XSD can be defined to support arbitrary
> xml sections (I forget the exact xsd data type but I know xsd supports
> this). The the mysql config can look for whatever options it expects.
> Alternately you could make a global property in server.properties that
> forces all mysql stores to be of one type or the other.
>
> One question I would ask is do you have any data on how the new schema
> is better? The MySQL is less used than the BDB backend, and if you
> have concrete data on how it is better and you know it is not worse in
> some other way, then maybe a better approach would be to just convert
> everything to your way and we could make a simple conversion tool.
>
> -Jay
>

[Jay Kreps]

Yeah I would exempt any backwards compatible schema changes like BLOB
=> LARGEBLOB or whatever from needing any kind of conversion script.
For a change that would need a conversion script I was thinking of
splitting of the mysql table into one table per partition. This is not
a backwards compatible change. Nor is it necessarily the case that
supporting both is good. If having many small tables is uniformly more
efficient on mysql then we should just do that--it will definitely
speed up the rebalancing apis that get partition specific k/v pairs.

For the config I think (1) is preferable as I believe it would make it
possible to keep the config storage engine agnostic, which means one
can implement a storage engine and use it without modifying voldemort
source code to add your properties.

-Jay