Access to Cluster and StoreDefinition from a StorageEngine or StorageConfiguration

[Mark Rambacher]

How would one go about gaining access to the Store Definition or other
metadata from within a StorageEngine?

As it currently stands, the StorageConfiguration is constructed by the
StorageService by passing in the VoldemortConfig. The
StorageConfiguration is never passed information in the metadata
(though the StorageService has that information at hand at the time of
configuration construction).

Comparably, the StorageConfiguration is not passed the
StorageDefinition at the time it constructs StorageEngines. This
means that the StorageEngine cannot configure itself based on any
properties found in the StorageDefinition.

I would propose making the following changes:
-> Change the required constructor for StorageConfiguration to take in
VoldemortConfig and metadata (approximately line 121 in
StorageService::initStorageConfig)
-> Change the StorageConfiguration::getStore method to take in a
StoreDefinition.

Why do I need these changes?
1. I would like to be have access to the Cluster and Routing
information within the store. This would allow me to partition the
data into separate tables based on the key in use.
2. I would like to be able to have store-specific configuration
properties (such as whether or not to partition based on keys or which
schema to use for writing data).

Before I go about making these changes, does anyone know if they are
needed or is there already a mechanism of getting this information
into the StorageConfigurations and StorageEngines?

- Mark

[Jay Kreps]

Hi Mark,

They aren't there now, but I think it would be a good idea to pass in
the full VoldemortMetadata object that has access to all the metadata
rather than the config object which just has the server-specific
configuration. There is a similar need for views which are
instantiated in a similar way. The only downside to this is that in
the current form you could introduce a dependency on the metadata for
another store, which would introduce a dependency on the order in
which the stores were loaded. So this would change the constructor
from XyzStorageConfiguration(VoldemortConfig config) to
XyzStorageConfiguration(VoldemortMetadata config). Would this work?

-Jay

> --
>
> You received this message because you are subscribed to the Google Groups "project-voldemort" group.
> To post to this group, send email to project-...@googlegroups.com.
> To unsubscribe from this group, send email to project-voldem...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/project-voldemort?hl=en.
>
>
>

[bhupesh bansal]

Hey Mark, 

As jay mentioned you should look at MetadataStore and VoldemortConfig and you should be able to wire them in your storageConfiguration. There is a InvalidMetadataCheckingStore which also need metadataStore to check for correct routing. you can look up that code for reference if needed.

One issue I have is with dynamic rebalancing feature partitions from one node would be needed to migrated to another node , so please make sure the storage engine supports adding/deleting new partitions on the fly.

Best

Bhupesh

[Mark Rambacher]

Jay/Bhupesh,

Does the VoldemortMetadata class you mention currently exist? I
cannot find it in the latest source. It appears to have been
removed. This class appears to have been close to what I require, but
it lacked access to the VoldemortConfig object.

Within a StorageEngine, I would like to know the Cluster information
and the StoreDefinition for the specific StorageEngine. Currently, I
do not know how to get at that information. Are there methods I
should be using to get at that information currently or is this
something that I would need to add?

I agree that the StorageEngine should be able to handle rebalancing of
partitions, but think this might become easier to implement. I think
within a StorageEngine you might want an API like
"getEntriesByPartition", which would be much faster to implement if
each partition was its own table. If the StorageEngine has access to
the same Cluster instance as the rest of the system, rebalancing
should not be an issue (it would be however if the Engine read the
metadata file directly). I am a little uncertain as to what you think
the problems this will face with rebalancing.

- Mark

On Dec 21, 12:52 pm, bhupesh bansal <bbansal....@gmail.com> wrote:
> Hey Mark,
>
> As jay mentioned you should look at MetadataStore and VoldemortConfig and
> you should be able to wire them in your storageConfiguration. There is a
> InvalidMetadataCheckingStore which also need metadataStore to check for
> correct routing. you can look up that code for reference if needed.
>
> One issue I have is with dynamic rebalancing feature partitions from one
> node would be needed to migrated to another node , so please make sure the
> storage engine supports adding/deleting new partitions on the fly.
>
> Best
> Bhupesh
>

> On Mon, Dec 21, 2009 at 9:24 AM, Jay Kreps <jay.kr...@gmail.com> wrote:
> > Hi Mark,
>
> > They aren't there now, but I think it would be a good idea to pass in
> > the full VoldemortMetadata object that has access to all the metadata
> > rather than the config object which just has the server-specific
> > configuration. There is a similar need for views which are
> > instantiated in a similar way. The only downside to this is that in
> > the current form you could introduce a dependency on the metadata for
> > another store, which would introduce a dependency on the order in
> > which the stores were loaded. So this would change the constructor
> > from XyzStorageConfiguration(VoldemortConfig config) to
> > XyzStorageConfiguration(VoldemortMetadata config). Would this work?
>
> > -Jay
>

> > On Mon, Dec 21, 2009 at 8:56 AM, Mark Rambacher <mramb...@gmail.com>

> > project-voldem...@googlegroups.com<project-voldemort%2Bunsu...@googlegroups.com>

> > .
> > > For more options, visit this group at
> >http://groups.google.com/group/project-voldemort?hl=en.
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "project-voldemort" group.
> > To post to this group, send email to project-...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > project-voldem...@googlegroups.com<project-voldemort%2Bunsu...@googlegroups.com>

[bhupesh bansal]

Hey Mark, 

The class is called MetadataStore , it currently does not have VoldemortConfig information (looks like a good idea to add it though) you will need to pass VoldemortConfig separately for now. 

you have getCluster() and getStoreDef(storename) function which you can use. 

My concern was that the cluster/storeDef info will change on the fly for rebalancing, so was just letting you know if you need to make some adjustments for that. 

Best

Bhupesh

To unsubscribe from this group, send email to project-voldem...@googlegroups.com.

[Mark Rambacher]

Bhupesh,

Thanks. As a final point of clarification, I believe you are
suggesting that:
- StoreConfiguration should have a constructor that takes a
MetadataStore (rather than VoldemortConfig)
- MetadataStore should have access to the VoldemortConfig.

Is that correct? If so, I do not have a problem making the change.
Just confirming we were on the same page. This still leaves
ViewStorageConfigurations initialized differently than other stores
(views also need access to the Store Repository).

Thanks,

Mark

On Dec 21, 2:25 pm, bhupesh bansal <bbansal....@gmail.com> wrote:
> Hey Mark,
>

> The class is called MetadataStore , it currently does not have
> VoldemortConfig information (looks like a good idea to add it though) you
> will need to pass VoldemortConfig separately for now.
>
> you have getCluster() and getStoreDef(storename) function which you can
> use.
>
> My concern was that the cluster/storeDef info will change on the fly for
> rebalancing, so was just letting you know if you need to make some
> adjustments for that.
>
> Best
> Bhupesh
>

> > <project-voldemort%2Bunsu...@googlegroups.com<project-voldemort%252Buns...@googlegroups.com>

>
> > > > .
> > > > > For more options, visit this group at
> > > >http://groups.google.com/group/project-voldemort?hl=en.
>
> > > > --
>
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "project-voldemort" group.
> > > > To post to this group, send email to
> > project-...@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > > project-voldem...@googlegroups.com<project-voldemort%2Bunsu...@googlegroups.com>

> > <project-voldemort%2Bunsu...@googlegroups.com<project-voldemort%252Buns...@googlegroups.com>

[Mark Rambacher]

Bhupesh,

After sleeping on it, would there be any harm or issue with adding the
MetadataStore to the VoldemortConfig class? This would be the
simplest approach as the StoreConfigurations all get passed a config
currently and could use the MetadataStore if they required. The
VoldmortServer constructor class creates the MetadataStore and could
add it to the configuration at that time.

Let me know what you think.

- Mark

[bhupesh bansal]

Mark,

VoldemortConfig have all the static configuration files, metadataStore is a real store which does meta data get/put operation on the fly.
I didn't put voldemort config into metadata store as it is read only for now and doesn't change on the fly.

please pass both voldemortConfig and metadataStore to the storeConfiguration class, Thats the cleanest way in my opinion.

Best
Bhupesh

To unsubscribe from this group, send email to project-voldem...@googlegroups.com.

[Jay Kreps]

Don't you think we should merge the two in some way since they both
encapsulate configuration information? I think their being seperate is
purely for historical reasons, no?

-Jay

[bhupesh bansal]

MetadataStore is a Store, we can keep VoldemortConfig inside it as a new key entry, It will help us avoid passing two parameters to constructor instead of only one (metadataStore) at many places, 

 I am just not sure what to do on put() on VoldemortConfig key. All the services read the VoldemortConfig value at start time and will not update theirselves if we update VoldemortConfig() entries. We should probably disallow updating VoldemortConfig key or log that this will not mean that VoldemortServer will change parameters on the fly .. 

Best

Bhupesh