Replication of data , when one node goes up?

[ams...@gmail.com]

Hi all,

i have one scenario ,in which i have 2 Voldemort server nodes up with 2 partition each.

My stores of XML looks like.

<stores>
  <!-- Note that "test" store requires 2 reads and writes,
     so to use this store you must have both nodes started and running -->
  <store>
    <name>test</name>
    <persistence>bdb</persistence>
    <routing>client</routing>
    <replication-factor>2</replication-factor>
    <preferred-reads>2</preferred-reads>    
    <required-reads>1</required-reads>
    <required-writes>1</required-writes>
    <key-serializer>
      <type>string</type>
    </key-serializer>
    <value-serializer>
      <type>string</type>
    </value-serializer>
    <retention-days>1</retention-days>
  </store>
</stores>

I used following steps:-

1. I have node 1 up and node 2 down.

2. i put a data(key,value pair) onto the server i.e on node 1.

3. Then i make node 2 up and node 1 down.

4. Then i get the value of the key for the server i.e node 2 , but i either that the old version or null pointer.

Is their a way by which i initiate the replication as soon as a node goes up ??

  

[Brendan Harris]

Hello,

If you want to avoid this situation, you should have 3+ nodes in the cluster and configure your store with replication-factor 3, required-reads 2 and required-writes 2. That will solve this problem.

You can also enable hinted-handoff, but this is an eventual consistency mechanism so it is not guaranteed to update the nodes immediately when they come back up. You can increase the slop.frequency to better ensure that it updates the nodes more readily. To do this, add <hinted-handoff-strategy>proximity-handoff</hinted-handoff-strategy> to your store config, add slop.enable=true to your server.properties and then optionally you can set slop.frequency.ms=<ms> (defaults to 300000) to your server.properties to set the interval at which it runs.

Brendan

[Brendan Harris]

On Tuesday, April 2, 2013 7:07:44 AM UTC-7, Brendan Harris wrote:

Hello,

You can also enable hinted-handoff ...

I should clarify this by saying that you _should_ enable hinted-handoff. Just don't rely on it for immediate consistency.

Brendan 

[ams team]

Hi Brendan,

I tried using <hinted-handoff-strategy>proximity-handoff</hinted-handoff-strategy> ,  slop.frequency.ms = 8000  and slop.enable=true ,

but it didn't rectify the problem.

If node1 is up and node 2 is down and i updated some data on node1.

and the i take node 2 up , waited for 15-20 seconds and then down my node 1 and tried to get data from server 2 , it resulted me with old data.

--
You received this message because you are subscribed to a topic in the Google Groups "project-voldemort" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/project-voldemort/tSvFM5csfrY/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to project-voldem...@googlegroups.com.
Visit this group at http://groups.google.com/group/project-voldemort?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Brendan Harris]

Hello,

Like I stated before, to truly solve this problem, you need 3 or more nodes and a replication-factor 3, required-reads/writes 2 configuration. The hinted-handoff is a secondary eventual consistency mechanism to help ensure that you do not lose data from a node being down for an extended period of time.

Brendan

[ams team]

Thanks Brendan,

I more query i have , when i do rebalancing by adding one more node , and if i have only that node up which i recently added and rest of the nodes are down , it doesn't allow me to do get , put operation on that node. If any previous node is up then it allows me to do get and put operation.

In my set up i have 2 nodes running and i added 3rd node with empty partition.

i got a exception like this when i only have recently added node up.


Exception in thread "main" voldemort.store.InsufficientOperationalNodesException: 1 get versionss required, but only 0 succeeded Original replication set :[0, 1] Known failed nodes before operation :[] Estimated live nodes in preference list :[0, 1] New failed nodes during operation :[0, 1]
    at voldemort.store.routed.action.PerformSerialRequests.execute(PerformSerialRequests.java:142)
    at voldemort.store.routed.Pipeline.execute(Pipeline.java:214)
    at voldemort.store.routed.PipelineRoutedStore.getVersions(PipelineRoutedStore.java:530)
    at voldemort.store.routed.PipelineRoutedStore.getVersions(PipelineRoutedStore.java:1)
    at voldemort.store.DelegatingStore.getVersions(DelegatingStore.java:86)
    at voldemort.store.DelegatingStore.getVersions(DelegatingStore.java:86)
    at voldemort.store.serialized.SerializingStore.getVersions(SerializingStore.java:144)
    at voldemort.store.DelegatingStore.getVersions(DelegatingStore.java:86)
    at voldemort.client.DefaultStoreClient.getVersions(DefaultStoreClient.java:163)
    at voldemort.client.DefaultStoreClient.put(DefaultStoreClient.java:213)
    at VolClient.main(VolClient.java:18)
Caused by: voldemort.store.UnreachableStoreException: Could not connect to http://localhost:8081/stores for test
    at voldemort.store.http.HttpStore.executeRequest(HttpStore.java:178)
    at voldemort.store.http.HttpStore.getVersions(HttpStore.java:203)
    at voldemort.store.http.HttpStore.getVersions(HttpStore.java:1)
    at voldemort.store.nonblockingstore.ThreadPoolBasedNonblockingStoreImpl$3.request(ThreadPoolBasedNonblockingStoreImpl.java:84)
    at voldemort.store.nonblockingstore.ThreadPoolBasedNonblockingStoreImpl$3.request(ThreadPoolBasedNonblockingStoreImpl.java:1)
    at voldemort.store.nonblockingstore.ThreadPoolBasedNonblockingStoreImpl$6.run(ThreadPoolBasedNonblockingStoreImpl.java:129)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8081 refused
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:158)
    at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
    at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:573)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
    at voldemort.store.http.HttpStore.executeRequest(HttpStore.java:166)
    ... 11 more
Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
    at java.net.Socket.connect(Socket.java:579)
    at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)

Why their is such kind of behaviour ?

[Brendan Harris]

Hello,

So, this is the trade-off between data availability and data consistency. If you want to ensure that you only ever get the most current data, this is the type of configuration you want. Now, if you want more fault tolerance, but you are willing to risk the chance of reading stale data if two nodes are down, you can change the required-reads/-writes 2 to preferred-reads/-writes 2. This will allow you to have two out of three nodes down, but you still have the risk of reading stale keys.

Brendan

[ams team]

Okay,

But now my issue is when i add a new node dynamically say it as node 3 , and now if my previous 2 nodes down , i am not able to do put and get operation on newly added node , i got the above exception in that case.

Why i am getting such exception ?

in stores.xml i have config like this :-

<stores>
  <!-- Note that "test" store requires 2 reads and writes,
     so to use this store you must have both nodes started and running -->
  <store>
    <name>test</name>
    <persistence>bdb</persistence>
    <routing>client</routing>

    <hinted-handoff-strategy>proximity-handoff</hinted-handoff-strategy>

    <replication-factor>2</replication-factor>
    <preferred-reads>2</preferred-reads>   
    <required-reads>1</required-reads>

    <preferred-writes>2</preferred-writes>

    <required-writes>1</required-writes>
    <key-serializer>
      <type>string</type>
    </key-serializer>
    <value-serializer>
      <type>string</type>
    </value-serializer>
    <retention-days>1</retention-days>
  </store>
</stores>

i also changed the replication factor to 3 , but i have no success in doing get and put operation when only newly dynamically added node is up.

[Brendan Harris]

That's odd. When you say you added the node "dynamically", what do you mean by that? Also, what version of voldemort are you using?

Brendan

[ams team]

dynamically here means that , when 2 nodes are already running and when i am adding a node through rebalancing using the link https://github.com/voldemort/voldemort/wiki/Voldemort-Rebalancing , section "

How to start rebalancing? "

I have to provide new cluster.xml with new node credentials in that with empty partition.

Voldemort version is 1.3.0

[Brendan Harris]

OK, can you do me a favor and paste an example of the current exception you are getting, as well as attach your cluster.xml (and make the host names anonymous, like "host1", "host2" and "host3" instead of their real host names or IPs)?

Thanks,

Brendan

[ams team]

Ok,

Error is

voldemort.store.InsufficientOperationalNodesException: 1 gets required, but only 0 succeeded Original replication set :[1, 0] Known failed nodes before operation :[] Estimated live nodes in preference list :[1, 0] New failed nodes during operation :[0, 1]

    at voldemort.store.routed.action.PerformSerialRequests.execute(PerformSerialRequests.java:142)
    at voldemort.store.routed.Pipeline.execute(Pipeline.java:214)

    at voldemort.store.routed.PipelineRoutedStore.get(PipelineRoutedStore.java:305)
    at voldemort.store.routed.PipelineRoutedStore.get(PipelineRoutedStore.java:1)
    at voldemort.store.logging.LoggingStore.get(LoggingStore.java:105)
    at voldemort.store.DelegatingStore.get(DelegatingStore.java:60)
    at voldemort.store.stats.StatTrackingStore.get(StatTrackingStore.java:66)
    at voldemort.store.stats.StatTrackingStore.get(StatTrackingStore.java:1)
    at voldemort.store.serialized.SerializingStore.get(SerializingStore.java:103)
    at voldemort.store.DelegatingStore.get(DelegatingStore.java:60)
    at voldemort.store.versioned.InconsistencyResolvingStore.get(InconsistencyResolvingStore.java:50)
    at voldemort.client.DefaultStoreClient.get(DefaultStoreClient.java:133)
    at voldemort.client.DefaultStoreClient.getValue(DefaultStoreClient.java:123)
    at VolClient.main(VolClient.java:22)
Caused by: voldemort.store.UnreachableStoreException: Could not connect to http://host1:8081/stores for test
    at voldemort.store.http.HttpStore.executeRequest(HttpStore.java:178)
    at voldemort.store.http.HttpStore.get(HttpStore.java:105)
    at voldemort.store.http.HttpStore.get(HttpStore.java:1)
    at voldemort.store.nonblockingstore.ThreadPoolBasedNonblockingStoreImpl$2.request(ThreadPoolBasedNonblockingStoreImpl.java:72)
    at voldemort.store.nonblockingstore.ThreadPoolBasedNonblockingStoreImpl$2.request(ThreadPoolBasedNonblockingStoreImpl.java:1)

    at voldemort.store.nonblockingstore.ThreadPoolBasedNonblockingStoreImpl$6.run(ThreadPoolBasedNonblockingStoreImpl.java:129)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)

Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://10.40.63.101:8081 refused

    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:158)
    at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
    at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:573)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
    at voldemort.store.http.HttpStore.executeRequest(HttpStore.java:166)
    ... 11 more
Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
    at java.net.Socket.connect(Socket.java:579)
    at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)

    ... 19 more

Cluster.xml is

<cluster>
  <name>mycluster</name>
  <server>
    <id>0</id>
    <host>host1</host>
    <http-port>8081</http-port>
    <socket-port>6666</socket-port>
    <admin-port>6667</admin-port>
    <partitions>0, 1</partitions>
  </server>
  <server>
    <id>1</id>
    <host>host2</host>
    <http-port>8082</http-port>
    <socket-port>6668</socket-port>
    <admin-port>6669</admin-port>
    <partitions>2, 3</partitions>
  </server>
  <server>
    <id>2</id>
    <host>host3</host>
    <http-port>8083</http-port>
    <socket-port>6660</socket-port>
    <admin-port>6661</admin-port>
    <partitions />
  </server>
</cluster>

Stores.xml

<stores>
  <!-- Note that "test" store requires 2 reads and writes,
     so to use this store you must have both nodes started and running -->
  <store>
    <name>test</name>
    <persistence>bdb</persistence>
    <routing>client</routing>
    <hinted-handoff-strategy>proximity-handoff</hinted-handoff-strategy>
    <replication-factor>2</replication-factor>

    <required-reads>1</required-reads>
    <preferred-reads>2</preferred-reads>
    <required-writes>1</required-writes>
    <preferred-writes>2</preferred-writes>

    <key-serializer>
      <type>string</type>
    </key-serializer>
    <value-serializer>
      <type>string</type>
    </value-serializer>
    <retention-days>1</retention-days>
  </store>
</stores>

[Brendan Harris]

  <server>
    <id>2</id>
    <host>host3</host>
    <http-port>8083</http-port>
    <socket-port>6660</socket-port>
    <admin-port>6661</admin-port>
    <partitions />

You have no partitions on node 2. That's the problem. You cannot store data on it, so if the two nodes that _do_ have partitions are down, the one node remaining up has no data.

It looks to me like you added the host to the cluster, but never rebalanced the cluster to move partitions onto it.

Brendan

[ams team]

I am adding a node through rebalancing using the link https://github.com/voldemort/voldemort/wiki/Voldemort-Rebalancing , section "How to start rebalancing"

, it clearly states that when i add a new node it should be added with 0 partitions .

and i followed all steps as specified in the link above , but i get a new node added with zero partitions in it , it didn't rebalance the partitions to all node ?

can you provide any other link through which i can rebalance it ?

REBALANCING SHOULD BE DONE WHEN 2 NODES ARE ALREADY RUNNING AND I AM ADDING 3RD NODE AT THAT TIME.

Brendan

--

You received this message because you are subscribed to a topic in the Google Groups "project-voldemort" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/project-voldemort/tSvFM5csfrY/unsubscribe?hl=en.

[Brendan Harris]

Hello,

OK, let me take a look at the rebalancing wiki page. It may have an error in it. I'll get back to you a little later, today.

Also, I advise that when you have a chance you recreate your cluster with a lot more partitions (more than 200, but less than 3000). See my posts in the following topic for more details on this:

https://groups.google.com/forum/?hl=en&fromgroups=#!topic/project-voldemort/3vrZfZgQp2Y

Brendan

[amstm07]

Hi Brendan,

what i did was , i have node 1 in which i have 16 partitions and then i added node 2 with empty partition and started rebalancing process

but still it didn't rebalance it.

In the link https://github.com/voldemort/voldemort/wiki/Voldemort-Rebalancing  ,

in section "How to start rebalancing " , when i do step 4 , the standard deviation came out to be 50% ,

and after performing step 6 , logs came as

Total number of primary partition moves : 0

so i am stuck why it is not moving any partition to new node.

Regards,

ams

[Brendan Harris]

Hello,

It seems to me like you did not do step 3:

Generate the new cluster xml. This can be done either by hand or if you want to do it automatically here is a tool to do to
./bin/voldemort-rebalance.sh —current-cluster [ current_cluster_path ] —target-cluster [ should be the same as current-cluster but with new nodes put in with empty partitions ] —current-stores [ current_stores_path ] —generate

After adding the new node with empty partitions, you next need to generate a new cluster.xml that moves partitions from the existing nodes to the new node. Then when you kick off the rebalance, you make that new cluster.xml your argument for the '--target-cluster' option of voldemort-rebalance.sh. Based on the plan output, it looks to me like you did not do this step.

Brendan

[amstm07]

I followed the step 3 , but i didn't find the new cluster.xml , so what i did was

"i typed command voldemort-rebalance.sh --help ",
 in this i got one parameter --output-dir , so in above command if you give
./bin/voldemort-rebalance.sh —current-cluster [ current_cluster_path ] —target-cluster [ should be the same as current-cluster but with new nodes put in with empty partitions ] —current-stores [ current_stores_path ] —generate --output-dir [directory path for output] , here you will find new cluster.xml named as final-cluster.xml , , and then i am able to successfully rebalance the voldemort.

One more thing , if i want to configure , stores.xml , as i want to increase replication factor as one node is added , so how to do that over the nodes when they are already running ?

i thing when nodes are running , and i change stores.xml their will be no effect.

[Brendan Harris]

Hello,

On Tuesday, April 16, 2013 11:25:31 PM UTC-7, amstm07 wrote:

"i typed command voldemort-rebalance.sh --help ",

 in this i got one parameter --output-dir , so in above command if you give
./bin/voldemort-rebalance.sh —current-cluster [ current_cluster_path ] —target-cluster [ should be the same as current-cluster but with new nodes put in with empty partitions ] —current-stores [ current_stores_path ] —generate --output-dir [directory path for output] , here you will find new cluster.xml named as final-cluster.xml , , and then i am able to successfully rebalance the voldemort.

Ah, I see. The documentation should definitely clarify this. I'll get it updated.

 

One more thing , if i want to configure , stores.xml , as i want to increase replication factor as one node is added , so how to do that over the nodes when they are already running ?

i thing when nodes are running , and i change stores.xml their will be no effect.

You can do a "voldemort-admin-tool.sh --set-metadata stores.xml --set-metadata-value <new_stores_xml_file>" with the new file containing the new settings. But then you have to restart the clients or force them to rebootstrap via the JMX operation. If you don't want to have to manually rebootstrap your clients, you can configure your clients with the "use_default_client" property set to "false" and then when you do the "--set-metadata" you can add the option "--stores <comma_seperated_list_of_modified_stores>" option, which will instruct the clients to rebootstrap automatically.

One thing to note, though, is that increasing the replication factor can be somewhat risky because you'll have keys that have not yet been rewritten with all replicas. So, if a node goes down, you might start getting null keys. The only way to guarantee the replication factor increase for all keys is to use the admin API to fetch all the keys and stream them back in. The only real _safe_ way of increasing the replication factor without "re-touching" all the keys is to go from replication-factor 2 to replication-factor 3 with required-reads and required-writes set to 2. That being said, as of now there's technically no true support in voldemort for increasing the replication-factor of stores.

Brendan

[amstm07]

Hi Brendan.

i am facing one issue ,

i am having voldemort server running on amazon instance , and i have voldemort client running on physical machine .

many times i get this type of exception

voldemort.versioning.ObsoleteVersionException: Key 0002756e697175652f722f32302f313a626f6f74737472617044617461 version(1:5373) ts:1366885602705 is obsolete, it is no greater than the current version of version(1:5373) ts:1366885599099.

what is error about ? how will it effect me and it comes when in an setting a data in voldemort and it is not able to set and throws this exception.

How to get rid of it ?

Regards.
ams

Brendan

--

You received this message because you are subscribed to a topic in the Google Groups "project-voldemort" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/project-voldemort/tSvFM5csfrY/unsubscribe?hl=en.

[Brendan Harris]

Hello,

It means that the key was already written by another thread (or app) between the time in which this thread retrieved the key and wrote it back. It's basically a concurrency problem. The keys' values are all versioned to provide better data integrity. When you do a put(), what happens under the hood is that the client actually gets the key's current version number, increments it by 1 and then put()s the value with the new version number. So, if another parallel thread does a put to the same key between the time in which this thread got the version, this thread's version will be obsolete by the time the actual write happens.

You can use the applyUpdate() method to do a looping retry of the put(), but it is usually wiser to implement your own conflict resolution that will re-get the key, compare the values and decide what to do about merging or overwriting. And, if you have a lot of key concurrency in your app(s), you should probably use a random-exponential or Fibonacci-style back-off to avoid hitting your server(s) too aggressively with retries.

Brendan

[amstm07]

Hi Brendan,

Thanks for your help and immediate replies.

I am facing issues while saving array of user defined objects in to voldemort.

Is there a way I can save the array\List\Set in the voldemort.

What would be the store.xml for the same. would it be possible for you to share the sample store.xml to save array\list\set in voldemort.

How the store client would be declared to store array\set\list like for storing string type

we declare StoreClient<String, String> strClient

Any info on this would help me in moving forward.

Thanks in advance

Regards,

ams

[Brendan Harris (a.k.a. stotch on irc.oftc.net)]

Hi ams,

Sorry for the lack of reply. I have been sick.

On Monday, May 6, 2013 7:20:00 AM UTC-7, amstm07 wrote:

Is there a way I can save the array\List\Set in the voldemort.

What would be the store.xml for the same. would it be possible for you to share the sample store.xml to save array\list\set in voldemort.

Well, you can store whatever you want in voldemort, you just need to define the schema to serialize it. I don't know the specifics of your data structure, so I cannot help you out much, there. You should look up the json specification (or use an online schema builder) to build the right schema. But, your value-serializer (assuming you're using "json" type) schema-info would probably look something like:

["<type>", "<type>", "<type>" ...]

Where <type> is the object type your are storing in that element of the array. But I think json only supports simple types, not object types. So, you might have to use avro instead and have a more complex schema. Alternatively, you can be schemaless by using "java-serialization" as your serializer type, but if you ever decide to evolve your data structure, you'll need to handle backward compatibility of the structure within your code.

  

How the store client would be declared to store array\set\list like for storing string type

we declare StoreClient<String, String> strClient

This really depends on your actual data structure types being used. But I think your value is a List (of objects of currently undefined type), not a String. Right?

Brendan

[amstm07]

Hi Btendan,

you are right , i am having List of object , let us suppose we have class rectangle and have data types length and breadth

class rectangle
{

Brendan

--

You received this message because you are subscribed to a topic in the Google Groups "project-voldemort" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/project-voldemort/tSvFM5csfrY/unsubscribe?hl=en.

[amstm07]

Hi Brendan,

Sorry , that last message was send by mistake.

you are right , i am having List of object , let us suppose we have class rectangle and have data types length and breadth

json only works for primitive types like string , integer etc.

class rectangle
{

      int length;

      int breath;
}

and we have list of this type of objects

List<rectangle> obj;

so how to store this List object in voldemort ?

can you share a sample store.xml , or just share what should we store in value-serializer field so that we put and get data from voldemort.

Regards,
ams

[Vinoth Chandar]

>> so how to store this List object in voldemort ?

Voldemort does not support lists explicitly as you might have noticed by now. People usually perform a get(), append and write it back.. You need to handle ObsoleteVersionException properly if you might be writing the same key from multiple clients.

Moving on to the schema (of which I am no authority), I think your concern (as Brendan clarified to me privately) is defining a json array in the schema.. I will try to confirm that this is not supported for sure.
But there are other alternatives - "java serialization" or "identity" (which is basically byte[]. your app does the conversion from json to byte[]). Would those work for you?

As Brendan pointed out, the downside is schema evolution is now completely your app's responsibility.

Thanks
Vinoth

[amstm07]

Hi Vinoth,

i solved the problem by taking value serializer as json and and in schema i took ["string"] i.e array of strings ,

i converted json array  to string array and stored that data there and when i retrieve the data i covert string back to json.

Regards,
ams

--

[Vinoth Chandar]

Glad that it worked out!!