Suitability of a distributed KV store

[Arpad Rajeczy von Burian]

Hi all,

I'm currently using Bleve for an app running on Google App Engine. I'm storing the indexes on Google Cloud Storage so I have a bunch of extra logic to backup and synchronise indexes between instances.

I just noticed the KVStore interface and I'm wondering if I can implement this for Google Cloud Datastore to simplify things. I have a few questions:

First of all has anyone done this or something similar already?

What's the characteristic usage of the KV store? Is it mostly at startup or are we talking many calls per query?

Would the inevitable extra latency be impractical, or would it just slow things down in general?

Regarding the interfaces:

• I'm not sure there's a way to implement KVReader.PrefixIterator on Datastore, but it appears that this isn't used other than tests and in upsidedown, so if we're using scorch it might be ok to have a dummy / brute force implementation?
• Likewise KVReader.RangeIterator - there a parallel in Datastore queries, but I would guess it's not going to be very efficient. Again appears to be used only in tests and upsidedown.

Thanks in advance for any thoughts and advice on this,

Arpad

[Naan Waana]

Hey, 

I was wondering if you managed to implement this? I have the same problem O:-)

Thanks!

[Ramachandran Duraisamy]

I got a chance to go through the below. It might be helpful. 

https://www.philipotoole.com/increasing-bleve-performance-sharding/