Transient or permanent?

[tv...@slb.com]

Not sure if this is a naive question, but is it safe to assume that data successfully stored in MemoryStore is permanent to the same level as e.g. Cloud Datastore and Cloud Storage? Or should it be considered only for transient data, sort of like the name implies. All of this is of course assuming I have no expiration set for the data in question.

Thanks,

T.

[Gopal Ashok]

Short answer, you should treat it as a transient store since data is only stored in Memory.

Cloud Memorystore is essentially managed Redis. There are two flavors Basic tier and Standard Tier. In both tiers we currently haven't enabled persistence so all the data is only in memory and not persisted to disk. 

Standard tier enables replication and is replicated to another instance. This gives you a higher level of availability and to some extent higher level of durability compared to basic instance incurring the same kind of failures. But you can still run into lost updates and inserts during failover since the replication is asynchronous.

If you can share the use case that will help.

Thanks

Gopal

Product Manager

Cloud Memorystore

--
You received this message because you are subscribed to the Google Groups "Google Cloud Memorystore Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-memorysto...@googlegroups.com.
To post to this group, send email to google-cloud-mem...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-memorystore-discuss/6298e668-33fd-4ce6-a3d6-df3559d78724%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tv...@slb.com]

Hi Gopal and thanks for getting back to me and confirming my assumption.

I'll give some context about the use case, maybe it could be an interesting scenario. 

We're building a CRUD style service for a somewhat complex Domain Object (DO) (some call them 'Business Object'). Each of these DOs can be several gigabytes large, and the service stores the data in a combination of Cloud Datastore and Cloud Storage. Due to the nature of these DOs, modifications have to be applied strictly in order. Sometimes we get bursts of writes/updates for a given DO, so to support some level of parallel updates/writes we store operations into a write-buffer and process them in the background (sorted by timestamp). Now, we still want strong consistency (or at least a minimum of latency) from when an UPDATE/WRITE is finished until it's available through the READ-APis, so we have implemented a system where a READ will check both the operations in the write-buffer (which rules out using PubSub as write buffer) and the data in permanent storage and apply them on the fly. This has worked well so far. 

We use MemoryStore as a cache in several parts of the system and it's doing an excellent job, but I was wondering if it could also be used as the write buffer described above (instead of Cloud Storage as we currently do). This would improve performance of both the service and the background processing, in addition to reducing cost. 

For the moment, we only use MemoryStore as a cache-aside for certain READ operations and invalidate these cached elements whenever we have a WRITE operation. We've been thinking about updating the cache both upon WRITE and READ operations as this would largely achieve the same as using MemoryStore as a write-buffer, but haven't found a way to resolve the potential race condition. 

T.

On Thursday, November 1, 2018 at 6:08:30 PM UTC-7, Gopal Ashok wrote:

Short answer, you should treat it as a transient store since data is only stored in Memory.

Cloud Memorystore is essentially managed Redis. There are two flavors Basic tier and Standard Tier. In both tiers we currently haven't enabled persistence so all the data is only in memory and not persisted to disk. 

Standard tier enables replication and is replicated to another instance. This gives you a higher level of availability and to some extent higher level of durability compared to basic instance incurring the same kind of failures. But you can still run into lost updates and inserts during failover since the replication is asynchronous.

If you can share the use case that will help.

Thanks

Gopal

Product Manager

Cloud Memorystore

On Thu, Nov 1, 2018 at 4:39 PM tvik via Google Cloud Memorystore Discuss <google-cloud-memorystore-dis...@googlegroups.com> wrote:

Not sure if this is a naive question, but is it safe to assume that data successfully stored in MemoryStore is permanent to the same level as e.g. Cloud Datastore and Cloud Storage? Or should it be considered only for transient data, sort of like the name implies. All of this is of course assuming I have no expiration set for the data in question.

Thanks,

T.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Memorystore Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-memorystore-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to google-cloud-memorystore-dis...@googlegroups.com.