feature request: support catching up immutable memtable without wal in secondary instance

[Jay Lee]

When using `OpenAsSecondary` and calling `TryCatchUpWithPrimary`, secondary will apply WAL first, and then check manifest to remove flushed memtables. If all records are written without WAL, `TryCatchUpWithPrimary` will only catch up flushed ssts.

The feature request is to also catch up immutable memtables in this case. Catching up should be implemented as reference clone instead of flushing.There are two benifits:

1. data are more up to date than before;

2. it's more efficient than before as no need to apply wal again.

Any suggestions?

[jay.z...@yahoo.com]

How the secondary can access the memtables? Are they have to be on the same host within the same process?

--
You received this message because you are subscribed to the Google Groups "rocksdb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rocksdb+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rocksdb/05ea9058-661f-4ef1-891a-8858b10423e3n%40googlegroups.com.

[Jay Lee]

Yes. `TryCatchUpWithPrimary`, or another method, can accept a pointer, which points to the primary db.

[Yanqin Jin]

Secondary instance must also support accessing the db from another process or even host.

If they are in the same process, curious why not access the primary to get most up-to-date data?

---

From: roc...@googlegroups.com <roc...@googlegroups.com> on behalf of Jay Lee <j...@pingcap.com>
Sent: Tuesday, November 9, 2021 11:39 PM
To: rocksdb <roc...@googlegroups.com>
Subject: Re: feature request: support catching up immutable memtable without wal in secondary instance

 

To view this discussion on the web visit https://groups.google.com/d/msgid/rocksdb/c67ee4d9-4616-4cdf-ae91-e1549f4bc60an%40googlegroups.com.