One more time about multiple processes access...

[kura...@mail.ru]

Good day!

I know that multiple processes access in LevelDB is impossible :-)

First of all - why (technically)?

But what about "1 writer, n readers" scheme? Is it theoretically possible? If not, why? Could it be implemented?

Maybe some forks, wrappers, etc. for this?

Thanks!

[Robert Escriva]

Great question! I haven't picked up the code in awhile (1.17 or 1.18),
but at the time, the architecture was such that a write was first
written to a log on disk, and then inserted into an in-memory skip list
to make that logged value immediately visible to readers. When that log
gets compacted into an SST, a new log (and subsequently, new nodes) are
allocated.

Then, there is also an in-memory representation of how the SSTs (*.ldb
on disk) should be overlaid to construct a coherent view of the data
that behaves as one LevelDB instance.

In order to make LevelDB multi-process, you'd need to figure out a way
to make both of these structures (at least) concurrently accessible.
For a multi-writer scheme, you need to be able to write concurrently and
to maintain the same guarantees as LevelDB, the writes need to hit all
processes concurrently. For a 1:n scheme, you'd need the readers to be
able to follow the latest version of both such that a reader gets a
coherent snapshot of both of the structures. In LevelDB this is a mutex
and a couple reference count changes.

In short, it could be done, but to do it without ending up with a
pattern where one process moderates access and the others have to ping
it for certain critical actions would be quite impressive. And if
you're going to have to effectively go into another process like that,
you might as well build a proper network server on top and up the
threadcount.

-Robert

[rui du]

NICE ANSWER

在 2017年8月20日星期日 UTC+8下午10:23:28，Robert Escriva写道：