Why WAL I/O pattern is considered not suitable for direct I/O?

Roman Leventov

unread,

Jun 13, 2018, 9:16:23 AM6/13/18

to roc...@googlegroups.com

https://github.com/facebook/rocksdb/wiki/Direct-IO says:

> use_direct_io_for_flush_and_compaction and use_direct_reads will only be applied to SST file I/O but not WAL I/O or MANIFEST I/O because the I/O pattern of these files are not suitable for direct I/O

I wonder why WAL I/O pattern is considered not suitable for direct I/O? E. g. PosgreSQL uses O_DIRECT for the WAL: https://github.com/postgres/postgres/blob/e3eb8be77ef82ccc8f87c515f96d01bf7c726ca8/src/backend/access/transam/xlog.c#L10073-L10086

Siying Dong

unread,

Jun 18, 2018, 8:05:57 PM6/18/18

to rocksdb

I removed this claim in the wiki page.

Santhosh Nayak

unread,

Jul 21, 2020, 2:30:57 AM7/21/20

to rocksdb

I want to follow up on this question. Though documentation says O_DIRECT it is not supported for Rocksdb WAL writes, Could someone help me understand the reason behind it?

Yi Wu

unread,

Jul 21, 2020, 4:55:52 PM7/21/20

to Santhosh Nayak, rocksdb

Curious about this as well. One thing I can see is O_DIRECT requires 4k aligned write and WAL write is usually small. Using O_DIRECT will increase the write-amp. But it is not a real problem since the WAL is buffered and written to file in batch when fsync is not used (see writable_file_max_buffer_size).

--
You received this message because you are subscribed to the Google Groups "rocksdb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rocksdb+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rocksdb/2b3e28b5-7198-451b-a163-ae1e72d3f258o%40googlegroups.com.

--

Thanks,

Yi

Yi Wu

unread,

Jul 21, 2020, 4:58:38 PM7/21/20

to Santhosh Nayak, rocksdb

sorry I'm wrong. the WAL is not buffered.

--

Thanks,

Yi

Reply all

Reply to author

Forward