Radhika
unread,Mar 18, 2025, 4:44:35 PMMar 18Sign in to reply to author
Sign in to forward
You do not have permission to delete messages in this group
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to rocksdb
We recently upgraded rocksDB from v5.7.2 to v8.3.2.
On replication, ie, calling GetUpdatesSince, we notice that CPU goes up by a lot(almost 30-40%), on `rocksdb::crc32c::Extend`.
These are x86 machines that have the __SSE42__ define, but not the __PCLMUL__ or NO_THREEWAY_CRC defines.
How do we get crc32c to be sped up for x86 machines on newer rocksDB versions? We have multiple threads that invoke replication, and as a result CPU shoots up by a very large amount.
The CPU perf stack is:
16.25% [.] rocksdb::crc32c::Extend ◆
3.40% [.] ZSTD_compressBlock_doubleFast ▒
1.80% [.] folly::TimeseriesHistogram<long, folly::LegacyStatsClock<std::chrono::duration<long, std::ratio<1l, 1l> > >, folly::MultiLevelTimeSeries<long, folly::LegacyStatsClock<std::chrono::duration<long, std::ratio<1l, 1l> > > > >::addValu▒
0.99% [.] ZSTD_decompressSequences_bmi2 ▒
0.71% [.] HUF_buildCTable_wksp ▒
0.63% [.] ZSTD_encodeSequences_bmi2 ▒
0.60% [.] rocksdb::log::Reader::ReadRecord ▒
0.58% [.] folly::NotificationQueue<folly::Function<void ()> >::Consumer::consumeMessages ▒
0.54% [.] rocksdb::InlineSkipList<rocksdb::MemTableRep::KeyComparator const&>::Insert<false> ▒
0.42% [.] rocksdb::TransactionLogIteratorImpl::UpdateCurrentWriteBatch ▒
0.42% [.] rocksdb::log::Reader::ReadPhysicalRecord ▒
0.37% [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate ▒
0.31% [.] rocksdb::(anonymous namespace)::SkipListRep::Iterator::Seek ▒
0.29% [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace ▒
0.27% [.] rocksdb::WriteBatch::WriteBatch