how to use the existing db

[Si]

Hi,

I tried to first preload a database with 5 million random inserted KVpairs, and then run other benchmarks. But when I use  -use_existing_db=true for the other benchmark, it shows error: open error: IO error: lock hold by current process, acquire time 1605646886 acquiring thread 140121872857792: /users/meditate/test/LOCK: No locks available

The commend lines are from the Fast20 paper's Appendix, like this.

First command creates db:

./db_bench –benchmarks=fillrandom –perf_level=3

-use_direct_io_for_flush_and_compaction=true -

use_direct_reads=true -cache_size=268435456 -key_size=48

-value_size=43 -num=5000000 -db=/users/meditate/si_rocksdb

Second command is prefix_dist benchmark using the existing db

./db_bench –benchmarks="mixgraph" -

use_direct_io_for_flush_and_compaction=true -

use_direct_reads=true -cache_size=268435456

-key_dist_a=0.002312 -key_dist_b=0.3467 -

keyrange_dist_a=14.18 -keyrange_dist_b=-2.917 -

keyrange_dist_c=0.0164 -keyrange_dist_d=-0.08082

-keyrange_num=30 -value_k=0.2615 -value_sigma=25.45

-iter_k=2.517 -iter_sigma=14.236 -mix_get_ratio=0.83

-mix_put_ratio=0.14 -mix_seek_ratio=0.03 -

sine_mix_rate_interval_milliseconds=5000 -sine_a=1000

-sine_b=0.000073 -sine_d=4500 –perf_level=2 -

reads=42000000 -num=5000000 -key_size=48 -db=./<

Directory of Generated Database with 50 million KV-pairs>

-use_existing_db=true

I am not sure when can I use  -use_existing_db=true. Some toy example also shows that lock problem. For example, db_bench --benchmarks="fillrandom" -db=/users/meditate/test --use_existing_db. I checked the wiki, https://github.com/facebook/rocksdb/wiki/Benchmarking-tools The explanation is "-use_existing_db (If true, do not destroy the existing database. If you set this flag and also specify a benchmark that wants a fresh database, that benchmark will fail.) type: bool default: false". In my understanding, "fillrandom" needs a fresh db. Is that correct?

Please help me to make Prefix_dist benchmark work. Thanks a lot!

Best,

Si

[曹志超]

Hi Si,

I think there were some changes in db_bench during the past several months causes the issues. I run the same command (first load and then run mixgraph), and causes the following issues: "db_bench: tools/db_bench_tool.cc:3622: void rocksdb::Benchmark::InitializeOptionsFromFlags(rocksdb::Options*): Assertion `db_.db == nullptr' failed.".

I may spend some time to debug.

Zhichao

Si <medi...@gmail.com> 于2020年11月17日周二 下午1:50写道：

--
You received this message because you are subscribed to the Google Groups "rocksdb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rocksdb+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rocksdb/008a7370-3208-4499-84d5-53f94d35e533n%40googlegroups.com.

[Si]

Thanks for your quick reply, Zhichao. I will stay tuned.