Re: Slow concurrent threads

297 views

Skip to first unread message

MARK CALLAGHAN

unread,

Jul 24, 2018, 2:28:15 PM7/24/18

to knut....@evry.com, MyRocks - RocksDB storage engine for MySQL

In my tests the worst case for a workload like this (data in memory, fetch one row by PK) is that InnoDB can be about 2X faster than MyRocks for QPS. If there are multiple indexes on the table then doing a force index might help. I have many results for MyRocks that can be found via http://smalldatum.blogspot.com/2017/12/myrocks-innodb-and-tokudb-summary.html. The results for sysbench point-query, from in-memory & fast server might be appropriate here. However my tests have used MyRocks from FB MySQL and Percona. I have yet to try MariaDB and I have no idea when I will do another round of perf tests.

2X worse QPS than InnoDB isn't my favorite result, however that is the worst-case. If your queries are more complex then the difference is usually much smaller. The value of MyRocks is less space-amplification and less write-amplification. If you use SSD then MyRocks (with or without compression) will use much less space than InnoDB thanks to less space-amp, and the SSD endurance will be better with MyRocks thanks to less write-amp.

With respect to concurrency, from your results I don't see that as a problem. The QPS for both InnoDB and MyRocks doesn't change when going from 20 to 200 concurrency requests. So both are either doing great or lousy with concurrency. The problem is that queries with MyRocks use more CPU -- at both low and high concurrency.

If you share the following we might have better advice:
* my.cnf
* output from SHOW GLOBAL VARIABLES
* the prefix of $datadir/.rocksdb/LOG which contains more configuration details, this is the RocksDB error/info log

On Tue, Jul 24, 2018 at 12:16 AM, <knut....@evry.com> wrote:

Info about project:
- I am bench marking a jetty java web server.
- I have mad a REST API that returns a JSON formatted text file containing all the contents of a table with 100 rows and some 30 columns of different datatypes.
- I am currently trying to decide between using InnoDB or RocksDB as my storage engine.
- Using 2 identical tables, one stored in a RocksDB and one in an InnoDB. They both contain 500 000 rows and i am extracting the 100 rows based on an INT id set as primary key.
- The select statement is in a stored procedure, which is identical on both databases.
- I am tracking the time in milliseconds it takes for the querry to execute using this code:
- The REST API uses the threadpool native to jetty.
- Each thread gets its own connection to the database from a connection pool.
- I have set the max number of threads and connections to 200 and i have confirmed that this works.
- The connection pool is managed by HikariCP
- The results shown below are from a standard MariaDB config (Program Files\MariaDB 10.3\data\my.ini) file.

Size of identical tables in different engines:
- InnoDB: 362Mib
- RocksDB: 236Mib

Code used to track the time executing a query takes:
```
long millis = System.currentTimeMillis();
preparedQuery.executeQuery();
long queryTime = System.currentTimeMillis() - millis;
System.out.println("Time spent to execute sql querry without storing it: " + queryTime + " ms");
```
Benchmarks:
(The following results are consistent over several tests)

The log output when using InnoDB when benchmarking 20 concurrent requests:
![image](https://user-images.githubusercontent.com/16221324/43073314-696e3c04-8e79-11e8-87c7-a3a77c882a86.png)
Throughput of the RESTAPI is now 2975 request/s

The log output when using InnoDB when benchmarking 200 concurrent requests:
![image](https://user-images.githubusercontent.com/16221324/43075344-d16ef09e-8e80-11e8-8c78-af468b0b9531.png)
Throughput of the RESTAPI is now 2911 request/s

The log output when using RocksDB when benchmarking 20 concurrent requests:
![image](https://user-images.githubusercontent.com/16221324/43073528-3d2e890e-8e7a-11e8-9359-302065db4164.png)
Throughput of the RESTAPI is now 810 request/s

The log output when using RocksDB when benchmarking 200 concurrent requests:
![image](https://user-images.githubusercontent.com/16221324/43073595-806ee63c-8e7a-11e8-9db4-9ba69ab8013c.png)
Throughput of the RESTAPI is now 750 request/s

As you can see, RocksDB seems to handle many concurrent reads very poorly, compared to InnoDB wich is consistent throughout most of the process.
I have been browsing the web for days trying to find some reasonable solutions to this, but have come up with nothing as most of the documentation i have found refers to old versions of RocksDB.

- Can someone explain why i see such an extensive preformance reduction when using RocksDB compared to InnoDB?
- Can someone refer me to a propper tutorial about tuning the config of RocksDB for concurrency and high read speeds?

--
You received this message because you are subscribed to the Google Groups "MyRocks - RocksDB storage engine for MySQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to myrocks-dev+unsubscribe@googlegroups.com.
To post to this group, send email to myroc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/myrocks-dev/e091841c-dfca-414d-816b-2478a396f85e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Mark Callaghan
mdca...@gmail.com

knut....@evry.com

unread,

Jul 25, 2018, 3:35:38 AM7/25/18

to MyRocks - RocksDB storage engine for MySQL

my.ini:

[mysqld]
datadir=C:/Program Files/MariaDB 10.3/data
port=3306
innodb_buffer_pool_size=4075M
max_connections = 10000


[client]
port=3306
plugin-dir=C:/Program Files/MariaDB 10.3/lib/plugin
 
global:
Variable_name;Value
alter_algorithm;DEFAULT
aria_block_size;8192
aria_checkpoint_interval;30
aria_checkpoint_log_activity;1048576
aria_encrypt_tables;OFF
aria_force_start_after_recovery_failures;0
aria_group_commit;none
aria_group_commit_interval;0
aria_log_file_size;1073741824
aria_log_purge_type;immediate
aria_max_sort_file_size;9223372036853727232
aria_page_checksum;ON
aria_pagecache_age_threshold;300
aria_pagecache_buffer_size;134217728
aria_pagecache_division_limit;100
aria_pagecache_file_hash_size;512
aria_recover_options;BACKUP,QUICK
aria_repair_threads;1
aria_sort_buffer_size;268434432
aria_stats_method;nulls_unequal
aria_sync_log_dir;NEWFILE
aria_used_for_temp_tables;ON
auto_increment_increment;1
auto_increment_offset;1
autocommit;ON
automatic_sp_privileges;ON
back_log;900
basedir;C:\Program Files\MariaDB 10.3\
big_tables;OFF
bind_address;
binlog_annotate_row_events;ON
binlog_cache_size;32768
binlog_checksum;CRC32
binlog_commit_wait_count;0
binlog_commit_wait_usec;100000
binlog_direct_non_transactional_updates;OFF
binlog_file_cache_size;16384
binlog_format;MIXED
binlog_optimize_thread_scheduling;ON
binlog_row_image;FULL
binlog_stmt_cache_size;32768
bulk_insert_buffer_size;8388608
character_set_client;latin1
character_set_connection;latin1
character_set_database;latin1
character_set_filesystem;binary
character_set_results;latin1
character_set_server;latin1
character_set_system;utf8
character_sets_dir;C:\Program Files\MariaDB 10.3\share\charsets\
check_constraint_checks;ON
collation_connection;latin1_swedish_ci
collation_database;latin1_swedish_ci
collation_server;latin1_swedish_ci
column_compression_threshold;100
column_compression_zlib_level;6
column_compression_zlib_strategy;DEFAULT_STRATEGY
column_compression_zlib_wrap;OFF
completion_type;NO_CHAIN
concurrent_insert;AUTO
connect_timeout;10
datadir;C:\Program Files\MariaDB 10.3\data\
date_format;%Y-%m-%d
datetime_format;%Y-%m-%d %H:%i:%s
deadlock_search_depth_long;15
deadlock_search_depth_short;4
deadlock_timeout_long;50000000
deadlock_timeout_short;10000
debug_no_thread_alarm;OFF
default_regex_flags;
default_storage_engine;InnoDB
default_tmp_storage_engine;
default_week_format;0
delay_key_write;ON
delayed_insert_limit;100
delayed_insert_timeout;300
delayed_queue_size;1000
div_precision_increment;4
encrypt_binlog;OFF
encrypt_tmp_disk_tables;OFF
encrypt_tmp_files;OFF
enforce_storage_engine;
event_scheduler;OFF
expensive_subquery_limit;100
expire_logs_days;0
explicit_defaults_for_timestamp;OFF
extra_max_connections;1
extra_port;0
flush;OFF
flush_time;0
foreign_key_checks;ON
ft_boolean_syntax;+ -><()~*:""&|
ft_max_word_len;84
ft_min_word_len;4
ft_query_expansion_limit;20
ft_stopword_file;(built-in)
general_log;OFF
general_log_file;PC36166.log
group_concat_max_len;1048576
gtid_binlog_pos;
gtid_binlog_state;
gtid_current_pos;
gtid_domain_id;0
gtid_ignore_duplicates;OFF
gtid_pos_auto_engines;
gtid_slave_pos;
gtid_strict_mode;OFF
have_compress;YES
have_crypt;NO
have_dynamic_loading;YES
have_geometry;YES
have_openssl;NO
have_profiling;YES
have_query_cache;YES
have_rtree_keys;YES
have_ssl;DISABLED
have_symlink;NO
histogram_size;0
histogram_type;SINGLE_PREC_HB
host_cache_size;1103
hostname;PC36166
idle_readonly_transaction_timeout;0
idle_transaction_timeout;0
idle_write_transaction_timeout;0
ignore_builtin_innodb;OFF
ignore_db_dirs;#rocksdb
init_connect;
init_file;
init_slave;
innodb_adaptive_flushing;ON
innodb_adaptive_flushing_lwm;10.000000
innodb_adaptive_hash_index;ON
innodb_adaptive_hash_index_parts;8
innodb_adaptive_max_sleep_delay;150000
innodb_autoextend_increment;64
innodb_autoinc_lock_mode;1
innodb_background_scrub_data_check_interval;3600
innodb_background_scrub_data_compressed;OFF
innodb_background_scrub_data_interval;604800
innodb_background_scrub_data_uncompressed;OFF
innodb_buf_dump_status_frequency;0
innodb_buffer_pool_chunk_size;134217728
innodb_buffer_pool_dump_at_shutdown;ON
innodb_buffer_pool_dump_now;OFF
innodb_buffer_pool_dump_pct;25
innodb_buffer_pool_filename;ib_buffer_pool
innodb_buffer_pool_instances;8
innodb_buffer_pool_load_abort;OFF
innodb_buffer_pool_load_at_startup;ON
innodb_buffer_pool_load_now;OFF
innodb_buffer_pool_size;4294967296
innodb_change_buffer_max_size;25
innodb_change_buffering;all
innodb_checksum_algorithm;crc32
innodb_checksums;ON
innodb_cmp_per_index_enabled;OFF
innodb_commit_concurrency;0
innodb_compression_algorithm;zlib
innodb_compression_default;OFF
innodb_compression_failure_threshold_pct;5
innodb_compression_level;6
innodb_compression_pad_pct_max;50
innodb_concurrency_tickets;5000
innodb_data_file_path;ibdata1:12M:autoextend
innodb_data_home_dir;
innodb_deadlock_detect;ON
innodb_default_encryption_key_id;1
innodb_default_row_format;dynamic
innodb_defragment;OFF
innodb_defragment_fill_factor;0.900000
innodb_defragment_fill_factor_n_recs;20
innodb_defragment_frequency;40
innodb_defragment_n_pages;7
innodb_defragment_stats_accuracy;0
innodb_disable_sort_file_cache;OFF
innodb_doublewrite;ON
innodb_encrypt_log;OFF
innodb_encrypt_tables;OFF
innodb_encryption_rotate_key_age;1
innodb_encryption_rotation_iops;100
innodb_encryption_threads;0
innodb_fast_shutdown;1
innodb_fatal_semaphore_wait_threshold;600
innodb_file_per_table;ON
innodb_fill_factor;100
innodb_flush_log_at_timeout;1
innodb_flush_log_at_trx_commit;1
innodb_flush_method;unbuffered
innodb_flush_neighbors;1
innodb_flush_sync;ON
innodb_flushing_avg_loops;30
innodb_force_load_corrupted;OFF
innodb_force_primary_key;OFF
innodb_force_recovery;0
innodb_ft_aux_table;
innodb_ft_cache_size;8000000
innodb_ft_enable_diag_print;OFF
innodb_ft_enable_stopword;ON
innodb_ft_max_token_size;84
innodb_ft_min_token_size;3
innodb_ft_num_word_optimize;2000
innodb_ft_result_cache_limit;2000000000
innodb_ft_server_stopword_table;
innodb_ft_sort_pll_degree;2
innodb_ft_total_cache_size;640000000
innodb_ft_user_stopword_table;
innodb_idle_flush_pct;100
innodb_immediate_scrub_data_uncompressed;OFF
innodb_io_capacity;200
innodb_io_capacity_max;2000
innodb_lock_schedule_algorithm;vats
innodb_lock_wait_timeout;50
innodb_locks_unsafe_for_binlog;OFF
innodb_log_buffer_size;16777216
innodb_log_checksums;ON
innodb_log_compressed_pages;ON
innodb_log_file_size;50331648
innodb_log_files_in_group;2
innodb_log_group_home_dir;.\
innodb_log_write_ahead_size;8192
innodb_lru_scan_depth;1024
innodb_max_dirty_pages_pct;75.000000
innodb_max_dirty_pages_pct_lwm;0.000000
innodb_max_purge_lag;0
innodb_max_purge_lag_delay;0
innodb_max_undo_log_size;10485760
innodb_monitor_disable;
innodb_monitor_enable;
innodb_monitor_reset;
innodb_monitor_reset_all;
innodb_old_blocks_pct;37
innodb_old_blocks_time;1000
innodb_online_alter_log_max_size;134217728
innodb_open_files;2000
innodb_optimize_fulltext_only;OFF
innodb_page_cleaners;4
innodb_page_size;16384
innodb_prefix_index_cluster_optimization;OFF
innodb_print_all_deadlocks;OFF
innodb_purge_batch_size;300
innodb_purge_rseg_truncate_frequency;128
innodb_purge_threads;4
innodb_random_read_ahead;OFF
innodb_read_ahead_threshold;56
innodb_read_io_threads;4
innodb_read_only;OFF
innodb_replication_delay;0
innodb_rollback_on_timeout;OFF
innodb_rollback_segments;128
innodb_scrub_log;OFF
innodb_scrub_log_speed;256
innodb_sort_buffer_size;1048576
innodb_spin_wait_delay;4
innodb_stats_auto_recalc;ON
innodb_stats_include_delete_marked;OFF
innodb_stats_method;nulls_equal
innodb_stats_modified_counter;0
innodb_stats_on_metadata;OFF
innodb_stats_persistent;ON
innodb_stats_persistent_sample_pages;20
innodb_stats_sample_pages;8
innodb_stats_traditional;ON
innodb_stats_transient_sample_pages;8
innodb_status_output;OFF
innodb_status_output_locks;OFF
innodb_strict_mode;ON
innodb_sync_array_size;1
innodb_sync_spin_loops;30
innodb_table_locks;ON
innodb_temp_data_file_path;ibtmp1:12M:autoextend
innodb_thread_concurrency;0
innodb_thread_sleep_delay;10000
innodb_tmpdir;
innodb_undo_directory;.\
innodb_undo_log_truncate;OFF
innodb_undo_logs;128
innodb_undo_tablespaces;0
innodb_use_atomic_writes;ON
innodb_use_native_aio;ON
innodb_version;10.3.8
innodb_write_io_threads;4
interactive_timeout;28800
join_buffer_size;262144
join_buffer_space_limit;2097152
join_cache_level;2
keep_files_on_create;OFF
key_buffer_size;134217728
key_cache_age_threshold;300
key_cache_block_size;1024
key_cache_division_limit;100
key_cache_file_hash_size;512
key_cache_segments;0
large_files_support;ON
large_page_size;0
large_pages;OFF
lc_messages;en_US
lc_messages_dir;
lc_time_names;en_US
license;GPL
local_infile;ON
lock_wait_timeout;86400
log_bin;OFF
log_bin_basename;
log_bin_compress;OFF
log_bin_compress_min_len;256
log_bin_index;
log_bin_trust_function_creators;OFF
log_disabled_statements;sp
log_error;C:\Program Files\MariaDB 10.3\data\PC36166.err
log_output;FILE
log_queries_not_using_indexes;OFF
log_slave_updates;OFF
log_slow_admin_statements;ON
log_slow_disabled_statements;sp
log_slow_filter;admin,filesort,filesort_on_disk,filesort_priority_queue,full_join,full_scan,query_cache,query_cache_miss,tmp_table,tmp_table_on_disk
log_slow_rate_limit;1
log_slow_slave_statements;ON
log_slow_verbosity;
log_tc_size;24576
log_warnings;2
long_query_time;10.000000
low_priority_updates;OFF
lower_case_file_system;ON
lower_case_table_names;1
master_verify_checksum;OFF
max_allowed_packet;16777216
max_binlog_cache_size;18446744073709547520
max_binlog_size;1073741824
max_binlog_stmt_cache_size;18446744073709547520
max_connect_errors;100
max_connections;10000
max_delayed_threads;20
max_digest_length;1024
max_error_count;64
max_heap_table_size;16777216
max_insert_delayed_threads;20
max_join_size;18446744073709551615
max_length_for_sort_data;1024
max_long_data_size;16777216
max_prepared_stmt_count;16382
max_recursive_iterations;4294967295
max_relay_log_size;1073741824
max_seeks_for_key;4294967295
max_session_mem_used;9223372036854775807
max_sort_length;1024
max_sp_recursion_depth;0
max_statement_time;0.000000
max_tmp_tables;32
max_user_connections;0
max_write_lock_count;4294967295
metadata_locks_cache_size;1024
metadata_locks_hash_instances;8
min_examined_row_limit;0
mrr_buffer_size;262144
multi_range_count;256
myisam_block_size;1024
myisam_data_pointer_size;6
myisam_max_sort_file_size;2146435072
myisam_mmap_size;18446744073709551615
myisam_recover_options;BACKUP,QUICK
myisam_repair_threads;1
myisam_sort_buffer_size;134216704
myisam_stats_method;NULLS_UNEQUAL
myisam_use_mmap;OFF
mysql56_temporal_format;ON
named_pipe;OFF
net_buffer_length;16384
net_read_timeout;30
net_retry_count;10
net_write_timeout;60
old;OFF
old_alter_table;DEFAULT
old_mode;
old_passwords;OFF
open_files_limit;52053
optimizer_prune_level;1
optimizer_search_depth;62
optimizer_selectivity_sampling_limit;100
optimizer_switch;index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
optimizer_use_condition_selectivity;1
performance_schema;OFF
performance_schema_accounts_size;-1
performance_schema_digests_size;-1
performance_schema_events_stages_history_long_size;-1
performance_schema_events_stages_history_size;-1
performance_schema_events_statements_history_long_size;-1
performance_schema_events_statements_history_size;-1
performance_schema_events_waits_history_long_size;-1
performance_schema_events_waits_history_size;-1
performance_schema_hosts_size;-1
performance_schema_max_cond_classes;80
performance_schema_max_cond_instances;-1
performance_schema_max_digest_length;1024
performance_schema_max_file_classes;50
performance_schema_max_file_handles;32768
performance_schema_max_file_instances;-1
performance_schema_max_mutex_classes;200
performance_schema_max_mutex_instances;-1
performance_schema_max_rwlock_classes;40
performance_schema_max_rwlock_instances;-1
performance_schema_max_socket_classes;10
performance_schema_max_socket_instances;-1
performance_schema_max_stage_classes;160
performance_schema_max_statement_classes;200
performance_schema_max_table_handles;-1
performance_schema_max_table_instances;-1
performance_schema_max_thread_classes;50
performance_schema_max_thread_instances;-1
performance_schema_session_connect_attrs_size;-1
performance_schema_setup_actors_size;100
performance_schema_setup_objects_size;100
performance_schema_users_size;-1
pid_file;C:\Program Files\MariaDB 10.3\data\PC36166.pid
plugin_dir;C:\Program Files\MariaDB 10.3\lib\plugin\
plugin_maturity;gamma
port;3306
preload_buffer_size;32768
profiling;OFF
profiling_history_size;15
progress_report_time;5
protocol_version;10
proxy_protocol_networks;
query_alloc_block_size;16384
query_cache_limit;1048576
query_cache_min_res_unit;4096
query_cache_size;1048576
query_cache_strip_comments;OFF
query_cache_type;OFF
query_cache_wlock_invalidate;OFF
query_prealloc_size;24576
range_alloc_block_size;4096
read_binlog_speed_limit;0
read_buffer_size;131072
read_only;OFF
read_rnd_buffer_size;262144
relay_log;
relay_log_basename;
relay_log_index;
relay_log_info_file;relay-log.info
relay_log_purge;ON
relay_log_recovery;OFF
relay_log_space_limit;0
replicate_annotate_row_events;ON
replicate_do_db;
replicate_do_table;
replicate_events_marked_for_skip;REPLICATE
replicate_ignore_db;
replicate_ignore_table;
replicate_wild_do_table;
replicate_wild_ignore_table;
report_host;
report_password;
report_port;3306
report_user;
rocksdb_access_hint_on_compaction_start;1
rocksdb_advise_random_on_open;ON
rocksdb_allow_concurrent_memtable_write;OFF
rocksdb_allow_mmap_reads;OFF
rocksdb_allow_mmap_writes;OFF
rocksdb_allow_to_start_after_corruption;OFF
rocksdb_blind_delete_primary_key;OFF
rocksdb_block_cache_size;536870912
rocksdb_block_restart_interval;16
rocksdb_block_size;4096
rocksdb_block_size_deviation;10
rocksdb_bulk_load;OFF
rocksdb_bulk_load_allow_unsorted;OFF
rocksdb_bulk_load_size;1000
rocksdb_bytes_per_sync;0
rocksdb_cache_index_and_filter_blocks;ON
rocksdb_checksums_pct;100
rocksdb_collect_sst_properties;ON
rocksdb_commit_in_the_middle;OFF
rocksdb_compact_cf;
rocksdb_compaction_readahead_size;0
rocksdb_compaction_sequential_deletes;0
rocksdb_compaction_sequential_deletes_count_sd;OFF
rocksdb_compaction_sequential_deletes_file_size;0
rocksdb_compaction_sequential_deletes_window;0
rocksdb_create_checkpoint;
rocksdb_create_if_missing;ON
rocksdb_create_missing_column_families;OFF
rocksdb_datadir;./#rocksdb
rocksdb_db_write_buffer_size;0
rocksdb_deadlock_detect;OFF
rocksdb_deadlock_detect_depth;50
rocksdb_debug_optimizer_no_zero_cardinality;ON
rocksdb_debug_ttl_ignore_pk;OFF
rocksdb_debug_ttl_read_filter_ts;0
rocksdb_debug_ttl_rec_ts;0
rocksdb_debug_ttl_snapshot_ts;0
rocksdb_default_cf_options;
rocksdb_delayed_write_rate;0
rocksdb_delete_obsolete_files_period_micros;21600000000
rocksdb_enable_2pc;ON
rocksdb_enable_bulk_load_api;ON
rocksdb_enable_thread_tracking;ON
rocksdb_enable_ttl;ON
rocksdb_enable_ttl_read_filtering;ON
rocksdb_enable_write_thread_adaptive_yield;OFF
rocksdb_error_if_exists;OFF
rocksdb_flush_log_at_trx_commit;1
rocksdb_force_compute_memtable_stats;ON
rocksdb_force_compute_memtable_stats_cachetime;60000000
rocksdb_force_flush_memtable_and_lzero_now;OFF
rocksdb_force_flush_memtable_now;OFF
rocksdb_force_index_records_in_range;0
rocksdb_git_hash;ba295cda29daee3ffe58549542804efdfd969784
rocksdb_hash_index_allow_collision;ON
rocksdb_ignore_unknown_options;ON
rocksdb_index_type;kBinarySearch
rocksdb_info_log_level;error_level
rocksdb_io_write_timeout;0
rocksdb_is_fd_close_on_exec;ON
rocksdb_keep_log_file_num;1000
rocksdb_large_prefix;OFF
rocksdb_lock_scanned_rows;OFF
rocksdb_lock_wait_timeout;1
rocksdb_log_file_time_to_roll;0
rocksdb_manifest_preallocation_size;4194304
rocksdb_manual_wal_flush;ON
rocksdb_master_skip_tx_api;OFF
rocksdb_max_background_jobs;2
rocksdb_max_latest_deadlocks;5
rocksdb_max_log_file_size;0
rocksdb_max_manifest_file_size;18446744073709551615
rocksdb_max_open_files;26026
rocksdb_max_row_locks;1048576
rocksdb_max_subcompactions;1
rocksdb_max_total_wal_size;0
rocksdb_merge_buf_size;67108864
rocksdb_merge_combine_read_size;1073741824
rocksdb_merge_tmp_file_removal_delay_ms;0
rocksdb_new_table_reader_for_compaction_inputs;OFF
rocksdb_no_block_cache;OFF
rocksdb_override_cf_options;
rocksdb_paranoid_checks;ON
rocksdb_pause_background_work;OFF
rocksdb_perf_context_level;0
rocksdb_persistent_cache_path;
rocksdb_persistent_cache_size_mb;0
rocksdb_pin_l0_filter_and_index_blocks_in_cache;ON
rocksdb_print_snapshot_conflict_queries;OFF
rocksdb_rate_limiter_bytes_per_sec;0
rocksdb_read_free_rpl_tables;
rocksdb_records_in_range;0
rocksdb_remove_mariabackup_checkpoint;OFF
rocksdb_reset_stats;OFF
rocksdb_seconds_between_stat_computes;3600
rocksdb_signal_drop_index_thread;OFF
rocksdb_sim_cache_size;0
rocksdb_skip_bloom_filter_on_read;OFF
rocksdb_skip_fill_cache;OFF
rocksdb_skip_unique_check_tables;.*
rocksdb_sst_mgr_rate_bytes_per_sec;0
rocksdb_stats_dump_period_sec;600
rocksdb_store_row_debug_checksums;OFF
rocksdb_strict_collation_check;ON
rocksdb_strict_collation_exceptions;
rocksdb_supported_compression_types;Zlib
rocksdb_table_cache_numshardbits;6
rocksdb_table_stats_sampling_pct;10
rocksdb_tmpdir;
rocksdb_trace_sst_api;OFF
rocksdb_two_write_queues;ON
rocksdb_unsafe_for_binlog;OFF
rocksdb_update_cf_options;
rocksdb_use_adaptive_mutex;OFF
rocksdb_use_clock_cache;OFF
rocksdb_use_direct_io_for_flush_and_compaction;OFF
rocksdb_use_direct_reads;OFF
rocksdb_use_fsync;OFF
rocksdb_validate_tables;1
rocksdb_verify_row_debug_checksums;OFF
rocksdb_wal_bytes_per_sync;0
rocksdb_wal_dir;
rocksdb_wal_recovery_mode;1
rocksdb_wal_size_limit_mb;0
rocksdb_wal_ttl_seconds;0
rocksdb_whole_key_filtering;ON
rocksdb_write_batch_max_bytes;0
rocksdb_write_disable_wal;OFF
rocksdb_write_ignore_missing_column_families;OFF
rowid_merge_buff_size;8388608
rpl_semi_sync_master_enabled;OFF
rpl_semi_sync_master_timeout;10000
rpl_semi_sync_master_trace_level;32
rpl_semi_sync_master_wait_no_slave;ON
rpl_semi_sync_master_wait_point;AFTER_COMMIT
rpl_semi_sync_slave_delay_master;OFF
rpl_semi_sync_slave_enabled;OFF
rpl_semi_sync_slave_kill_conn_timeout;5
rpl_semi_sync_slave_trace_level;32
secure_auth;ON
secure_file_priv;
secure_timestamp;NO
server_id;1
session_track_schema;ON
session_track_state_change;OFF
session_track_system_variables;autocommit,character_set_client,character_set_connection,character_set_results,time_zone
session_track_transaction_info;OFF
shared_memory;OFF
shared_memory_base_name;MYSQL
skip_external_locking;ON
skip_name_resolve;OFF
skip_networking;OFF
skip_show_database;OFF
slave_compressed_protocol;OFF
slave_ddl_exec_mode;IDEMPOTENT
slave_domain_parallel_threads;0
slave_exec_mode;STRICT
slave_load_tmpdir;C:\windows\SERVIC~2\NETWOR~1\AppData\Local\Temp
slave_max_allowed_packet;1073741824
slave_net_timeout;60
slave_parallel_max_queued;131072
slave_parallel_mode;conservative
slave_parallel_threads;0
slave_parallel_workers;0
slave_run_triggers_for_rbr;NO
slave_skip_errors;OFF
slave_sql_verify_checksum;ON
slave_transaction_retries;10
slave_transaction_retry_errors;1213,1205
slave_transaction_retry_interval;0
slave_type_conversions;
slow_launch_time;2
slow_query_log;OFF
slow_query_log_file;PC36166-slow.log
socket;MySQL
sort_buffer_size;2097152
sql_auto_is_null;OFF
sql_big_selects;ON
sql_buffer_result;OFF
sql_log_bin;ON
sql_log_off;OFF
sql_mode;STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
sql_notes;ON
sql_quote_show_create;ON
sql_safe_updates;OFF
sql_select_limit;18446744073709551615
sql_slave_skip_counter;0
sql_warnings;OFF
ssl_ca;
ssl_capath;
ssl_cert;
ssl_cipher;
ssl_crl;
ssl_crlpath;
ssl_key;
standard_compliant_cte;ON
storage_engine;InnoDB
stored_program_cache;256
strict_password_validation;ON
sync_binlog;0
sync_frm;ON
sync_master_info;10000
sync_relay_log;10000
sync_relay_log_info;10000
system_time_zone;W. Europe Daylight Time
system_versioning_alter_history;ERROR
system_versioning_asof;DEFAULT
table_definition_cache;400
table_open_cache;2000
table_open_cache_instances;8
tcp_keepalive_interval;0
tcp_keepalive_probes;0
tcp_keepalive_time;0
thread_cache_size;256
thread_concurrency;10
thread_handling;pool-of-threads
thread_pool_idle_timeout;60
thread_pool_max_threads;65536
thread_pool_min_threads;1
thread_pool_mode;windows
thread_pool_oversubscribe;3
thread_pool_prio_kickup_timer;1000
thread_pool_priority;auto
thread_pool_size;8
thread_pool_stall_limit;500
thread_stack;299008
time_format;%H:%i:%s
time_zone;SYSTEM
timed_mutexes;OFF
tmp_disk_table_size;18446744073709551615
tmp_memory_table_size;16777216
tmp_table_size;16777216
tmpdir;C:\windows\SERVIC~2\NETWOR~1\AppData\Local\Temp
transaction_alloc_block_size;8192
transaction_prealloc_size;4096
tx_isolation;REPEATABLE-READ
tx_read_only;OFF
unique_checks;ON
updatable_views_with_limit;YES
use_stat_tables;NEVER
userstat;OFF
version;10.3.8-MariaDB
version_comment;mariadb.org binary distribution
version_compile_machine;x64
version_compile_os;Win64
version_malloc_library;system
version_source_revision;eaab98f7022732b26b4ea590e1bb44308e253be8
version_ssl_library;YaSSL 2.4.4
wait_timeout;28800

Message has been deleted

knut....@evry.com

unread,

Jul 25, 2018, 3:38:18 AM7/25/18

to MyRocks - RocksDB storage engine for MySQL

log

2018/07/25-09:12:10.225315 16a4 [WARN] [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\utilities\transactions\pessimistic_transaction_db.cc:191] Transaction write_policy is d

2018/07/25-09:12:10.226974 16a4 RocksDB version: 5.10.0

2018/07/25-09:12:10.227007 16a4 Git sha rocksdb_build_git_sha:@@

2018/07/25-09:12:10.227023 16a4 Compile date Jul 2 2018

2018/07/25-09:12:10.227038 16a4 DB SUMMARY

2018/07/25-09:12:10.230179 16a4 CURRENT file: CURRENT

2018/07/25-09:12:10.230209 16a4 IDENTITY file: IDENTITY

2018/07/25-09:12:10.230268 16a4 MANIFEST file: MANIFEST-000085 size: 795 Bytes

2018/07/25-09:12:10.230287 16a4 SST files in ./#rocksdb dir, Total Num: 10, files: 000027.sst 000028.sst 000029.sst 000031.sst 000033.sst 000036.sst 000050.sst 000061.sst 000072.sst

2018/07/25-09:12:10.232190 16a4 Write Ahead Log file in ./#rocksdb: 000086.log size: 74 ;

2018/07/25-09:12:10.232209 16a4 Options.error_if_exists: 0

2018/07/25-09:12:10.232218 16a4 Options.create_if_missing: 1

2018/07/25-09:12:10.232224 16a4 Options.paranoid_checks: 1

2018/07/25-09:12:10.232229 16a4 Options.env: 00000255455C0E00

2018/07/25-09:12:10.232235 16a4 Options.info_log: 000002554D8C1B10

2018/07/25-09:12:10.232244 16a4 Options.max_file_opening_threads: 16

2018/07/25-09:12:10.232250 16a4 Options.statistics: 0000025544C7EDF0

2018/07/25-09:12:10.232256 16a4 Options.use_fsync: 0

2018/07/25-09:12:10.232262 16a4 Options.max_log_file_size: 0

2018/07/25-09:12:10.232268 16a4 Options.max_manifest_file_size: 18446744073709551615

2018/07/25-09:12:10.232274 16a4 Options.log_file_time_to_roll: 0

2018/07/25-09:12:10.232280 16a4 Options.keep_log_file_num: 1000

2018/07/25-09:12:10.232286 16a4 Options.recycle_log_file_num: 0

2018/07/25-09:12:10.232292 16a4 Options.allow_fallocate: 1

2018/07/25-09:12:10.232298 16a4 Options.allow_mmap_reads: 0

2018/07/25-09:12:10.232303 16a4 Options.allow_mmap_writes: 0

2018/07/25-09:12:10.232309 16a4 Options.use_direct_reads: 0

2018/07/25-09:12:10.232315 16a4 Options.use_direct_io_for_flush_and_compaction: 0

2018/07/25-09:12:10.232331 16a4 Options.create_missing_column_families: 0

2018/07/25-09:12:10.232338 16a4 Options.db_log_dir:

2018/07/25-09:12:10.232344 16a4 Options.wal_dir: ./#rocksdb

2018/07/25-09:12:10.232350 16a4 Options.table_cache_numshardbits: 6

2018/07/25-09:12:10.232355 16a4 Options.max_subcompactions: 1

2018/07/25-09:12:10.232362 16a4 Options.max_background_flushes: -1

2018/07/25-09:12:10.232367 16a4 Options.WAL_ttl_seconds: 0

2018/07/25-09:12:10.232373 16a4 Options.WAL_size_limit_MB: 0

2018/07/25-09:12:10.232379 16a4 Options.manifest_preallocation_size: 4194304

2018/07/25-09:12:10.232385 16a4 Options.is_fd_close_on_exec: 1

2018/07/25-09:12:10.232390 16a4 Options.advise_random_on_open: 1

2018/07/25-09:12:10.232397 16a4 Options.db_write_buffer_size: 0

2018/07/25-09:12:10.232403 16a4 Options.write_buffer_manager: 000002554D8D2050

2018/07/25-09:12:10.232409 16a4 Options.access_hint_on_compaction_start: 1

2018/07/25-09:12:10.232414 16a4 Options.new_table_reader_for_compaction_inputs: 0

2018/07/25-09:12:10.232420 16a4 Options.random_access_max_buffer_size: 1048576

2018/07/25-09:12:10.232426 16a4 Options.use_adaptive_mutex: 0

2018/07/25-09:12:10.232431 16a4 Options.rate_limiter: 0000000000000000

2018/07/25-09:12:10.232440 16a4 Options.sst_file_manager.rate_bytes_per_sec: 0

2018/07/25-09:12:10.232447 16a4 Options.wal_recovery_mode: 1

2018/07/25-09:12:10.232490 16a4 Options.enable_thread_tracking: 1

2018/07/25-09:12:10.232497 16a4 Options.enable_pipelined_write: 0

2018/07/25-09:12:10.232503 16a4 Options.allow_concurrent_memtable_write: 0

2018/07/25-09:12:10.232509 16a4 Options.enable_write_thread_adaptive_yield: 0

2018/07/25-09:12:10.232514 16a4 Options.write_thread_max_yield_usec: 100

2018/07/25-09:12:10.232520 16a4 Options.write_thread_slow_yield_usec: 3

2018/07/25-09:12:10.232526 16a4 Options.row_cache: None

2018/07/25-09:12:10.232531 16a4 Options.wal_filter: None

2018/07/25-09:12:10.232537 16a4 Options.avoid_flush_during_recovery: 0

2018/07/25-09:12:10.232543 16a4 Options.allow_ingest_behind: 0

2018/07/25-09:12:10.232548 16a4 Options.preserve_deletes: 0

2018/07/25-09:12:10.232554 16a4 Options.two_write_queues: 1

2018/07/25-09:12:10.232562 16a4 Options.manual_wal_flush: 1

2018/07/25-09:12:10.232568 16a4 Options.max_background_jobs: 2

2018/07/25-09:12:10.232576 16a4 Options.max_background_compactions: -1

2018/07/25-09:12:10.232582 16a4 Options.avoid_flush_during_shutdown: 0

2018/07/25-09:12:10.232588 16a4 Options.writable_file_max_buffer_size: 1048576

2018/07/25-09:12:10.232594 16a4 Options.delayed_write_rate : 16777216

2018/07/25-09:12:10.232599 16a4 Options.max_total_wal_size: 0

2018/07/25-09:12:10.232605 16a4 Options.delete_obsolete_files_period_micros: 21600000000

2018/07/25-09:12:10.232610 16a4 Options.stats_dump_period_sec: 600

2018/07/25-09:12:10.232616 16a4 Options.max_open_files: 26026

2018/07/25-09:12:10.232622 16a4 Options.bytes_per_sync: 0

2018/07/25-09:12:10.232627 16a4 Options.wal_bytes_per_sync: 0

2018/07/25-09:12:10.232633 16a4 Options.compaction_readahead_size: 0

2018/07/25-09:12:10.232639 16a4 Compression algorithms supported:

2018/07/25-09:12:10.232647 16a4 Snappy supported: 0

2018/07/25-09:12:10.232652 16a4 Zlib supported: 1

2018/07/25-09:12:10.232657 16a4 Bzip supported: 0

2018/07/25-09:12:10.232662 16a4 LZ4 supported: 0

2018/07/25-09:12:10.232667 16a4 ZSTDNotFinal supported: 0

2018/07/25-09:12:10.232672 16a4 ZSTD supported: 0

2018/07/25-09:12:10.232679 16a4 Fast CRC32 supported: Not supported on x86

2018/07/25-09:12:10.236748 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\version_set.cc:2834] Recovering from manifest file: MANIFEST-000085

2018/07/25-09:12:10.251312 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\column_family.cc:429] --------------- Options for column family [default]:

2018/07/25-09:12:10.251352 16a4 Options.comparator: RocksDB_SE_v3.10

2018/07/25-09:12:10.251359 16a4 Options.merge_operator: None

2018/07/25-09:12:10.251364 16a4 Options.compaction_filter: None

2018/07/25-09:12:10.251370 16a4 Options.compaction_filter_factory: Rdb_compact_filter_factory

2018/07/25-09:12:10.251377 16a4 Options.memtable_factory: SkipListFactory

2018/07/25-09:12:10.251382 16a4 Options.table_factory: BlockBasedTable

2018/07/25-09:12:10.251457 16a4 table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0000025544B911A0)

cache_index_and_filter_blocks: 1

cache_index_and_filter_blocks_with_high_priority: 0

pin_l0_filter_and_index_blocks_in_cache: 1

index_type: 0

hash_index_allow_collision: 1

checksum: 1

no_block_cache: 0

block_cache: 0000025544C73D50

block_cache_name: LRUCache

block_cache_options:

capacity : 536870912

num_shard_bits : 6

strict_capacity_limit : 0

high_pri_pool_ratio: 0.000

block_cache_compressed: 0000000000000000

persistent_cache: 0000000000000000

block_size: 4096

block_size_deviation: 10

block_restart_interval: 16

index_block_restart_interval: 1

metadata_block_size: 4096

partition_filters: 0

use_delta_encoding: 1

filter_policy: nullptr

whole_key_filtering: 1

verify_compression: 0

read_amp_bytes_per_bit: 0

format_version: 2

enable_index_compression: 1

2018/07/25-09:12:10.251607 16a4 Options.write_buffer_size: 67108864

2018/07/25-09:12:10.251619 16a4 Options.max_write_buffer_number: 2

2018/07/25-09:12:10.251629 16a4 Options.compression: NoCompression

2018/07/25-09:12:10.251636 16a4 Options.bottommost_compression: Disabled

2018/07/25-09:12:10.251642 16a4 Options.prefix_extractor: nullptr

2018/07/25-09:12:10.251648 16a4 Options.memtable_insert_with_hint_prefix_extractor: nullptr

2018/07/25-09:12:10.251654 16a4 Options.num_levels: 7

2018/07/25-09:12:10.251658 16a4 Options.min_write_buffer_number_to_merge: 1

2018/07/25-09:12:10.251663 16a4 Options.max_write_buffer_number_to_maintain: 2

2018/07/25-09:12:10.251669 16a4 Options.compression_opts.window_bits: -14

2018/07/25-09:12:10.251675 16a4 Options.compression_opts.level: -1

2018/07/25-09:12:10.251680 16a4 Options.compression_opts.strategy: 0

2018/07/25-09:12:10.251686 16a4 Options.compression_opts.max_dict_bytes: 0

2018/07/25-09:12:10.251691 16a4 Options.level0_file_num_compaction_trigger: 4

2018/07/25-09:12:10.251697 16a4 Options.level0_slowdown_writes_trigger: 20

2018/07/25-09:12:10.251702 16a4 Options.level0_stop_writes_trigger: 36

2018/07/25-09:12:10.251708 16a4 Options.target_file_size_base: 67108864

2018/07/25-09:12:10.251713 16a4 Options.target_file_size_multiplier: 1

2018/07/25-09:12:10.251719 16a4 Options.max_bytes_for_level_base: 268435456

2018/07/25-09:12:10.251724 16a4 Options.level_compaction_dynamic_level_bytes: 0

2018/07/25-09:12:10.251730 16a4 Options.max_bytes_for_level_multiplier: 10.000000

2018/07/25-09:12:10.251740 16a4 Options.max_bytes_for_level_multiplier_addtl[0]: 1

2018/07/25-09:12:10.251746 16a4 Options.max_bytes_for_level_multiplier_addtl[1]: 1

2018/07/25-09:12:10.251751 16a4 Options.max_bytes_for_level_multiplier_addtl[2]: 1

2018/07/25-09:12:10.251757 16a4 Options.max_bytes_for_level_multiplier_addtl[3]: 1

2018/07/25-09:12:10.251762 16a4 Options.max_bytes_for_level_multiplier_addtl[4]: 1

2018/07/25-09:12:10.251768 16a4 Options.max_bytes_for_level_multiplier_addtl[5]: 1

2018/07/25-09:12:10.251774 16a4 Options.max_bytes_for_level_multiplier_addtl[6]: 1

2018/07/25-09:12:10.251779 16a4 Options.max_sequential_skip_in_iterations: 8

2018/07/25-09:12:10.251785 16a4 Options.max_compaction_bytes: 1677721600

2018/07/25-09:12:10.251791 16a4 Options.arena_block_size: 8388608

2018/07/25-09:12:10.251796 16a4 Options.soft_pending_compaction_bytes_limit: 68719476736

2018/07/25-09:12:10.251802 16a4 Options.hard_pending_compaction_bytes_limit: 274877906944

2018/07/25-09:12:10.251808 16a4 Options.rate_limit_delay_max_milliseconds: 100

2018/07/25-09:12:10.251813 16a4 Options.disable_auto_compactions: 1

2018/07/25-09:12:10.251821 16a4 Options.compaction_style: kCompactionStyleLevel

2018/07/25-09:12:10.251827 16a4 Options.compaction_pri: kByCompensatedSize

2018/07/25-09:12:10.251833 16a4 Options.compaction_options_universal.size_ratio: 1

2018/07/25-09:12:10.251839 16a4 Options.compaction_options_universal.min_merge_width: 2

2018/07/25-09:12:10.251844 16a4 Options.compaction_options_universal.max_merge_width: 4294967295

2018/07/25-09:12:10.251850 16a4 Options.compaction_options_universal.max_size_amplification_percent: 200

2018/07/25-09:12:10.251856 16a4 Options.compaction_options_universal.compression_size_percent: -1

2018/07/25-09:12:10.251862 16a4 Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize

2018/07/25-09:12:10.251868 16a4 Options.compaction_options_fifo.max_table_files_size: 1073741824

2018/07/25-09:12:10.251874 16a4 Options.compaction_options_fifo.allow_compaction: 0

2018/07/25-09:12:10.251882 16a4 Options.compaction_options_fifo.ttl: 0

2018/07/25-09:12:10.251892 16a4 Options.table_properties_collectors: Rdb_tbl_prop_coll_factory;

2018/07/25-09:12:10.251898 16a4 Options.inplace_update_support: 0

2018/07/25-09:12:10.251903 16a4 Options.inplace_update_num_locks: 10000

2018/07/25-09:12:10.251909 16a4 Options.memtable_prefix_bloom_size_ratio: 0.000000

2018/07/25-09:12:10.251915 16a4 Options.memtable_huge_page_size: 0

2018/07/25-09:12:10.251921 16a4 Options.bloom_locality: 0

2018/07/25-09:12:10.251926 16a4 Options.max_successive_merges: 0

2018/07/25-09:12:10.251932 16a4 Options.optimize_filters_for_hits: 0

2018/07/25-09:12:10.251938 16a4 Options.paranoid_file_checks: 0

2018/07/25-09:12:10.251943 16a4 Options.force_consistency_checks: 0

2018/07/25-09:12:10.251948 16a4 Options.report_bg_io_stats: 0

2018/07/25-09:12:10.254134 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\column_family.cc:429] --------------- Options for column family [__system__]:

2018/07/25-09:12:10.254153 16a4 Options.comparator: RocksDB_SE_v3.10

2018/07/25-09:12:10.254160 16a4 Options.merge_operator: Rdb_system_merge_op

2018/07/25-09:12:10.254166 16a4 Options.compaction_filter: None

2018/07/25-09:12:10.254171 16a4 Options.compaction_filter_factory: Rdb_compact_filter_factory

2018/07/25-09:12:10.254176 16a4 Options.memtable_factory: SkipListFactory

2018/07/25-09:12:10.254182 16a4 Options.table_factory: BlockBasedTable

2018/07/25-09:12:10.254242 16a4 table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0000025544B911A0)

cache_index_and_filter_blocks: 1

cache_index_and_filter_blocks_with_high_priority: 0

pin_l0_filter_and_index_blocks_in_cache: 1

index_type: 0

hash_index_allow_collision: 1

checksum: 1

no_block_cache: 0

block_cache: 0000025544C73D50

block_cache_name: LRUCache

block_cache_options:

capacity : 536870912

num_shard_bits : 6

strict_capacity_limit : 0

high_pri_pool_ratio: 0.000

block_cache_compressed: 0000000000000000

persistent_cache: 0000000000000000

block_size: 4096

block_size_deviation: 10

block_restart_interval: 16

index_block_restart_interval: 1

metadata_block_size: 4096

partition_filters: 0

use_delta_encoding: 1

filter_policy: nullptr

whole_key_filtering: 1

verify_compression: 0

read_amp_bytes_per_bit: 0

format_version: 2

enable_index_compression: 1

2018/07/25-09:12:10.254251 16a4 Options.write_buffer_size: 67108864

2018/07/25-09:12:10.254257 16a4 Options.max_write_buffer_number: 2

2018/07/25-09:12:10.254262 16a4 Options.compression: NoCompression

2018/07/25-09:12:10.254268 16a4 Options.bottommost_compression: Disabled

2018/07/25-09:12:10.254273 16a4 Options.prefix_extractor: nullptr

2018/07/25-09:12:10.254279 16a4 Options.memtable_insert_with_hint_prefix_extractor: nullptr

2018/07/25-09:12:10.254284 16a4 Options.num_levels: 7

2018/07/25-09:12:10.254289 16a4 Options.min_write_buffer_number_to_merge: 1

2018/07/25-09:12:10.254295 16a4 Options.max_write_buffer_number_to_maintain: 2

2018/07/25-09:12:10.254300 16a4 Options.compression_opts.window_bits: -14

2018/07/25-09:12:10.254306 16a4 Options.compression_opts.level: -1

2018/07/25-09:12:10.254311 16a4 Options.compression_opts.strategy: 0

2018/07/25-09:12:10.254317 16a4 Options.compression_opts.max_dict_bytes: 0

2018/07/25-09:12:10.254322 16a4 Options.level0_file_num_compaction_trigger: 4

2018/07/25-09:12:10.254328 16a4 Options.level0_slowdown_writes_trigger: 20

2018/07/25-09:12:10.254333 16a4 Options.level0_stop_writes_trigger: 36

2018/07/25-09:12:10.254339 16a4 Options.target_file_size_base: 67108864

2018/07/25-09:12:10.254345 16a4 Options.target_file_size_multiplier: 1

2018/07/25-09:12:10.254400 16a4 Options.max_bytes_for_level_base: 268435456

2018/07/25-09:12:10.254413 16a4 Options.level_compaction_dynamic_level_bytes: 0

2018/07/25-09:12:10.254419 16a4 Options.max_bytes_for_level_multiplier: 10.000000

2018/07/25-09:12:10.254427 16a4 Options.max_bytes_for_level_multiplier_addtl[0]: 1

2018/07/25-09:12:10.254432 16a4 Options.max_bytes_for_level_multiplier_addtl[1]: 1

2018/07/25-09:12:10.254437 16a4 Options.max_bytes_for_level_multiplier_addtl[2]: 1

2018/07/25-09:12:10.254443 16a4 Options.max_bytes_for_level_multiplier_addtl[3]: 1

2018/07/25-09:12:10.254449 16a4 Options.max_bytes_for_level_multiplier_addtl[4]: 1

2018/07/25-09:12:10.254454 16a4 Options.max_bytes_for_level_multiplier_addtl[5]: 1

2018/07/25-09:12:10.254460 16a4 Options.max_bytes_for_level_multiplier_addtl[6]: 1

2018/07/25-09:12:10.254465 16a4 Options.max_sequential_skip_in_iterations: 8

2018/07/25-09:12:10.254470 16a4 Options.max_compaction_bytes: 1677721600

2018/07/25-09:12:10.254476 16a4 Options.arena_block_size: 8388608

2018/07/25-09:12:10.254481 16a4 Options.soft_pending_compaction_bytes_limit: 68719476736

2018/07/25-09:12:10.254487 16a4 Options.hard_pending_compaction_bytes_limit: 274877906944

2018/07/25-09:12:10.254493 16a4 Options.rate_limit_delay_max_milliseconds: 100

2018/07/25-09:12:10.254498 16a4 Options.disable_auto_compactions: 1

2018/07/25-09:12:10.254504 16a4 Options.compaction_style: kCompactionStyleLevel

2018/07/25-09:12:10.254511 16a4 Options.compaction_pri: kByCompensatedSize

2018/07/25-09:12:10.254517 16a4 Options.compaction_options_universal.size_ratio: 1

2018/07/25-09:12:10.254526 16a4 Options.compaction_options_universal.min_merge_width: 2

2018/07/25-09:12:10.254532 16a4 Options.compaction_options_universal.max_merge_width: 4294967295

2018/07/25-09:12:10.254537 16a4 Options.compaction_options_universal.max_size_amplification_percent: 200

2018/07/25-09:12:10.254542 16a4 Options.compaction_options_universal.compression_size_percent: -1

2018/07/25-09:12:10.254549 16a4 Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize

2018/07/25-09:12:10.254555 16a4 Options.compaction_options_fifo.max_table_files_size: 1073741824

2018/07/25-09:12:10.254560 16a4 Options.compaction_options_fifo.allow_compaction: 0

2018/07/25-09:12:10.254566 16a4 Options.compaction_options_fifo.ttl: 0

2018/07/25-09:12:10.254576 16a4 Options.table_properties_collectors: Rdb_tbl_prop_coll_factory;

2018/07/25-09:12:10.254592 16a4 Options.inplace_update_support: 0

2018/07/25-09:12:10.254599 16a4 Options.inplace_update_num_locks: 10000

2018/07/25-09:12:10.254622 16a4 Options.memtable_prefix_bloom_size_ratio: 0.000000

2018/07/25-09:12:10.254630 16a4 Options.memtable_huge_page_size: 0

2018/07/25-09:12:10.254635 16a4 Options.bloom_locality: 0

2018/07/25-09:12:10.254641 16a4 Options.max_successive_merges: 0

2018/07/25-09:12:10.254647 16a4 Options.optimize_filters_for_hits: 0

2018/07/25-09:12:10.254652 16a4 Options.paranoid_file_checks: 0

2018/07/25-09:12:10.254658 16a4 Options.force_consistency_checks: 0

2018/07/25-09:12:10.254664 16a4 Options.report_bg_io_stats: 0

2018/07/25-09:12:10.348052 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\version_set.cc:3100] Recovered from manifest file:./#rocksdb/MANIFEST-000085 succeeded,manifest_file_number is 85, next_file_number is 92, last_sequence is 6502534, log_number is 0,prev_log_number is 0,max_column_family is 1

2018/07/25-09:12:10.348077 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\version_set.cc:3108] Column family [default] (ID 0), log number is 83

2018/07/25-09:12:10.348085 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\version_set.cc:3108] Column family [__system__] (ID 1), log number is 83

2018/07/25-09:12:10.352662 16a4 EVENT_LOG_v1 {"time_micros": 1532502730348583, "job": 1, "event": "recovery_started", "log_files": [86]}

2018/07/25-09:12:10.356693 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl_open.cc:518] Recovering log #86 mode 1

2018/07/25-09:12:10.359231 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\version_set.cc:2609] Creating manifest 92

2018/07/25-09:12:10.443881 16a4 EVENT_LOG_v1 {"time_micros": 1532502730443858, "cf_name": "__system__", "job": 1, "event": "table_file_creation", "file_number": 93, "file_size": 1119, "table_properties": {"data_size": 56, "index_size": 34, "filter_size": 0, "raw_key_size": 32, "raw_average_key_size": 16, "raw_value_size": 12, "raw_average_value_size": 6, "num_data_blocks": 1, "num_entries": 2, "filter_policy_name": "", "__indexstats__": "[...1 records...]", "kDeletedKeys": "0", "kMergeOperands": "0"}}

2018/07/25-09:12:10.452243 16a4 EVENT_LOG_v1 {"time_micros": 1532502730452220, "job": 1, "event": "recovery_finished"}

2018/07/25-09:12:10.480100 18fc [WARN] [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl.cc:455] ------- DUMPING STATS -------

2018/07/25-09:12:10.480124 18fc [WARN] [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl.cc:456]

** DB Stats **

Uptime(secs): 0.2 total, 0.2 interval

Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s

Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s

Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent

Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s

Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s

Interval stall: 00:00:0.000 H:M:S, 0.0 percent

** Compaction Stats [default] **

Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop

----------------------------------------------------------------------------------------------------------------------------------------------------------

L0 3/0 116.09 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

L1 3/0 154.55 MB 0.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

Sum 6/0 270.65 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

Uptime(secs): 0.2 total, 0.2 interval

Flush(GB): cumulative 0.000, interval 0.000

AddFile(GB): cumulative 0.000, interval 0.000

AddFile(Total Files): cumulative 0, interval 0

AddFile(L0 Files): cumulative 0, interval 0

AddFile(Keys): cumulative 0, interval 0

Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds

Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds

Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** File Read Latency Histogram By Level [default] **

** Compaction Stats [__system__] **

Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop

----------------------------------------------------------------------------------------------------------------------------------------------------------

L0 4/0 4.38 KB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0 1 0.025 0 0

L1 1/0 3.43 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

Sum 5/0 7.81 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0 1 0.025 0 0

Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0 1 0.025 0 0

Uptime(secs): 0.2 total, 0.2 interval

Flush(GB): cumulative 0.000, interval 0.000

AddFile(GB): cumulative 0.000, interval 0.000

AddFile(Total Files): cumulative 0, interval 0

AddFile(L0 Files): cumulative 0, interval 0

AddFile(Keys): cumulative 0, interval 0

Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds

Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds

** File Read Latency Histogram By Level [__system__] **

** Level 0 read latency histogram (micros):

Count: 4 Average: 4.7500 StdDev: 3.27

Min: 2 Median: 2.0000 Max: 10

Percentiles: P50: 2.00 P75: 6.00 P99: 9.84 P99.9: 9.98 P99.99: 10.00

------------------------------------------------------

( 1, 2 ] 2 50.000% 50.000% ##########

( 4, 6 ] 1 25.000% 75.000% #####

( 6, 10 ] 1 25.000% 100.000% #####

** Compaction Stats [default] **

Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop

----------------------------------------------------------------------------------------------------------------------------------------------------------

L0 3/0 116.09 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

L1 3/0 154.55 MB 0.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

Sum 6/0 270.65 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

Uptime(secs): 0.2 total, 0.0 interval

Flush(GB): cumulative 0.000, interval 0.000

AddFile(GB): cumulative 0.000, interval 0.000

AddFile(Total Files): cumulative 0, interval 0

AddFile(L0 Files): cumulative 0, interval 0

AddFile(Keys): cumulative 0, interval 0

Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds

Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds

** File Read Latency Histogram By Level [default] **

** Compaction Stats [__system__] **

Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop

----------------------------------------------------------------------------------------------------------------------------------------------------------

L0 4/0 4.38 KB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0 1 0.025 0 0

L1 1/0 3.43 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

Sum 5/0 7.81 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0 1 0.025 0 0

Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0

Uptime(secs): 0.2 total, 0.0 interval

Flush(GB): cumulative 0.000, interval 0.000

AddFile(GB): cumulative 0.000, interval 0.000

AddFile(Total Files): cumulative 0, interval 0

AddFile(L0 Files): cumulative 0, interval 0

AddFile(Keys): cumulative 0, interval 0

Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds

Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds

** File Read Latency Histogram By Level [__system__] **

** Level 0 read latency histogram (micros):

Count: 4 Average: 4.7500 StdDev: 3.27

Min: 2 Median: 2.0000 Max: 10

Percentiles: P50: 2.00 P75: 6.00 P99: 9.84 P99.9: 9.98 P99.99: 10.00

------------------------------------------------------

( 1, 2 ] 2 50.000% 50.000% ##########

( 4, 6 ] 1 25.000% 75.000% #####

( 6, 10 ] 1 25.000% 100.000% #####

2018/07/25-09:12:10.480221 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl_open.cc:1165] DB pointer 000002554D951890

2018/07/25-09:12:10.480675 18fc [WARN] [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl.cc:406] STATISTICS:

rocksdb.block.cache.miss COUNT : 1

rocksdb.block.cache.hit COUNT : 1

rocksdb.block.cache.add COUNT : 1

rocksdb.block.cache.add.failures COUNT : 0

rocksdb.block.cache.index.miss COUNT : 1

rocksdb.block.cache.index.hit COUNT : 1

rocksdb.block.cache.index.add COUNT : 1

rocksdb.block.cache.index.bytes.insert COUNT : 29

rocksdb.block.cache.index.bytes.evict COUNT : 0

rocksdb.block.cache.filter.miss COUNT : 0

rocksdb.block.cache.filter.hit COUNT : 0

rocksdb.block.cache.filter.add COUNT : 0

rocksdb.block.cache.filter.bytes.insert COUNT : 0

rocksdb.block.cache.filter.bytes.evict COUNT : 0

rocksdb.block.cache.data.miss COUNT : 0

rocksdb.block.cache.data.hit COUNT : 0

rocksdb.block.cache.data.add COUNT : 0

rocksdb.block.cache.data.bytes.insert COUNT : 0

rocksdb.block.cache.bytes.read COUNT : 29

rocksdb.block.cache.bytes.write COUNT : 29

rocksdb.bloom.filter.useful COUNT : 0

rocksdb.persistent.cache.hit COUNT : 0

rocksdb.persistent.cache.miss COUNT : 0

rocksdb.sim.block.cache.hit COUNT : 0

rocksdb.sim.block.cache.miss COUNT : 0

rocksdb.memtable.hit COUNT : 0

rocksdb.memtable.miss COUNT : 0

rocksdb.l0.hit COUNT : 0

rocksdb.l1.hit COUNT : 0

rocksdb.l2andup.hit COUNT : 0

rocksdb.compaction.key.drop.new COUNT : 0

rocksdb.compaction.key.drop.obsolete COUNT : 0

rocksdb.compaction.key.drop.range_del COUNT : 0

rocksdb.compaction.key.drop.user COUNT : 0

rocksdb.compaction.range_del.drop.obsolete COUNT : 0

rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0

rocksdb.number.keys.written COUNT : 0

rocksdb.number.keys.read COUNT : 0

rocksdb.number.keys.updated COUNT : 0

rocksdb.bytes.written COUNT : 0

rocksdb.bytes.read COUNT : 0

rocksdb.number.db.seek COUNT : 0

rocksdb.number.db.next COUNT : 0

rocksdb.number.db.prev COUNT : 0

rocksdb.number.db.seek.found COUNT : 0

rocksdb.number.db.next.found COUNT : 0

rocksdb.number.db.prev.found COUNT : 0

rocksdb.db.iter.bytes.read COUNT : 0

rocksdb.no.file.closes COUNT : 0

rocksdb.no.file.opens COUNT : 1

rocksdb.no.file.errors COUNT : 0

rocksdb.l0.slowdown.micros COUNT : 0

rocksdb.memtable.compaction.micros COUNT : 0

rocksdb.l0.num.files.stall.micros COUNT : 0

rocksdb.stall.micros COUNT : 0

rocksdb.db.mutex.wait.micros COUNT : 0

rocksdb.rate.limit.delay.millis COUNT : 0

rocksdb.num.iterators COUNT : 0

rocksdb.number.multiget.get COUNT : 0

rocksdb.number.multiget.keys.read COUNT : 0

rocksdb.number.multiget.bytes.read COUNT : 0

rocksdb.number.deletes.filtered COUNT : 0

rocksdb.number.merge.failures COUNT : 0

rocksdb.bloom.filter.prefix.checked COUNT : 0

rocksdb.bloom.filter.prefix.useful COUNT : 0

rocksdb.number.reseeks.iteration COUNT : 0

rocksdb.getupdatessince.calls COUNT : 0

rocksdb.block.cachecompressed.miss COUNT : 0

rocksdb.block.cachecompressed.hit COUNT : 0

rocksdb.block.cachecompressed.add COUNT : 0

rocksdb.block.cachecompressed.add.failures COUNT : 0

rocksdb.wal.synced COUNT : 0

rocksdb.wal.bytes COUNT : 0

rocksdb.write.self COUNT : 0

rocksdb.write.other COUNT : 0

rocksdb.write.timeout COUNT : 0

rocksdb.write.wal COUNT : 0

rocksdb.compact.read.bytes COUNT : 0

rocksdb.compact.write.bytes COUNT : 1119

rocksdb.flush.write.bytes COUNT : 0

rocksdb.number.direct.load.table.properties COUNT : 10

rocksdb.number.superversion_acquires COUNT : 0

rocksdb.number.superversion_releases COUNT : 0

rocksdb.number.superversion_cleanups COUNT : 0

rocksdb.number.block.compressed COUNT : 0

rocksdb.number.block.decompressed COUNT : 0

rocksdb.number.block.not_compressed COUNT : 0

rocksdb.merge.operation.time.nanos COUNT : 0

rocksdb.filter.operation.time.nanos COUNT : 0

rocksdb.row.cache.hit COUNT : 0

rocksdb.row.cache.miss COUNT : 0

rocksdb.read.amp.estimate.useful.bytes COUNT : 0

rocksdb.read.amp.total.read.bytes COUNT : 0

rocksdb.number.rate_limiter.drains COUNT : 0

rocksdb.number.iter.skip COUNT : 0

rocksdb.blobdb.num.put COUNT : 0

rocksdb.blobdb.num.write COUNT : 0

rocksdb.blobdb.num.get COUNT : 0

rocksdb.blobdb.num.multiget COUNT : 0

rocksdb.blobdb.num.seek COUNT : 0

rocksdb.blobdb.num.next COUNT : 0

rocksdb.blobdb.num.prev COUNT : 0

rocksdb.blobdb.num.keys.written COUNT : 0

rocksdb.blobdb.num.keys.read COUNT : 0

rocksdb.blobdb.bytes.written COUNT : 0

rocksdb.blobdb.bytes.read COUNT : 0

rocksdb.blobdb.write.inlined COUNT : 0

rocksdb.blobdb.write.inlined.ttl COUNT : 0

rocksdb.blobdb.write.blob COUNT : 0

rocksdb.blobdb.write.blob.ttl COUNT : 0

rocksdb.blobdb.blob.file.bytes.written COUNT : 0

rocksdb.blobdb.blob.file,bytes.read COUNT : 0

rocksdb.blobdb.blob.file.synced COUNT : 0

rocksdb.blobdb.blob.index.expired COUNT : 0

rocksdb.blobdb.gc.num.files COUNT : 0

rocksdb.blobdb.gc.num.new.files COUNT : 0

rocksdb.blobdb.gc.failures COUNT : 0

rocksdb.blobdb.gc.num.keys.overwritten COUNT : 0

rocksdb.blobdb.gc.num.keys.expired COUNT : 0

rocksdb.blobdb.gc.num.keys.relocated COUNT : 0

rocksdb.blobdb.gc.bytes.overwritten COUNT : 0

rocksdb.blobdb.gc.bytes.expired COUNT : 0

rocksdb.blobdb.gc.bytes.relocated COUNT : 0

rocksdb.blobdb.fifo.num.files.evicted COUNT : 0

rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0

rocksdb.blobdb.fifo.bytes.evicted COUNT : 0

rocksdb.db.get.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.db.write.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.compaction.times.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.subcompaction.setup.times.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.table.sync.micros statistics Percentiles :=> 50 : 13279.000000 95 : 13279.000000 99 : 13279.000000 100 : 13279.000000

rocksdb.compaction.outfile.sync.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.wal.file.sync.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.manifest.file.sync.micros statistics Percentiles :=> 50 : 9900.000000 95 : 16275.000000 99 : 16275.000000 100 : 16275.000000

rocksdb.table.open.io.micros statistics Percentiles :=> 50 : 133.000000 95 : 133.000000 99 : 133.000000 100 : 133.000000

rocksdb.db.multiget.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.read.block.compaction.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.read.block.get.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.write.raw.block.micros statistics Percentiles :=> 50 : 1.000000 95 : 3.800000 99 : 3.960000 100 : 4.000000

rocksdb.l0.slowdown.count statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.memtable.compaction.count statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.num.files.stall.count statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.hard.rate.limit.delay.count statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.soft.rate.limit.delay.count statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.numfiles.in.singlecompaction statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.db.seek.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.db.write.stall statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.sst.read.micros statistics Percentiles :=> 50 : 2.000000 95 : 9.200000 99 : 9.840000 100 : 10.000000

rocksdb.num.subcompactions.scheduled statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.bytes.per.read statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.bytes.per.write statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.bytes.per.multiget statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.bytes.compressed statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.bytes.decompressed statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.compression.times.nanos statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.decompression.times.nanos statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.read.num.merge_operands statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.key.size statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.value.size statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.write.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.get.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.multiget.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.seek.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.next.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.prev.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.blob.file.write.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.blob.file.read.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.blob.file.sync.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.gc.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.compression.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.blobdb.decompression.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

rocksdb.db.flush.micros statistics Percentiles :=> 50 : 0.000000 95 : 0.000000 99 : 0.000000 100 : 0.000000

2018/07/25-09:12:10.480752 18fc (Original Log Time 2018/07/25-09:12:10.480741) [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl_compaction_flush.cc:1571] Compaction nothing to do

2018/07/25-09:12:10.596964 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl.cc:532] SetOptions() on column family [default], inputs:

2018/07/25-09:12:10.596984 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl.cc:535] disable_auto_compactions: false

2018/07/25-09:12:10.596992 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl.cc:539] [default] SetOptions() succeeded

2018/07/25-09:12:10.596999 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:114] write_buffer_size: 67108864

2018/07/25-09:12:10.597006 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:116] max_write_buffer_number: 2

2018/07/25-09:12:10.597013 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:119] arena_block_size: 8388608

2018/07/25-09:12:10.597019 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:121] memtable_prefix_bloom_ratio: 0.000000

2018/07/25-09:12:10.597028 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:124] memtable_huge_page_size: 0

2018/07/25-09:12:10.597035 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:127] max_successive_merges: 0

2018/07/25-09:12:10.597042 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:130] inplace_update_num_locks: 10000

2018/07/25-09:12:10.597048 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:132] disable_auto_compactions: 0

2018/07/25-09:12:10.597055 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:134] soft_pending_compaction_bytes_limit: 68719476736

2018/07/25-09:12:10.597062 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:136] hard_pending_compaction_bytes_limit: 274877906944

2018/07/25-09:12:10.597068 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:138] level0_file_num_compaction_trigger: 4

2018/07/25-09:12:10.597079 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:140] level0_slowdown_writes_trigger: 20

2018/07/25-09:12:10.597085 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:142] level0_stop_writes_trigger: 36

2018/07/25-09:12:10.597092 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:144] max_compaction_bytes: 1677721600

2018/07/25-09:12:10.597098 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:146] target_file_size_base: 67108864

2018/07/25-09:12:10.597105 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:148] target_file_size_multiplier: 1

2018/07/25-09:12:10.597111 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:150] max_bytes_for_level_base: 268435456

2018/07/25-09:12:10.597117 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:152] max_bytes_for_level_multiplier: 10.000000

2018/07/25-09:12:10.597124 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:166] max_bytes_for_level_multiplier_additional: 1, 1, 1, 1, 1, 1, 1

2018/07/25-09:12:10.597128 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:168] max_sequential_skip_in_iterations: 8

2018/07/25-09:12:10.597132 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:170] paranoid_file_checks: 0

2018/07/25-09:12:10.597136 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:172] report_bg_io_stats: 0

2018/07/25-09:12:10.597140 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:174] compression: 0

2018/07/25-09:12:10.600429 18fc [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\compaction_job.cc:1493] [__system__] [JOB 4] Compacting 4@0 + 1@1 files to L1, score 1.00

2018/07/25-09:12:10.600458 18fc [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\compaction_job.cc:1497] [__system__] Compaction start summary: Base version 7 Base level 0, inputs: [93(1119B) 83(1123B) 72(1123B) 61(1123B)], [50(3511B)]

2018/07/25-09:12:10.600491 18fc EVENT_LOG_v1 {"time_micros": 1532502730600470, "job": 4, "event": "compaction_started", "files_L0": [93, 83, 72, 61], "files_L1": [50], "score": 1, "input_data_size": 7999}

2018/07/25-09:12:10.614900 18fc [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\compaction_job.cc:1171] [__system__] [JOB 4] Generated table #100: 31 keys, 3511 bytes

2018/07/25-09:12:10.614953 18fc EVENT_LOG_v1 {"time_micros": 1532502730614924, "cf_name": "__system__", "job": 4, "event": "table_file_creation", "file_number": 100, "file_size": 3511, "table_properties": {"data_size": 2078, "index_size": 39, "filter_size": 0, "raw_key_size": 614, "raw_average_key_size": 19, "raw_value_size": 1637, "raw_average_value_size": 52, "num_data_blocks": 1, "num_entries": 31, "filter_policy_name": "", "__indexstats__": "[...6 records...]", "kDeletedKeys": "0", "kMergeOperands": "1"}}

2018/07/25-09:12:10.615068 18fc [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\compaction_job.cc:1245] [__system__] [JOB 4] Compacted 4@0 + 1@1 files to L1 => 3511 bytes

2018/07/25-09:12:10.619930 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl.cc:532] SetOptions() on column family [__system__], inputs:

2018/07/25-09:12:10.619944 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl.cc:535] disable_auto_compactions: false

2018/07/25-09:12:10.619952 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\db_impl.cc:539] [__system__] SetOptions() succeeded

2018/07/25-09:12:10.619959 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:114] write_buffer_size: 67108864

2018/07/25-09:12:10.619966 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:116] max_write_buffer_number: 2

2018/07/25-09:12:10.619973 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:119] arena_block_size: 8388608

2018/07/25-09:12:10.619979 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:121] memtable_prefix_bloom_ratio: 0.000000

2018/07/25-09:12:10.619988 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:124] memtable_huge_page_size: 0

2018/07/25-09:12:10.619994 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:127] max_successive_merges: 0

2018/07/25-09:12:10.620001 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:130] inplace_update_num_locks: 10000

2018/07/25-09:12:10.620008 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:132] disable_auto_compactions: 0

2018/07/25-09:12:10.620014 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:134] soft_pending_compaction_bytes_limit: 68719476736

2018/07/25-09:12:10.620021 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:136] hard_pending_compaction_bytes_limit: 274877906944

2018/07/25-09:12:10.620028 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:138] level0_file_num_compaction_trigger: 4

2018/07/25-09:12:10.620034 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:140] level0_slowdown_writes_trigger: 20

2018/07/25-09:12:10.620041 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:142] level0_stop_writes_trigger: 36

2018/07/25-09:12:10.620048 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:144] max_compaction_bytes: 1677721600

2018/07/25-09:12:10.620058 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:146] target_file_size_base: 67108864

2018/07/25-09:12:10.620067 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:148] target_file_size_multiplier: 1

2018/07/25-09:12:10.620080 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:150] max_bytes_for_level_base: 268435456

2018/07/25-09:12:10.620087 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:152] max_bytes_for_level_multiplier: 10.000000

2018/07/25-09:12:10.620098 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:166] max_bytes_for_level_multiplier_additional: 1, 1, 1, 1, 1, 1, 1

2018/07/25-09:12:10.620105 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:168] max_sequential_skip_in_iterations: 8

2018/07/25-09:12:10.620112 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:170] paranoid_file_checks: 0

2018/07/25-09:12:10.620118 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:172] report_bg_io_stats: 0

2018/07/25-09:12:10.620125 16a4 [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\options\cf_options.cc:174] compression: 0

2018/07/25-09:12:10.621523 18fc (Original Log Time 2018/07/25-09:12:10.621416) [d:\winx64-packages\build\src\storage\rocksdb\rocksdb\db\compaction_job.cc:640] [__system__] compacted to: base level 1 max bytes base 268435456 files[0 1 0 0 0 0 0] max score 0.00, MB/sec: 0.5 rd, 0.2 wr, level 1, files in(4, 1) out(1) MB in(0.0, 0.0) out(0.0), read-write-amplify(2.6) write-amplify(0.8) OK, records in: 39, records dropped: 8 output_compression: NoCompression

2018/07/25-09:12:10.621536 18fc (Original Log Time 2018/07/25-09:12:10.621457) EVENT_LOG_v1 {"time_micros": 1532502730621432, "job": 4, "event": "compaction_finished", "compaction_time_micros": 14554, "output_level": 1, "num_output_files": 1, "total_output_size": 3511, "num_input_records": 39, "num_output_records": 31, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 1, 0, 0, 0, 0, 0]}

2018/07/25-09:12:10.621773 18fc EVENT_LOG_v1 {"time_micros": 1532502730621763, "job": 4, "event": "table_file_deletion", "file_number": 93}

2018/07/25-09:12:10.621962 18fc EVENT_LOG_v1 {"time_micros": 1532502730621954, "job": 4, "event": "table_file_deletion", "file_number": 83}

2018/07/25-09:12:10.622127 18fc EVENT_LOG_v1 {"time_micros": 1532502730622120, "job": 4, "event": "table_file_deletion", "file_number": 72}

2018/07/25-09:12:10.622301 18fc EVENT_LOG_v1 {"time_micros": 1532502730622294, "job": 4, "event": "table_file_deletion", "file_number": 61}

2018/07/25-09:12:10.622478 18fc EVENT_LOG_v1 {"time_micros": 1532502730622471, "job": 4, "event": "table_file_deletion", "file_number": 50}

Mark Callaghan

unread,

Jul 25, 2018, 10:22:08 AM7/25/18

to MyRocks - RocksDB storage engine for MySQL

First, thank you for evaluating MyRocks and for starting this thread with sufficient details on your problem. MyRocks needs better documentation. We will get there. Until then there are mailing lists.

The my.cnf settings that we suggest are at https://github.com/facebook/mysql-5.6/wiki/my.cnf-tuning. See "MyRocks configuration example for users other than linkbench". However, I use these on FB MySQL, not MariaDB. The most useful one is rocksdb_default_cf_options which will avoid the problems I mention below. Setting rocksdb_max_open_files=-1 might also help. I don't think you need to change the other options listed on that wiki page. I think a simple change is to add this to your my.cnf/my.ini

rocksdb_default_cf_options=block_based_table_factory={cache_index_and_filter_blocks=1;filter_policy=bloomfilter:10:false;whole_key_filtering=1};level_compaction_dynamic_level_bytes=true;optimize_filters_for_hits=true;compaction_pri=kMinOverlappingRatio

And then optionally set this:
rocksdb_max_open_files=-1

----- you can stop reading here, it is just an editorial on the configuration options that are set by default

Point-query performance should improve if you enable bloom filters. Although it is unlikely to match InnoDB. I know that bloom filters are disabled because of this in the LOG output:
> filter_policy: nullptr

If you know the defaults, then this is another way to check without looking at LOG. If rocksdb_default_cf_options aren't set then you get the default, which includes no bloom filter.

# bin/mysql -uroot -ppw -e 'show global variables like "%_cf%"'

+-----------------------------+-------+

| Variable_name | Value |

+-----------------------------+-------+

| rocksdb_compact_cf | |

| rocksdb_default_cf_options | |

| rocksdb_override_cf_options | |

| rocksdb_update_cf_options | |

You have not enabled compression and compression is a great feature for MyRocks. I know that because of this in LOG:

2018/07/25-09:12:10.251629 16a4 Options.compression: NoCompression

2018/07/25-09:12:10.251636 16a4 Options.bottommost_compression: Disabled

I also know that compression isn't enabled because rocksdb_default_cf_options hasn't been set. Enabling compression usually doesn't help with query performance, but it does reduce space used. Not sure that matters for you. Alas, MariaDB only supports zlib for compression. zstd is a much better choice. And zlib is likely to hurt performance.

This is disabled by default. It is a great feature to enable. Perhaps one day RocksDB will fix the default. Until then it can be set via rocksdb_default_cf_options.

2018/07/25-09:12:10.251724 16a4 Options.level_compaction_dynamic_level_bytes: 0

Another option with a default that isn't the best. Again, it can be changed via rocksdb_default_cf_options, a better value is kMinOverlappingRatio

Mark Callaghan

unread,

Jul 25, 2018, 11:43:47 AM7/25/18

to MyRocks - RocksDB storage engine for MySQL

A note to myself to make debugging this faster in the future - http://smalldatum.blogspot.com/2018/07/default-options-in-myrocks.html

Reply all

Reply to author

Forward

0 new messages