dnet_recovery not recovery

[Рустам Нарманов]

One instance have lost one hdd
hdd replaced
After 7 days of work dnet_recovery i have big value in covered_keys_failures, what wrong?
cat /var/tmp/dnet_recovery_dc/stats.txt
============================================= monitor ==============================================
recovered_keys_success: 99
recovered_keys_failures: 18611592
recovered_keys_total: 18611691
result: 1
total_keys: 18611491
main_started: 2019-08-27 13:31:50.315455
main_started...iterating: 0:00:00.000192
main_iterating...transpose: 3 days, 16:17:56.761948
main_transpose...merge: 0:00:00.000182
main_merge...final_merge: 0:19:26.339115
main_final_merge...fill_buckets: 0:01:02.231643
main_fill_buckets...recover: 0:32:45.604601
main_recover...finished: 1:52:40.656285
main_finished: 2019-08-31 08:35:41.909421
============================================= iterate ==============================================
======================================= 10.13.177.203:1021:2 =======================================
=============================================== 304 ================================================
iterated_keys: 3811138
iteration_speed: 178.41
iterations: 1
total_keys: 3474594
process_started: 2019-08-29 06:29:27.120548
process_started...iterate: 0:00:00.031382
process_iterate...sort: 5:56:02.403491
process_sort...finished: 0:00:14.003748
process_finished: 2019-08-29 12:25:43.559169
======================================= 10.13.177.230:1021:2 =======================================
=============================================== 102 ================================================
iterated_keys: 24407372
iteration_speed: 163.85
iterations: 1
total_keys: 22086987
process_started: 2019-08-29 12:25:44.673090
process_started...iterate: 0:00:00.000100
process_iterate...sort: 1 day, 17:22:37.747832
process_sort...finished: 0:01:23.890464
process_finished: 2019-08-31 05:49:46.311486
======================================= 10.13.177.242:1021:2 =======================================
=============================================== 1314 ===============================================
iterated_keys: 24354499
iteration_speed: 165.39
iterations: 1
total_keys: 22086878
process_started: 2019-08-27 13:31:50.432869
process_started...iterate: 0:00:00.000143
process_iterate...sort: 1 day, 16:54:19.477272
process_sort...finished: 0:03:16.117899
process_finished: 2019-08-29 06:29:26.028183
============================================= recover ==============================================
local_read_bytes: 2498043
local_reads_success: 4
local_reads_failures: 441
local_reads_total: 445
recovered_keys_success: 99
recovered_keys_failures: 18611592
recovered_keys_total: 18611691
recovers_in_progress: 0
recovery_speed: 105.33
remote_writes: 6
remote_written_bytes: 3785118
skipped_keys: 240
recover_started: 2019-08-31 06:43:01.406464
recover_started...finished: 1:52:40.502890
recover_finished: 2019-08-31 08:35:41.909354

[Evgeniy Polyakov]

Hi Rustam

Sorry for long delay

02.09.2019, 20:04, "'Рустам Нарманов' via reverbrain" <rever...@googlegroups.com>:

> One instance have lost one hdd
> hdd replaced
> After 7 days of work dnet_recovery i have big value in covered_keys_failures, what wrong?
> cat /var/tmp/dnet_recovery_dc/stats.txt
> ============================================= monitor ==============================================
> recovered_keys_success: 99
> recovered_keys_failures: 18611592

You should check server and recovery logs to find out why these keys could not be recovered