Does RedisTimeSeries support AOF persistent

32 views
Skip to first unread message

Andy Mei

unread,
Feb 20, 2020, 10:54:11 AM2/20/20
to RedisTimeSeries Discussion Forum
Hi all,

  Love your work on RTS.  We love the benchmarking result of RTS with 1TB of raw financial tick data.

  We encounter following error after RTS server restart.  I wonder if RTS supports RDB or AOF persistent?  Thanks.

776:M 20 Feb 2020 15:46:09.092 # Error loading data from RDB (short read or EOF). Read performed by module 'timeseries' about type 'TSDB-TYPE' after reading '0' bytes of a value.

Andy

Guy Korland

unread,
Feb 20, 2020, 11:22:55 AM2/20/20
to RedisTimeSeries Discussion Forum
It seems like an issue with the RDB deserialization, which version of RTS are you using?
Can you attach the RDB?

Aaron Lee

unread,
Feb 20, 2020, 1:56:12 PM2/20/20
to RedisTimeSeries Discussion Forum
# Server
redis_version:5.0.5
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:79857a93f73253c0
redis_mode:standalone
os:Linux 4.14.146-120.181.amzn2.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:7.3.1
process_id:1282
run_id:a4f4e66633cdd443d1190aabcb591fab44bd6ec2
tcp_port:6379
uptime_in_seconds:6602
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:5166438
executable:/usr/local/bin/redis-server
config_file:/etc/redis/redis.conf

# Clients
connected_clients:32
client_recent_max_input_buffer:2
client_recent_max_output_buffer:0
blocked_clients:0

# Memory
used_memory:2092608328
used_memory_human:1.95G
used_memory_rss:2169384960
used_memory_rss_human:2.02G
used_memory_peak:7891646504
used_memory_peak_human:7.35G
used_memory_peak_perc:26.52%
used_memory_overhead:1431602
used_memory_startup:791416
used_memory_dataset:2091176726
used_memory_dataset_perc:99.97%
allocator_allocated:2092676304
allocator_active:2093699072
allocator_resident:2179067904
total_system_memory:33069961216
total_system_memory_human:30.80G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.00
allocator_frag_bytes:1022768
allocator_rss_ratio:1.04
allocator_rss_bytes:85368832
rss_overhead_ratio:1.00
rss_overhead_bytes:-9682944
mem_fragmentation_ratio:1.04
mem_fragmentation_bytes:76839520
mem_not_counted_for_evict:2798
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:574276
mem_aof_buffer:2798
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:6048
rdb_bgsave_in_progress:0
rdb_last_save_time:1582224653
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:171
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:1273856
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:175
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:270184448
aof_current_size:15038791875
aof_base_size:14387324147
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:20

# Stats
total_connections_received:847
total_commands_processed:2260126
instantaneous_ops_per_sec:0
total_net_input_bytes:36730287027
total_net_output_bytes:9320493313
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
evicted_keys:0
keyspace_hits:115969
keyspace_misses:1343
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:29872
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0

# Replication
role:master
connected_slaves:0
master_replid:76a0ba0dfe36f57995859870243d52bb8731dc8b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:99.035822
used_cpu_user:717.799139
used_cpu_sys_children:166.188894
used_cpu_user_children:2618.338202

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=1373,expires=0,avg_ttl=0

Andy Mei

unread,
Feb 24, 2020, 6:14:36 PM2/24/20
to RedisTimeSeries Discussion Forum
Hey Guy,

  Please see version below.  Redis failed with original error message when we try to attach the RDB file.

redis_version:5.0.5
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:79857a93f73253c0
redis_mode:standalone
os:Linux 4.14.146-120.181.amzn2.x86_64 x86_64


Aaron Lee

unread,
Mar 2, 2020, 7:19:02 PM3/2/20
to RedisTimeSeries Discussion Forum

Hey I've attached the redis.log and here's the appendonly.aof:


Let me know if you can't access it.

Thanks!
redis.log
Reply all
Reply to author
Forward
0 new messages