redis-cli restore command with size of more than 512 MB

[Gyanranjan Muduli]

Hi Team,

I  am trying  to restore the dump data  with size more than 512 MB, but getting Error :  Connection reset by peer.

i am expecting to restore around 10GB dump data to redis. 

Kindly suggest on this.

Regards

Gyan

[sundb]

What is your client-query-buffer-limit set to?

[Gyanranjan Muduli]

it is 1073741824 bytes

Thanks

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/ca440116-be58-4875-91a1-d3c9e8310d6fn%40googlegroups.com.

[sundb]

timeout 0

client-query-buffer-limit 5gb

Can you set it to this configuration, and then test it again?

Or let me know your reproduction steps, and I will test.

[Gyanranjan Muduli]

Hi Team,

I  have tried with 5GB as well as 10 GB, but getting  same error.

    cat 523.txt | redis-cli -p 6379 -n 1 -a ****** -x restore 523 0
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Error: Connection reset by peer

Regards

Gyan

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/18af6d53-abda-4e82-ba76-698e0fe442e1n%40googlegroups.com.

[Gyanranjan Muduli]

Hi Team,

I  have tried with 5GB as well as 10 GB, but getting  same error.

FYI = 523.txt(it 's dump  data  of size 923 MB taken  from redis  with single key 523)

cat 523.txt | redis-cli -p 6379 -n 1 -a ****** -x restore 523 0
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Error: Connection reset by peer

Regards

Gyan

[Gyanranjan Muduli]

Hi Team,

Could  you please help on it.

Regards

Gyan

[sundb]

Sorry for the late reply.

You can change the configuration ```loglevel verbose```, then you can see the reason for the disconnection.

Gyanranjan Muduli <gyanra...@gmail.com> 于2021年4月21日周三 下午6:21写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/CAETQcjy868A8Vm%3DG4V-Rf_Yvwm-5jr1Vnuc9cXoHbjf%3D%2BhsCgA%40mail.gmail.com.

[sundb]

Is there a useful disconnection message?

sundb <sund...@gmail.com> 于2021年4月22日周四 下午6:36写道：

[Gyanranjan Muduli]

Please find below details

871:M 22 Apr 2021 18:15:13.893 - DB 0: 14 keys (0 volatile) in 16 slots HT.
871:M 22 Apr 2021 18:15:13.894 - DB 1: 1 keys (0 volatile) in 4 slots HT.
871:M 22 Apr 2021 18:15:13.894 - DB 2: 6 keys (0 volatile) in 8 slots HT.
871:M 22 Apr 2021 18:15:13.894 - DB 3: 34 keys (0 volatile) in 64 slots HT.
871:M 22 Apr 2021 18:15:13.894 - DB 5: 16 keys (0 volatile) in 16 slots HT.
871:M 22 Apr 2021 18:15:13.894 - DB 6: 79 keys (0 volatile) in 128 slots HT.
871:M 22 Apr 2021 18:15:13.894 - DB 8: 1 keys (0 volatile) in 4 slots HT.
871:M 22 Apr 2021 18:15:13.894 - DB 9: 3992 keys (0 volatile) in 4096 slots HT.
871:M 22 Apr 2021 18:15:13.894 - DB 11: 250 keys (0 volatile) in 256 slots HT.
871:M 22 Apr 2021 18:15:13.923 - Accepted 127.0.0.1:54134
871:M 22 Apr 2021 18:15:18.043 - Protocol error (invalid bulk length) from client: id=6815 addr=127.0.0.1:54134 fd=9 name= age=5 idle=0 flags=N db=1 sub=0 psub=0 multi=-1 qbuf=16384 qbuf-free=16384 obl=42 oll=0 omem=0 events=r cmd=select user=default. Query buffer during protocol error: '$1007655076....0............q.......@d.{..G.zt?...../.5.&@.DP.' (... more 16220 bytes ...) 'y_......c....o...kt>;).#.+I.F.s..5HJ....vH......C...=C..[g3.2...'
871:M 22 Apr 2021 18:15:18.904 - DB 0: 14 keys (0 volatile) in 16 slots HT.
871:M 22 Apr 2021 18:15:18.904 - DB 1: 1 keys (0 volatile) in 4 slots HT.
871:M 22 Apr 2021 18:15:18.905 - DB 2: 6 keys (0 volatile) in 8 slots HT.
871:M 22 Apr 2021 18:15:18.905 - DB 3: 34 keys (0 volatile) in 64 slots HT.
871:M 22 Apr 2021 18:15:18.905 - DB 5: 16 keys (0 volatile) in 16 slots HT.
871:M 22 Apr 2021 18:15:18.905 - DB 6: 79 keys (0 volatile) in 128 slots HT.
871:M 22 Apr 2021 18:15:18.905 - DB 8: 1 keys (0 volatile) in 4 slots HT.
871:M 22 Apr 2021 18:15:18.905 - DB 9: 3992 keys (0 volatile) in 4096 slots HT.
871:M 22 Apr 2021 18:15:18.905 - DB 11: 250 keys (0 volatile) in 256 slots HT.

Thanks

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/CAJWqF5BT2tn-10qVs6vS_GKS39882tvQK3h_7nyuc9cq-E4XvQ%40mail.gmail.com.

[Gyanranjan Muduli]

Only getting below

protocol error (invalid bulk length) from client: id=6815 addr=127.0.0.1:54134 fd=9 name= age=5 idle=0 flags=N db=1 sub=0 psub=0 multi=-1 qbuf=16384 qbuf-free=16384 obl=42 oll=0 omem=0 events=r cmd=select user=default. Query buffer during protocol error: '$1007655076....0............q.......@d.{..G.zt?...../.5.&@.DP.' (... more 16220 bytes ...) 'y_......c....o...kt>;).#.+I.F.s..5HJ....vH......C...=C..[g3.2...'

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/CAJWqF5D-Ckr38Fucc2gVRQYqSH6s67yMSUH%3D1%3DhCdNmk0dVANw%40mail.gmail.com.

[sundb]

I almost know why, proto-max-bulk-len 512M, your configuration should be like this, make its value bigger.

[Gyanranjan Muduli]

It is working when i  increase it to 1GB.
Is there any impact on performance  if we will increase it more ?

Regards

Gyan

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/ddb79ffb-a0b9-41e4-8baf-89b7ca70dc61n%40googlegroups.com.

[sundb]

Is the performance impact you're talking about the users online?

If you just increase proto-max-bulk-len it will not affect the restore speed, it will only affect other connected users.