Redis 6 cluster resharding with ACL

[Wilson Lindelof]

Hey all. I'm trying to figure out how the --cluster reshard command works with the new ACLs on. I've got an existing cluster that works, and added a new master and slave. I can run cluster nodes on it to see that it is in the cluster, but has no slots assigned. Using a rc2 client and server, I try to run a command like the following:

redis-cli -c -h <ip> -p <port> --user <username> --pass <password> --cluster reshard <ip>:<port> --cluster-from <node-id> --cluster-to <node-id> --cluster-slots 500 --cluster-yes

But the command will eventually tell me that it refused connection to the other nodes in the cluster, with one message per node like:

Could not connect to Redis at <ip>:6379: Connection refused

I see that it is trying to connect on the 6379 port, so its not using the cluster bus to communicate but instead connecting to the redis port. I have the default user turned off, so I believe it refused the connection because of the auth failure. 

How can I tell the reshard command the username and passwords to use when it tries to connect to these nodes in the resharding? Thanks!

[Benjamin Sergeant]

I believe that the cli communicates directly with the nodes, on the normal 6379 port, not everything goes over the gossip port.

Did you add a user/password to the old cluster nodes ?

I would expect a different error than 'Connection refused' when the auth failed (like permission denied).

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/543da52d-60ae-40ce-94ce-320ecfed7501%40googlegroups.com.

[Benjamin Sergeant]

commit eda703ab284f1ce491fb2c376fd08fe8aa956c62

Author: antirez <ant...@gmail.com>

Date:   Mon Sep 23 19:57:13 2019 +0200

    redis-cli: support for ACL style user/pass AUTH.

Support came from this commit it seems. You could also try to use this environment variable REDIS_CLI_AUTH_ENV / but it should be equivalent.

[Benjamin Sergeant]

Have you set masteruser and masterauth in your config file ?

user default off nopass ~* +@all

user bob on >robert ~* +@all

masteruser bob

masterauth robert

It is not mentioned yet in the doc which is probably slowly transitioning to redis 6  https://redis.io/topics/replication

With those options, I can run a cluster with ACL and no default user.

[Benjamin Sergeant]

Wilson you might have run into a real bug actually.

I just noticed in the source code that the MIGRATE command, required to reshard, should take an AUTH2 argument when ACL are on.

(from https://redis.io/commands/migrate)

• AUTH -- Authenticate with the given password to the remote instance.
• AUTH2 -- Authenticate with the given username and password pair (Redis 6 or greater ACL auth style).

The redis-cli does not have support for this yet I think, but I'm sure it will come in soon. I might try to fix it and make a PR for it.

    if (config.auth) {                                                         

        argv[offset] = "AUTH";                                                 

        argv_len[offset] = 4;                                                  

        offset++;                                                              

        argv[offset] = config.auth;                                            

        argv_len[offset] = strlen(config.auth);                                

        offset++;                                                              

    }                                                                          

[Benjamin Sergeant]

I made a ticket for it as I strongly believe it's a bug -> https://github.com/antirez/redis/issues/7166

[Wilson Lindelof]

Thanks Benjamin, I'll follow this issue you raised.

To unsubscribe from this group and stop receiving emails from it, send an email to redi...@googlegroups.com.

[Benjamin Sergeant]

BTW in the (likely short) meantime if you like to take risks (who doesn't with data migration !) I have a resharding tool written in python that supports this.

Install with

curl -sL https://raw.githubusercontent.com/machinezone/rcc/master/tools/install.sh | sh

(you'll need Python-3.6 + a C compiler)

Then to migrate a slot from one host to another:

rcc migrate --password robert --user bob --src-addr redis://127.0.0.1:11002 --dst-addr redis://127.0.0.1:11000 12182

I just added a top like thing to it that scrapes INFO ; the labels are still wip.

node                 role    vers       clients  rss        cpu(s)    cpu(u)      recv      sent    ops

-------------------  ------  -------  ---------  -------  --------  --------  --------  --------  -----

172.25.13.194:6379   master  5.9.103       1569  59.64M     524365  187262    20193.6   25025.2   10952

172.25.223.216:6379  master  5.9.103       1567  38.04M     424357  151316    15876.5   16192.2    8397

172.25.76.195:6379   master  5.9.103       1566  52.35M     509350  199373     9583.74   9828.09   6625

172.26.11.251:6379   master  5.9.103       1567  81.04M     415993  151441     8052.81   8310.19   6983

172.26.163.119:6379  master  5.9.103       1567  72.15M     455918  164208     8562.1    9892.12   7478

172.26.165.170:6379  master  5.9.103       1567  99.35M     432522  176379     9502.55  10060.5    7057

172.26.182.31:6379   master  5.9.103       1574  106.29M    453076  195209     9440.52  10531.2    7791

172.26.199.125:6379  master  5.9.103       1568  58.36M     486969  167407    10028.6   14815.5    7968

172.26.78.90:6379    master  5.9.103       1567  60.79M     413430  149223     9196.23   9452.31   6452

172.28.109.130:6379  slave   5.9.103         42  52.77M     147446   95098.1   9308.64      2.51   6440

172.28.118.46:6379   slave   5.9.103         42  32.26M     188363  121348    20269.5       2.52  10908

172.28.157.217:6379  slave   5.9.103         42  23.10M     146265   99892.1  15941         2.49   8365

172.28.93.42:6379    slave   5.9.103         42  32.62M     197033  125681     9654.72      2.51   6607

172.29.59.206:6379   slave   5.9.103         42  68.28M     140138   86713.3   8128.7       2.54   6961

172.30.215.37:6379   slave   5.9.103         42  39.60M     165981  102012     8571.57      2.52   6783

172.30.237.224:6379  slave   5.9.103         42  79.64M     177644  106364     9567.14      2.52   6768

172.30.99.136:6379   slave   5.9.103         42  77.42M     155858   99479.9   9202.74      2.52   7070

172.31.195.50:6379   slave   5.9.103         42  34.62M     176102  107096     9978.23      2.52   6883

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/042c4da6-df06-488d-9066-ab879f7bfcc1%40googlegroups.com.