On Thu, Apr 09, 2020 at 09:44:09AM -0700, Ultrabug wrote:
> Option 1- extend the protocol to allow clients to specify a shard_id to
> connect to
>
> Maybe we could add a key in the protocol so that clients could specify the
> shard_id they want to connect to making connections-to-shard predictable.
> I have no clue how hard it is or the consequences, so please go ahead.
>
How do you specify a shard_id while connecting? A shard is chosen
during connection (in case of DPDK by HW hash). Moving a connection after it was
created is pretty hard especially if we want to continue supporting DPDK.
> ---
>
> Option 2 -change the way nodes assign shards to client connections on scylla
>
> Maybe we could have nodes assign shards in a round-robin manner but per
> client.
> This would save us from this eternal race and competition between multiple
> connections originating from multiple clients.
>
Connection is created before a client is knows. Load balancing already
works per server port which is the only thing that identifies a
connection during connect.
Seastar supports "port" load balancer that allows a client to connect to
a specific shard by choosing a local port carefully. A connection is
created to a shard_id = src_port % smp::count. This is how internal rpc
connects to a right shard if a cluster is homogeneous. CQL port right
now does not uses this load balancer in the fear that it will negatively
affect non shard aware drivers).
--
Gleb.