Multiple coordinators

[Timur Luchkin]

Hi.

I want to have multiple coordinators with possibility to point it to the Replicas workers.

I have a setup with automatic SQL migrations (Liquibase) and many tables with daily/hourly timescale partitions (which can be both added and/or dropped).  So number of DDL changes can be pretty high per day. 

What options do I have to implement these multiple coordinators without stopping replicas and rsync them every 10 minutes? 

[jelte....@microsoft.com]

You can add another worker node to the cluster with shouldhaveshards=false and use that almost like a second coordinator. It won't have any shards of distributed tables, but you can still use it to query the other nodes. The only thing you cannot do on this node is DDL. You can set shouldhaveshards=false using citus_set_node_property: https://docs.citusdata.com/en/v11.1/develop/api_udf.html#citus-set-node-property

[Timur Luchkin]

Probably I missed something.  

Lets assume we have 2 coordinators : master (CM1) and replica (CR1). And we have 2 workers with 2 masters (WM1 and WM2) and 2 corresponding replicas (WR1 and WR2). 

I want to have a possibility to run queries on workers replicas (WR1 and WR2).  

So you're suggesting me to add worker replicas (WR1 and WR2) to the coordinator with citus_set_node_property(shouldhaveshards=false). But how it will help me? 

[jelte....@microsoft.com]

Ah yes, I misread your initial email. I now understand you want to have a read only copy of your main cluster, with a replica for the coordinator and replicas for each worker.

In that case you should first add all replicas as "secondary" nodes to the cluster with citus_add_node and specifying noderole='secondary',  groupid={groupid of matching primary} and nodecluster='read_only_1': https://docs.citusdata.com/en/v11.1/develop/api_udf.html#citus-add-node

Then you should change the following two settings in the postgresql.conf file of your coordinator replica:
citus.use_secondary_nodes = 'always'

citus.cluster_name = 'read_only_1'

Once you've done that the replica coordinator will connect to the replica workers when executing distributed queries.

[Timur Luchkin]

Thanks Jelte. This is exactly what I need. Let me check and reply back.