Open Source Citus backup/restore options

[apyh...@gmail.com]

Hello.

We are investigating possibilities to use open source citus to scale our database (> 100 GB raw data). 
So far I have several questions, related to backup, recovery and failover.
1) If I have shards stored with shard_replication_factor set to > 1, and one of DB node goes down, I have to manually
run master_copy_shard() to resync shard when node is up. Is there any preferred way to automate this?

2) As I understand, we want to backup list of shards on each  node to be able to restore after node failure (the easiest way
is to backup schema). Then when we take new server up,
we can restore node from backup, mark shards placed on this node as inactive  and call master_copy_shard() to resync data.

Something like

update pg_dist_shard_placement set shardstate =3 where nodeport='5432' and nodename='node1';
select master_copy_shard_placement(shardid,nodename,nodeport,'node1','5432') from pg_dist_shard_placement where shardid in  (select shardid from pg_dist_shard_placement where nodeport=5432 and nodename='node1' ) and (not (nodeport=5432 and nodename='node1'))  ;

What is the preferred solution  to recover from one node failure?

3) What is suggested way to dump coordinator node? pg_dump seems to ignore pg_dist_* tables, even if explicitly say to backup them.

4) What is suggested way to recover whole cluster?

We would like to avoid PITR due to inadequate speed to apply even one-day WAL volume and large volume of wal files. 
--
Best regards,
Alexander Pyhalov,
system administrator of Southern Federal University IT department