how tp do dynamic scaling using pg_shard and VMs

[Priyesh Karatha]

I am doing data base scaling using postgresql. Currently i am using pg_shard for scaling and i am able to do sharding and replication. i have tested the example that mentioned in Readme file of pg_shard. But i need dynamically scale a cluster as new machines are added or old ones are retired.I am using google cloud VM to setup database .So once one VM is filled with data i want to setup new instance with same configuration. ie,if the current machine size is 4GB and is filled then it should create one more VM with 4GB size and next entries should come there. I have gone through http://slideplayer.com/slide/4896815/ and after reading this i understood that it is possible to do but the steps are not mentioned anywhere. How to achieve this using pg_shard?

[Sumedh Pathak]

Hi Priyesh,

Thanks for your question, and sorry for the delay in getting back to you. As I understand, you want to incrementally add new nodes as the current ones get filled up with data. We can do this via our Enterprise edition of Citus, which includes the shard re-balancer utility. This utility will notice the new node, and move shards from the old nodes to the new one until the data distribution is even again. We also recommend starting with a high number of initial shards, so that re-distributing these shards for dynamic scaling is much easier.  See our documentation for more details:https://www.citusdata.com/documentation/citusdb-documentation/admin_guide/cluster_management.html. If you want to know how to get the shard rebalancer utility, please get in touch with us at sup...@citusdata.com. What use-case are you looking to use pg_shard/Citus for?

Finally, we are launching Citus 5.0 soon, which is open-source and includes all the pg_shard functionality as well as Citus query parallelization. Please keep an eye on our website, or send a note to in...@citusdata.com for more information.