How to implement Distributed Replication in iDempiere

[Víctor Arvelaiz]

We have a project where the client has different branches distributed nationwide, due to the deficiency in the internet service, he wants each branch to have a local server and each branch to replicate its transactions to the central server, and from the central server to replicate to each branch.

A distributed data approach.  In one instance we implemented replication between two servers and in that instance we used SymmetricSD and a different table correlation sequence on each server.

We are consulting because we want to optimize the process. We are concerned about implementing ActiveMQ because it does not store the replication queue in DB and when the ActiveMQ server is restarted the queue is lost, this happened to us in 2018 when we evaluated implementing replication with ActiveMQ and that is why we are inclined to use SymmetricSD.

I would like to read your comments to know the approach you would implement.

thanks in advance

[Pedro Rozo]

Victor,  low level replication with SymmetricSD  is tricky, because you would need to update not just basic tables, but also all the dependent objects: example: if you want to replicate a invoice, you would need to track and replicate changes to the related objects; products, price list, bpartners and so on, and send them in the correct order to the other side .. plus update the internal auditing tables of idempiere ..to keep both sides consistent . .. trying to do that just with database access is timeconsuming and not 100% safe for all the erp info.

However, another different approach is 'smart selective replication'  You might be interested in play with our recent SmartJSP release in the idempiere community: SmartIntegrator:   https://sourceforge.net/projects/smartintegrator/

that addresses that common pattern ... you would need to add some additional features/logic for the most advanced scenarios and perform more testing  .. but that provides smart replication (incoming adn outgoing) including object dependencies and others verification using the idempiere API  out of the box ..for transportation layer we use apache kafka backend (better than active mq) .. and we have optimized the objects with simplified json (optional)... all the documents. code and so on is here .. compatible with id 9 and id 10  

Hope that helps 

Regards, 

Pedro R

[Víctor Arvelaiz]

Thank you Pedro.. 

I will review what you have told me and go a little deeper....

A query, with this solution if the Server is restarted, resumes the queue that is pending to synchronize? or is lost?....

that is exactly one of the weak points with ActiveMQ and that is why we used SymmetricSD at the time, the solution that you refer me to, how does it behave in that case?