How to control same changestream mongodb notification data from multiple client

[sanjeev kumar]

Hi All,

I have created a java class which using changestream api of mongodb and able to get notification of changes happening in mongodb collection. We need to process this notification data to process the business logic. 

I have deployed this java class in the cloud environment where we have four instances of this java class running. Now after testing i found that these four instances are getting same data.

can anybody suggest me how to get the notification data in all the servers and no two servers will get the same data.

My java code copied below printing same data in all the servers.

public String startProcess() {

MongoClient mongoClient = new MongoClient(new MongoClientURI("mongodb://localhost:27017,localhost:27018,localhost:27019/user?replSet=simpliReplica"));

MongoDatabase database = mongoClient.getDatabase("mypoc");

MongoCollection<Document> collection = database.getCollection("user");

try {

Block<ChangeStreamDocument<Document>> printBlock = new Block<ChangeStreamDocument<Document>>() {

public void apply(final ChangeStreamDocument<Document> changeStreamDocument) {

System.out.println(" MyService:::"+changeStreamDocument.getFullDocument());

}

};

// collection.watch - Establishes a Change Stream on a collection.This will identify any changes happening to the  collection.

collection.watch(asList(Aggregates.match(Filters.in("operationType", asList("insert", "update", "replace", "delete")))))

.fullDocument(FullDocument.UPDATE_LOOKUP).forEach(printBlock);

} catch (IOException e) {

e.printStackTrace();

}

}

}

.

Regards

Sanjeev

[Wan Bachtiar]

Now after testing i found that these four instances are getting same data.

Hi Sanjeev,

By default, processes listening to the same change streams notification will receive the same data. In order to process the notifications without any duplication, you would need to handle this on the application/client side. 

can anybody suggest me how to get the notification data in all the servers and no two servers will get the same data.

The answer would be depending on your environments and requirements. There are a few methods that you could try:

• Create a worker pool, where you have only one instance listening to the Change Streams then delegate the task to one of the worker instances for processing.

• Add an aggregation filter to each of the instances i.e. $match pipeline stage. For example, one instance is only going to handle notification where user has a specific id, etc.

• Create a collection for locking. First instance able to insert a lock in a collection can process the notification.

Each of the methods above have their own pros and cons depending on your application use case.

Regards,
Wan.

​