Mongodb oplog unmarshall into a json object

[Gordon Hu]

Hi,

I have a project where we would to leverage Mongodb oplog  to get our mongodb in synch with our redis cache.

Below is what the log logs like

{

"ts": "Timestamp(1415124334, 1)",

"op": "i",

"ns": "test.products",

"o": {

"_id": ObjectId("5459156e7950e8c18c094992"),

"item": "card",

"qty": 15

}

}

I would like to convert it into json or POJO so I can use  Camel to route the message to appropriate handler.

The problem I am having is the document is not a json object. 

So does anyone know how to do the following:

1) Convert the above into json (Timstamp is a BSON object)

2) Convert the above into  a POJO

Thanks,

Gordon

[Tugdual Grall]

Hello,

I am not sure to understand the challenge here.

If you query the oplog collection using one of the MongoDB driver, for example Java it will return a object that you can easily serialize into a JSON object:

For example an opload entry will look like:

---

{

"ts" : {

"$ts" : 1413959603,

"$inc" : 18241

},

"h" : 2815469582040261600,

"v" : 2,

"op" : "i",

"ns" : "movielens.ratings",

"o" : {

"_id" : {

"$oid" : "54474fb33004ade75db1f9ec"

},

"user_id" : 2419,

"item_id" : 1580,

"preference" : "4",

"timestamp" : 974247385

}

}

---

If you need to you can use the "BSON Type" of each field to process them the way you want, for example using the BSONTimestamp ( http://api.mongodb.org/java/2.6/org/bson/types/BSONTimestamp.html ) , or ObjectID ( http://api.mongodb.org/java/2.6/org/bson/types/ObjectId.html )

Regards
Tug
@tgrall

[Gordon Hu]

Thanks for the reply,

So the idea is that we are using rabbitmq to poll the Mongodb  oplog and route the log to different queues based on the operation(insert, update...etc)

Camel will thenbe  hooked up with  rabbitmq consumers to consume the oplog message.

The problem I am facing now is how to turn the below oplog data into json or pojo since "ts": Timestamp(1415124334, 1) 

is not json so it cannot be converted into a pojo to be processed by Camel.

Any suggestions/ideas?

Gordon

[Tugdual Grall]

Do you mind sharing some code around RabbitMQ and/or Camel to be sure I can test/reproduce and find a solution