Hi,
we use MongoDB to store some cookie information for every request in a collection. Up to know we create for every customer for every day a single collection. Every night a cronjob simple drops the collections which are older than the cookie lifetime.
For evaluation purposes this is going to be a nightmare. Best way would be to store everything in one collection and remove the documents which are older than the cookie lifetime. We have approx. 10 mio hits per day and keep those documents about 30 days - so after all our collection has 300 mio documents.
We tried this with a 2.x version of MongoDB and noticed that the index was rebuild every time a single document was removed and this breaks our cluster.
What would be the best way to remove those old documents?
Kind regards
Marcus