MongoDB - How to delete expired documents except last one

[Josip Grgurica]

Hi,

We're working on audit log solution for one of our projects and would appreciate your help. We have media entity which contains media id (numeric), action (string, eg. PUBLISH) and occurred on date with time representing date and time when action happened. Requirement is to delete logs that are older than 90 days but to keep log with last action on media entity.

We are using Mongo 3.2 and we need help how to organize our collection since we need to support reads and writes on collection that will contain 200-300 millions of documents.

We tried several approaches but couldn't figure out a simple way on how to do it. Any suggestions on how to organize schema for this scenario?

Thanks,
Josip

[Weishan Ang]

Hi,

How about using TTL documents?

[Kevin Adistambha]

Hi Josip,

Could you elaborate further on your requirements? For example:

• A step by step walkthrough of your process, and the use case associated with it
• Example documents (“media entity”, “action”, “log”, “lastaction”, etc.)
• What approaches have you tried so far

Best regards,
Kevin

​

[Josip Grgurica]

Hi,

@Weishan Ang: Thanks you on the suggestion but TTL documents don't solve requirement to keep  log with last action on media entity.

@Kevin Adistambha: We have decided to use separate collection for this problem but thank you on help.