Adding TTL to existing collection with large data volume

[bob]

Hi,

I am currently using MongoDB 2.0.6. My document does not contain any data field as of now. I want to use TTL from MongoDB 2.2.0 to remove my expired documents.

we have a field "md" which stores modification data of document as integer, e.g. 20120925 etc. As per the document http://docs.mongodb.org/manual/tutorial/expire-data/, To implement TTL, we need indexing on variable of type 'bsonDate'. My data in store is big and update operations can not be performed to add any bsonDate type variable.

Is there any trick around it or a way other than data migration (like read document and store with 'md' as bsonDate; where migration will take long time)?

Thanks You in advanced.
Bob.

[bob]

Isn't there any way for this?

[Scott Hernandez]

You would need to update all your data to be dates and that will take
a while to process if the data is large (and need to read off disk).

There is no trick or shortcut.

> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb

[Luciano Issoe]

Hi Scott,

I think the process of creating the TTL index from a large collection can take a very long time by itself. 

In both cases ( creating TTL indexes or replacing a field on every document and then create the index ) should take long times and should be avoided.

In my opinion, one should try to live with two collections and phase out the one without TTL, as soon as possible.

What is your say on that? 

Regards,

Luciano

[Scott Hernandez]

You can do it any number of ways. That is one approach.

Yes, you will need to build the index and that requires reading all the docs.

[bob]

Hi team,

We used another approach for this and it worked well.

1. We added TTL indexing on md field which is of type integer ( TTL set for 30 days).

2. we started new inserts and updates with 'md' as date type.

3. after 30 days, we started deleting all the document's with 'md' as data type 'integer' which affects performance a bit but kind of acceptable. 

4. we fully migrated with TTL on 'md' field which now is of type 'date'

MongoDB working fine after migration for 6 months now. :)

Thanks a lot for your guidance guys !!

bob.

On Tuesday, 25 September 2012 20:50:10 UTC+5:30, bob wrote: