Replacement of stored procedures

[jasbir singh]

I have a dataset of approx 5 million records. For every scenario there is 5 million record attached to it. If i update at ALL level then all the records get dirty and fresh data will be stored in all 5 million records. But i have a problem in which I only update suppose 5000 records. For this scenario what is the best way to get the non dirty data ? Earlier in Relational DB i was running a stored procedure in which i was replicating non dirty data from old scenario to new created scenario. What will be the best approach to solve it in MongoDB and how?

Regards,

Jasbir Singh

[Wan Bachtiar]

Hi Jasbir,

To help digging more on this:

1. Can you give more information on the document schema ?
2. Does ‘dirty’ in this case means that the record has been modified ?
3. Would the records that have been marked dirty ever be reset back to non-dirty state at some later stage ?

Based on the information that you have described, perhaps a new field could be added to your document schema to distinguish between dirty and non-dirty records.

For example, the value of the dirty field could be a last modified date:

{
 ...
 lastModified:  ISODate("2015-09-23T04:11:18.965Z")
}

On an update operation, also set the field to mark all the documents updated as ‘dirty’ state.

The following example updates all documents matching status=test, sets lastModified field to the current date, and sets example field to 99.

/* Get a current date */
var newLastModified = new ISODate();
db.collection.update(
   { status: 'test' },
   { $set: {
        lastModified: newLastModified,

        /* Any fields that you want to update */
        example: 99
        }
   },
   { multi: true }
)

Afterwards, an example to query the records that were not updated :

/* Find the latest updated record */
var updatedDoc = db.collection.findOne({
                         $query: {}, 
                         $fields: { lastModified:1 }, 
                         $orderby: { lastModified:-1 }
                     }
                  )

/* Find all records that are older than the last batch of updates */
db.collection.find({ lastModified: { $lt: updatedDoc.lastModified } } );

Kind Regards,

Wan.

​