Can not load some documents when restart MongoDB process.

[陆峰]

After hard restart the server and MongoDB process, that MongoDB can not load some documents. I have not kill the MongoDB process before restart server because that server has no response and can not login with ssh. so I hard restart the that server, but I know that MongoDB has not get any update operations when I restart the server.

when I restart the server and MongoDB, I want to get all the documents in that MongoDB. I got an error and can not load documents in one collection. 

Fri Jan  3 10:22:52.662 Assertion: 10320:BSONElement: bad type 110

0x5867d3 0x554703 0x495426 0x4a1166 0x540bcd 0x7ff26077eb1d 0x7ff260781573 0x6a85800616e 

 mongo(_ZN5mongo15printStackTraceERSo+0x23) [0x5867d3]

 mongo(_ZN5mongo11msgassertedEiPKc+0x93) [0x554703]

 mongo(_ZNK5mongo11BSONElement4sizeEv+0x126) [0x495426]

 mongo(_ZNK5mongo7BSONObj8getFieldERKNS_10StringDataE+0x86) [0x4a1166]

 mongo() [0x540bcd]

 /usr/lib64/libv8.so.3.18.5.14(+0x2bdb1d) [0x7ff26077eb1d]

 /usr/lib64/libv8.so.3.18.5.14(+0x2c0573) [0x7ff260781573]

 [0x6a85800616e]

Error: 10320 BSONElement: bad type 110

> 

> it

Fri Jan  3 10:23:41.744 mongo got signal 11 (Segmentation fault), stack trace: 

Fri Jan  3 10:23:41.744 mongo got signal 11 (Segmentation fault), stack trace: 

Fri Jan  3 10:23:41.771 0x5867d3 0x48be64 0x7ff25e3e3fd0 0x49484e 0x53af60 0x53fd0f 0x540c42 0x7ff2606eb281 0x7ff2606eb6aa 0x7ff26054a142 0x7ff26073890b 0x6a85800616e 

 mongo(_ZN5mongo15printStackTraceERSo+0x23) [0x5867d3]

 mongo(_Z12quitAbruptlyi+0xe4) [0x48be64]

 /lib64/libc.so.6(+0x37fd0) [0x7ff25e3e3fd0]

 mongo(_ZN5mongo10toHexLowerEPKvi+0xae) [0x49484e]

 mongo(_ZN5mongo7V8Scope5newIdERKNS_3OIDE+0x40) [0x53af60]

 mongo(_ZN5mongo7V8Scope16mongoToV8ElementERKNS_11BSONElementEb+0x6ff) [0x53fd0f]

 mongo() [0x540c42]

 /usr/lib64/libv8.so.3.18.5.14(+0x22a281) [0x7ff2606eb281]

 /usr/lib64/libv8.so.3.18.5.14(+0x22a6aa) [0x7ff2606eb6aa]

 /usr/lib64/libv8.so.3.18.5.14(+0x89142) [0x7ff26054a142]

 /usr/lib64/libv8.so.3.18.5.14(+0x27790b) [0x7ff26073890b]

 [0x6a85800616e]

And I can not repair that database 

> db.repairDatabase()

{

        "errmsg" : "exception: BufBuilder attempted to grow() to 268435456 bytes, past the 64MB limit.",

        "code" : 13548,

        "ok" : 0

}

May be some database file was destroyed. 

How can I fix this problem?

Many thanks.

[Eoin Brazil]

Hi,

Just to understand this setup, it is a stand alone non-replicated non-sharded single MongoD instance that is running ?

An exception "BSONElement" in MongoDB has generally two causes when a bad BSON type error will be throw, i) the pointer has got mangled or ii) there is invalid BSON being stored in the database.

Can you run the validate command against the collection and post the results, in particular the item of interest is 'validate.invalidObjects' to see if the database contains any corrupt BSON documents.

> db.collection.validate(true)

or

> db.runCommand( { validate: "collection", full: true } )

To help understand what is happening with your database can you post the log output information. The best way to get this information is for you to increase the logLevel to 2, run your query again, then drop the logLevel back to its previous value, and post the server log here.

To increase the log level use these commands

> use admin
> db.runCommand( { setParameter: 1, logLevel: 2 } )

To set the log levels back to their previous value

> use admin
> db.runCommand( { setParameter: 1, logLevel: 0 } )

The final thing would be to perform a mongodump on the database to see if you can export the database to disk without errors.

$ mongodump -d database --repair

These logs will help in understanding exactly what is happening and allow for a better understanding of what exactly the problem is.

Best regards,

Eoin