repair doesnt work, always get a ns not found

Kaustubh Padegaonkar

unread,

Jan 4, 2011, 6:53:12 AM1/4/11

to mongod...@googlegroups.com

I have lots of map reduce collections that I want to delete, so I use this script:

function f() {
   var names = db.getCollectionNames();
   for(var i = 0; i < names.length; i++){
   if(names[i].indexOf("tmp.") == 0){
   db[names[i]].drop();}}};

After running this function in the shell, I get:

"nIndexesWas" : 1,
"errmsg" : "exception: drop: dropIndexes for collection failed - consider trying repair cause: Invalid BSONObj size: 0 (0x00000000) first element: EOO",
"code" : 12503,
"ok" : 0

So I stopped the daemon, started it with the following command:

sudo mongod --repair --dbpath /var/lib/mongodb/

on which I get this output:

sudo mongod --repair --dbpath /var/lib/mongodb/
Tue Jan 4 17:20:24 MongoDB starting : pid=19128 port=27017 dbpath=/var/lib/mongodb/ 64-bit

** NOTE: This is a development version (1.7.4-pre-) of MongoDB.
** Not recommended for production.

Tue Jan 4 17:20:24 db version v1.7.4-pre-, pdfile version 4.5
Tue Jan 4 17:20:24 git version: d9307addab35b2262f283183177fd260a1ad2ac3
Tue Jan 4 17:20:24 sys info: Linux domU-12-31-39-06-79-A1 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Tue Jan 4 17:20:24 [initandlisten] ****
Tue Jan 4 17:20:24 [initandlisten] ****
Tue Jan 4 17:20:24 [initandlisten] need to upgrade database prolinkd with pdfile version 4.5, new version: 4.5
Tue Jan 4 17:20:24 [initandlisten] starting upgrade
Tue Jan 4 17:20:24 [initandlisten] prolinkd repairDatabase prolinkd
Tue Jan 4 17:20:24 [initandlisten] prolinkd.system.namespaces Assertion failure magic == 0x41424344 db/pdfile.h 252
0x5455d1 0x55107f 0x778657 0x7761c8 0x6353e4 0x744b44 0x74b3cb 0x74c3d1 0x588dae 0x5892e8 0x716e19 0x718b3d 0x76452d 0x88c5a8 0x88e615 0x89089c 0x8913d8 0x898156 0x7f9a55b46c4d 0x4de429
mongod(_ZN5mongo12sayDbContextEPKc+0xb1) [0x5455d1]
mongod(_ZN5mongo8assertedEPKcS1_j+0x10f) [0x55107f]
mongod(_ZNK5mongo7Forward4nextERKNS_7DiskLocE+0x187) [0x778657]
mongod(_ZN5mongo11BasicCursor7advanceEv+0xd8) [0x7761c8]
mongod(_ZN5mongo14processGetMoreEPKcixRNS_5CurOpEiRb+0x294) [0x6353e4]
mongod(_ZN5mongo15receivedGetMoreERNS_10DbResponseERNS_7MessageERNS_5CurOpE+0x294) [0x744b44]
mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0x135b) [0x74b3cb]
mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_b+0x81) [0x74c3d1]
mongod(_ZN5mongo14DBClientCursor11requestMoreEv+0x30e) [0x588dae]
mongod(_ZN5mongo14DBClientCursor4moreEv+0x58) [0x5892e8]
mongod(_ZN5mongo6Cloner2goEPKcRSsRKSsbbbb+0x3d9) [0x716e19]
mongod(_ZN5mongo9cloneFromEPKcRSsRKSsbbbb+0x3d) [0x718b3d]
mongod(_ZN5mongo14repairDatabaseESsRSsbb+0x4fd) [0x76452d]
mongod(_ZN5mongo11doDBUpgradeERKSsSsPNS_14DataFileHeaderE+0x68) [0x88c5a8]
mongod() [0x88e615]
mongod(_ZN5mongo14_initAndListenEiPKc+0x4ec) [0x89089c]
mongod(_ZN5mongo13initAndListenEiPKc+0x18) [0x8913d8]
mongod(main+0x6c16) [0x898156]
/lib/libc.so.6(__libc_start_main+0xfd) [0x7f9a55b46c4d]
mongod(__gxx_personality_v0+0x3c1) [0x4de429]
Tue Jan 4 17:20:24 [initandlisten] getmore prolinkd.system.namespaces cid:8525351411634740776 getMore: {} exception 0 assertion db/pdfile.h:252 bytes:20 nreturned:0 3ms
Tue Jan 4 17:20:24 allocating new datafile /var/lib/mongodb/$tmp_repairDatabase_0/prolinkd.ns, filling with zeroes...
Tue Jan 4 17:20:24 done allocating datafile /var/lib/mongodb/$tmp_repairDatabase_0/prolinkd.ns, size: 16MB, took 0 secs
Tue Jan 4 17:20:24 allocating new datafile /var/lib/mongodb/$tmp_repairDatabase_0/prolinkd.0, filling with zeroes...
Tue Jan 4 17:20:24 done allocating datafile /var/lib/mongodb/$tmp_repairDatabase_0/prolinkd.0, size: 64MB, took 0 secs
Tue Jan 4 17:20:24 allocating new datafile /var/lib/mongodb/$tmp_repairDatabase_0/prolinkd.1, filling with zeroes...
Tue Jan 4 17:20:24 done allocating datafile /var/lib/mongodb/$tmp_repairDatabase_0/prolinkd.1, size: 128MB, took 0 secs
Tue Jan 4 17:20:24 [initandlisten] building new index on { _id: 1 } for prolinkd.user
Tue Jan 4 17:20:24 [initandlisten] done for 14 records 0secs
Tue Jan 4 17:20:24 [initandlisten] building new index on { _id: 1 } for prolinkd.profile
Tue Jan 4 17:20:24 [initandlisten] done for 920 records 0.006secs
Tue Jan 4 17:20:24 [initandlisten] building new index on { _id: 1 } for prolinkd.statistics
Tue Jan 4 17:20:24 [initandlisten] done for 831 records 0.004secs
Tue Jan 4 17:20:24 [initandlisten] building new index on { _id: 1 } for prolinkd.connection
Tue Jan 4 17:20:24 [initandlisten] done for 1643 records 0.007secs
Tue Jan 4 17:20:24 [initandlisten] building new index on { _id: 1 } for prolinkd.update
Tue Jan 4 17:20:24 [initandlisten] done for 343 records 0.001secs
Tue Jan 4 17:20:24 [initandlisten] prolinkd.queue Assertion failure magic == 0x41424344 db/pdfile.h 252
0x5455d1 0x55107f 0x760185 0x7605d8 0x77e741 0x64cb7c 0x781b1a 0x785dd6 0x789da4 0x78cc24 0x78d827 0x638bf7 0x747dba 0x74b37e 0x74c3d1 0x586578 0x56bbca 0x716014 0x71785c 0x718b3d
mongod(_ZN5mongo12sayDbContextEPKc+0xb1) [0x5455d1]
mongod(_ZN5mongo8assertedEPKcS1_j+0x10f) [0x55107f]
mongod(_ZN5mongo11DataFileMgr7findAllEPKcRKNS_7DiskLocE+0x585) [0x760185]
mongod(_ZN5mongo13findTableScanEPKcRKNS_7BSONObjERKNS_7DiskLocE+0xe8) [0x7605d8]
mongod(_ZNK5mongo9QueryPlan9newCursorERKNS_7DiskLocEi+0x811) [0x77e741]
mongod(_ZN5mongo11UserQueryOp5_initEv+0x3fc) [0x64cb7c]
mongod(_ZN5mongo12QueryPlanSet6Runner6initOpERNS_7QueryOpE+0x11a) [0x781b1a]
mongod(_ZN5mongo12QueryPlanSet6Runner3runEv+0x396) [0x785dd6]
mongod(_ZN5mongo12QueryPlanSet5runOpERNS_7QueryOpE+0x284) [0x789da4]
mongod(_ZN5mongo16MultiPlanScanner9runOpOnceERNS_7QueryOpE+0x64) [0x78cc24]
mongod(_ZN5mongo16MultiPlanScanner5runOpERNS_7QueryOpE+0x17) [0x78d827]
mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0xf37) [0x638bf7]
mongod() [0x747dba]
mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0x130e) [0x74b37e]
mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_b+0x81) [0x74c3d1]
mongod(_ZN5mongo14DBClientCursor4initEv+0x148) [0x586578]
mongod(_ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0x32a) [0x56bbca]
mongod(_ZN5mongo6Cloner4copyEPKcS2_bbbbNS_5QueryE+0x584) [0x716014]
mongod(_ZN5mongo6Cloner2goEPKcRSsRKSsbbbb+0xe1c) [0x71785c]
mongod(_ZN5mongo9cloneFromEPKcRSsRKSsbbbb+0x3d) [0x718b3d]
Tue Jan 4 17:20:24 [initandlisten] assertion 0 assertion db/pdfile.h:252 ns:prolinkd.queue query:{}
Tue Jan 4 17:20:24 [initandlisten] exception in initAndListen std::exception: nextSafe(): { $err: "assertion db/pdfile.h:252" }, terminating
Tue Jan 4 17:20:24 dbexit:
Tue Jan 4 17:20:24 [initandlisten] shutdown: going to close listening sockets...
Tue Jan 4 17:20:24 [initandlisten] shutdown: going to flush oplog...
Tue Jan 4 17:20:24 [initandlisten] shutdown: going to close sockets...
Tue Jan 4 17:20:24 [initandlisten] shutdown: waiting for fs preallocator...
Tue Jan 4 17:20:24 [initandlisten] shutdown: closing all files...
Tue Jan 4 17:20:24 closeAllFiles() finished
Tue Jan 4 17:20:24 [initandlisten] shutdown: removing fs lock...
Tue Jan 4 17:20:24 dbexit: really exiting now

I use ubuntu-server 10.04, and have installed mongo from the repositories mentioned on the mongodb downloads page. I get the same result with mongodb-stable, and mongodb-snapshot.

How do I repair the db now?

Thanks.

Nat

unread,

Jan 4, 2011, 7:04:42 AM1/4/11

to mongodb-user

It looks like your system crashed earlier. Is this database a
development database? If so, it may be easier to start from scratch
again.

Eliot Horowitz

unread,

Jan 4, 2011, 8:57:58 AM1/4/11

to mongod...@googlegroups.com

Firstly, I would definitely not use a 1.7. nightly if at all possible.

Given you are, if that's not working, the easiest thing might be to
mongodump the good collections, wipe data, then mongorestore.

> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=en.
>

Kaustubh Padegaonkar

unread,

Jan 4, 2011, 9:45:09 AM1/4/11

to mongod...@googlegroups.com

Well, thats true, i am using a 1.7.2 version, but its because I tried it on mongodb-stable, and I just wanted to check what happens on other versions. This is an internal test server, but with some amount of "production" data. Ill do as Eliot says.

Thanks for your help :)

Nadir Boussoukaia

unread,

Apr 24, 2014, 3:09:17 PM4/24/14

to mongod...@googlegroups.com

on my DB even mongodb --repair was failing.

What I did is simply removed the journal files then run again the mongod -f /etc/mongodb.conf --repair

then removed the lock file /mnt/ebs/mongodb/mongod.lock (and a chown -R mongodb:mongodb <db folder> )

Asya Kamsky

unread,

Apr 24, 2014, 4:19:55 PM4/24/14

to mongodb-user

Please don't reply to really old threads, especially when it's probably about very different versions of OS, MongoDB, etc.

And it is absolutely not advised to do what you described - it's very likely to result in data loss or corruption.

Asya

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---

You received this message because you are subscribed to the Google Groups "mongodb-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.

Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/cb0250a6-fdcf-4bf3-bf95-a2555c1deeec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all

Reply to author

Forward