MongoDB crashes after maxing out HD

[MKN Web Solutions]

I'm sure this is an easy fix by removing something, but I want to come here before I spend any time going through documentation.

MongoDB running on VBox, I maxed the space and memory (Created a new table) from the VM causing everything to come to a halt.  I ended up doing a forced reboot.  I set up a new virtual HD with double the space and moved all MongoDB files into it.  I set up the new path in the config; still wouldn't work so I ran a repair, which seemed to pick up all my dbs correcty, but every time I try to init mongod I get the following:

root@WebOS:/media/mongo/mongodb# mongod --config /etc/mongodb.conf -vvvv

Wed Aug 15 13:08:58 isInRangeTest passed

Wed Aug 15 13:08:58 shardKeyTest passed

Wed Aug 15 13:08:58 shardObjTest passed

Wed Aug 15 13:08:58 BackgroundJob starting: DataFileSync

Wed Aug 15 13:08:58 versionCmpTest passed

Wed Aug 15 13:08:58 versionArrayTest passed

Wed Aug 15 13:08:58 [initandlisten] MongoDB starting : pid=1823 port=27017 dbpath=/media/mongo/mongodb master=1 64-bit host=WebOS

Wed Aug 15 13:08:58 [initandlisten] db version v2.0.7, pdfile version 4.5

Wed Aug 15 13:08:58 [initandlisten] git version: 875033920e8869d284f32119413543fa475227bf

Wed Aug 15 13:08:58 [initandlisten] build info: Linux ip-XXX SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41

Wed Aug 15 13:08:58 [initandlisten] options: { config: "/etc/mongodb.conf", dbpath: "/media/mongo/mongodb", master: "true", vvvv: true }

Wed Aug 15 13:08:58 [initandlisten] flushing directory /media/mongo/mongodb

Wed Aug 15 13:08:58 [initandlisten] journal dir=/media/mongo/mongodb/journal

Wed Aug 15 13:08:58 [initandlisten] recover : no journal files present, no recovery needed

Wed Aug 15 13:08:58 [initandlisten] flushing directory /media/mongo/mongodb/journal

Wed Aug 15 13:08:58 [initandlisten] flushing directory /media/mongo/mongodb/journal

Wed Aug 15 13:08:58 [initandlisten] runQuery called local.system.namespaces { name: /^local.temp./ }

Wed Aug 15 13:08:58 [initandlisten] Accessing: local for the first time

Wed Aug 15 13:08:58 [initandlisten] query local.system.namespaces query: { name: /^local.temp./ } reslen:20 0ms

Wed Aug 15 13:08:58 [initandlisten] enter repairDatabases (to check pdfile version #)

Wed Aug 15 13:08:58 [initandlisten] xx

Wed Aug 15 13:08:58 [initandlisten] Accessing: xx for the first time

Wed Aug 15 13:08:58 [initandlisten] local

Wed Aug 15 13:08:58 [initandlisten] xx

Wed Aug 15 13:08:58 [initandlisten] Accessing: xx for the first time

Wed Aug 15 13:08:58 [initandlisten] it

Wed Aug 15 13:08:58 [initandlisten] Accessing: it for the first time

Wed Aug 15 13:08:58 [initandlisten] xx

Wed Aug 15 13:08:58 [initandlisten] Accessing: xx for the first time

Wed Aug 15 13:08:58 [initandlisten] xx

Wed Aug 15 13:08:58 [initandlisten] Accessing: xx for the first time

Wed Aug 15 13:08:58 [initandlisten] xx

Wed Aug 15 13:08:58 [initandlisten] Accessing: xx for the first time

Wed Aug 15 13:08:58 [initandlisten] done repairDatabases

Wed Aug 15 13:08:58 [initandlisten] master=true

Wed Aug 15 13:08:58 [initandlisten] Accessing: local for the first time

Wed Aug 15 13:08:58 [initandlisten] runQuery called local.oplog.$main { query: {}, orderby: { $natural: -1 } }

Wed Aug 15 13:08:58 [initandlisten]  local.oplog.$main Assertion failure isOk() db/pdfile.h 300

0x57a616 0x58515b 0x85ed70 0x8c3e95 0x8a760a 0x8cba8e 0x94d7d0 0x8ce7ea 0x8cec03 0x8cf10c 0x8d9b1e 0x8dd81e 0x8df045 0x8e03f7 0x944455 0x888fd7 0x88dbb9 0x88f4c1 0x5e380d 0x5b70ec 

 mongod(_ZN5mongo12sayDbContextEPKc+0x96) [0x57a616]

 mongod(_ZN5mongo8assertedEPKcS1_j+0xfb) [0x58515b]

 mongod(_ZNK5mongo16NamespaceDetails10lastRecordERKNS_7DiskLocE+0x140) [0x85ed70]

 mongod(_ZN5mongo19ReverseCappedCursorC1EPNS_16NamespaceDetailsERKNS_7DiskLocE+0xf5) [0x8c3e95]

 mongod(_ZN5mongo13findTableScanEPKcRKNS_7BSONObjERKNS_7DiskLocE+0x57a) [0x8a760a]

 mongod(_ZNK5mongo9QueryPlan9newCursorERKNS_7DiskLocEi+0x29e) [0x8cba8e]

 mongod(_ZN5mongo11UserQueryOp5_initEv+0x780) [0x94d7d0]

 mongod(_ZN5mongo7QueryOp4initEv+0x11a) [0x8ce7ea]

 mongod(_ZN5mongo12QueryPlanSet6Runner6initOpERNS_7QueryOpE+0x23) [0x8cec03]

 mongod(_ZN5mongo12QueryPlanSet6Runner4initEv+0x2ec) [0x8cf10c]

 mongod(_ZN5mongo12QueryPlanSet6Runner22runUntilFirstCompletesEv+0x1e) [0x8d9b1e]

 mongod(_ZN5mongo12QueryPlanSet5runOpERNS_7QueryOpE+0x11e) [0x8dd81e]

 mongod(_ZN5mongo16MultiPlanScanner9runOpOnceERNS_7QueryOpE+0x525) [0x8df045]

 mongod(_ZN5mongo16MultiPlanScanner5runOpERNS_7QueryOpE+0x17) [0x8e03f7]

Wed Aug 15 13:08:58 BackgroundJob starting: PeriodicTask::Runner

Wed Aug 15 13:08:58 BackgroundJob starting: ClientCursorMonitor

Wed Aug 15 13:08:58 BackgroundJob starting: snapshot

 mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0xfa5) [0x944455]

 mongod() [0x888fd7]

 mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x559) [0x88dbb9]

 mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs+0x81) [0x88f4c1]

 mongod(_ZN5mongo14DBClientCursor4initEv+0xad) [0x5e380d]

 mongod(_ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0x3ac) [0x5b70ec]

Wed Aug 15 13:08:58 [initandlisten] assertion 0 assertion db/pdfile.h:300 ns:local.oplog.$main query:{ query: {}, orderby: { $natural: -1 } }

Wed Aug 15 13:08:58 [initandlisten]  ntoskip:0 ntoreturn:1

Wed Aug 15 13:08:58 [initandlisten] query local.oplog.$main query: { query: {}, orderby: { $natural: -1 } } ntoreturn:1 exception: assertion db/pdfile.h:300 reslen:61 3ms

Wed Aug 15 13:08:58 [initandlisten] User Assertion: 13106:nextSafe(): { $err: "assertion db/pdfile.h:300" }

Wed Aug 15 13:08:58 [initandlisten] exception in initAndListen: 13106 nextSafe(): { $err: "assertion db/pdfile.h:300" }, terminating

Wed Aug 15 13:08:58 dbexit: 

Wed Aug 15 13:08:58 [initandlisten] shutdown: going to close listening sockets...

Wed Aug 15 13:08:58 [initandlisten] shutdown: going to flush diaglog...

Wed Aug 15 13:08:58 [initandlisten] shutdown: going to close sockets...

Wed Aug 15 13:08:58 [initandlisten] shutdown: waiting for fs preallocator...

Wed Aug 15 13:08:58 [initandlisten] shutdown: lock for final commit...

Wed Aug 15 13:08:58 [initandlisten] shutdown: final commit...

Wed Aug 15 13:08:58 [initandlisten] shutdown: closing all files...

Wed Aug 15 13:08:58 [initandlisten] closeAllFiles() finished

Wed Aug 15 13:08:58 [initandlisten] journalCleanup...

Wed Aug 15 13:08:58 [initandlisten] removeJournalFiles

Wed Aug 15 13:08:58 [initandlisten] flushing directory /media/mongo/mongodb/journal

Wed Aug 15 13:08:58 [initandlisten] removeJournalFiles end

Wed Aug 15 13:08:58 [initandlisten] shutdown: removing fs lock...

Wed Aug 15 13:08:58 dbexit: really exiting now

[MKN Web Solutions]

You're all too slow! Just kidding of course.. I fixed the issue by just removing the "master" option.  I'm assuming it cleared the oplog - which I don't mind at all since this is a dev instance.

Thanks,

Mike