mongodb init script failed to start on debian wheezy

[Antoine H]

Hello,

I have just got a brand new server with a fresh debian wheezy installation. I have installed the .deb provided by 10gen but the init script failed. I don't know why but it seams that it's a common issue on wheezy. Some people on forum... suggests to install an older version (2.2.3). I try almost all version (maybe 15) from 2.2 to 2.4. And it still doesn't start automatically.

I launch mannualy /usr/bin/mongod -f /etc/mongodb.conf and it works like a charm. Unfortunatly it doesn't suit me because It will be my production server. What could I do about it ?

Add "/usr/bin/mongod -f /etc/mongodb.conf" in rc.local file ? It feels so wrong !

Someone have any idea or any clue please ? I prefer the simplier and cleanier installation as possible.

Thanks in advance,

A.

[Russell Bateman]

But, why did it fail? Did you try the init script statement yourself to diagnose it? Are you sure it's not running? Try the following and tell us the result. If you've really got a problem, it will error out part way through, possibly with a permissions or ownership problem.

~ $ sudo bash
~ # ps -ef | grep [m]ongo
~ # rm /var/lib/mongodb/mongod.lock
~ # start-stop-daemon --start --chuid mongodb --exec /usr/bin/mongod -- --config /etc/mongodb.conf
all output going to: /var/log/mongodb/mongodb.log
^Z
[1]+  Stopped                 start-stop-daemon --start --chuid mongodb --exec /usr/bin/mongod -- --config /etc/mongodb.conf
~ # bg
[1]+ start-stop-daemon --start --chuid mongodb --exec /usr/bin/mongod -- --config /etc/mongodb.conf &
~ # ps -ef | grep [m]ongo
mongodb  30690 30536  3 08:21 pts/8    00:00:00 /usr/bin/mongod --config /etc/mongodb.conf
~ #

--just trying to help.

--
--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb
 
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Antoine H]

Yes, I saw and I really appreciate your help !

I already try to see if mongo processus was launch and it's not the case. (But I have retry with your command of course).

I get no log if I try to launch the service with "service mongodb start" or with "/etc/init.d/mongodb start".

I try you command and I didn't get the same result as you. And I didn't understand the result. Maybe it will give you a clue ?

# start-stop-daemon --start --chuid mongodb --exec /usr/bin/mongod -- --config /etc/mongodb.conf

Wed Sep  4 16:44:24.501 terminate() called, printing stack (if implemented for platform):

0xdddd81 0x6cfbae 0x2d71543f996 0x2d71543f9c3 0x2d71543fbee 0xdf5351 0xdd41f1 0xdd4aa9 0xdd3145 0xdd32a9 0x70106c 0x70150c 0x7018b2 0x7018f1 0x701d43 0x6dde65 0x6dfc29 0x2d714bd8ead 0x6cf999 

 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdddd81]

 /usr/bin/mongod(_ZN5mongo11myterminateEv+0x3e) [0x6cfbae]

 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x63996) [0x2d71543f996]

 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x639c3) [0x2d71543f9c3]

 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x63bee) [0x2d71543fbee]

 /usr/bin/mongod(_ZN5boost11filesystem36detail6statusERKNS0_4pathEPNS_6system10error_codeE+0x231) [0xdf5351]

 /usr/bin/mongod(_ZN5mongo11ProcessInfo16checkNumaEnabledEv+0x41) [0xdd41f1]

 /usr/bin/mongod(_ZN5mongo11ProcessInfo10SystemInfo17collectSystemInfoEv+0x3a9) [0xdd4aa9]

 /usr/bin/mongod(_ZN5mongo11ProcessInfo20initializeSystemInfoEv+0xa5) [0xdd3145]

 /usr/bin/mongod(_ZN5mongo36_mongoInitializerFunction_SystemInfoEPNS_18InitializerContextE+0x9) [0xdd32a9]

 /usr/bin/mongod(_ZN5boost6detail8function17function_invoker1IPFN5mongo6StatusEPNS3_18InitializerContextEES4_S6_E6invokeERNS1_15function_bufferES6_+0xc) [0x70106c]

 /usr/bin/mongod(_ZNK5mongo11Initializer7executeERKSt6vectorISsSaISsEERKSt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEE+0x13c) [0x70150c]

 /usr/bin/mongod(_ZN5mongo21runGlobalInitializersERKSt6vectorISsSaISsEERKSt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEE+0x32) [0x7018b2]

 /usr/bin/mongod(_ZN5mongo26runGlobalInitializersOrDieERKSt6vectorISsSaISsEERKSt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEE+0x21) [0x7018f1]

 /usr/bin/mongod(_ZN5mongo26runGlobalInitializersOrDieEiPKPKcS3_+0x3b3) [0x701d43]

 /usr/bin/mongod() [0x6dde65]

 /usr/bin/mongod(main+0x9) [0x6dfc29]

 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x2d714bd8ead]

 /usr/bin/mongod() [0x6cf999]

Wed Sep  4 16:44:24.506 Got signal: 6 (Aborted).

Wed Sep  4 16:44:24.510 Backtrace:

0xdddd81 0x6d0d29 0x2d714bec4f0 0x2d714bec475 0x2d714bef6f0 0x6cfbb3 0x2d71543f996 0x2d71543f9c3 0x2d71543fbee 0xdf5351 0xdd41f1 0xdd4aa9 0xdd3145 0xdd32a9 0x70106c 0x70150c 0x7018b2 0x7018f1 0x701d43 0x6dde65 

 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdddd81]

 /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x6d0d29]

 /lib/x86_64-linux-gnu/libc.so.6(+0x324f0) [0x2d714bec4f0]

 /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x2d714bec475]

 /lib/x86_64-linux-gnu/libc.so.6(abort+0x180) [0x2d714bef6f0]

 /usr/bin/mongod(_ZN5mongo11myterminateEv+0x43) [0x6cfbb3]

 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x63996) [0x2d71543f996]

 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x639c3) [0x2d71543f9c3]

 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x63bee) [0x2d71543fbee]

 /usr/bin/mongod(_ZN5boost11filesystem36detail6statusERKNS0_4pathEPNS_6system10error_codeE+0x231) [0xdf5351]

 /usr/bin/mongod(_ZN5mongo11ProcessInfo16checkNumaEnabledEv+0x41) [0xdd41f1]

 /usr/bin/mongod(_ZN5mongo11ProcessInfo10SystemInfo17collectSystemInfoEv+0x3a9) [0xdd4aa9]

 /usr/bin/mongod(_ZN5mongo11ProcessInfo20initializeSystemInfoEv+0xa5) [0xdd3145]

 /usr/bin/mongod(_ZN5mongo36_mongoInitializerFunction_SystemInfoEPNS_18InitializerContextE+0x9) [0xdd32a9]

 /usr/bin/mongod(_ZN5boost6detail8function17function_invoker1IPFN5mongo6StatusEPNS3_18InitializerContextEES4_S6_E6invokeERNS1_15function_bufferES6_+0xc) [0x70106c]

 /usr/bin/mongod(_ZNK5mongo11Initializer7executeERKSt6vectorISsSaISsEERKSt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEE+0x13c) [0x70150c]

 /usr/bin/mongod(_ZN5mongo21runGlobalInitializersERKSt6vectorISsSaISsEERKSt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEE+0x32) [0x7018b2]

 /usr/bin/mongod(_ZN5mongo26runGlobalInitializersOrDieERKSt6vectorISsSaISsEERKSt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEE+0x21) [0x7018f1]

 /usr/bin/mongod(_ZN5mongo26runGlobalInitializersOrDieEiPKPKcS3_+0x3b3) [0x701d43]

 /usr/bin/mongod() [0x6dde65]

Thanks for your help,

A.

[Russell Bateman]

Yes, the behavior you describe when using upstart is exactly what I'd expect (having seen it plenty of times myself). Remember, the --quiet flag is keeping you out of the loop on what's going wrong.

I assume we're looking at MongoDB 2.4.6 (the latest stable Debian package), right?

Okay, let's trim it even further. The mongod is aborting for some reason which may become an issue only a 10gen responder can speculate on, but let's launch the mongod directly as root (instead of as user mongodb) to see just how bad this is:

~ $ sudo bash
~ # rm /var/lib/mongodb/mongod.lock
~ # /usr/bin/mongod --config /etc/mongodb.conf

What will probably happen is just more of what you've pasted here, but at least we'll get this trimmed down to something that can't be casually diagnosed.

[Ae]

Yes, I use the lasted stable.

I already try to launch it as root and it works fine. (I forgot to create a mongodb user... hum hum).

I didn't have to rm the lock file. It was not locked.

root@ks211010:~# /usr/bin/mongod --config /etc/mongodb.conf

all output going to: /var/log/mongodb/mongodb.log

I check the log, and it work like a charm (I just have hidded my host)...

Wed Sep  4 17:08:47.821 [initandlisten] MongoDB starting : pid=20643 port=27017 dbpath=/var/lib/mongodb 64-bit host=XXXXXXXXXXXX

Wed Sep  4 17:08:47.822 [initandlisten] db version v2.4.6

Wed Sep  4 17:08:47.822 [initandlisten] git version: b9925db5eac369d77a3a5f5d98a145eaaacd9673

Wed Sep  4 17:08:47.822 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49

Wed Sep  4 17:08:47.822 [initandlisten] allocator: tcmalloc

Wed Sep  4 17:08:47.822 [initandlisten] options: { config: "/etc/mongodb.conf", dbpath: "/var/lib/mongodb", logappend: "true", logpath: "/var/log/mongodb/mongodb.log" }

Wed Sep  4 17:08:47.880 [initandlisten] journal dir=/var/lib/mongodb/journal

Wed Sep  4 17:08:47.880 [initandlisten] recover : no journal files present, no recovery needed

Wed Sep  4 17:08:47.965 [websvr] admin web console waiting for connections on port 28017

Wed Sep  4 17:08:47.965 [initandlisten] waiting for connections on port 27017

Wed Sep  4 17:08:47.821 [initandlisten] MongoDB starting : pid=20643 port=27017 dbpath=/var/lib/mongodb 64-bit host=XXXXXXXXXXXXX

Wed Sep  4 17:08:47.822 [initandlisten] db version v2.4.6

Wed Sep  4 17:08:47.822 [initandlisten] git version: b9925db5eac369d77a3a5f5d98a145eaaacd9673

Wed Sep  4 17:08:47.822 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49

Wed Sep  4 17:08:47.822 [initandlisten] allocator: tcmalloc

Wed Sep  4 17:08:47.822 [initandlisten] options: { config: "/etc/mongodb.conf", dbpath: "/var/lib/mongodb", logappend: "true", logpath: "/var/log/mongodb/mongodb.log" }

Wed Sep  4 17:08:47.880 [initandlisten] journal dir=/var/lib/mongodb/journal

Wed Sep  4 17:08:47.880 [initandlisten] recover : no journal files present, no recovery needed

Wed Sep  4 17:08:47.965 [websvr] admin web console waiting for connections on port 28017

Wed Sep  4 17:08:47.965 [initandlisten] waiting for connections on port 27017

I'm able to use the mongo shell.

# mongo

MongoDB shell version: 2.4.6

connecting to: test

Welcome to the MongoDB shell.

For interactive help, type "help".

For more comprehensive documentation, see

        http://docs.mongodb.org/

Questions? Try the support group

        http://groups.google.com/group/mongodb-user

> 

bye

Thanks again.

[Russell Bateman]

The Debian package installation creates user mongodb if it does not exist in /etc/passwd. You should/must not have to create it yourself.

Of course, you should not launch mongod as root, at least, not in a production environment. You should work through the problem to find a solution. From the stack trace it's unclear to me what's going on. What we do know is:

- MongoDB has been installed and is not corrupt.
- root can execute mongod successfully.
- Executing mongod as user mongodb is problematic.

If you're satisfied for now (because it's your development host), let's drop this. If not, try the next thing:

~$ sudo bash
~$ cat /etc/passwd | grep mongodb
mongodb:x:117:65534::/home/mongodb:/bin/false

If you see something like the above, edit /etc/passwd and change false to bash (temporarily), exit, then try launching as user mongodb to see if you get the same thing. Do this:

$ sudo su - mongodb
$ /usr/bin/mongod --config /etc/mongodb.conf

At this point, we're looking to see if it still gets the same messy stack traces or not.

[Ae]

Oh yes, I wonder why I didn't think about it...

So yes, I change and add a bash shell to launch mongo and I get this result :

$ /usr/bin/mongo --config /etc/mongodb.conf

terminate called after throwing an instance of 'boost::filesystem3::filesystem_error'

  what():  boost::filesystem::status: Permission denied: "/sys/devices/system/node/node1"

Abandon

I saw someone having the same problem here (and i'm indeed on an OVH server) : http://stackoverflow.com/questions/18416671/starting-service-mongodb-fail-on-permission-denied-sys-devices-system-node-no

But I don't want to install an ubuntu to solve my problem like he suggests. I'm searching the last stable debian, with the last stable mongodb package. I want to use it in production.

Thanks you again.

[Russell Bateman]

It doesn't look like a problem that 10gen would "unsolve" suddenly in 2.4.x, but maybe an earlier version will work. Note that, on my stock Ubuntu (Mint) system, /sys/devices/system/node/node0 and underneath it is r-x/rwx by all.

Best of luck.

[Ae]

It's strange because, this file didn't exist on the serveur. So the only way is to switch to an another distribution ?

Thank you for your help Russell.

[Christiano Anderson]

Antoine,

I have noticed some issues with the Debian packages provided by 10gen (now MongoDB Inc). After following the instructions at http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian/ , the MongoDB doesn't start at the first time.

Sometimes I need to run:

~# /etc/init.d/mongodb stop

And after that:

~# /etc/init.d/mongodb start

After this procedure, I am able to log into the Mongo Shell and use without any other issue.

I have a pool of 6 servers running Debian Wheezy (fresh install) and I have noticed this issue in 3 machines. In the other 3 machines, the procedure worked fine.

Since I know DPKG pretty well, I will check the packages structure provided by 10gen. Something is wrong with it.

Cheers

Christiano Anderson 

http://christiano.me/
http://twitter.com/dump/

[Ae]

Hello,

Sorry to answer so late, but an answer is better than nothing.

It seam's that it come from the OVH special Kernel (my server provider). So I switch to a standard one and it works like a charm.

Thanks for your help !

A.