SOS-PROBLEM :mongo | wait for mongo server to start

[dim kar]

Hello to everyone,

I have tried to install the edx-platform from: 

              https://github.com/edx/configuration/wiki/edX-Ubuntu-12.04-64-bit-Installation

using the manual installation 

but I have an error.. during the installation.. it failed at 

TASK: [mongo | wait for mongo server to start] ******************************** 
failed: [localhost] => {"elapsed": 300, "failed": true, "item": ""}
msg: Timeout when waiting for 127.0.0.1:27017

FATAL: all hosts have already failed -- aborting

and my pc stacks and freezes.

Do you know what is the solution for this problem? What should I do to fix that issue?

[dalfa]

Just my two cents: I saw a similar issue while not shutting down properly the VM (and lock files are not deleted). The steps described here http://blog.codref.com/vagrant-dirty-shutdown-and-mongodb/ cannot make things worst, so I would give it a try and relaunch the process.

[KBS]

Hi Dim,

Could you solve the problem? I am also facing the same.  If you have its solution, it will be very kind of you to let me know.

Thank you.

[Mahyar Damavand]

Can you login to the server and look at the logs for mongo. They should be
at '/edx/var/log/mongodb.log'. Sometimes mongodb failes to start if there
is not sufficient space for its database on disk or because there is not
enough memory on the box.

you should take a look here too:
http://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/latest/installation/devstack/troubleshooting_vm_installation.html#starting-open-edx-servers-after-a-mongodb-failure

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/4d88ce15-881b-4ab3-8a70-8869a21271e3%40googlegroups.com.

[KBS]

Thank you  Mahayar Damavand for the reply.

I checked the log in /edx/var/log/mongo/mongodb.log  and it is as here:

------------------------------------------------------------------

2017-01-23T00:58:42.740-0800 ***** SERVER RESTARTED *****

2017-01-23T00:58:42.770-0800 [initandlisten] MongoDB starting : pid=20885 port=27017 dbpath=/edx/var/mongo/mongodb 64-bit host=ubuntu-new

2017-01-23T00:58:42.770-0800 [initandlisten] db version v2.6.12

2017-01-23T00:58:42.770-0800 [initandlisten] git version: d73c92b1c85703828b55c2916a5dd4ad46535f6a

2017-01-23T00:58:42.770-0800 [initandlisten] build info: Linux build5.ny.cbi.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=12017-01-23T00:58:42.771-0800 [initandlisten] allocator: tcmalloc

2017-01-23T00:58:42.771-0800 [initandlisten] options: { config: "/etc/mongod.conf", net: { port: 27017 }, storage: { dbPath: "/edx/var/mongo/mongodb", journal: { enabled: tr} }, systemLog: { destination: "file", logAppend: true, path: "/edx/var/log/mongo/mongodb.log" } }

2017-01-23T00:58:42.777-0800 [initandlisten] journal dir=/edx/var/mongo/mongodb/journal

2017-01-23T00:58:42.777-0800 [initandlisten] recover : no journal files present, no recovery needed

2017-01-23T00:58:42.777-0800 [initandlisten]

2017-01-23T00:58:42.777-0800 [initandlisten] ERROR: Insufficient free space for journal files

2017-01-23T00:58:42.777-0800 [initandlisten] Please make at least 3379MB available in /edx/var/mongo/mongodb/journal or use --smallfiles

2017-01-23T00:58:42.777-0800 [initandlisten]

2017-01-23T00:58:42.780-0800 [initandlisten] exception in initAndListen: 15926 Insufficient free space for journals, terminating

2017-01-23T00:58:42.780-0800 [initandlisten] dbexit:

2017-01-23T00:58:42.780-0800 [initandlisten] shutdown: going to close listening sockets...

2017-01-23T00:58:42.780-0800 [initandlisten] shutdown: going to flush diaglog...

2017-01-23T00:58:42.780-0800 [initandlisten] shutdown: going to close sockets...

2017-01-23T00:58:42.780-0800 [initandlisten] shutdown: waiting for fs preallocator...

2017-01-23T00:58:42.780-0800 [initandlisten] shutdown: lock for final commit...

2017-01-23T00:58:42.780-0800 [initandlisten] shutdown: final commit...

2017-01-23T00:58:42.780-0800 [initandlisten] shutdown: closing all files...

2017-01-23T00:58:42.780-0800 [initandlisten] closeAllFiles() finished

2017-01-23T00:58:42.780-0800 [initandlisten] journalCleanup...

2017-01-23T00:58:42.780-0800 [initandlisten] removeJournalFiles

2017-01-23T00:58:42.780-0800 [initandlisten] shutdown: removing fs lock...

2017-01-23T00:58:42.780-0800 [initandlisten] dbexit: really exiting now

--------------------------------------------

and the error that I got is:

----------------------------------------------------------------------------------------

 TASK: [mongo | stop mongod service] ******************************************* 

ok: [localhost]

TASK: [mongo | move mongodb to {{ mongo_data_dir }}] ************************** 

ok: [localhost]

TASK: [mongo | copy mongodb key file] ***************************************** 

skipping: [localhost]

TASK: [mongo | copy configuration template] *********************************** 

ok: [localhost]

TASK: [mongo | start mongo service] ******************************************* 

changed: [localhost]

TASK: [mongo | wait for mongo server to start] ******************************** 

failed: [localhost] => {"elapsed": 300, "failed": true}

msg: Timeout when waiting for 127.0.0.1:27017

FATAL: all hosts have already failed -- aborting

PLAY RECAP ******************************************************************** 

INFO:ansible.callback_plugins.datadog_tasks_timing:mongo | wait for mongo server to start ----------------------------------------- 300.46s

INFO:ansible.callback_plugins.datadog_tasks_timing:mongo | install mongo server and recommends -------------------------------------- 8.65s

INFO:ansible.callback_plugins.datadog_tasks_timing:user | Install role-independent useful system packages --------------------------- 7.04s

INFO:ansible.callback_plugins.datadog_tasks_timing:common | Install role-independent useful system packages ------------------------- 6.95s

INFO:ansible.callback_plugins.datadog_tasks_timing:server_utils | Install system packages ------------------------------------------- 6.88s

INFO:ansible.callback_plugins.datadog_tasks_timing:mongo | install python pymongo for mongo_user ansible module --------------------- 5.23s

INFO:ansible.callback_plugins.datadog_tasks_timing:common | pip install virtualenv -------------------------------------------------- 3.14s

INFO:ansible.callback_plugins.datadog_tasks_timing:common | Update CA Certificates -------------------------------------------------- 2.95s

INFO:ansible.callback_plugins.datadog_tasks_timing:edxlocal | install memcached ----------------------------------------------------- 0.88s

INFO:ansible.callback_plugins.datadog_tasks_timing:mysql | install mysql 56 and dependencies ---------------------------------------- 0.88s

INFO:ansible.callback_plugins.datadog_tasks_timing:

Playbook edx_sandbox finished: Mon Jan 23 01:03:42 2017, 107 total tasks.  0:05:55 elapsed. 

           to retry, use: --limit @/home/kbsingh/edx_sandbox.retry

localhost                  : ok=63   changed=6    unreachable=0    failed=1   

------------------------------------------------------------------------------------------------------------------------------------

I think that hard-disk space and memory is not the problem as these are 500GB and 4GB respectively.

Is there any other possible reason of error?? 

[Ned Batchelder]

We should move operations questions to the openedx-ops list: https://groups.google.com/forum/#!forum/openedx-ops

--Ned.

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/0a4e1b5d-8cb2-413e-a86d-527797ef3d97%40googlegroups.com.

[abdul...@arbisoft.com]

@dim

The issue may be:

- Unavailability of enough system resources.
- The setup may be running previously and was not shutdown properly or interrupted, causing locks to be sitting around.

Try the following:

- on your terminal,

vagrant halt
vagrant destroy

 then,

- delete the .vagrant folder, then on the terminal,

vagrant up
vagrant provision

Best,
Moeez