mongod don't start after server crash

[Алибек Каримов]

after server crashed, mongod service don't start. OS: Windows Server 2008 R2 start mongo version: 3.0.0 repair option don't help, journal directory is empty after crash update mongo to 3.0.2, this is all in log file:

2015-04-28T14:56:45.154+0600 I CONTROL  Hotfix KB2731284 or later update is     installed, no need to zero-out data files
2015-04-28T14:56:45.154+0600 W -        [initandlisten] Detected unclean shutdown - C:\Program Files\MongoDB\Server\3.0\data\mongod.lock is not empty.
2015-04-28T14:56:45.154+0600 W STORAGE  [initandlisten] Recovering data from the last clean checkpoint.
2015-04-28T14:56:45.154+0600 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-04-28T14:56:45.154+0600 I -        [initandlisten] Assertion: 28595:2: No such file or directory
2015-04-28T14:56:45.154+0600 I STORAGE  [initandlisten] exception in initAndListen: 28595 2: No such file or directory, terminating
2015-04-28T14:56:45.154+0600 I CONTROL  [initandlisten] dbexit:  rc: 100

How can I fix this?

[SteveD]

We experience this issue often on Azure VMs running Windows Server 2012 R2 Datacenter. In our use case, the VMs can shut down in an improper fashion due to some Azure maintenance and cause the services to not shut down cleanly due to a timeout. The default service timeout, in my OS, is 5 seconds and I believe it is the same for yours. 5 seconds is unacceptable.

I believe increasing the Windows service timeouts in the registry will resolve this for you. I increased it to 2 minutes to give the MongoDB service enough time to shutdown cleanly. Here is a snippet of Powershell code that I use to make the edits to all of my MongoDB servers remotely.

Invoke-Command -ConnectionUri $uri -Credential $localCredentials `
        -ScriptBlock {set-itemproperty -path HKLM:\SYSTEM\CurrentControlSet\Control -name WaitToKillServiceTimeout -value 120000}

This potential solution has thus far been working for 2 weeks in our QA environment with zero lock file issues as compared to many in the same period before the fix.

As with any modifications to the registry, you should really understand what it is you are doing before making changes. I suggest you try this in a development environment first.

[SDillon]

To be clear, my solution is how to provide a way to stop the issue from ever happening to begin with.

On Wednesday, April 29, 2015 at 4:51:53 AM UTC-4, Алибек Каримов wrote:

[Stephen Steneker]

Hi Stephen,

Note that your solution changes a global timeout setting for services.

FYI, it looks like a related code fix for long shutdown times in Windows will be included in MongoDB 3.0.3:

 https://jira.mongodb.org/browse/SERVER-17820

There are RCs out for 3.0.3 if you would like to test in a staging/QA environment.

Regards,

Stephen

[Stephen Dillon]

Correct. That is what the change is; a services timeout modification. In a pre 3.0.3 world, it workef for us thus far when no other solution existed or could be offered by Support.

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/4C_z6SY9e4o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/17d27e8c-2740-40b7-87de-370b03122321%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.