Entire Server Hang with MongoDB

[Gavin]

Hi

I'm running the latest stable release (1.2.2) 64-bit on linux. I'm
just started playing with MongoDB for about 2 days now.

While loading data into a test database I created, after some time,
the entire server just hung. By "hung" I mean: my existing ssh
session into the machine (which was running the mongo terminal) is
unresponsive...typing anything does nothing, ctrl-c (to attempt to
break out of the mongo terminal does nothing).

Additionally, attempting to open a new ssh connection into this server
also hangs...

~$ ssh -v mongodb01
OpenSSH_4.7p1 Debian-8ubuntu1.2, OpenSSL 0.9.8g 19 Oct 2007
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to mongodb01 [10.243.21.156] port 22.
debug1: Connection established.
debug1: identity file /home/gdas/.ssh/identity type -1
debug1: identity file /home/gdas/.ssh/id_rsa type -1
debug1: identity file /home/gdas/.ssh/id_dsa type -1

...and just sits there.

Right now I'm just waiting, but if this doesn't let me in, I will need
to power cycle the server and likely have to repair the database.

About 30 minutes before the hang, I checked on the size of the
collection I was loading (it was approximately 20 million
documents...I don't think that should be a problem since this is mongo
db afterall...and I need to load upwards of 100 million documents).

Any ideas what could cause this? There were only 2 processes running
on the server, Mongo db and a python script which is what is loading
the data (reading from a remote mysql database). Ther server is 64-
bit, has 8GB of memory and 2 cores.

As a related note, I have personally seen this sort of behavior once
before with a mysql server (server hang, not accepting new ssh
connections). In diagnosing that problem we found that the "hang"
would occur when mysql exhausted all of the space in /tmp. To
alleviate that issue we set a property in the mysql config file that
told it to use a different directory for temp operations...on a volume
that had more space. We never saw that problem again. I'm wondering
if the same thing might be happening with mongoDB. Does MongoDB use
the operating system /tmp folder? If so, is this a configurable
property?

The /tmp issue is just a shot in the dark since I've seen something
similar happen before... I have no idea if that is the same issue
here...any and all ideas are appreciated.

Thanks much!
Gavin

[Eliot Horowitz]

Mongo doesn't use tmp at all so that's not it.
Can you hit the mongo web console or open a new connection to the db
to get the curren op?

> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com
> .
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en
> .
>

[Gavin]

Great ideas...wish I had thought to do that (darn!)...

The machine ended up rebooting on it's own...kernel panic I
assume...unfortunately, the linux system logs are not showing any
information about what happened... the restart is logged, what
happened just before the restart is not logged.

If this happens again, I will be sure to try the web console as well
as trying to connect to the db remotely

Thanks!

On Feb 13, 12:16 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> Mongo doesn't use tmp at all so that's not it.
> Can you hit the mongo web console or open a new connection to the db  
> to get the curren op?
>

> > .- Hide quoted text -
>
> - Show quoted text -