Long time to drop (tiny) database?

[Kris Pruden]

Hi,

I'm using mongodb in a project where we have written a series of
automated functional tests. These tests do a variety of things, but
they all follow the same sequence:

1. Connect to mongodb and create a new database
2. Do some stuff..
3. Drop the database

I've noticed that dropping the database is taking a shockingly long
time to complete - anywhere from a half-dozen to nearly 30 seconds.
The databases in these cases are absolutely tiny, not more than a
dozen records. Here's an example of the log output:

Mon Dec 14 16:23:16 query truth.$cmd ntoreturn:1 reslen:77 nscanned:0
{ dropDatabase: 1 } nreturned:1 10120ms

If I'm reading that right, this entry corresponds to the drop command
and took a hair over ten seconds to complete.

Any theories on why this might be the case? This is *really* slowing
down our tests...

FYI I was using 1.0.0. I just upgraded to 1.2.0 and am still seeing
the same thing...

This is on Mac OS 10.5, using the 64-bit pre-built binary (downloaded
from the website).

Not sure if it's relevant, but our code is talking to the database
using the java driver.

Thanks!

Kris

[Mathias Stearn]

Databases are rather expensive to create and destroy since they correspond to actual files on the filesystem.

Would it be possible to refactor your tasks to use the same db but create and destroy collections rather than databases? You can use "dotted" collections to maintain logical separation:

db.TASK_NAME.COLLECTION_NAME.insert({});

--

You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Kris Pruden]

On Dec 14, 2009, at 5:02 PM, Mathias Stearn wrote:

> Databases are rather expensive to create and destroy since they
> correspond to actual files on the filesystem.

That makes sense.

> Would it be possible to refactor your tasks to use the same db but
> create and destroy collections rather than databases? You can use
> "dotted" collections to maintain logical separation:

We are using collections in our application, but to ensure that each
test gets a clean environment, we drop the entire database at the end
of each test.

Looking at the logs again, I see these entries:

Mon Dec 14 16:35:27 allocating new datafile data/truth.ns, filling
with zeroes...
Mon Dec 14 16:35:27 done allocating datafile data/truth.ns, size:
16MB, took 0.014 secs
Mon Dec 14 16:35:27 allocating new datafile data/truth.0, filling with
zeroes...
Mon Dec 14 16:35:27 done allocating datafile data/truth.0, size: 64MB,
took 0.061 secs

I'm assuming these are the datafiles, and if that's correct each test
is creating then deleting 80MB worth of data on disk. I'm running on
a laptop (1.5-yo MBP), and disk performance isn't the greatest, so
that could explain the bad performance.

I just found the --nssize and --smallfiles options. Using these
brings the time down into the 1-3 second range, which is tolerable.
It looks like the --smallfiles options brings the initial datafile
size down to 32MB. Is there any way I can make it even smaller?

Thanks,

Kris

[Mathias Stearn]

--noprealloc will avoid creating the second file until it is needed.

http://www.mongodb.org/display/DOCS/Developer+FAQ#DeveloperFAQ-Preallocation explains our preallocation strategy.