Why isn't MongoDB reusing ample space already allocated?

[Mark]

Hi. Long time listener, first time caller.

I'm sure this has come up but I can't find anything that answers my
question specifically. I understand that MongoDB allocates space
pretty aggressively up front for performance reasons and that it
doesn't give back to the filesystem by default since chances are it
will probably need that space again. What I don't understand is why
MongoDB continues to grab 2GB chunks when there is already a ton of
free space.

I've run a simple example(below) just inserting, removing, compacting
and querying db.stats(). I expected allocated disk to grow with
initial insert, remove and compact. I didnt expect to see filesize
grow on subsequent iterations of the same. I assumed it would reuse
the space it knew it had free. That seems not to be the case. Before
the last insert, there is approx fileSize-storageSize = 12GB
available.

Why doesn't MongoDB use that space?

I had the impression that the repair command was on its way out. But
if its the only way to reclaim space, then it means that we need to
always ensure we have 2x free space to run it as a regular maintenance
job. Is that correct or I missing something?

Thanks in advance.

switched to db mango
> //loading 2 mil thingies
> db.stats()
{
"db" : "mango",
"collections" : 3,
"objects" : 2047904,
"avgObjSize" : 1571.746865087426,
"dataSize" : 3218786692,
"storageSize" : 3577475072,
"numExtents" : 32,
"indexes" : 1,
"indexSize" : 77688352,
"fileSize" : 4226809856,
"nsSizeMB" : 16,
"ok" : 1
}
> db.thingies.remove()
> db.thingies.runCommand('compact')
{ "ok" : 1 }
> db.stats()
{
"db" : "mango",
"collections" : 3,
"objects" : 4,
"avgObjSize" : 45,
"dataSize" : 180,
"storageSize" : 721420288,
"numExtents" : 3,
"indexes" : 1,
"indexSize" : 8176,
"fileSize" : 6373244928,
"nsSizeMB" : 16,
"ok" : 1
}
> //let's add the 2 mill again.
> db.stats()
{
"db" : "mango",
"collections" : 3,
"objects" : 2047904,
"avgObjSize" : 1571.7465193680953,
"dataSize" : 3218785984,
"storageSize" : 3872567296,
"numExtents" : 6,
"indexes" : 1,
"indexSize" : 79675120,
"fileSize" : 10666115072,
"nsSizeMB" : 16,
"ok" : 1
}
> //allocating more space.
> db.thingies.remove()
> db.thingies.runCommand('compact')
{ "ok" : 1 }
> db.stats()
{
"db" : "mango",
"collections" : 3,
"objects" : 4,
"avgObjSize" : 45,
"dataSize" : 180,
"storageSize" : 1495941120,
"numExtents" : 3,
"indexes" : 1,
"indexSize" : 8176,
"fileSize" : 12812550144,
"nsSizeMB" : 16,
"ok" : 1
}
> //load them one more time
> db.stats()
{
"db" : "mango",
"collections" : 3,
"objects" : 2047904,
"avgObjSize" : 1571.7464666312483,
"dataSize" : 3218785876,
"storageSize" : 3291062272,
"numExtents" : 4,
"indexes" : 1,
"indexSize" : 79748704,
"fileSize" : 14958985216,
"nsSizeMB" : 16,
"ok" : 1
}
>

//we now have 14+ GB allocated to Mongo but only 3 in use.

[Tony Hannan]

There is a bug where compact always allocates more space: https://jira.mongodb.org/browse/SERVER-3791.

Maybe you are running into this bug.

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Mark]

Hi Tony.

Thanks for the reply. You're right. That would be one problem and I'm
sure we're hitting it. But another issue is why new inserts result in
new disk allocation when there is already an abundance of it free and
allocated to the app. Do you know why this is?

Mark.

> >http://groups.google.com/group/mongodb-user?hl=en.- Hide quoted text -
>
> - Show quoted text -

[Tony Hannan]

Hi Marc,

You're right, looks like a bug. Could you please create a SERVER ticket with reproducible case at jira.mongodb.org.

Thanks,

Tony

[Mark]

Sure thing.

Thanks.

> > > >http://groups.google.com/group/mongodb-user?hl=en.-Hide quoted text -