DIfference Between storageSize and fileSize of MongoDB?

[arnold]

Hi Guys,

We have started using MongoDB few days ago but We couldn't understand
why there is so Dis-Similarities with Total Disk Space Used by
MongoDB(test.o,test.1..etc) and Data-base size Used by MongoDB. My
Concern is as follows:

When we execute the query db.stats(); we saw the details as follows:

DataSize : 232.68MB
storageSize : 292.58MB
Indexsize : 204.51MB
avgObjSize : 171MB
fileSize:2.0GB

If we check in the database storage location, total file size (test.
0+test.1+test.2..etc) is "2.0GB". In the event of adding the [datasize
+storagesize+indexsize+avgobjsize = 900.77MB] why it's not equal to
the physical space used by the Disk "2.0GB". Apart from datasize,
storagesize, indexsize, avgobjsize what else space is used for
MongoDB. Please explain us about difference between "StorageSize" &
"fileSize"...

Advanced Thanks,

[Shekhar Vemuri]

http://www.mongodb.org/display/DOCS/Excessive+Disk+Space

How long have you had the database up? What kind of operations have you been running? Mostly inserts? Updates and deletes?

Sent from my iPhone.

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>

[arnold]

Thanks for your attention. We had been started using the database
from the past one month. Mostly we perform "inserts" occasionally
"Updates" & "delete" .

On Dec 21, 11:17 am, Shekhar Vemuri <shekh...@gmail.com> wrote:
> http://www.mongodb.org/display/DOCS/Excessive+Disk+Space
>
> How long have you had the database up? What kind of operations have you been running? Mostly inserts? Updates and deletes?
>
> Sent from my iPhone.
>

[Tyler Brock]

Your average object size seems impossibly large. The maximum size for
an object is 16mb.
Are you sure you did the math correctly when converting from bytes to
MB?

Are you using GridFS?

-Tyler

[Tyler Brock]

Anyway, this is probably due to mongodb pre-allocating storage space.
If this is a concern, and you are running a small dataset not in
production, you can turn it off by using the --noprealloc flag when
starting mongod.
You can read more about it here: http://www.mongodb.org/display/DOCS/Excessive+Disk+Space

[arnold]

Hi @Tyler Brock,
The avgobjsize its 171Bytes.

[arnold]

@Tyler Brock,
We understood that the problem is due to pre-
allocating storage space but we are baffling with the "storageSize".
So, please explain about strorageSize specifically and What does it
embrace.

Advance Thanks,

[Tyler Brock]

dataSize = size of datastorageSize = size of data w/ padding
fileSize = size on disk w/ preallocation

Make sense?

-Tyler

[arnold]

Yup, This makes sense to me. Thanks for your precious information.

[arnold]

Hi,

We have been using MongoDB from few weeks ago. One thing we
could not understood is Pre-allocated file. Is there any way to check
whether the pre-allocated file is Currently used by MongoDB or not. We
have planned to implement "Quota File". So we need to ensure the pre-
allocated file is currently used or not.

For Example:-

I have the database file as follows:
sgserver.0 ---> 16MB
sgserver.1 ---> 128MB
sgserver.2 ---> 256MB
sgserver.3 ---> 512MB

Here,
How to check the pre-allocated file sgserver.3 is used by
MongoDB or not[ie This database is used by any BSON or its Un-used].
Please provide us with your valuable suggestion.

Advance Thanks,

[Tyler Brock]

As per the documentation here: http://www.mongodb.org/display/DOCS/Excessive+Disk+Space

"As soon as a datafile starts to be used, the next one will be
preallocated."

Therefore, the last file to be created is not in use.

If you want to disable this behavior simply use the --noprealloc flag
when running mongod.

-Tyler

[Tyler Brock]

Ninja,

Perhaps you have deleted some documents? Database files don't shrink when you do so.

-Tyler

On Monday, June 18, 2012 10:04:58 PM UTC-4, ninja kiwi wrote:

I have a question:

I have checked my padding for the document is 1.

Yet my storagesize is much bigger than my datasize: 38GB >> 27GB

Any idea of where does the 11GB occupied space come from? (i did remove some documents, is that the reason the padding remains there and i need to 'repair' my database?)

Or another question: if my padding factor is 1, is it my storagesize has to be equal to my datasize?

Thanks.

[Tyler Brock]

Ok Arnold,

Are you all set understanding the sizes that were displayed? Please let us know.

-Tyler

> > > > To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.

[ninja kiwi]

Ohya, that's a very important fact (which is not intuitive). The database files dont shrink by itself after the document files are removed. 

Thanks~