Enabling --smallfiles and --noprealloc

[Matt]

We have a current mongodb cluster in which we have about 20 or so
existing authenticated databases. To save disk space, for databases
created in the future, we're planning to restart mongod with --
smallfiles and --noprealloc enabled.

Are there any caveats or known problems with enabling these options on
an existing instance?

Thanks!

[Spencer Brody]

You can enable those flags on an existing instance, though they'll only affect new files - existing data won't be affected.

Be careful about using --noprealloc though, we generally advice against using it in production deployments, as it can lead to performance degradation and disk fragmentation as your data set grows.

-Spencer

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Matt]

Thanks for the clarification.

My understanding is that --smallfiles will reduce the size of the pre-
allocated files by 1/2? Assuming that is correct, will they still
double in size on each allocation up to a size of 2GB?

I'm trying to limit the size of a given database to 4GB using a
combination of --smallfiles, --quota and --quotaFiles.

-Matt

[Spencer Brody]

Yes, smallfiles makes the files start at half the size.  The max size is also cut in half, so they'll only grow to 1GB.

If you're concerned about your DB growing too large, take a look at capped collections (http://www.mongodb.org/display/DOCS/Capped+Collections).  That will allow you to limit the max size of a given collection.

Cheers,

-Spencer

--

[Matt]

We're using capped collections in a few places but our environment is
more akin to a hosted mongodb environment like MongoLab, MongoHQ etc.
Each of our users is allocated their own authenticated database with a
hard-limit max size of 4GB. The goal at the moment is to minimize the
amount of disk space required when a given database is created and as
it grows. --noprealloc would be an ideal addition but I see your
point about introducing the risk of disk fragmentation as the various
databases grow.

Just so I make sure I'm doing my math right, at database creation time
with --smallfiles enabled, two 32MB files will be allocated, 1 for the
namespace collection and an initial empty file for the user data. So
for a hard limit of 4GB per database (not including namespace file) I
would need the following:

--smallFiles
--quota
--quotaFiles=8

Is that correct?

-Matt

[Spencer Brody]

After doing a small test on my machine, it looks like when someone first creates a database using --smallfiles and insert one document, you get one 16MB file and one 32MB file (ignoring the namespace file).  For a limit of 4GB per database, I think that --quoteFiles=8 is correct (16+32+64+128+256+512+1024+2048 = 4080MB)

Note that I did this using mongo version 1.9.0.  I believe some of these defaults have changed in recent versions, so it may be worthwhile to run a small test to verify the behavior on whichever version you are using.

Cheers,

-Spencer

-Matt

[Elmer Zhang]

I have enabled --smallfiles and --noprealloc. The size of db files start from 16M and only grow to 512MB.

Allocating a 512MB data file only took about 0.5 seconds. 

[Spencer Brody]

I just ran an instance of 1.8.2 running with --smallfiles and --quotafiles=8, then kept inserting a large string until the db was full.

It looks like I was wrong and the max size is 512Mb per file, sorry about the confusion there.

Given that, it looks like you will want to use quotafiles=12 to create a 4Gb cap.

I don't believe this behavior has been changed since 1.8.2, but if you're running with something later, it should be easy to write something to confirm this behavior.  I just did:

str = "AAAAA"

while (str.length < 1024*1024){ str+=str}

for (var i =0; i < 5000; i++) { db.foo.insert({a:str})}

That should fill up your database pretty quickly if you're using --smallfiiles and --quotafiles=12, then you can verify that the file sizes look as you expect.

Hope that helps clear things up!

-Spencer

On Mon, Aug 1, 2011 at 4:09 PM, Matt <ma...@ziplinegames.com> wrote:

Hi Spencer,
 Apologies for bugging you directly (I'll repost on the forum as
appropriate) but I was wondering if you could confirm for me the
maximum pre-allocated file size when using --smallfiles.  You
indicated below that they grow to 1GB but a later post suggested that
they grow only to 512MB.

Also, is this behavior any different in 1.9+ than in 1.8.x?

Thanks much for your help.

Cheers!

-Matt Towers

[Ramesh T]

i am using latest  mongodb v3.0.1

how to enable the --smallfile

give me suggest how to run smallfile on ubuntu.But I saw config file there is no smallfile..is we need to add for version