Limits of database size and collection size with WiredTiger

[Yos Tj]

What are the limits(maximum size) of database size and collection size with WiredTiger?
I cannot see them at MongoDB Limits and Thresholds(https://docs.mongodb.org/v3.0/reference/limits/).

[Asya Kamsky]

The docs state that Wired Tiger is not subject to any limits here:

https://docs.mongodb.org/manual/reference/limits/#Database-Size and https://docs.mongodb.org/manual/reference/limits/#Data-Size mention limits for MMAPV1 and say that "The WiredTiger storage engine is not subject to this limitation."

If you can tell us specific limit(s) you are worried you may exceed, we can answer that more directly.

Asya

On Sun, Dec 27, 2015 at 7:58 AM, Yos Tj <mysql2...@gmail.com> wrote:

What are the limits(maximum size) of database size and collection size with WiredTiger?
I cannot see them at MongoDB Limits and Thresholds(https://docs.mongodb.org/v3.0/reference/limits/).

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/770260b7-2ec3-448d-bbc3-393612bedb63%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Asya Kamsky
Lead Product Manager
MongoDB
Download MongoDB - mongodb.org/downloads
Free MongoDB Monitoring - cloud.mongodb.com
Free Online Education - university.mongodb.com
Get Involved - mongodb.org/community
We're Hiring! - https://www.mongodb.com/careers

[Yos Tj]

Hello Asya Kamsky,

Sorry for my late reply.

 > "The WiredTiger storage engine is not subject to this limitation."

I'm afraid it's about data set(query result) size, not collection/database size.
Typically a query result is smaller than its base collection,
but not necessarily so. Actually, for MMAPv1, the maximum database size is 32TB,
and the maximum data size is 64TB.

 > If you can tell us specific limit(s) you are worried you may exceed,
 > we can answer that more directly.

That's not yet determined, but I hope a single mongod(i.e, without sharding)
can handle about 300GB-1TB of collection, or 20-30TB of database
with WiredTiger storage engine, stably.

Later I'm going to consider sharding to avoid performance issue,
but for now, I'd like to know the limit(s) of single mongod.

Regards,

2015年12月29日火曜日 2時10分08秒 UTC+9 Asya Kamsky:

[Asya Kamsky]

20-30TB of compressed or uncompressed data?

What sort of server do you have that you think you will get good performance from a single node with this much data?

Also, I presume you will use replication - syncing this much data can take a *really* long time.

Asya

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/c68a8a6a-d367-47cb-bd48-83c96dd4dd6c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Yos Tj]

Thank you for your reply.

 > 20-30TB of compressed or uncompressed data?

Uncompressed. I need to take compression rate into consideration,
which depends on compression algorithm and compressed data.

 > What sort of server do you have that you think
 > you will get good performance from a single node with this much data?
 > Also, I presume you will use replication - syncing this much data
 > can take a *really* long time.

Indeed, it's not reallistic...

How about handling a few 300GB(after compression) collections
in a single mongod with wired tiger storage engine?

Regards,

[Yos Tj]

Hello,

After all, are there any limits of database and collection size with WiredTiger?
Please ignore performance issues for this time. I'm just asking the limits which
mongod has internally.

Some people may want to preserve very large amount of cold(rarely accessed) data
for the possibility of future use. In such case, cost is prior to performance.
They will prefer to do without sharding, which needs more servers and costs more
than simply adding disks to existing server.

Regards,