Can Mongo DB handle a databse like this?

47 views
Skip to first unread message

Razvan T

unread,
Sep 9, 2015, 9:21:57 PM9/9/15
to mongodb-user
Hello!

I need to store a maximum of 200 billion records, each having 2 fields: A, and B, both strings of no more than 255 characters.
I'll be inserting records into this database at a pace of about 50 000 per second.
About once a second, I'll also be querying the database. All the queries will be the same: I'll need all the records where the field A=X, for a given X string.

1. Is it possible to use Mongo DB to store such a database?
2. What kind of hardware would I need to store it?

Thanks,

sallgeud

unread,
Sep 11, 2015, 7:32:33 PM9/11/15
to mongodb-user
It'll certainly work. With the right hardware and properly threaded inserting, it could work on 1 system, but you'll probably want more. 

It seems like what you really have is the most raw form of a key/value store, if A is unique.  If not, you may still benefit from a less capable system, as your query and data is so simple. You could fairly easily make a SQL database do what you want as well, so this isn't a problem that Mongo will help with beyond the fact that if your data gets bigger, you'll want sharding / scaling capabilities that are much easier in Mongo than in SQL databases (mysql being the easiest, but still painful). 

Our company has worked with sql, couchbase, elastic.co, redis, memcache and many other key value stores. We prefer mongo, but it's not always the answer. Nearly all of those would probably also work. These are all also very easy to test. I suggest running performance test on a few of them with a sample of your data and choosing the one that works best.
Reply all
Reply to author
Forward
0 new messages