Should we use caching engine (like Memcache) above MongoDB

[Martin Poirier]

We are developping new feature for our application using MongoDB. Since we are use too a MySQL / Memcache environment to reduce server load on the database I'm woundering if it's also a good practice with Mongo DB ? Or should we just more focus on using sharding properly ?

[Sam Millman]

http://www.mongodb.org/display/DOCS/Philosophy

There is a nice graph there on the right which kinda sums it up. Really Mongo is a lot like memcached, it is just ever so slightly heavier to deal with more complexity etc.

So for the min I would focus on sharding, schema and querying and if you find slow bits in future just add a bit of memcached.

On 15 October 2011 14:45, Martin Poirier <mpoi...@woozworld.com> wrote:

We are developping new feature for our application using MongoDB. Since we are use too a MySQL / Memcache environment to reduce server load on the database I'm woundering if it's also a good practice with Mongo DB ? Or should we just more focus on using sharding properly ?

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Martin Poirier]

According to this philosophy is it valid to use MongoDB as the session storage engine ? We are currently using memcache but we could have session lost on some server failling. We were going to manage a store to MySQL at every 'n' write but this is causing the issue of losing some data in case of faillure, having too many technology for the same "service" and developping it !. We tought about going to Redis for this specic needs. Is Mongo still a viable alternative for this ?

Thanks

2011/10/15 Sam Millman <sam.m...@gmail.com>

[Karl Seguin]

Yes, using memcached with MongoDB makes sense...but only once you reach a point where you need a fast, in-memory, distributed key=>value store.

They server difference purposes. They are build differently. They compliment each other, but also overlap a little. You can use MongoDB as a key=>value store, which might be the first place to start...that way you have a single backend database. Once you have an actual need for a more specialized cache, based on real measurements, then you can introduce memcached.

I disagree with Sammaye. I would say that, in relation to this discussion, MongoDB is like MySQL. And the harmony you have with MySQL + memcached is the same harmony you'll have with MongoDB + memcached...but again, I wouldn't introduce memcached until I knew, for certain that I needed it to handle the load..and I think you'll find that you'll get more performance and scale out of  MongoDB than you did with MySQL.

For sessions, yes, store it in MongoDB. Retrieval by _id is fast. If ever you have so many users that it isn't fast enough, add memcached infront of it.

[Sam Millman]

From testing I have done that would have to be a huge user group. My test went upto 500m users so unless your facebook or google your not gonna need memcached any time soon.

--

You received this message because you are subscribed to the Google Groups "mongodb-user" group.

To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/7ESTA-t1WSsJ.

[Martin Poirier]

Do you mean 500 000 or 500 000 000.

Because we are a few times above 500 000  but way under 500 000 000

2011/10/15 Sam Millman <sam.m...@gmail.com>

[Sam Millman]

500,000,000.

With upto 1,500,000 I doubt very much you will need to worry about memcached, in my automated tests from scripts I calculate mongo should handle perfectly well without memcached. Of course as a disclaimer I will say this is a test on my hardware and my servers and with automated script, but still promising. I tested this on a graph social network wall (much like facebook).

The time you might need to worry is if you are doing big queries, but then you might wanna think harder about your shcema design tbh.

[Martin Poirier]

We are in the same kind of project (social network) we are using mongo for storage, neo4j for the graph part and we still have mysql for all the old feature (we don't want to mitgrate for now).

We are doing a close beta for our paying user only so we should have time to tweak everything before the big opening.

I thought about using sharding with 1 shard (if possible) so the setup will already be there if we need to add more shard, from what I understand we can add shard without downtime, is it right ?

I'm not the sys admin of the system but I pretty lead the decision to go with mongo so...

Martin

2011/10/15 Sam Millman <sam.m...@gmail.com>

[Eliot Horowitz]

Depends why your'e using memcache.

If you are using in mysql to cache user profiles because to reassemble
a user profile from a bunch of tables is slow, then I wouldn't use in
mongo.
If you are using to cache full html, or computed fields, it still
might make sense.