Created index before or after mass data load?

260 views
Skip to first unread message

chriskessel

unread,
Jul 16, 2010, 3:25:02 PM7/16/10
to mongodb-user
I'm loading in, via mongoimport, 350 million records. I have an index
on 1 field. Would it be better to create the index before or after the
load? I created it before and I'm wondering if that's causing the load
perf degradation I'm seeing. I started ~4000/sec being added, but I'm
down to ~700/sec now and I'm only a few million into the load.

Riyad

unread,
Jul 17, 2010, 11:39:38 AM7/17/10
to mongodb-user
Chris,

I would recommend creating the index after - creating it before
requires Mongo to modify the index list on every single import which
*will* cause your inserts to get slower and slower and slower as that
index list grows.

When you create the index at the end, Mongo can grind front to back
through all the entries and create that index list for you.

Slow loading of *huge* data sets have come up before, and when the
indices are removed, people are generally not dying of old age infront
of their keyboards anymore :)
Reply all
Reply to author
Forward
0 new messages