Mongoimport csv data - delay while selecting the collection

45 views
Skip to first unread message

EvalDocDB

unread,
Jun 1, 2010, 1:33:01 PM6/1/10
to mongodb-user
I am doing a mongoimport on a csv file and it did output the message
"imported 7216081 objects". When I do a query on the collection using
the db.collection.find().count() and the data returned was less than
the import reported.
It looks like the data is in memory and is flushed to disk every 60
seconds, in my case it took 45 minutes to get all the count in the
collection after the import has completed. Can the process be improved
using some settings or is this normal.

My environment is Sharded with 3 servers (1.5.2 release)

One more question : Is there a way to import nested collections using
mongoimport from csv file


Eliot Horowitz

unread,
Jun 1, 2010, 4:10:16 PM6/1/10
to mongod...@googlegroups.com
Might be an issue with sharded getlasterror not blocking correctly.
Its not that its syncing to disk every minute, its that the inserts
are queued up in the server.
import is supposed to wait till they're applied, but i think the
shraded version of that isn't quite right.

No way to do nesting with csv.
Kind of a weird api.
Suggestions/patches welcome

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

Reply all
Reply to author
Forward
0 new messages