Issue with mongoexport

[hdev ml]

Hi Mongo Dev,

I exported all my collections and one of the collections had around 350GB of data.

I cleaned the mongo setup, reinstalled with 4 shards. I tried to re-import the data from this exported file through mongoimport and it is giving me issues while importing.

It is finding JSON data with spurious commas e.g.

{ "_id" : { "millis" : 1288603364761, "ptn" : "-12979677046105402" }, "sessionId" : "e327d914-3924-4bb3-8ad0-99a35bc997d8",  }

Note the "," in the second last position.

Because of this the import is failing for a number of rows. Same file had a pattern of having commas like },  }. I replaced those and now encountered this problem.

I am using v 1.6.5, 64 bit on GNU Linux

I do not have the actual data that was stored in the collection. So not sure how you will debug this issue. But this can be possibly be a huge issue for any developer, because you are not sure whether you lost the data or it is just adding a spurious comma in addition to the actual data.

Harshad

[Scott Hernandez]

Unfortunately mongoexport is not the best utility for data fidelity
independent of this issue. You should use mongodump (and mongorestore)
to backup and restore data.

Can you do a dump that way?

> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=en.
>

[hdev ml]

Right now I cannot. The cluster has been cleaned out. I wish I could point out specific instances for mongo dev to fix any issue.

[Scott Hernandez]

Were you able to remove those commas and import your data?

Do you know which documents caused that extra comma at the end? Is
there anything you can tell us to help us track this down? What
platform/version of mongo were you using?

I have created an issue here: http://jira.mongodb.org/browse/SERVER-2742

Please feel free to add any information that might help.

[hdev ml]

The only pattern I find is that these are all small documents. Now I am not sure whether there is any data lost here because, these 3 fields are common to all documents.

Some more examples.

{ "_id" : { "millis" : 1288737773444, "ptn" : "-12979737494758278" }, "sessionId" : "8b46df44-eef0-454e-91fc-b7712d864015",  }
{ "_id" : { "millis" : 1288737773573, "ptn" : "-12979742432515071" }, "sessionId" : "7e325729-d466-4615-bf38-649eec675c33",  }

I have added the same comment in JIRA