Which MongoDB types are not preserved by mongoimport/mongoexport?

192 views

Skip to first unread message

Dan Dascalescu

unread,

Jun 12, 2019, 2:34:25 PM6/12/19

to mongodb-user

Re-posting here from StackOverflow hoping for an answer.

I'd be much happier to use mongodump and mongorestore, but mongorestore don't support upserting, which makes it extremely slow to update collections from a production DB to a test DB that needs to reflect prod.

Therefore, I've looked at mongoexport/import. My collection doesn't use any exotic types (only double, int, date, string, bool, object and array), so I should not experience any corruption. However, the documentation is very skittish around which "rich BSON types" exactly might not be preserved. Can that please be clarified? Especially given this paragraph, which seems new-ish:

> To preserve type information, mongoexport and mongoimport uses the strict mode representation for certain types.

Kevin Adistambha

unread,

Jun 14, 2019, 1:47:23 AM6/14/19

to mongodb-user

Hi Dan,

It’s true that since BSON is a superset of JSON with additional data types, there are cases where JSON cannot reflect the correct datatype. However, MongoDB Extended JSON provides a workaround for this.

By default, mongoexport will output extended JSON, so this shouldn’t be an issue.

I did a quick test using some extended datatypes:

> db.test.find()
{
  "_id": ObjectId("5d032fce3741299f7e32aec0"),
  "date": ISODate("2019-06-14T05:25:34.096Z"),
  "ts": Timestamp(1, 0),
  "minkey": [object MinKey],
  "maxkey": [object MaxKey],
  "long": NumberLong("1"),
  "decimal": NumberDecimal("1")
}

then exporting it:

$ mongoexport -d test -c test
2019-06-14T15:25:41.362+1000    connected to: localhost
{"_id":{"$oid":"5d032fce3741299f7e32aec0"},"date":{"$date":"2019-06-14T05:25:34.096Z"},"ts":{"$timestamp":{"t":1,"i":0}},"minkey":{"$minKey":1},"maxkey":{"$maxKey":1},"long":{"$numberLong":"1"},"decimal":{"$numberDecimal":"1"}}
2019-06-14T15:25:41.362+1000    exported 1 record

also re-importing it into another collection:

$ mongoexport -d test -c test | mongoimport -d test -c test2 --drop

The test2 collection contains the correct datatypes:

> db.test2.find()
{
  "_id": ObjectId("5d032fce3741299f7e32aec0"),
  "date": ISODate("2019-06-14T05:25:34.096Z"),
  "ts": Timestamp(1, 0),
  "minkey": [object MinKey],
  "maxkey": [object MaxKey],
  "long": NumberLong("1"),
  "decimal": NumberDecimal("1")
}

If you’re using types listed in the Extended JSON page, it should be ok to use mongoexport. I would suggest testing using your particular dataset to confirm this.

I would mention that one possible issue in using mongoexport/mongoimport would be the need to translate the BSON data into extended JSON, and the reverse during import. This would of course be slower than dumping/restoring the actual BSON data, but this may be unavoidable in your case since you need the upsert feature.

To further clarify this, I have opened DOCS-12805. Please feel free to watch the ticket for updates.