pymongo's bson takes too much time to do simple json encondig!

229 views
Skip to first unread message

Aliane Abdelouahab

unread,
Mar 9, 2016, 3:41:00 PM3/9/16
to mongodb-user
Doing benchmarks with this:

tornado     serialize: 1.225  deserialize: 2.117  total: 3.342
ujson       serialize: 0.935  deserialize: 0.413  total: 1.348
simplejson  serialize: 1.944  deserialize: 0.794  total: 2.738
mongodb bson serialize: 17.280  deserialize: 3.438  total: 20.718
stdlib json serialize: 1.079  deserialize: 2.053  total: 3.132

If the json is too big, the operation will never end!

Bernie Hackett

unread,
Mar 9, 2016, 8:20:20 PM3/9/16
to mongodb-user
Yes, json_util is not fast. It uses python's json module to convert JSON strings to python dictionaries, then iterates over those dictionaries to turn the mongo extended JSON format into PyMongo specific objects. This is slow. If you don't need your mongo extended JSON in PyMongo format, just use python's json module or simplejson.

Bernie Hackett

unread,
Mar 9, 2016, 8:32:53 PM3/9/16
to mongodb-user
In fact, your benchmark doesn't need json_util at all. All json_util does in your case is add a bunch of overhead looking for mongo extended JSON types.

Aliane Abdelouahab

unread,
Mar 10, 2016, 1:59:57 AM3/10/16
to mongodb-user
ah, got it, thank you ^_^
Reply all
Reply to author
Forward
0 new messages