Cursor toArray performance

[Paul Gee]

I'm using version the latest 1.x drivers to a single installation of Mongo 3.2.

I'm performing a query which returns around 2500 records (each Json record is about 500 bytes). The query is indexed and I've checked it isn't doing a full scan.

I'm calling the toArray() method on the cursor and it is taking around 200ms to build the array and populate all running on the same machine,

I'm wondering is there a way to improve the performance, check whether the data set is in memory or whether this seems pretty quick.

Are there any quicker ways of retrieving a result set?

Any advice is welcome. 

[Craig Wilson]

In general, performance can depend on what type you are materializing. Using a LazyBsonDocument or a RawBsonDocument will make things much faster, but you'll pay a penalty later if you use them in the same way as you would a fully deserialized entity.

Could you provide the code you are using? Also, could you tell us your use case?

Craig

[Paul Gee]

Thanks Craig for the RawBsonDocument suggestion.

The use case is as follows:

I have a collection of vehicle locations and details which are constantly being updated as the vehicle moves.

I have a geospatial index on the collection and clients using a WCF REST api can query for vehicles in a polygon. They are then returned an array of Json records of vehicles which are contained within the polygon.

I think the delay I am introducing is that toArray is deserializing to an internal structure and then I'm passing the array of objects to the WCF Json serializer to be converted back to Json to be passed to the calling client.

I'd like to bypass the serialization if possible.

The downside of the RawBsonDocument is that every field would be returned to the client.

[Craig Wilson]

Yeah, so your problem is likely that you are doing double work. It'd be great to be able to go straight from bson into json and avoid the overhead of the WCF Json serializer.  We don't have anything built-in to handle that scenario and, more importantly, the WCF Json serializer might make different assumptions than our JsonWriter does.

You have a good separation between your boundaries, so it sucks to mess that up. I'm not sure I have a good suggestion here. RawBsonDocument implements the ToString() method which writes out json. In theory, that may make it possible to ignore the WCF Json serializer completely.  It might be smart to add some stopwatches in there and figure out exactly where the time is spent.

Craig

[Paul Gee]

Hmm, I did a quick test and the overhead of calling ToString() on the RawBsonDocument for each document returned slowed things right down.

I figure this type of issue would have come up before :(

Slightly off topic but is there any significant performance different in using version 2.0+ of the driver compared to version 1.x?

[Craig Wilson]

We haven't seen anything indicating that 2.x has suffered performance regressions. That are certainly edge cases here and there where 2.x outperforms 1.x and 1.x outperforms 2.x, but nothing substantial.

This type of thing doesn't come up all that often because there is usually some business logic between the database and the user. So the deserialization has to happen and then the json writing. When you want to go straight from database to user, you might need to do a little more work to kill the overhead and be able to go straight from bson -> json. If you want to file a ticket, we can investigate and probably add something into the driver to make this scenario simpler.  jira.mongodb.org under the CSHARP project.

[Paul Gee]

Thanks Craig for a comprehensive answer. I'll certainly raise a ticket.

It's because our scenario involves going from the database to a client based app which processes the raw fields.

Cheers for all your help.

Paul.