Overflow sort stage buffered data usage of 33716050 bytes exceeds internal limit of 33554432 bytes

[Santosh Kumar]

I am getting the error "Overflow sort stage buffered data usage of 33716050 bytes exceeds internal limit of 33554432 bytes" if offset in the query is 80 and limit is 20 (pagination).

But when the offset is 0 and limit is 20 then same query is working fine. 

   I am not able to understand if i can fetch the first 20 records and next records till 80 then why it's not able to fetch records after 80. How does sort and limit works in Mongo DB.

* The sorting field is not indexed. I know that if I will index this sort field then the above error is solved but still my question is why it's not throwing error for the first 20 records. Also,  I am using the latest version of Mongo DB (3.0).

[Amar]

Hi Santosh,

Using skip 80 and limit 20 is going to cause 100 documents to be sorted, then the first 80 documents to be skipped and return the next 20 documents. On the other hand, skip 0 and limit 20 would cause only 20 documents to be sorted and returned. There is a limitation of 32 MB for sort so depending on the size of your documents those 100 documents to be sorted might exceed that limit.

As you mentioned, using an index eliminates the 32 MB limit. Another option might be using aggregation which has a 100 MB limit on sort with the option to set allowDiskUse to allow the usage of disk for the sort.

Regards,

Amar

​