On Jan 15, 2014, at 10:56 AM, Sean Corfield <
se...@corfield.org> wrote:
> You probably need to use `collection/find` and manage the sorting etc yourself so you can process the entire collection (untested):
>
> (with-open [cursor (doto (collection/find collection {})
> (.batchSize 512)
> (.sort (convert/to-db-object {:timestamp 1})))]
I was able to make it work with this:
(->> (with-collection collection
(find {})
(sort {:timestamp 1})
(paginate :page page :per-page our-batch-size)
(batch-size our-batch-size))
(map filters/old-profile-entry)))
… where this code is called with new `page` values until it returns an empty sequence. I was inspired to do this by reading in some Mongo documentation that sorting won't work, even with an index, if <some resource> exceeds 32M.