How to limit query when using Kaminari?

[Jörgen Nilsson]

Heya,

I've searched all morning for a proper answer or solution but can't
find any. This is more a Kaminari question than MongoID, but they feel
quite related in this case.

All I want to do is to limit my query to 1000 documents. But it seems
like Kaminari overrides or just skips the limit specified.

I've tried a bunch of different variations, but none of them are
working and I just can't find a good answer or solution, via Google,
Stackoverflow or GitHub for example.


Query variations, countries is an array of country iso short codes
['CA', 'SE', 'DE'] for example
-------------------------------------------------
@accounts = Account.any_in(country_short:
countries).limit(100).page(params[:page]).per(20)
@accounts = Account.all(conditions: {:country_short.in => countries},
limit: 100).page(params[:page]).per(20)
@accounts = Account.where(:country_short.in =>
countries).limit(100).page(params[:page]).per(20)


These quires returns the expected result, but without the pagination
part of course.
-------------------------------------------------
@accounts = Account.any_in(country_short: countries).limit(100)
@accounts = Account.all(conditions: {:country_short.in => countries},
limit: 100)
@accounts = Account.where(:country_short.in => countries).limit(100)


Versions and gems used
--------------------------------------------------
bson (1.6.2)
bson_ext (1.6.2)
mongo (1.6.2)
mongoid (2.4.7)
kaminari (0.13.0)

Would be great if someone could shed some light on this matter. Is it
possible at all? Is it a bug? Am I doing something very wrong? If it
is a bug, is there a work around?

[Durran Jordan]

Kaminari's #page method appears to be overriding the limit - have you tried putting limit after the call to page?

https://github.com/amatsuda/kaminari/blob/master/lib/kaminari/models/mongoid_extension.rb#L24

2012/4/12 Jörgen Nilsson <jor...@modondo.com>

[Jörgen Nilsson]

Jupz, I've tried that as well. The query runs without any problems but
the limit is not respected and a count() always shows the same result.
A side effect is that the pagination helper miscalculates the current
page. So if I access ?page=5 then page 4 is shown as active in the
pagination helper. When I remove the limit option, this behavior goes
away.

I've tried using scopes for both "filter" and limit, but without
success. I've also tried a plain
Account.all.limit(100).page(1).per(20) without any luck.

Might be something else that I'm missing. If I run @accounts =
Account.all.limit(100) I can iterate over 100 documents (@accounts)
but the count of @accounts says 219 (current number of documents in
the collection)

This must be a very common problem for quite a lot of people? :D

Thanks for your quick response!

On Apr 12, 11:35 am, Durran Jordan <dur...@gmail.com> wrote:
> Kaminari's #page method appears to be overriding the limit - have you tried
> putting limit after the call to page?
>

> https://github.com/amatsuda/kaminari/blob/master/lib/kaminari/models/...
>

[rubish gupta]

Count doesn't respect limit property by default in mongodb. To get a count respecting limit, you need to add another parameter to the count.

Account.count                        == Account.limit(100).count     #=> true

Account.limit(100).count(true)  == 100                                  #=> true

I don't understand the thing with original question though, if you want 100 items at a time why don't you make the page size to be 100. I am sure I am missing something here.

2012/4/12 Jörgen Nilsson <jor...@modondo.com>

[Jörgen Nilsson]

> I don't understand the thing with original question though, if you want 100
> items at a time why don't you make the page size to be 100. I am sure I am
> missing something here.
>

Let say I have 50 000 documents in my collection. I only want to fetch
the latest 100 inserted documents. These 100 documents I then want to
paginate displaying 20 documents per page using pagination. This will
would give me 5 pagination pages.

When using limit I get all 50 000 documents and therefor get an
ridiculously amount of pagination pages.

So the question is how can get the latest 100 documents and paginate
them 20 documents (link to each document) per page without fetching
the entire collection?

This must be a very common task. It feels like I'm just missing some
key setting or something.

[Daniel Doubrovkine]

I had fixed the count(true) part in Kaminari in  https://github.com/amatsuda/kaminari/pull/194, so that's properly limiting the number of items.

But using limit and then paginating results is not something that's supported in Kamari. I created https://github.com/amatsuda/kaminari/issues/228. I think it could be done by changing some of Kaminari's count methods to examine the current count vs. the existing limit on the criteria.

2012/4/16 Jörgen Nilsson <jor...@modondo.com>

--

dB. | Moscow - Geneva - Seattle - New York
dblock.org - @dblockdotorg

[Jörgen Nilsson]

Daniel, thanks for your answer and nicely done filing the issue on
GitHub.

You or anyone else wouldn't know an easy to to paginate for example
200 documents out of 40 000. I really don't want to paginate 40 000
documents when I only need to fetch 200.

Any ideas or maybe know of another pagination gem that works well with
Mongoid / Monogdb where it is possible to paginate a limited result?

On Apr 17, 1:33 pm, Daniel Doubrovkine <dbl...@dblock.org> wrote:
> I had fixed the count(true) part in Kaminari inhttps://github.com/amatsuda/kaminari/pull/194, so that's properly limiting

> the number of items.
>
> But using limit and then paginating results is not something that's

> supported in Kamari. I createdhttps://github.com/amatsuda/kaminari/issues/228. I think it could be done

[Daniel Doubrovkine]

Pagination doesn't fetch more than it shows, so paginating 4M documents or 10 takes the same amount of time. The only difference will be in displaying how many pages you have.

2012/4/19 Jörgen Nilsson <jor...@modondo.com>

[Jörgen Nilsson]

Jupz, that is very true.

So the question is, how do I show a pagination list/menu from 1-10
(first 10 pages only) when fetching 10m documents?

When I think about it, I might have gone at it the wrong way. In
reality I only want to show 1-10 and if I would click on 10, the
pagination list should show something like 5-14 or 10-19.

This should be possible and hopefully fairly easy to achieve?

Still, it would be much easier to only fetch the 100 or so documents I
really want to show, but sometimes you just have to compromise :)

[Daniel Doubrovkine]

You do it by making changes in Kaminari that let you write Model.all.limit(100).page(3).per(10)

Inside the Kaminari code examine the current limit option and store it as a max value. Then use min of @max and total count instead of total count where appropriate. Great place for an open-source contribution! This is really not a mongoid question at this point. Good luck :)

cheers

dB.

2012/4/19 Jörgen Nilsson <jor...@modondo.com>

[Nick Hoffman]

On Thursday, 19 April 2012 15:03:21 UTC-4, Jörgen Nilsson wrote:

So the question is, how do I show a pagination list/menu from 1-10
(first 10 pages only) when fetching 10m documents?

When I think about it, I might have gone at it the wrong way. In
reality I only want to show 1-10 and if I would click on 10, the
pagination list should show something like 5-14 or 10-19.

That's easy to do with the various options that Kaminari provides with its view helpers. In Kaminari's README, scroll down to the section titled "Helpers", and look into the "window", "outer_window", "left", etc options.

https://github.com/amatsuda/kaminari/

[Jörgen Nilsson]

Thank you for the pointers! I'll see what I can do. Jupz, you are
right, it is now beyond Mongoid :)

[Jörgen Nilsson]

One of the first things I did was to create my own Kaminari theme so
that Kaminari would play nice with Twitter Bootstrap pagination.
I've adjusted the helper and it now seems to work as intended. Thanks
for the pointers!

Still, it kind of bothers me that Kaminari seems fetch millions of
document and operate on them when I only need around 500 documents in
return. Now, this will not be a problem since I don't have or will
have millions of documents in my collection for quite a foreseeable
future.

[Jörgen Nilsson]

One of the first things I did was to create my own Kaminari theme so
that Kaminari would play nice with Twitter Bootstrap pagination.
I've adjusted the helper and it now seems to work as intended. Thanks
for the pointers!

Still, it kind of bothers me that Kaminari seems fetch millions of
document and operate on them when I only need around 500 documents in
return. Now, this will not be a problem since I don't have or will
have millions of documents in my collection for quite a foreseeable
future.

On Apr 22, 7:19 pm, Nick Hoffman <n...@deadorange.com> wrote:

[Durran Jordan]

I'm not sure what you mean by "fetch millions of documents and operate on them". Performing a count with no query will just return the total, performing a count with a query will scan only the index. The only documents that actually get returned are the ones within the skip and limit for each page since we are using a cursor and not bringing everything back in memory.

Correct me if I am wrong, but it seems like you really don't want traditional pagination anyways, which is really not usable. When would I as a user ever say "oh I'm going to click on page n, because I guess what I might be looking for is there." Unless you have a business requirement for some reason where page numbers really matter, I'd probably just go with a "more" button or paged scrolling that simply just gives to the next n documents at a time. If you have a lot of documents and you need users to find something specific, this is where search comes into play.

2012/4/24 Jörgen Nilsson <jor...@modondo.com>