Pagination in boss_db?

[David Welton]

Would it be worth including something like this directly in boss_db ?

paginate(Model, Conditions, Args) ->
Page = proplists:get_value(page, Args, 1),
PageSize = proplists:get_value(page_size, Args, ?DEFAULT_PAGE_SIZE),
ArgList = proplists:delete(page_size, proplists:delete(page, Args)) ++
[{offset, PageSize * (Page - 1)}, {limit, PageSize}],
Total = boss_db:count(Model, Conditions),
TotalPages = (Total div PageSize) + (case Total rem PageSize of
0 -> 0;
_ -> 1
end),
{Page, TotalPages, boss_db:find(Model, Conditions, ArgList)}.

--
David N. Welton

http://www.welton.it/davidw/

http://www.dedasys.com/

[kotedo]

+1

> --
> You received this message because you are subscribed to the Google Groups "ChicagoBoss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to chicagoboss...@googlegroups.com.
> Visit this group at http://groups.google.com/group/chicagoboss.
> To view this discussion on the web visit https://groups.google.com/d/msgid/chicagoboss/CA%2Bb9R_vQ9me_eTTwad7-fNqhyoqHD5N0xd8dLqAq8CUtwHECKg%40mail.gmail.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Tim McNamara]

+1

Easy pagination in core seems very useful.

To view this discussion on the web visit https://groups.google.com/d/msgid/chicagoboss/80ED8459-BE68-49A9-8EA1-02D43E7D6731%40gmail.com.

[David Welton]

Ok, here it is:

https://github.com/ChicagoBoss/boss_db/pull/153

> https://groups.google.com/d/msgid/chicagoboss/CAMUs9-N7%3DZS%3DChSKJ8sV5RfHB%2BkjSs6%2Be7W4_pW%3DZk4KCHjBxQ%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/groups/opt_out.



--

[Evgeny M]

Pagination with mnesia could be quite tricky.

mnesia adapter contains the following function:

count(Conn, Type, Conditions) ->

    length(find(Conn, Type, Conditions, all, 0, id, ascending)).

So every time you do Count on mnesia boss_db will load full query result in memory, with all data of all fields.

And as you do boss_db:find(Model, Conditions, ArgList) again, you will do the same query and load the same data again. 

You may not notice this on developers box when there's only so much of data, but on real server you will have huge hit on performance and memory. 

Of course there's no such problem on postgres or mysql, but count(*)..where ...  is quite heavy operation even in RDBMS world

среда, 22 января 2014 г., 19:17:20 UTC+4 пользователь David Welton написал:

[David Welton]

> Of course there's no such problem on postgres or mysql, but count(*)..where
> ... is quite heavy operation even in RDBMS world

Suggestions to improve things? I think having a simple paginate is a
good first step - people who need more can optimize according to their
needs.

[Graeme Defty]

HI Evgeny,

Yes, Mnesia does not have a 'count' function, so I did a 'quick and dirty' to implement the boss_db functionality. It was not something I needed so I did not worry about it too much at the time. (Actually I thought I was the only one using Mnesia!)

Short of enhancing Mnesia itself I am not sure how to improve this.  :-(

g

--

You received this message because you are subscribed to the Google Groups "ChicagoBoss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chicagoboss...@googlegroups.com.
Visit this group at http://groups.google.com/group/chicagoboss.

To view this discussion on the web visit https://groups.google.com/d/msgid/chicagoboss/abd60772-8956-49b6-ba66-273e057fb286%40googlegroups.com.

[Samuel Rose]

On Thu, Jan 23, 2014 at 8:51 AM, Graeme Defty <graeme...@gmail.com> wrote:
> HI Evgeny,
>
> Yes, Mnesia does not have a 'count' function, so I did a 'quick and dirty'
> to implement the boss_db functionality. It was not something I needed so I
> did not worry about it too much at the time. (Actually I thought I was the
> only one using Mnesia!)
>
> Short of enhancing Mnesia itself I am not sure how to improve this. :-(
>
> g
>
>

For mnesia, I wonder if
http://www.erlang.org/doc/apps/mnesia/Mnesia_chap4.html#id74480
pattern matching can help here? First to limit, then to select where
pattern does not match the first id's in the first limit.

Just a thought anyway.

> https://groups.google.com/d/msgid/chicagoboss/CAKF5fiCtr9iQqtCoJaMheCCHcvetOCKwgFQmsLt-wLJV6EnM_w%40mail.gmail.com.

[chan sisowath]

here mnesia count with select
https://github.com/mihawk/boss_db/commit/cf5e1ce91477fce731d0c4f504cad55141b09698

To view this discussion on the web visit https://groups.google.com/d/msgid/chicagoboss/CA%2BH5g6oBmrkN3yFkK8CiC4fANNLJ%3DGEYPCMWseOi5kXm1tJq9g%40mail.gmail.com.