Using cache for improving pagination recipe in the manual?

[Giacomo Dorigo]

I have read the manual recipe on pagination:

http://web2py.com/books/default/chapter/29/14/other-recipes#Pagination

That works fine if we just want to have a page aware if it has a previous or next one, but what if we want also to get the list of all next pages? i.e. by putting a link for each next page, near the next "button"?

Let's say I have 300 records in a table, I want to serve 10 per time. If I am serving page 1, there are other 29 pages more, so I want to be able to put links at least to page 2, 3, 4, 5, 6.... so to make navigation easier.

The recipe in the manual is quite good, because it allows to get only the 10 elements I need to serve in the current page, but my page do not know how many pages there are in total.

I was thinking to solve this creating a PageThread class taking care of storing the information about the whole. My idea was to let this class making the query for all items in the table and then creating the single pages to serve.

But the problem here is that when the controller will instantiate this class it will reset it all times, so it will make the query to the whole table every time, so it's wasted.

Probably this can be solved using the web2py built-in count() or is there a possibility to use cache?

Does anybody already solved this problem somewhere?

This is a sample of the code of the two classes I thought about:

class PageThread(object):
    def __init__(self, items, items_per_page=10):
        self.items = items
        self.items_count = len(self.items)
        self.items_per_page = items_per_page
        self.__set_number_of_pages()
        self.__load_pages()
           
    def __set_number_of_pages(self):
        self.number_of_pages = self.items_count/self.items_per_page + 1 if self.items_count % self.items_per_page else self.items_count/self.items_per_page

    def __load_pages(self):
        self.pages = [Page(p_id, self.items[(p_id-1)*self.items_per_page: p_id*self.items_per_page]) for p_id in range(1, self.number_of_pages+1)]


class Page(object):
    def __init__(self, id, items):
        # id is the unique number of the page in the PageThread
        self.id = id
        self.items = items

    def __str__(self):
        return 'Page n.{}, {} items: {}'.format(self.id, len(self.items), self.items)

While this was the original Page that just knows if it has previous or next:

class Page(object):
    def __init__(self, page, items_per_page=10):
        self.id = int(page)
        self.items_per_page = items_per_page
        # range gives you an item more in order to check if there is a next page
        self.range = ((self.id-1)*self.items_per_page, self.id*self.items_per_page+1)


    def load_items(self, rows):
        # where rows are those returned from a select query to the db
        if len(rows) > self.items_per_page :
            self.has_next = True
            self.items = rows[:-1]
        else:
            self.has_next = False
            self.items = rows


    def get_page_navigation_div(self, **request_vars):
        contents = []
        request_vars.update({'items_per_page': self.n_items})
        if self.has_previous:
            request_vars.update({'page': self.id-1})
            contents.append( A('<<< previous', _class='nav-previous', _href=URL(vars=request_vars)) )
        for n in range(1, self.id):
            request_vars.update({'page': n})
            contents.append( A(' %s' % n, _class='nav-page', _href=URL(vars=request_vars)) )
        if self.has_next:
            request_vars.update({'page': self.id+1})
            contents.append( A('next >>>', _class='nav-next', _href=URL(vars=request_vars)) )
        return DIV(*contents, _id='page_navigation')

In the controller you use like this:

page = Page(request.vars.page, 10)
page.load_items(db(your_query).select(limitby=page.range))
#do stuff
return dict(page=page)

[Giacomo Dorigo]

In the end I solved with two simple classes, without using cache, but with two queries, one counting total items and one limited by limitby. Also I have looked in slices and I found some pagination plugins.

Anyway here the basic code:

class PageThread(object):

    def __init__(self, query, items_per_page=10):

        self.query = query

        self.items_count = db(self.query).count()

        self.items_per_page = items_per_page

        self.__set_number_of_pages()

            

    def __set_number_of_pages(self):

        self.number_of_pages = self.items_count/self.items_per_page + 1 if self.items_count % self.items_per_page else self.items_count/self.items_per_page

    def serve_page(self, page_id):

        page_items = db(self.query).select(limitby=((page_id-1)*self.items_per_page, page_id*self.items_per_page))

        p = Page(page_id, page_items)

        p.previous_pages = [n for n in range(1, page_id)]

        p.next_pages = [n for n in range(page_id+1, self.number_of_pages+1)]

        return p

class Page(object):

    def __init__(self, id, items):

        # id is the unique number of the page in the PageThread

        self.id = id

        self.items = items

    def __str__(self):

        return 'Page n.{}, {} items: {}'.format(self.id, len(self.items), self.items)

    def get_page_navigation_div(self, **request_vars):

        request_vars.update({'items_per_page': len(self.items)})

        previous_pages_subdiv = DIV(_class='previous-pages-subdiv')

        next_pages_subdiv = DIV(_class='next-pages-subdiv')

        if self.previous_pages:

            request_vars.update({'page': self.id-1})

            previous_pages_subdiv.append( A('<<< previous', _class='nav-previous-link', _href=URL(vars=request_vars)) )

        for n in self.previous_pages:

            request_vars.update({'page': n})

            previous_pages_subdiv.append( A(' %s' % n, _class='nav-previous-pages', _href=URL(vars=request_vars)) )

        for n in self.next_pages:

            request_vars.update({'page': n})

            next_pages_subdiv.append( A(' %s' % n, _class='nav-next-pages', _href=URL(vars=request_vars)) )

        if self.next_pages:

            request_vars.update({'page': self.id+1})

            next_pages_subdiv.append( A(' next >>>', _class='nav-next-link', _href=URL(vars=request_vars)) )

        return DIV(previous_pages_subdiv, next_pages_subdiv, _class='page-navigation-div')

    def get_style(self):

        style = STYLE("""\

        .page-navigation-div {

            width: 1170px;

        }

        .page-navigation-div div {

            display: inline;

        }

        .page-navigation-div .next-pages-subdiv{

            float: right;   

        }

        """)

        return style

[Niphlod]

if you're presenting 10 rows at a time and expecting the user to navigate 30 pages, "you're doing it wrong", from the UX side of things.

On the other side, there's no reason to cache "in advance" something the user will never ask. Let the cache be the cache. If you need to "warm it up", just do

cached_it = False #you logic to see if the cache needs to be updated

if not cached_it:

    for i in range(number_of_pages):

        rtn = db(query).select(limitby=(number_of_pages*no_of_records, (number_of_pages+1)*no_of_records))

    cached_it = True

but mind that is quite silly.

[Giacomo Dorigo]

Well, the 10 rows for 30 p was just an example for understanding :)
The idea of the was in order to do only 1 query but definitely you are right, it wasn't worth of!

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/-awUNbuI_t8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Derek]

This is one of those cases of mixing your controller with your view. Please don't do that. 

Look here for that...

http://en.wikipedia.org/wiki/Link_prefetching

and please keep your html out of your controllers. they should only go in views.