paginate and query parameters

[Talin]

OK I've spent about an hour searching the docs and looking at the
source code to the paginate decorator, and I can't figure this one out.

How do I use paginate in conjunction with a controller method that
takes multiple parameters? I have a search page that takes a url in the
form "page.html?search=xxx". I'd like to be able to use the pagination
decorator - however, I can't figure out how to get the pagination
decorator to combine my "search=" query parameter with its own query
params.You would think that get_href would take optional params
indicating additional query params - just like tg.url does.

(And you can't use tg.url to glue together the pagination url and your
own query, because that just adds an extra '?' making the URL invalid.)

-- Talin

[Adam Jones]

get_href already uses tg.url, but could probably be fixed by adding
keyword support and having it pass those keywords to tg.url. Here is
what would need to be changed:

In the file paginate.py:

- def get_href(self, page, order=None, reverse_order=None):

+ def get_href(self, page, order=None, reverse_order=None, **kw):

- return turbogears.url('', self.input_values)

+ return turbogears.url('', self.input_values, **kw)

Hope that helps, I haven't had a chance to test this so it may not work
first try.

-Adam

[Nicky Ayoub]

We had to make some changes to paginate.py. Specifically we added the following code into class Paginate in the __init__(). It wraps the  incoming list and dict structures so that the GET url used on the links can handle the nested structures. A colleague of my has a version that is recursive. The code here only goes a level deep. It was needed to pass form data along with the pagination. It's taken from Randall's patch for pagination.

Look at http://trac.turbogears.org/turbogears/ticket/1115 for more.

...
        # If we have a nested dict from a form, it must be denormalized before
        # it can be put into a GET URL.
        #     {'form_field':{'text':'hello', 'hidden':''}
        # Goes to:
        #     {'form_field.text':'hello', 'form_field.hidden':''}
        # I'm going to be specific so as to minimize breakage.
        #
        #  -- Nicky's change. Break out lists like Randall did for dicts.
        for (ikey, ivalue) in input_values.items():
            if isinstance(ivalue, dict) and len(ivalue) > 1:
                del input_values[ikey]
                for (sub_ikey, sub_ivalue) in ivalue.items():
                    if isinstance(sub_ivalue, list) and len(ivalue) > 1 :
                        for idx, sub_sub_ivalue in enumerate (sub_ivalue):
                            new_key ='%s.%s-%d' % (ikey,sub_ikey,idx) # Magic list format
                            input_values[new_key] = sub_sub_ivalue
                    else:
                        new_key = '%s.%s' % (ikey, sub_ikey)
                        input_values[new_key] = sub_ivalue
# This is probable needed for completeness...
            if isinstance(ivalue, list) and len(ivalue) > 1 :
                for idx, sub_ivalue in enumerate (ivalue):
                    new_key ='%s-%d' % (ikey,idx) # Magic list format
                    input_values[new_key] = sub_ivalue

--
--
Nicky Ayoub
G-Mail Account

[Alberto Valverde]

I'd strongly suggest using formencode.variabledecode.variable_encode for encoding the data NestedVariables decodes (using formencode.variabledecode.variable_decode) from the request parameters in TG's filter. 

Just in case one day this encoding format changes in FE (and to avoid wheel reinvention ;) )

Alberto

[Nicky Ayoub]

AMAZING !  I took the patch in #1115 and replaced the encoding with the formencode version. Works like a charm.
I had as sneaky suspicion that the code was in there somewhere...
Thanks for the tip!

In the standard paginate.py I added the following as the first line of Paginate.__init__():
input_values = variable_encode(input_values)
Don't forget to import...

Nicky

[Nicky Ayoub]

Here's the simple 2 line patch that I forgot to add to my previous response.

On 10/9/06, Nicky Ayoub <nicky...@gmail.com > wrote:

AMAZING !  I took the patch in #1115 and replaced the encoding with the formencode version. Works like a charm.
I had as sneaky suspicion that the code was in there somewhere...
Thanks for the tip!

In the standard paginate.py I added the following as the first line of Paginate.__init__():
input_values = variable_encode(input_values)
Don't forget to import...

Nicky

[Randall]

I'm thrilled to see this because my solution is a terrible hack and
I've applied it to my version of paginate.py. I was hoping it would
handle input from DatePicker and friends, but they're still coming in
as datetime objects when they should be strings. Any idea why they're
not being converted?

[Randall]

I see now that variable_decode doesn't do what I thought. I guessed it
dealt with the the validators and called validator._from_python, but it
doesn't. Maybe, before putting values in the GET URI, the input should
be fetched from cherrypy.request. The current method will explode if
you do any data manipulation in the validator. Using cherrypy.request
seems pretty foolproof. What do you think?

[Randall]

This code takes input_values from cherrpy.request.params instead of kw
and seems to do what's needed.

input_values =
variable_encode(cherrypy.request.params.copy())
input_values.pop('self', None)
for input_key in input_values.keys():
if input_key.startswith('tg_paginate'):
del input_values[input_key]

[Ian Wilson]

Hello,
I am kind of confused as to what the solution to this problem is.
Should I apply all thost patches from the trac page or just the last
one? I tried to use just the last one which included my non-keyword
argument from my controller in the get_href url but the
next/prev/last/first urls are all still input_values-less. How can I
fix those as well?
Ie.
@expose(template='...'
@validate(validators=dict(id=vs.Int(not_empty=True)))
@paginate(var_name='images', max_pages=10, limit=2,
allow_limit_override=False)
@error_handler(index)
def editImages(self, id, tg_errors=None, **kwargs):

Going to http://localhost:8080/admin/brand/editImages?id=1 makes:

A page url: http://localhost:8080/admin/brand/editImages?tg_paginate_limit=2&id=1&tg_paginate_no=2

But the next/last urls:
http://localhost:8080/admin/brand/editImages?tg_paginate_limit=2

I looked at the code and it looks like the input values should be okay
when they get to making the urls but it is not working. How can I get
the id in that next/last urls?

Thanks.

-Ian

[Ian Wilson]

Well never mind apparently that patch is not for my version or
something because it actually breaks a lot of things. The moral of
the story is do not expect non-keyword arguments to get put into urls
generated by paginate. Which I guess I can live with... Its
painful,not intuitive, having a validator say that sometihng cannot be
empty and then in the def having id=None which seems to express that
an empty id is okay.

-Ian

[Randall]

Ian, the last patch was based on revision 2013 and I'm using it in a
production environment without issue. As far as I know, it should
still work with the current version. I haven't figured out what's
causing your problems yet, but I'm looking into it. If it's not to
much trouble, could you send me a quickstarted app that demonstrates
the problems. Thanks.

Randall

[Randall]

Looks like patch 6 introduced new problems. I really need to write
unit tests, but I've run into trouble there:

http://groups.google.com/group/turbogears/browse_thread/thread/4380df8d5b5823ec/7f0e5025b4263014?lnk=raot#7f0e5025b4263014

Patch 5 should work for you. Note that until I can write a full set of
tests for it, I don't consider the paginate changes stable.

Randall

[Ian Wilson]

Yeah okay well sorry I can't be of more help right now I am getting
run over at work by more work. I think using it without the patch
actually seems to be working as long as I use keyword arguments
everywhere and as you mention in that post the order matters as
well(paginate versus validate). And if it means anything it has been
incredibly useful to me. That is wierd that the test util is so
broken.

-Ian

[Ian Wilson]

I'll try that when I get to work.

[Ian Wilson]

Hey Randall sorry it took me so long to get back.

So the quickstart you sent me works but I realized that my example was
not working because my template tries to use the urls in tg.paginate
like this:

<div>
<span><a
href="${tg.paginate.href_first}">First</a></span>
<span><a href="${tg.paginate.href_prev}">Prev</a></span>
<span py:for="page in tg.paginate.pages">
<a py:if="page != tg.paginate.current_page"
href="${tg.paginate.get_href(page)}">${page}</a>
<b py:if="page ==
tg.paginate.current_page">${page}</b>
</span>
<span><a href="${tg.paginate.href_next}">Next</a></span>
<span><a href="${tg.paginate.href_last}">Last</a></span>
</div>

Somehow the input_values got updated and put in self but the following
lines didn't get tended to: So this seems to be the problem(These
lines numbers are probably not correct in reference to the current
source):
160c117
< self.href_next = turbogears.url(cherrypy.request.path,
self.input_values)
---
> self.href_next = turbogears.url(cherrypy.request.path, input_values)
164c121
< self.href_last = turbogears.url(cherrypy.request.path,
self.input_values)
---
> self.href_last = turbogears.url(cherrypy.request.path, input_values)
173c130
< self.href_prev = turbogears.url(cherrypy.request.path,
self.input_values)
---
> self.href_prev = turbogears.url(cherrypy.request.path, input_values)
177c134
< self.href_first = turbogears.url(cherrypy.request.path,
self.input_values)
---
> self.href_first = turbogears.url(cherrypy.request.path, input_values)

-Ian