Re: Incrementing through result set 20 at a time

[Peter Bengtsson]

If you keep them "inside" tornado you run two risks:

1. If you extract, say, 1000 elements straight away from the database but only show the first 100. Then, suppose your users only bother to look at the first 100. That means you have fetched the 900 extra for no good use. Sure, sooner or later your users might look at all of them but that might be a very long time to hold on to the data in some sort of global variable.

2. What if you have 2 tornado processors running. One on port :8001 and one on :8002 and you put Nginx or HAProxy in front of this. Then your users might hit the first server for the first 100 rows and the next server for the next request of the next 100.

If you have a really slow database, consider fetching 1000 and sticking them into a memcache or redis and then build your template so that it fetches from memcache instead.

On Friday, April 5, 2013 7:19:55 AM UTC-7, Quinlan Morake wrote:

What would the best way to increment through a result set, <X> at a time, be? 

I have a large result set of which I'd like to display 20 per page, and would like to put a navigation bar at the bottom, <a href links or something of the like, to go to the rest. What would the best way to go about it be? I would ideally not like to load the results from the datasource each time, but keep them in "inside" tornado.

Regards,

Quinlan

[Russ Weeks]

Hi, Quinlan,

Why not use memcached, redis or a plain old Python dict to implement a basic HTTP session.  Keep the current position of the result set in the user's HTTP session.  You say 'result set', so I'm assuming your data is coming from a database.  Use your database driver's skip and limit functionality to navigate through the result set based on the current position.

-Russ

On Fri, Apr 5, 2013 at 7:19 AM, Quinlan Morake <lca...@gmail.com> wrote:

What would the best way to increment through a result set, <X> at a time, be? 

I have a large result set of which I'd like to display 20 per page, and would like to put a navigation bar at the bottom, <a href links or something of the like, to go to the rest. What would the best way to go about it be? I would ideally not like to load the results from the datasource each time, but keep them in "inside" tornado.

Regards,

Quinlan

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[aliane abdelouahab]

in pymongo (and i guess in other dbms driver) there is something like:
db.collection.find().limit(n).skip(m)
where m initialised to zero and then becomes m = n+m

[aliane abdelouahab]

and if it reached:
db.collection.count()
then you hide the next button

[Waldecir Santos]

peter is this what are you looking ? 

motor-iterating-over-results-the-grand-conclusion

Grato,

Waldecir

[A. Jesse Jiryu Davis]

If you search on Google for "foo" you hit a page with a URL like this:

https://www.google.com/search?q=foo

The "next" link on that page is coded to bring you here:

https://www.google.com/search?q=foo&start=10

If you go straight to that URL you'll see that it says "page 2" at the top. This lets Google's servers be stateless; all the information they need is in the URL. So, ignore that link to my blog about Motor; just do:

@tornado.web.asynchronous

@gen.engine
def get(self):

    page_number = self.get_argument('page_number', 0)

    collection = self.db.collection  # get a db reference somehow

    results = yield motor.Op(collection.find({ ... query ... }
        ).skip(page_number * 10).limit(10).to_list, length=10)

    self.render('template.html', results=results, page_number=page_number)

In your template, the "next" button points to "/search?page_number={{ page_number + 1 }}", the "previous" button to page_number - 1 if page_number > 0.

[Quinlan Morake]

Yes thanks for replies everyone. Decided to use memcache, am familiar with it and have used it before, just didn't think about it at the time. Sorry for my late reply, I was pulled off this project to another, and have now returned to it. 

Quinlan