pagination with 500000 items

[Luma]

hi

I've got a pagination with about 500000 items, and I don't know if it
will be slow. I need only the first 1000, so what I'm trying to do is
@item_pages, @items = paginate(:products, :order => "title", :per_page
=> 15, :limit => 1000)

But paginate seems to have no limit option. Will the pagination be
fast without this option?

Luma

[Adam Gardiner]

Luma wrote:
> But paginate seems to have no limit option. Will the pagination be
> fast without this option?

Pagination works by running two queries, one that returns a count of the
total number of results, and one that returns the requested page of data
only. This second query returning the page of data already uses LIMIT
and OFFSET in the SQL query, so there is no need for an additional limit
option.

Cheers,

Adam
--
Posted via http://www.ruby-forum.com/.

[Lon]

In reality no user of your application will page through 500,000 items.

If you only need the first 1,000 items, why not limit the results to 1,000
and paginate those in viewable chunks?

You can use a combination of tagging and searching to help users get the
"right" 1,000 records to look through.

[Roderick van Domburg]

Luma wrote:
> But paginate seems to have no limit option. Will the pagination be
> fast without this option?

The built-in paginator is a known performance and scalability bottleneck
and is likely to be removed from the Rails core in upcoming releases.
Luckily, there's a couple of very good plugins out there that've tackled
the issue: paginating_find [1] and will_paginate [2].

[1] http://cardboardrocket.com/pages/paginating_find
[2] http://errtheblog.com/post/929

--
Roderick van Domburg
http://www.nedforce.nl

[toby privett]

> I've got a pagination with about 500000 items, and I don't know if it
> will be slow. I need only the first 1000, so what I'm trying to do is
> @item_pages, @items = paginate(:products, :order => "title", :per_page
> => 15, :limit => 1000)

The built in rails pagination will explode with that many items. I
strongly recomment find_by_sql with a :limit parameter.

I recently used some the info here:
http://www.igvita.com/blog/2006/09/22/eager-find-by-sql-pagination-in-rails/
along with the paginating_find plugin from here:
http://cardboardrocket.com/pages/paginating_find

Workign well with 100s of 1000s or records

[Pratik]

Just use will_paginate and not look back.

--
Cheers!
- Pratik
http://m.onkey.org

[Jet]

I second that..
The Paginator in rails has an issue with getting all the records for
you..
We use paginating_find plugin here...works good so far.

cheers...

On Jul 29, 8:55 pm, Roderick van Domburg <rails-mailing-l...@andreas-

[Rick Olson]

> The built-in paginator is a known performance and scalability bottleneck
> and is likely to be removed from the Rails core in upcoming releases.
> Luckily, there's a couple of very good plugins out there that've tackled
> the issue: paginating_find [1] and will_paginate [2].

It's gone in edge rails, though there's a plugin for backwards
compatibility purposes.

--
Rick Olson
http://lighthouseapp.com
http://weblog.techno-weenie.net
http://mephistoblog.com

[Camo]

Watch this http://railscasts.com/episodes/51

Will explains things for you

[Luma]

thanks for that many answers. I'll try the plugins.

[tonypm]

Hi,

thanks all for the pagination tips, and thanks too for pointing out
the RailsCasts. For some reason I had not picked up on these yet,
but have just been and had a look and they are really cool.

Tonypm