pagination issue with memcached - Thinking Sphinx

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion pagination issue with memcached
Received-SPF: pass (google.com: domain of jvn...@gmail.com designates 209.85.216.177 as permitted sender) client-ip=209.85.216.177;
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=hhqDOyjboctyCLyfUlilKdHRSvKVOUJDDE92L/8iKsJRgz37Dwshg1RaDMJ/pW0PGx
         CvpVublkL5Ux5NGdSrL1GwvjKdvNNmrzzeu76Vx8OSNbbr9gJN+CPMm3xKj3M0xdnwTq
         RuSMdGM4Z9JB8Ypix116VCefbZYf2WdRJpTxg=
MIME-Version: 1.0
In-Reply-To: <eefdf004-9b25-41ba-ae43-001f34ea8...@i4g2000prf.googlegroups.com>
References: <970e4f1d-e9d8-459d-8df4-20d51118a...@q40g2000prg.googlegroups.com>
	<eefdf004-9b25-41ba-ae43-001f34ea8...@i4g2000prf.googlegroups.com>
Date: Tue, 21 Sep 2010 10:06:17 +0800
Message-ID: <AANLkTinA8wkJrK00+G2bmS_dvyE=o-BRQ-1VEp9t+...@mail.gmail.com>
Subject: Re: [ts] Re: pagination issue with memcached
From: Jim Ruther Nill <jvn...@gmail.com>
To: thinking-sphinx@googlegroups.com
Content-Type: multipart/alternative; boundary=00c09f83a311750acf0490bb7975

--00c09f83a311750acf0490bb7975
Content-Type: text/plain; charset=ISO-8859-1

I think there's no way around this. memcache the object and memcache the
total count.


On Tue, Sep 21, 2010 at 8:00 AM, badnaam <asitkmis...@gmail.com> wrote:

> Update
>
> OK, the issue is related to the face that the collection needs to be
> of type Will_Paginte::Collection. TS Search by default returns the
> results of this type, that why it works when the block is executed,
> but memcached returns just an array and hence it doesn't work.
>
> I tried converting the memcached Array to a will_paginate:collection
> type like...
>
> if results.class == ThinkingSphinx::Search
>            return results
>        else
>            pag_results = WillPaginate::Collection.create(page,
> per_page) do |pager|
>                pager.replace(results)
>                unless pager.total_entries
>                    # the pager didn't manage to guess the total
> count, do it manually
>                    pager.total_entries = results.count ##this is bad
>                end
>
>            end
>            return pag_results
>        end
>
> But the problem is results does not contain the entire search count,
> only the first paginated lot and without doing the search all again, I
> can't calculate the exact count, which would defeat the purpose of
> memcaching it, since everytime there is a cache hit I would have to
> run the search again just to find the total count.
>
> Bump!
>
> On Sep 19, 1:48 pm, badnaam <asitkmis...@gmail.com> wrote:
> > I memcached (with Dalli client) some of my search results like..
> >
> > def self.city_search  city, per_page, page, order
> >         Rails.cache.fetch("#{city}_#{page}_#{order}") do
> >             Post.search(:include => [:poster, :category],:geo =>
> > location_lat_lng_radian(city),
> >               :with => {"@geodist" => 0.0..(Constants::PROXIMITY *
> > Constants::METERS_PER_MILE), :status => Post::STATUS['approved']},
> >               :latitude_attr => :lat, :longitude_attr
> > => :lng, :per_page => per_page, :page => page, :order =>
> > (ModelHelpers.determine_order_search order))
> >         end
> >     end
> >
> > ##In controller
> > @search_results  = Search.city_search params[:city], params[:per_page]
> > || Constants::LISTINGS_PER_PAGE, params[:page] || 1, params[:order] ||
> > 'distance'
> >
> > The result is presented in the view with pagination like..
> >
> > <%= page_entries_info @search_results, :entry_name => "Search Result"
> > %>
> >   <%= will_paginate @search_results, :class => 'pagination' %>
> >
> > The first hit i.e. when the block is exececuted and the return type of
> > the collection is "ThinkingSphinx::Search" this all works fine. The
> > 2nd time when there is a cache hit and the collection returned from
> > the city_search method is of type "Array" I get  the following error.
> >
> > If I convert the results of the block to an array so that the view
> > always get an Array (regardless of cached or non cached hit), the
> > first hit i.e. non-cached hit, get the same exact error.
> >
> > Will paginate's page_entires_info expect a collection..so why am I
> > seeing this? Not sure if this is a will paginate or thinkingsphinx
> > issue.
> >
> > undefined method `total_pages' for #<Array:0xa068c40>
> > /usr/local/lib/ruby/gems/1.8/gems/will_paginate-2.3.14/lib/
> > will_paginate/view_helpers.rb:171:in `page_entries_info'
> > /home/username/Apps/myapp/app/views/searches/_search_results.html.erb:
> > 7:in
> >
> `_run_erb_app47views47searches47_search_results46html46erb_locals_object_paginate_search_results'
> > /home/username/Apps/myapp/app/views/searches/index.html.erb:4:in
> > `_run_erb_app47views47searches47index46html46erb'
> > /home/username/Apps/myapp/app/controllers/searches_controller.rb:28:in
> > `index'
>
> --
> You received this message because you are subscribed to the Google Groups
> "Thinking Sphinx" group.
> To post to this group, send email to thinking-sphinx@googlegroups.com.
> To unsubscribe from this group, send email to
> thinking-sphinx+unsubscribe@googlegroups.com<thinking-sphinx%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/thinking-sphinx?hl=en.
>
>


-- 
-------------------------------------------------------------
visit my blog at http://jimlabs.heroku.com

--00c09f83a311750acf0490bb7975
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I think there&#39;s no way around this. memcache the object and memcache th=
e total count.<div><br></div><div><br><div class=3D"gmail_quote">On Tue, Se=
p 21, 2010 at 8:00 AM, badnaam <span dir=3D"ltr">&lt;<a href=3D"mailto:asit=
kmis...@gmail.com">asitkmis...@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;">Update<br>
<br>
OK, the issue is related to the face that the collection needs to be<br>
of type Will_Paginte::Collection. TS Search by default returns the<br>
results of this type, that why it works when the block is executed,<br>
but memcached returns just an array and hence it doesn&#39;t work.<br>
<br>
I tried converting the memcached Array to a will_paginate:collection<br>
type like...<br>
<br>
if results.class =3D=3D ThinkingSphinx::Search<br>
 =A0 =A0 =A0 =A0 =A0 =A0return results<br>
 =A0 =A0 =A0 =A0else<br>
 =A0 =A0 =A0 =A0 =A0 =A0pag_results =3D WillPaginate::Collection.create(pag=
e,<br>
per_page) do |pager|<br>
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pager.replace(results)<br>
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0unless pager.total_entries<br>
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0# the pager didn&#39;t manage to gu=
ess the total<br>
count, do it manually<br>
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pager.total_entries =3D results.cou=
nt ##this is bad<br>
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0end<br>
<br>
 =A0 =A0 =A0 =A0 =A0 =A0end<br>
 =A0 =A0 =A0 =A0 =A0 =A0return pag_results<br>
 =A0 =A0 =A0 =A0end<br>
<br>
But the problem is results does not contain the entire search count,<br>
only the first paginated lot and without doing the search all again, I<br>
can&#39;t calculate the exact count, which would defeat the purpose of<br>
memcaching it, since everytime there is a cache hit I would have to<br>
run the search again just to find the total count.<br>
<br>
Bump!<br>
<div><div></div><div class=3D"h5"><br>
On Sep 19, 1:48=A0pm, badnaam &lt;<a href=3D"mailto:asitkmis...@gmail.com">=
asitkmis...@gmail.com</a>&gt; wrote:<br>
&gt; I memcached (with Dalli client) some of my search results like..<br>
&gt;<br>
&gt; def self.city_search =A0city, per_page, page, order<br>
&gt; =A0 =A0 =A0 =A0 Rails.cache.fetch(&quot;#{city}_#{page}_#{order}&quot;=
) do<br>
&gt; =A0 =A0 =A0 =A0 =A0 =A0 Post.search(:include =3D&gt; [:poster, :catego=
ry],:geo =3D&gt;<br>
&gt; location_lat_lng_radian(city),<br>
&gt; =A0 =A0 =A0 =A0 =A0 =A0 =A0 :with =3D&gt; {&quot;@geodist&quot; =3D&gt=
; 0.0..(Constants::PROXIMITY *<br>
&gt; Constants::METERS_PER_MILE), :status =3D&gt; Post::STATUS[&#39;approve=
d&#39;]},<br>
&gt; =A0 =A0 =A0 =A0 =A0 =A0 =A0 :latitude_attr =3D&gt; :lat, :longitude_at=
tr<br>
&gt; =3D&gt; :lng, :per_page =3D&gt; per_page, :page =3D&gt; page, :order =
=3D&gt;<br>
&gt; (ModelHelpers.determine_order_search order))<br>
&gt; =A0 =A0 =A0 =A0 end<br>
&gt; =A0 =A0 end<br>
&gt;<br>
&gt; ##In controller<br>
&gt; @search_results =A0=3D Search.city_search params[:city], params[:per_p=
age]<br>
&gt; || Constants::LISTINGS_PER_PAGE, params[:page] || 1, params[:order] ||=
<br>
&gt; &#39;distance&#39;<br>
&gt;<br>
&gt; The result is presented in the view with pagination like..<br>
&gt;<br>
&gt; &lt;%=3D page_entries_info @search_results, :entry_name =3D&gt; &quot;=
Search Result&quot;<br>
&gt; %&gt;<br>
&gt; =A0 &lt;%=3D will_paginate @search_results, :class =3D&gt; &#39;pagina=
tion&#39; %&gt;<br>
&gt;<br>
&gt; The first hit i.e. when the block is exececuted and the return type of=
<br>
&gt; the collection is &quot;ThinkingSphinx::Search&quot; this all works fi=
ne. The<br>
&gt; 2nd time when there is a cache hit and the collection returned from<br=
>
&gt; the city_search method is of type &quot;Array&quot; I get =A0the follo=
wing error.<br>
&gt;<br>
&gt; If I convert the results of the block to an array so that the view<br>
&gt; always get an Array (regardless of cached or non cached hit), the<br>
&gt; first hit i.e. non-cached hit, get the same exact error.<br>
&gt;<br>
&gt; Will paginate&#39;s page_entires_info expect a collection..so why am I=
<br>
&gt; seeing this? Not sure if this is a will paginate or thinkingsphinx<br>
&gt; issue.<br>
&gt;<br>
&gt; undefined method `total_pages&#39; for #&lt;Array:0xa068c40&gt;<br>
&gt; /usr/local/lib/ruby/gems/1.8/gems/will_paginate-2.3.14/lib/<br>
&gt; will_paginate/view_helpers.rb:171:in `page_entries_info&#39;<br>
&gt; /home/username/Apps/myapp/app/views/searches/_search_results.html.erb:=
<br>
&gt; 7:in<br>
&gt; `_run_erb_app47views47searches47_search_results46html46erb_locals_obje=
ct_paginate_search_results&#39;<br>
&gt; /home/username/Apps/myapp/app/views/searches/index.html.erb:4:in<br>
&gt; `_run_erb_app47views47searches47index46html46erb&#39;<br>
&gt; /home/username/Apps/myapp/app/controllers/searches_controller.rb:28:in=
<br>
&gt; `index&#39;<br>
<br>
--<br>
You received this message because you are subscribed to the Google Groups &=
quot;Thinking Sphinx&quot; group.<br>
To post to this group, send email to <a href=3D"mailto:thinking-sphinx@goog=
legroups.com">thinking-sphinx@googlegroups.com</a>.<br>
To unsubscribe from this group, send email to <a href=3D"mailto:thinking-sp=
hinx%2Bunsubscribe@googlegroups.com">thinking-sphinx+unsubscribe@googlegrou=
ps.com</a>.<br>
For more options, visit this group at <a href=3D"http://groups.google.com/g=
roup/thinking-sphinx?hl=3Den" target=3D"_blank">http://groups.google.com/gr=
oup/thinking-sphinx?hl=3Den</a>.<br>
<br>
</div></div></blockquote></div><br><br clear=3D"all"><br>-- <br>-----------=
--------------------------------------------------<br>visit my blog at <a h=
ref=3D"http://jimlabs.heroku.com">http://jimlabs.heroku.com</a><br>
</div>

--00c09f83a311750acf0490bb7975--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
