In which I ask your help designing Motor's cursor API - Tornado Web Server

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion In which I ask your help designing Motor's cursor API
Received-SPF: neutral (google.com: 209.85.214.176 is neither permitted nor denied by best guess record for domain of je...@emptysquare.net) client-ip=209.85.214.176;
MIME-Version: 1.0
In-Reply-To: <CANmUDYtLXjj3QeaO-_tJb9VmVE7YN=C5wH9UCkZ_sbQEcR9...@mail.gmail.com>
References: <d6c72fce-21f7-491f-958d-e48fa8293e85@googlegroups.com>
 <CAFkYKJ6M4-GXOexyZx9NQvt6TOBXK0t0yYmD8=QgcHGrHjJ...@mail.gmail.com>
 <CANmUDYshkoeS5N225C3DgfMwM0uSXcdkMVo7zm4Y_iwPJ7x...@mail.gmail.com>
 <CAFRUCtZqfvbgra3vti=RXdTpr5=Frad3taJvc-QagN_yS4g...@mail.gmail.com>
 <CANmUDYvVBZd=tFMO1+hj8hAQy_S8-bYCd4SxhrWrh3DVktw...@mail.gmail.com>
 <CAFRUCta1geKdbxVw7k5c6j1UiMmzD1vz-pYaoJ1NnMZQEWX...@mail.gmail.com>
 <CAFRUCtY8RVsaMOZe5Vv4fBjsAov-3fEWV7VkEc41XoJ_pLr...@mail.gmail.com>
 <CAFkYKJ74yOGEjpWy3j20uV9ieMT+7Lzb+8_jjoEykGcu6AP...@mail.gmail.com>
 <CAFRUCtYAcVxuxgzYbSG39p4p1Dj8B6zQSEZx6C0Y2ksHvVV...@mail.gmail.com> <CANmUDYtLXjj3QeaO-_tJb9VmVE7YN=C5wH9UCkZ_sbQEcR9...@mail.gmail.com>
From: "A. Jesse Jiryu Davis" <je...@emptysquare.net>
Date: Wed, 14 Nov 2012 06:30:27 -0600
Message-ID: <CAFRUCtbwFd-mMu3DoEL1JLHc_dCvShDmE2CDLpWWC6_q=X3...@mail.gmail.com>
Subject: Re: [tornado] In which I ask your help designing Motor's cursor API
To: python-tornado@googlegroups.com
Content-Type: multipart/alternative; boundary=bcaec54d426444ae1804ce73b433

--bcaec54d426444ae1804ce73b433
Content-Type: text/plain; charset=ISO-8859-1

Yup, makes sense - I think it'll be as simple as:

for document in (yield motor.Op(cursor.to_list, 10)):


On Wed, Nov 14, 2012 at 1:39 AM, Shane Spencer <sh...@bogomip.com> wrote:

> I'm thinkin....
>
> On Tue, Nov 13, 2012 at 8:25 PM, A. Jesse Jiryu Davis
> <je...@emptysquare.net> wrote:
> > Thanks Ben. Yeah, I was worried about the extra yield-overhead, then
> > realized I could get rid of it.
> >
> > Shane, even without a code example to illustrate your idea, I'm coming
> > around to the opinion that to_list should take a "length" argument that
> > defaults to None (get all documents), I'll add it here soon:
> >
> >
> http://emptysquare.net/motor/pymongo/api/motor/motor_cursor.html#motor.MotorCursor.to_list
> >
>
> >>> @gen.engine
> ... def do_find():
> ...     cursor = db.test_collection.find({'i': {'$lt': 5}}) #returns 45
> items
> ...     while cursor.alive:
> ...         # continue from current cursor position
> ...         for documents in (yield motor.Op(cursor.to_list,
> {'length': 10})): #called 5 times
> ...             # pre stuff
> ...             # do something sync or async with the 10 documents ...
> ...             # post stuff
>
> I like the idea of quickly getting at a small set of the whole cursor
> like this.  It allows me to say.. run AsyncHTTPClient in pseudo
> parallel using information from each document and wait for them all to
> return.. or use the small set of documents in an 'in' based query to
> help aggregate data.
>

--bcaec54d426444ae1804ce73b433
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Yup, makes sense - I think it&#39;ll be as simple as:<br><br>for document i=
n (yield motor.Op(cursor.to_list, 10)):<br><div class=3D"gmail_extra"><br><=
br><div class=3D"gmail_quote">On Wed, Nov 14, 2012 at 1:39 AM, Shane Spence=
r <span dir=3D"ltr">&lt;<a href=3D"mailto:sh...@bogomip.com" target=3D"_bla=
nk">sh...@bogomip.com</a>&gt;</span> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">I&#39;m thinkin....<br>
<div class=3D"im"><br>
On Tue, Nov 13, 2012 at 8:25 PM, A. Jesse Jiryu Davis<br>
&lt;<a href=3D"mailto:je...@emptysquare.net">je...@emptysquare.net</a>&gt; =
wrote:<br>
&gt; Thanks Ben. Yeah, I was worried about the extra yield-overhead, then<b=
r>
&gt; realized I could get rid of it.<br>
&gt;<br>
&gt; Shane, even without a code example to illustrate your idea, I&#39;m co=
ming<br>
&gt; around to the opinion that to_list should take a &quot;length&quot; ar=
gument that<br>
&gt; defaults to None (get all documents), I&#39;ll add it here soon:<br>
&gt;<br>
&gt; <a href=3D"http://emptysquare.net/motor/pymongo/api/motor/motor_cursor=
.html#motor.MotorCursor.to_list" target=3D"_blank">http://emptysquare.net/m=
otor/pymongo/api/motor/motor_cursor.html#motor.MotorCursor.to_list</a><br>


&gt;<br>
<br>
</div>&gt;&gt;&gt; @gen.engine<br>
... def do_find():<br>
... =A0 =A0 cursor =3D db.test_collection.find({&#39;i&#39;: {&#39;$lt&#39;=
: 5}}) #returns 45 items<br>
... =A0 =A0 while cursor.alive:<br>
... =A0 =A0 =A0 =A0 # continue from current cursor position<br>
... =A0 =A0 =A0 =A0 for documents in (yield motor.Op(cursor.to_list,<br>
{&#39;length&#39;: 10})): #called 5 times<br>
... =A0 =A0 =A0 =A0 =A0 =A0 # pre stuff<br>
... =A0 =A0 =A0 =A0 =A0 =A0 # do something sync or async with the 10 docume=
nts ...<br>
... =A0 =A0 =A0 =A0 =A0 =A0 # post stuff<br>
<br>
I like the idea of quickly getting at a small set of the whole cursor<br>
like this. =A0It allows me to say.. run AsyncHTTPClient in pseudo<br>
parallel using information from each document and wait for them all to<br>
return.. or use the small set of documents in an &#39;in&#39; based query t=
o<br>
help aggregate data.<br>
</blockquote></div><br></div>

--bcaec54d426444ae1804ce73b433--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
