Sending intermediate content

[jose]

Is it possible to send intermediate content to a browser with pylons?
the use-case is if I have a long process I would like to send some
content to the browser, run my long process and then send the rest of
the content in real-time. Currently an action waits for the must
return the Response, meaning that all the content is sent all at once
rather then incremental.
Thanks for any and all help

Jose

[Ben Bangert]

You'll want to return a generator as your response like so:

def action(self):
req = request.current_obj()
def long_func():
env = req.environ
yield "some content"
# do something that takes awhile
yield "more content"
resp = Response()
resp.content = long_func()
return resp

Note that you need to bring special vars like request, c, g, and
session into scope for the generator closure so that they'll be
available when the generator is used for the response.

Hopefully that makes sense?

HTH,
Ben

[Ben Bangert]

Right now in the controller, it is called with the full WSGI interface.
The WSGIController then calls your action, and returns the output in a
Response object. To do intermediate content, you need to put the
content returning stuff into an iterator. This way it will start
sending as it has the information.

Here's how you can send an iterator in an action that's sending data as
it can:

def action(self):
def long_func():
yield "some data right away"
# do something that take a long time
yield "some more data"
# etc.

resp = Response()
resp.content = long_func()
return resp

Note that if you need access to the special vars request, session, c,
or g in your generator function, you should bring the actual object
into scope for the function. This is because the iterator could likely
be executing outside the Pylons environment during which the special
var proxies will not proxy. So:

def action(self):
session = session.current_obj()

req = request.current_obj()
def long_func():
env = req.environ

# etc....

The closure over the scope with the actual object will ensure its
presence when the generator yields its response.

Hopefully this all makes sense?

HTH,
Ben

[Ben Bangert]

On Aug 16, 2006, at 10:18 AM, Ben Bangert wrote:

and no... I have no idea how I ended up posting two copies of this
response.

- Ben

[jose]

Thanks Ben, yes this does make sense, I'll give it a try later and post
back if I have any issues
Jose

[jose]

I just gave this a try and it didn't give me the result I was
expecting. The code I tried is:

def index(self):

req = request.current_obj()
def long_func():
env = req.environ

yield 'I am starting now'
#simulate a very long process
for i in range(10000):
print i
yield 'I should be finishing now'
res = Response()
res.content = long_func()
return res

I know that this really does not do anything, I am just trying to get
the use-case to work. What I expected to see in the browser was first
the "I am starting now" statement, followed by by the print stuff in
the cmd window followed by the last statement being sent to the
browser. Instead it looks like either its all being sent at once, or
the browser is waiting for the page to finish before anything gets
rendered. Do you know of any way to test to see which is it?
Jose

[Ian Bicking]

jose wrote:
> I just gave this a try and it didn't give me the result I was
> expecting. The code I tried is:
>
> def index(self):
> req = request.current_obj()
> def long_func():
> env = req.environ
> yield 'I am starting now'
> #simulate a very long process
> for i in range(10000):
> print i
> yield 'I should be finishing now'
> res = Response()
> res.content = long_func()
> return res
>
> I know that this really does not do anything, I am just trying to get
> the use-case to work. What I expected to see in the browser was first
> the "I am starting now" statement, followed by by the print stuff in
> the cmd window followed by the last statement being sent to the
> browser. Instead it looks like either its all being sent at once, or
> the browser is waiting for the page to finish before anything gets
> rendered. Do you know of any way to test to see which is it?

If you really want to send out content over time, you need to use the
writer that start_response returns; the iterator allows for incremental
production of a response, but it's up to the server exactly when/how to
send it out and it might be buffered.

I don't know if Pylons normally gives access to the writer
start_response returns?

--
Ian Bicking | ia...@colorstudy.com | http://blog.ianbicking.org

[jose]

Thanks Ian, I'll wait for Ben's response
Jose

[benchline]

I just tried this code as well and it gave me the same result. The
page did not show any of the text until the whole request was finished.
I am using the paster serve server.

Could it be that I have a version that is before Ben removed all the
explicit list() calls?

If so, what is the easy_install command to run to upgrade to the right
version?

Thanks,

Paul

jose wrote:
> I just gave this a try and it didn't give me the result I was
> expecting. The code I tried is:
>
> def index(self):
> req = request.current_obj()
> def long_func():
> env = req.environ
> yield 'I am starting now'
> #simulate a very long process
> for i in range(10000):
> print i
> yield 'I should be finishing now'
> res = Response()
> res.content = long_func()
> return res

> > > return the Response, meaning that all the content is sent all at once

[askel]

Ben Bangert wrote:
> def action(self):
> req = request.current_obj()
> def long_func():
> env = req.environ
> yield "some content"
> # do something that takes awhile
> yield "more content"
> resp = Response()
> resp.content = long_func()
> return resp

I know that Response is actually paste.wsgiwrappers.WSGIResponse, but
it'd be great if it'd accept an iterator as content parameter so last
three lines could be replaced by

return Response(long_func())

[Ben Bangert]

On Aug 17, 2006, at 9:03 AM, benchline wrote:

> I just tried this code as well and it gave me the same result. The
> page did not show any of the text until the whole request was finished.
> I am using the paster serve server.
>
> Could it be that I have a version that is before Ben removed all the
> explicit list() calls?
>
> If so, what is the easy_install command to run to upgrade to the right
> version?

Ah, yes. There was the update, I believe that made it into 0.9.1
though. I should mention that due to how debug mode catches the info
should an exception be caught, you can only send a stream when debug
mode is off.

Try uncommenting the
set debug = false

At the bottom of the development.ini file, and let me know if the
iterator works.

HTH,
Ben

[benchline]

That did it. After uncommenting that line and restarting the paster
server the following function (slightly modified from the previous
example to send all items to the browser) worked as expected sending
each yield statement output to the browser immediately.

def test(self):
import time

req = request.current_obj()
def long_func():
env = req.environ
yield 'I am starting now'

for i in range(100):
yield "<br>%s" % i
time.sleep(.1)

yield 'I should be finishing now'
res = Response()
res.content = long_func()
return res

Thanks Ben.

Paul