How to have a view getting data from a long running time controller ?

[António Ramos]

Hello i have a controller that scans a lot of databases searching for some data.

The view in the end gets that data a makes a dashboard.

If the controller takes 10 minutes , the view is blank for 10 minutes and that is not a good thing for the user.

How can i have the controller to send the data to the view and the view to display data as it is being calculated from the controller?

Is it possible in a simple manner?

Regards

António

[黄祥]

just an idea, perhaps you can use loading animation (javascript, etc) in the view during waiting the long awaited data so that the user dont get bored, after the data is calculated, you can show the data in the view (replace the loading animation).

best regards,

stifan

[António Ramos]

but the idea is to get some of the data as it is being calculated in the controller.

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Anthony]

On Tuesday, February 13, 2018 at 4:43:11 PM UTC-5, Ramos wrote:

but the idea is to get some of the data as it is being calculated in the controller.

One option would be to use the web2py scheduler to run the calculations as a background task. As results are generated, they could temporarily be stored in a special database table. From the browser, you can then make periodic Ajax requests (perhaps every few seconds) to check for new results in the special table.

As an alternative to the Ajax polling, you could use server push (e.g., Websockets or Server Sent Events) to push results to the browser. If you want to take that approach, consider external pubsub servers such as Nchan or Centrifugo, or realtime services such as Realtime.co, Ably, Pusher, Pubnub, or Firebase.

Anthony

[Dave S]

On Tuesday, February 13, 2018 at 1:43:11 PM UTC-8, Ramos wrote:

but the idea is to get some of the data as it is being calculated in the controller.

For that I would use a stream.  There are a couple of examples that have popped up here on occasion, although I can't at the moment link to them.  Which stream module you use may depend on whether your results are text/html, or binary data (like a video).  I think I have an example at home for that I used for returning a jpg, but my glasses aren't good enough to see it from here.

/dps
 

2018-02-13 21:38 GMT+00:00 黄祥 <steve.van...@gmail.com>:

just an idea, perhaps you can use loading animation (javascript, etc) in the view during waiting the long awaited data so that the user dont get bored, after the data is calculated, you can show the data in the view (replace the loading animation).

best regards,

stifan

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

[Val K]

Just a very simple example using https://github.com/axios/axios

def long_load():

    ret = DIV()

    ret.append(SCRIPT(_src=URL('static/js','axios.min.js')))

    ret.append(SCRIPT(

    """

        function DownLoadTick(e){
            document.getElementById("result").innerHTML = e.target.response;             

        };

              function long_load(){

            axios({

              url: 'streamer',

              timeout: 1000000,

              onDownloadProgress: DownLoadTick,

              withCredentials: true

            });

        };    

    """    

    ))

    ret.append(DIV(_id='result'))

    ret.append(BUTTON('click me', _onclick = 'long_load()'))

    return dict(ret=ret)

def streamer():

    i=0

    while True:

        # do something for a long time    

        time.sleep(1)

        yield DIV('part %s' % i).xml()

        i+=1

        if i>5: 

            yield DIV('Done!').xml()

            break

[António Ramos]

Will try it...

I knew i would need a Yield !!

Many thanks for your time

António

[Anthony]

Note, that will consume one of your web server processes/threads for the entire duration of the processing -- might be better with an async server like gevent.

Anthony

[Val K]

Yes, I played with Gunicorn+gevent workers - just "time.sleep(0)" in the slow controller releases worker to service other requests.

[António Ramos]

I tried your simple example but when i click on the button i get "Server Error" above the button

then in admin i see this

any help

Regards

--

Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.

[Val K]

As I see it's PY3, I tested on PY2. As I know in PY3 all strings are unicode, so, It is necessary to dig in this direction

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

[António Ramos]

i think i misplaced axios.min.js

second try and got this error

and after 3 seconds i got this 

Regards

António

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.

[Val K]

try  encode yield:

yield DIV(...).xml().encode('utf-8')

[António Ramos]

Stil some error about no attribute encode

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.

[Val K]

I don't have PY3, so can't test. If it's bytes, then it 's already encoded and all should work OK. Try ...xml().decode().encode('utf-8')  or try to yield something simple  - just 'hello'

[António Ramos]

Thank you.

It works ...

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.

[Val K]

keep in mind, that production servers have buffering. For example, if you are using  nginx you should turn off 'proxy_buffering' for the 'streamer' location.