Suggestions on "Loading" for possible long load-times?

[Kurtis]

Hey,

I've got a part of my site that is just prototyped at the moment. It's
not particularly optimized since it's still in the works.

I don't think it's going to take a considerable amount of time to load
while we have few users. But, when we start migrating our old users to
the new system, I can imagine there will be a delay before the
calculations are finished and the data is ready to display.

Unfortunately, this is real-time calculations so it's not a simple
matter of caching it to death. I'm not ready to do any major
optimization at this point anyways since it's still a work-in-
progress.

Anyways, I want to display a "loading" page while the data is
prepared. I don't mind if it's a matter of a redirect, done using
AJAX, or just using some sort of a Javascript technique that will
display the "Loading" dialog while the rest of the data is sent to the
browser. What are some suggestions on doing this with Django? I prefer
a simple, straight-forward method that won't cause a lot of
complication with the existing code base if at all possible.

Thanks!

[Russell Keith-Magee]

There's a very common pattern for this kind of activity:

1) User requests /start page to get some data that is expensive to calculate

2) View for /start creates a Task object that represents the work to
be executed in the background.

3) An external process is running, waiting for new Task objects to be
created. Exactly how this is serviced is up to you, but the important
part is that the expensive calculation isn't performed in the view
code -- it's handled by a process external to the web server. Celery
is a common way to handle this sort of task; you can also knock
something together quickly with a script running under a cron (run
once per minute, find all new tasks, process each, then quit. Not
ideal for production, but it works as a proof of concept).

4) Once the task has been *created* (Not finished -- just created),
/start redirects to a /loading page.

5) /loading contains an ajax lookup that polls the completion status
of the task

6) The task executes and completes; on completion, the task is marked
as "complete".

7) Once the task is complete, /loading reports the completion, and
redirects to /finished to display the results of the calculation.

At every step, the web server's interaction is short lived -- create a
task; check if the task is complete; display the result of the task --
so it scales well. If you're careful about the design of your task
processor, you can make sure that this processor scales too; e.g., add
a second Celery processor for the task, and you've just doubled your
capacity.

Hope this helps!

Yours,
Russ Magee %-)

[Kurtis Mullins]

Thanks a lot Russel! It sounds like a pretty reasonable approach to me.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

[Matt Schinckel]

Russ mentioned it in passing, but I'll point it out for clarity: Celery is a great framework for handling background tasks.

[Andre Terra]

What Matt meant to say is that Celery is *the* framework for handling background tasks, especially in the Python/Django world.


Cheers,
AT

On Wed, Nov 2, 2011 at 8:55 PM, Matt Schinckel <ma...@schinckel.net> wrote:

Russ mentioned it in passing, but I'll point it out for clarity: Celery is a great framework for handling background tasks.

--

You received this message because you are subscribed to the Google Groups "Django users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/django-users/-/TwtPwcwjzvkJ.