Web2py celery plugin

[Stefan van den Eertwegh]

Hi all,

Does anyone know when the web2py-celery plugin comes out?

In the google code GIT it says last change sinds 2011.

I need background processes running which are started by web2py.

I tried redis (rq) with web2py but it gives me errors when passing database connection through the scheduled process.

Many thanks,

Stefan

[Niphlod]

you won't ever get to start workers from web2py, either with the embedded scheduler, with rq or with celery.........that's pretty much a requirement (being completely separated)

PS: celery integration was done on a so old release of celery that it won't ever be production ready

PS2: you don't need any web2py plugin to push task to celery.

[Stefan van den Eertwegh]

Hi Niphlod,

Thanks for you answer.

Why i am asking this is because i want to generate around 150 pdf's by weasyprint and per each pdf i want to queue a mail with it as an attachment.

Could this be done by the build in web2py Scheduler?

PS: i tried the scheduler, but it tries to use the generic.html view but has not enough settings provided in the layout.html to use it, it fails.

is is not so that it executes the function from the model with database privileges and not uses a view? Or is it also using the default generic view?

Thanks for responding!

Best regards,

Stefan

Op vrijdag 2 oktober 2015 11:21:07 UTC+2 schreef Niphlod:

[Niphlod]

I'd say you're loosing yourself in a glass of water. In order to schedule something, that something must work. If your "something" doesn't, it really doesn't matter what tool are you issuing commands to.

[Luis Valladares]

But how can you exec a web2py task (who may uses web2py internal components like Request, response, db, etc..) with a external celery app? i mean, if i wish to execute a task, for example: To insert a new register in a table of my web2py database, and in enqueue this task to a remote (or local) celery worker to do it, how i can get this worker to know about all the "environment" of web2py? the only idea i have its creating a worker that is aware of the web2py environment, but i dont know how to do this.

There is anyway to use celery queue to execute tasks that requires the web2py environment?

[Mark Graves]

Luis,

I've never personally used celery to queue tasks to execute in a web2py environment.

I found myself in a similar situation recently and just used a cron task running a python script in the web2py environment from the command line.

Is there a particular reason this workflow would not work for you?

-Mark

[Luis Valladares]

Hello mark! thanks for you answer.

Your approach sounds really nice, but exactly how you run a cron task inside web2py environment? you start the web2py client with the console option and there execute the task? may you give me an example of this?

[Mark Graves]

As documented in the web2py book,

The "web2py.py" script can take many command-line arguments specifying the maximum number of threads, enabling of SSL, etc. For a complete list type:

>>> python web2py.py -h

For example, a shell script called async_events.sh can be called at reboot as follows:

#! /bin/bash
cd /home/www-data/web2py
python web2py.py -S YOUR_APPLICATION_NAME -M -R applications/YOUR_APPLICATION_NAME/private/async_events.py

The contents of async_events.py are a simple python script run as if it were a request from web2py

For example:

import time 

 

def create_pdfs(db):
    while True:
        ALL YOUR PDF PROCESSING LOGIC HERE

            time.sleep(60) 

 

 

create_pdfs(db)

The above is an example script where your create_pdfs function handles all the processing logic.

The time.sleep(60) means run every minute.

You would run this task once @reboot so python would handle the interval between tasks.

 

From there, you can use all the internal web2py API including mail.send() etc

-Mark

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/YHsRsh0uQGo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luis Valladares]

Thanks Mark! I think this will work, i will try it on the job and come here if i have any troubles.