Pyramid Integration with Celery

[John Anderson]

To make integration with celery a little bit easier I created the package https://github.com/sontek/pyramid_celery

It creates the command 'pceleryd' for running the celery daemon and will load your pyramid environment for your tasks so that they can access the database/pyramid registry.

Its still fairly minimal and I'll keep growing it but I included a demo app so you guys can see how to use it.

[Robert Forkel]

Nice! Just the last bit I needed to push myself to actually using celery!
Right now (with no release on pypi yet) the instructions for running
the demo don't work without tweaking, but it was easy enough to get
around this.
Thanx and regards
robert

> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To post to this group, send email to pylons-...@googlegroups.com.
> To unsubscribe from this group, send email to
> pylons-discus...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/pylons-discuss?hl=en.

[John Anderson]

On 01/11/2012 01:12 AM, Robert Forkel wrote:
> Nice! Just the last bit I needed to push myself to actually using celery!
> Right now (with no release on pypi yet) the instructions for running
> the demo don't work without tweaking, but it was easy enough to get
> around this.
> Thanx and regards
> robert

What was missing and I'll add the instructions? I'll make a release on
pypi in a week or two, just wanted to get some people on the mailing
list using it out of git for a little bit before I made it "official".

[John Anderson]

Sorry, pyramid_celery uses the standard .ini configuration, so if you wanted to setup celery to use a redis back, just put the following in your dev.ini:

CELERY_RESULT_BACKEND = redis

CELERY_REDIS_HOST = localhost

CELERY_REDIS_PORT = 6379

CELERY_REDIS_DB = 0

BROKER_BACKEND = redis

BROKER_TRANSPORT = redis

BROKER_URL = redis://localhost:6379/0

On Mon, Jan 16, 2012 at 4:29 AM, Laurent DAVERIO <lau...@daverio.net> wrote:

Hello list,

Le 11/01/12 08:35, John Anderson a écrit :

> What was missing and I'll add the instructions?   I'll make a release on
> pypi in a week or two, just wanted to get some people on the mailing
> list using it out of git for a little bit before I made it "official".

I too am a first-time user of pyramid_celery, celery and rabbitmq.


I managed to run RabbitMQ and connect it to Celery, creating a
"celeryconfig.py" file in the process :

> (pyramid)[daverio@myhost cmr2]$ celeryd --loglevel=INFO
> [2012-01-16 11:26:14,081: WARNING/MainProcess]
>
>  -------------- cel...@myhost.fr v2.4.6
> ---- **** -----
> --- * ***  * -- [Configuration]
> -- * - **** ---   . broker:      amqp://cmr@localhost:5672//
> - ** ----------   . loader:      celery.loaders.default.Loader
> - ** ----------   . logfile:     [stderr]@INFO
> - ** ----------   . concurrency: 4
> - ** ----------   . events:      OFF
> - *** --- * ---   . beat:        OFF
> -- ******* ----
> --- ***** ----- [Queues]
>  --------------   . celery:      exchange:celery (direct) binding:celery
>
>
> [Tasks]
>   . cmr2.tasks.add


However, it appears that "pceleryd" is not using it, as queue parameters
seem to revert to their default values:

> (pyramid)[daverio@myhost cmr2]$ pceleryd development.ini
> [2012-01-16 11:27:19,218: WARNING/MainProcess]
>
>  -------------- cel...@myhost.fr v2.4.6
> ---- **** -----
> --- * ***  * -- [Configuration]
> -- * - **** ---   . broker:      amqp://guest@localhost:5672//
> - ** ----------   . loader:      pyramid_celery.PyramidLoader
> - ** ----------   . logfile:     [stderr]@WARNING
> - ** ----------   . concurrency: 4
> - ** ----------   . events:      OFF
> - *** --- * ---   . beat:        OFF
> -- ******* ----
> --- ***** ----- [Queues]
>  --------------   . celery:      exchange:celery (direct) binding:celery
>
>
> [2012-01-16 11:27:19,248: WARNING/MainProcess] cel...@myhost.fr has started.
> [2012-01-16 11:27:22,263: ERROR/MainProcess] Consumer: Connection Error: Socket closed. Trying again in 2 seconds...


So, maybe you could add a line or two about how to configure Celery with
pyramid_celery.

Thanks for all,

Laurent.

[Carlos Daniel Ruvalcaba Valenzuela]

Nice, this should really help ease development with celery and
pyramid, a pypi release should speedup testing also making the license
terms clearer would be good also :)

Overall nice work, I will test it soon on a small app I have on
production to see how it handles.

Regards,
Carlos Daniel Ruvalcaba