Unable to set up celery in supervisord

[Ankush Thakur]

​I'm trying to set up celery as a supervisord job (for my Django project) and getting an error. Most likely it's because of wrong import paths (or some other environment setting), but I have no idea what. Please help!

Here's my directory structure ('>' means down one level):

/home/ankush/jremind

>env

>jremind

>>celerybeat.pid  

>>celerybeat-schedule  

>>jremind  

>>manage.py  

>>remind  

>>requirements.txt

>logs

  

The supervisord conf file is:​

command=/home/ankush/jremind/env/bin/celery --app=remind.celery:app worker --loglevel=INFO

environment=PYTHONPATH=/home/ankush/jremind/jremind

directory=/home/ankush/jremind/jremind

stdout_logfile=/home/ankush/jremind/logs/celeryd.log

stderr_logfile=/home/ankush/jremind/logs/celeryd.log

autostart=true

autorestart=true

startsecs=10

stopwaitsecs=600

Now, when I try to start the process, it exists too quickly, and I'm left with the following in the logs:

. . .

    self._conf = force_mapping(obj)

  File "/home/ankush/jremind/env/lib/python3.4/site-packages/celery/datastructures.py", line 50, in force_mapping

    if isinstance(m, (LazyObject, LazySettings)):

  File "/home/ankush/jremind/env/lib/python3.4/site-packages/django/utils/functional.py", line 204, in inner

    self._setup()

  File "/home/ankush/jremind/env/lib/python3.4/site-packages/django/conf/__init__.py", line 43, in _setup

    self._wrapped = Settings(settings_module)

  File "/home/ankush/jremind/env/lib/python3.4/site-packages/django/conf/__init__.py", line 120, in __init__

    raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")

django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.

I'm not sure why I'm getting this error, because the secret key is most definitely there and the app runs fine. What am I doing wrong?

Best,

Ankush​

[George Silva]

If you are getting variables from the environment, supervisor that special environment directive. The variables need to specified in the supervisor conf file, such as:

command=/home/ankush/jremind/env/bin/celery --app=remind.celery:app worker --loglevel=INFO

environment=PYTHONPATH=/home/ankush/jremind/jremind,SECRET_KEY="foo",VARIABLE_A="a"

directory=/home/ankush/jremind/jremind

stdout_logfile=/home/ankush/jremind/logs/celeryd.log

stderr_logfile=/home/ankush/jremind/logs/celeryd.log

autostart=true

autorestart=true

startsecs=10

stopwaitsecs=600

Supervisor won't be able to read them otherwise. There are other alternatives, for example, if you are using a crafted command, using bash:

command=/home/foo/command.sh

environment=PYTHONPATH=/home/ankush/jremind/jremind

directory=/home/ankush/jremind/jremind

stdout_logfile=/home/ankush/jremind/logs/celeryd.log

stderr_logfile=/home/ankush/jremind/logs/celeryd.log

autostart=true

autorestart=true

startsecs=10

stopwaitsecs=600

command.sh

!#/bin/bash

export SECRET_KEY="foo"

export VARIABLE_A="a"

echo $SECRET_KEY

echo $VARIABLE_A

--

George R. C. Silva
Sigma Geosistemas LTDA

----------------------------

http://www.sigmageosistemas.com.br/

[monoBOT]

2016-07-21 19:20 GMT+01:00 George Silva <george...@gmail.com>:

If you are getting variables from the environment, supervisor that special environment directive. The variables need to specified in the supervisor conf file, such as:

​I totally recomend this tutorial about that ... its very well detailed, so you can modify to your own specifications http://michal.karzynski.pl/blog/2013/06/09/django-nginx-gunicorn-virtualenv-supervisor/

​

--

monoBOT

Visite mi sitio(Visit my site): monobotsoft.es/blog/

[Ankush Thakur]

Well, setting up the line this way gives me the following error:

Starting supervisor: Error: Format string 'PYTHONPATH=/home/ankush/jremind/jremind,JREMIND_SECRET_KEY="SDFDSF@$%#$%$#TWFDFGFG%^$%ewrw$#%TFRETERTERT$^",JREMIND_DATABASE="jremind",JREMIND_USERNAME="root",JREMIND_PASSWORD="root"' for 'environment' is badly formatted

What do you think is badly formatted here?

Best,

Ankush

[George Silva]

The secret key contains % chars. Chance them to something else or escape them.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f9639cef-0a04-4982-92b4-9c8a9c5a1158%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[George Silva]

Check the docs. There's plenty of information regarding this.

It's probably a bad formatted character, misplaced comma or whatever.

--

You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f9639cef-0a04-4982-92b4-9c8a9c5a1158%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Ankush Thakur]

Wooohooo! That did it. Many thanks! :-) :-) :-)

Umm, but, say, isn't kind of clunky, that I have to copy all the variables over to the supervisor config? Isn't there a neater way to do it?

Regards,

Ankush Thakur

--
You received this message because you are subscribed to a topic in the Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-users/0yxoRdSyctA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAGyPVTvOfBvbh_Cpz3ZgRb%3DcfQ3K8FuoS5Xi57FjTdO9aCyQ4g%40mail.gmail.com.

[Ankush Thakur]

Unfortunately the docs are anything but beginner-friendly on Celery + Supervisor: http://docs.celeryproject.org/en/latest/tutorials/daemonizing.html#supervisord If you follow the link, you land on a GitHub page with three files and no explanations. :(

Besides, if you see https://github.com/celery/celery/blob/3.1/extra/supervisord/celeryd.conf, there's no mention of environment variables.

Regards,

Ankush Thakur

--
You received this message because you are subscribed to a topic in the Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-users/0yxoRdSyctA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAGyPVTv7X_9Waim0BU7uknipwPAujp5ik7AHSCBmqksRNKaszg%40mail.gmail.com.

[George Silva]

I'm familiar with the two ways I've explained to you. I'm not sure if there are others.

Actually, it's a single place where you have to define the variables (the supervisor conf) so I don't think it's that bad.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CALX%3DrK%2BtutoBh3pDMLaH7Xm8MVDf%3DxXgArY7ESWrvniUgsL66w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Ankush Thakur]

Yes, I guess that's good enough. Thanks for helping out! :-)

Regards,

Ankush Thakur

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAGyPVTuXk-sG7Kd85t-87RbyYirza_zV-PCPBXJGr0cD8ZoE5g%40mail.gmail.com.