Celery Worker trying to connect to rabbitmq instead of redis?

2,285 views
Skip to first unread message

Artūras Šlajus

unread,
Aug 29, 2015, 7:43:27 AM8/29/15
to sentry
Hey there.

I'm getting:

[2015-08-29 11:41:52,881: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Trying again in 32.00 seconds...

in my celery worker log though I have "BROKER_URL = 'redis://localhost:6379/0'" in my sentry.conf.py.

Any ideas why is it trying to connect to rabbitmq?

My sentry.conf.py:

from sentry.conf.server import *
import os.path
CONF_ROOT = os.path.dirname(__file__)
DATABASES = {
    'default': {
        # You can swap out the engine for MySQL easily by changing this value
        # to ``django.db.backends.mysql`` or to PostgreSQL with
        # ``sentry.db.postgres``
        # If you change this, you'll also need to install the appropriate python
        # package: psycopg2 (Postgres) or mysql-python
        'ENGINE': 'sentry.db.postgres',
        'NAME': 'sentry',
        'USER': 'tlp',
        'PASSWORD': 'redacted',
        'HOST': '',
        'PORT': '',
    }
}
SENTRY_USE_BIG_INTS = True
SENTRY_ADMIN_EMAIL = 'art...@tinylabproductions.com'
SENTRY_SINGLE_ORGANIZATION = True
SENTRY_REDIS_OPTIONS = {
    'hosts': {
        0: {
            'host': '127.0.0.1',
            'port': 6379,
        }
    }
}
SENTRY_CACHE = 'sentry.cache.redis.RedisCache'
CELERY_ALWAYS_EAGER = False
BROKER_URL = 'redis://localhost:6379/0'
SENTRY_RATELIMITER = 'sentry.ratelimits.redis.RedisRateLimiter'
SENTRY_BUFFER = 'sentry.buffer.redis.RedisBuffer'
SENTRY_QUOTAS = 'sentry.quotas.redis.RedisQuota'
SENTRY_TSDB = 'sentry.tsdb.redis.RedisTSDB'
SENTRY_FILESTORE = 'django.core.files.storage.FileSystemStorage'
SENTRY_FILESTORE_OPTIONS = {
    'location': '/tmp/sentry-files',
}
SENTRY_URL_PREFIX = 'redacted'  # No trailing slash!
SENTRY_WEB_HOST = '127.0.0.1'
SENTRY_WEB_PORT = 8080
SENTRY_WEB_OPTIONS = {
    'daemon': True,
    'pidfile': '/home/tlp/run/sentry/main.pid',
    'accesslog': '/home/tlp/run/sentry/main.access.log',
    'errorlog': '/home/tlp/run/sentry/main.error.log',
    # 'workers': 3,  # the number of gunicorn workers
    # 'secure_scheme_headers': {'X-FORWARDED-PROTO': 'https'},
}
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = ''
EMAIL_HOST_USER = ''
EMAIL_PORT = 25
EMAIL_USE_TLS = False
SERVER_EMAIL = 'err...@tinylabproductions.com'
MAILGUN_API_KEY = ''
SECRET_KEY = 'redacted'

Worker is being run with:

/home/tlp/sentry/bin/sentry --config=/home/tlp/sentry/sentry.conf.py celery worker --pidfile=/home/tlp/run/sentry/celery-worker.pid --detach --logfile=/home/tlp/run/sentry/celery-worker.log

And beat is being run with:

 /home/tlp/sentry/bin/sentry --config=/home/tlp/sentry/sentry.conf.py celery beat --pidfile=/home/tlp/run/sentry/celery-beat.pid --detach --logfile=/home/tlp/run/sentry/celery-beat.log

David Cramer

unread,
Aug 29, 2015, 7:27:08 PM8/29/15
to gets...@googlegroups.com, Artūras Šlajus
You should confirm its actually using the config file you’re expecting (via SENTRY_CONF=…)
--
You received this message because you are subscribed to the Google Groups "sentry" group.
To unsubscribe from this group and stop receiving emails from it, send an email to getsentry+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Artūras Šlajus

unread,
Aug 30, 2015, 1:24:02 AM8/30/15
to gets...@googlegroups.com
I've been launching it with monit:

   check process sentry-main
     with pidfile /home/tlp/run/sentry/main.pid
     start program = "/home/tlp/sentry/bin/sentry --config=/home/tlp/sentry/sentry.conf.py start"
       as uid tlp and gid tlp
       with timeout 10 seconds
     stop program = "/bin/bash -c 'kill `cat /home/tlp/run/sentry/main.pid`'"
       as uid tlp and gid tlp
  
   check process sentry-celery-worker
     with pidfile /home/tlp/run/sentry/celery-worker.pid
     start program = "/home/tlp/sentry/bin/sentry --config=/home/tlp/sentry/sentry.conf.py celery worker --pidfile=/home/tlp/run/sentry/celery-worker.pid --detach --logfile=/home/tlp/run/sentry/celery-worker.log"
       as uid tlp and gid tlp
       with timeout 10 seconds
     stop program = "/bin/bash -c 'kill `cat /home/tlp/run/sentry/celery-worker.pid`'"
       as uid tlp and gid tlp
  
   check process sentry-celery-beat
     with pidfile /home/tlp/run/sentry/celery-beat.pid
     start program = "/home/tlp/sentry/bin/sentry --config=/home/tlp/sentry/sentry.conf.py celery beat --pidfile=/home/tlp/run/sentry/celery-beat.pid --detach --logfile=/home/tlp/run/sentry/celery-beat.log"
       as uid tlp and gid tlp
       with timeout 10 seconds
     stop program = "/bin/bash -c 'kill `cat /home/tlp/run/sentry/celery-beat.pid`'"
       as uid tlp and gid tlp

The way I see it I specify the config file via --config option. Isn't that enough?

David Cramer

unread,
Aug 30, 2015, 1:30:43 AM8/30/15
to gets...@googlegroups.com, Artūras Šlajus, gets...@googlegroups.com
Nope that works fine. All I can really say is that its loading whatever config you tell it and you have that set to RabbitMQ.

I know this because we default the BROKER_URL to something that is neither Rabbit or Redis, though we generate the config with a default of redis. Look for duplicate values, ensure the config is correct, try running it in the foreground to confirm, etc.

Artūras Šlajus

unread,
Aug 30, 2015, 2:11:27 AM8/30/15
to David Cramer, gets...@googlegroups.com
This is weird.

If I run the worker without the --detach option, everything works fine and the "missing workers" message is gone from the web ui. If I run it with --detach - things never work. Any ideas?
Reply all
Reply to author
Forward
0 new messages