django_rq or django settings issue ?

[bonet]

Hi all,

I'm facing a strange behavior when trying to create Django_rq queues configuration dynamically from an environment variable.

my env var is :

CUSTOM_QUEUES=default:q1:q2

into my settings.py I have:

from getenv import env



# Cache
CACHES_DEFAULT = "redis://127.0.0.1:6379/1"
CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": env("CACHE_URL", CACHES_DEFAULT),
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
        },
        'TIMEOUT': 3600
    },
}


RQ_QUEUES_REDIS = {'USE_REDIS_CACHE': 'default'}
RQ_QUEUES = {k:RQ_QUEUES_REDIS for k in env('CUSTOM_QUEUES', "default").split(':')}

I'm expecting the denerated RQ_QUEUES dict to be like this one which is working :

RQ_QUEUES = {
    'default': {
        'USE_REDIS_CACHE': 'default',
    },
    'q1': {
        'USE_REDIS_CACHE': 'default',
    },
    'q2': {
        'USE_REDIS_CACHE': 'default',
    },
}

even though the configuration seems to run well and I can see the queues from django_rq web page, my workers cannot connect and actually throw this error as like there's no queue key in redis

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/work/virtualenv/runaway_dev/lib/python3.6/site-packages/django/core/management/__init__.py", line 350, in execute_from_command_line
    utility.execute()
  File "/home/work/virtualenv/runaway_dev/lib/python3.6/site-packages/django/core/management/__init__.py", line 342, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/work/virtualenv/runaway_dev/lib/python3.6/site-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/work/virtualenv/runaway_dev/lib/python3.6/site-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/home/work/virtualenv/runaway_dev/lib/python3.6/site-packages/django_rq/management/commands/rqworker.py", line 79, in handle
    queues = get_queues(*args)
  File "/home/work/virtualenv/runaway_dev/lib/python3.6/site-packages/django_rq/queues.py", line 166, in get_queues
    queue_params = QUEUES[queue_names[0]]
KeyError: 'default'

By The Way: if I use the Static configuration above it just works well, so my suspect is more focused on Django_rq not creating the queues but maybe I'm also doing something unproper on the settings of Django. 

Any help would be really appreciated. thanks.

F.

[Melvyn Sopacua]

On Wednesday 21 June 2017 09:12:17 bonet wrote:

 

> I'm expecting the denerated RQ_QUEUES dict to be like this one which

> is working :

>

> RQ_QUEUES = {

> 'default': {

> 'USE_REDIS_CACHE': 'default',

> },

> 'q1': {

> 'USE_REDIS_CACHE': 'default',

> },

> 'q2': {

> 'USE_REDIS_CACHE': 'default',

> },

> }

>

So did you test that?

 

from django.conf import settings

from django.test import SimpleTestCase

 

 

class ConfigTest(SimpleTestCase):

def test_rq_dict(self):

self.assertTrue(hasattr(settings, 'RQ_QUEUES'))

expect = {

'default': {

'USE_REDIS_CACHE': 'default',

},

'q1': {

'USE_REDIS_CACHE': 'default',

},

'q2': {

'USE_REDIS_CACHE': 'default',

}

}

self.assertDictEqual(expect, settings.RQ_QUEUES)

--

Melvyn Sopacua