Channels group_send hangs in extrenal process

[Yaro Kifor]

Hello,

I am a little new to channels, but I can't seem to figure out this issue. I am trying to use group_send from an external process and I am using Redis as my channel backend. Here is what my process code looks like:

 

from channels.layers import get_channel_layer
from multiprocessing import Process
from asgiref.sync import AsyncToSync, SyncToAsync

class GenericAgent(Process):
    def __init__(self, *args, **kwargs):

        self.channel_layer = get_channel_layer()

        self.send_message = AsyncToSync(self.channel_layer.group_send)

        super().__init__(*args, **kwargs)

    def run(self):

        while True:

            print("Sending data")

            self.send_message('test', {'test': 0})

           time.sleep(1)

Anytime I run the process it will print out "Sending data" once and do nothing else. Any channel that is part of the test group will not receive anything as well. If anyone has a clue to what the issue might be please let know your help is greatly appreciated.

Here is my pip freeze:

aioredis==1.2.0

asgiref==2.3.2

async-timeout==3.0.1

attrs==18.2.0

autobahn==18.11.1

Automat==0.7.0

certifi==2018.4.16

channels==2.1.5

channels-redis==2.3.1

chardet==3.0.4

constantly==15.1.0

daphne==2.2.3

Django==2.0.6

easy-thumbnails==2.5

future==0.15.2

hiredis==0.2.0

hyperlink==18.0.0

idna==2.7

imageio==2.3.0

incremental==17.5.0

interop==0.0.0

LatLon23==1.0.7

monotonic==1.2

msgpack==0.5.6

numpy==1.14.4

Pillow==5.1.0

psycopg2-binary==2.7.5

PyHamcrest==1.9.0

pyproj==1.9.5.1

pytz==2018.4

rawpy==0.10.1

requests==2.19.1

six==1.11.0

sorl-thumbnail==12.4.1

Twisted==18.9.0

txaio==18.8.1

urllib3==1.23

zope.interface==4.6.0

Thanks in advance,

Yaro 

[Andrew Godwin]

I've never tried using the async-to-sync stuff with multiprocessing - first, I'd try removing that and see if it still works (just run from a management command or script directly).

If it still fails, I'd suggest running a redis MONITOR to see if any traffic makes it to Redis at all.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/4916f184-9db9-4d34-a9cd-3d6ff5238aff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yaro Kifor]

Huh that's weird. I've removed the async-to-sync stuff and now the script doesn't hang. I've checked redis MONITOR and I do see traffic now which nice, but for some reason my consumer isn't seeing any of it. Maybe I messed up the group_add method. Here's the redis MONITOR output.

1542149506.032331 [0 172.19.0.4:45506] "BRPOPLPUSH" "asgi:specific.eYdlWGyC!" "asgi:specific.eYdlWGyC!$inflight" "5"
1542149512.043776 [0 172.19.0.4:45506] "EVAL" "\n            local backed_up = redis.call('LRANGE', ARGV[2], 0, -1)\n            for i = #backed_up, 1, -1 do\n                redis.call('LPUSH', ARGV[1], backed_up[i])\n            end\n            redis.call('DEL', ARGV[2])\n        " "0" "asgi:specific.eYdlWGyC!" "asgi:specific.eYdlWGyC!$inflight"
1542149512.043867 [0 lua] "LRANGE" "asgi:specific.eYdlWGyC!$inflight" "0" "-1"
1542149512.043884 [0 lua] "DEL" "asgi:specific.eYdlWGyC!$inflight"
1542149512.044194 [0 172.19.0.4:45506] "BRPOPLPUSH" "asgi:specific.eYdlWGyC!" "asgi:specific.eYdlWGyC!$inflight" "5"
1542149518.056667 [0 172.19.0.4:45506] "EVAL" "\n            local backed_up = redis.call('LRANGE', ARGV[2], 0, -1)\n            for i = #backed_up, 1, -1 do\n                redis.call('LPUSH', ARGV[1], backed_up[i])\n            end\n            redis.call('DEL', ARGV[2])\n        " "0" "asgi:specific.eYdlWGyC!" "asgi:specific.eYdlWGyC!$inflight"
1542149518.056757 [0 lua] "LRANGE" "asgi:specific.eYdlWGyC!$inflight" "0" "-1"
1542149518.056773 [0 lua] "DEL" "asgi:specific.eYdlWGyC!$inflight"
1542149518.057120 [0 172.19.0.4:45506] "BRPOPLPUSH" "asgi:specific.eYdlWGyC!" "asgi:specific.eYdlWGyC!$inflight" "5"
1542149524.068806 [0 172.19.0.4:45506] "EVAL" "\n            local backed_up = redis.call('LRANGE', ARGV[2], 0, -1)\n            for i = #backed_up, 1, -1 do\n                redis.call('LPUSH', ARGV[1], backed_up[i])\n            end\n            redis.call('DEL', ARGV[2])\n        " "0" "asgi:specific.eYdlWGyC!" "asgi:specific.eYdlWGyC!$inflight"
1542149524.068919 [0 lua] "LRANGE" "asgi:specific.eYdlWGyC!$inflight" "0" "-1"
1542149524.068935 [0 lua] "DEL" "asgi:specific.eYdlWGyC!$inflight"
1542149524.069345 [0 172.19.0.4:45506] "BRPOPLPUSH" "asgi:specific.eYdlWGyC!" "asgi:specific.eYdlWGyC!$inflight" "5"
1542149530.080909 [0 172.19.0.4:45506] "EVAL" "\n            local backed_up = redis.call('LRANGE', ARGV[2], 0, -1)\n            for i = #backed_up, 1, -1 do\n                redis.call('LPUSH', ARGV[1], backed_up[i])\n            end\n            redis.call('DEL', ARGV[2])\n        " "0" "asgi:specific.eYdlWGyC!" "asgi:specific.eYdlWGyC!$inflight"
1542149530.080999 [0 lua] "LRANGE" "asgi:specific.eYdlWGyC!$inflight" "0" "-1"
1542149530.081015 [0 lua] "DEL" "asgi:specific.eYdlWGyC!$inflight"
1542149530.081372 [0 172.19.0.4:45506] "BRPOPLPUSH" "asgi:specific.eYdlWGyC!" "asgi:specific.eYdlWGyC!$inflight" "5"

I'll let you know where I get with this. Thanks for the help so far.

Yaro

You received this message because you are subscribed to a topic in the Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-users/Y2xNv_aOVqE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFwN1upK-iXuxbZNTcPu78OzPnDAfS8-E%3DfOHnPUio0u1dppRw%40mail.gmail.com.

[Yaro Kifor]

I've ensured my consumer is setup properly, added a type field to the message and added a callback method for that type. My callback method isn't getting called. I've tested my callback method from a different source and it worked there. In my tested I noticed that redis MONITOR logs this:

1542152017.290859 [0 lua] "EXPIRE" "asgi:specific.cNbsZzOu!" "60"
1542152017.290866 [0 lua] "LLEN" "asgi:specific.aqRkFBbT!"
1542152017.290874 [0 lua] "LPUSH" "asgi:specific.aqRkFBbT!" "\x83\xa4type\xa4test\xa4test\xa4test\xb0__asgi_channel__\x91\xbespecific.aqRkFBbT!FCpiHVGLaITQ

 

While from my GenericAgent process, redis MONITOR doesn't show anything like this (see my previous email). I am not sure what my process is missing. Is this a bug or am I going about this all wrong?

Yaro