Channels - Groups and discovery

[Josh Smeaton]

I've finally had the chance to use channels for a project (hack day multiplayer game - hope to release and blog about it some time soon), and I wanted to document some of the rough edges I hit and ask some questions about them. 

Specifically though, I find the mapping of payload.type to a method on the consumer confusing and somewhat brittle.

The design we went with was to have a PlayerConsumer(AsyncWebsocketConsumer) and a GameConsumer(SyncConsumer) running as a worker. The GameConsumer starts the engine in a thread, and lets the engine fetch the channel layer. The player and engine then communicate like so:

# PlayerConsumer: receive game state updates

await self.channel_layer.group_add(

self.group_name,

self.channel_name

)

# PlayerConsumer: publish player joining event to game

await self.channel_layer.send(

'game_engine',

{

'type': 'player.new',

'player': self.user.email,

'channel': self.channel_name,

}

)

# GameConsumer: publish state update

async_to_sync(self.channel_layer.group_send)(

self.group_name,

{

'type': 'game_update',

'state': state_json,

}

)

This works, provided PlayerConsumer has a method:

async def game_update(self, event):

And the GameConsumer has a method:

def player_new(self, event):

But if these two consumers are in completely different code bases/packages, there is no real way to know what the interface is between these consumers. Worse, it's extremely easy for a bad actor to crash a listening consumer. Either of the following events will crash the consumer receiving the message:

await self.channel_layer.send(

'game_engine',

{

'type': 'i_am_a_bad_consumer'

}

)

async_to_sync(self.channel_layer.group_send)(

self.group_name,

{

'type': 'gme_update', # typo

'msg': 'hi',

}

)

I'm not so concerned about arbitrary asgi applications gatecrashing my app as I ultimately have control over what is going to run. But each consumer participating in a group must support the superset of all message types that may be sent to that group if it wants to avoid crashing. And it has to support the superset of message types without being able to discover what they might be. Oh, and they're strings that **might** match a method I have if periods are converted to underscores.

This is all very good for adhoc eventing, but I think at some point you're going to want to publish an explicit Interface. A GroupInterface might define a collection of message types that are valid for members participating in that group, and then a consumer could subscribe to **that** rather than just a name. The interface members might be optional or required, I haven't thought that far ahead, but at least each member would be known. Attempting to publish an unknown message to a group would crash the **sender** rather than the **receiver**.

Has anything like this come up before?

[Andrew Godwin]

Nobody else has suggested this particular approach yet, and while it would definitely make writing applications much more reliable, there's no particularly easy way to distribute an interface that I can think of (even the ASGI specification ends up being enforced on the receiver side, though ultimately it's the conformance test suites that I rely on for safety).

Without a simple solution to that I think it's more the sort of thing I would like to make sure people can hook into themselves to provide validation rather than trying to ship something in the core release.

There's also the idea of adding an easy way to ignore messages you're not interested in if you're using groups as a firehose rather than targeted broadcast - that does mean silent failures, though, which is my least-favourite kind. At least with the current design pattern something will complain if it's not working right (even though that thing is on the receiving side not the sending side).

Andrew

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/939460e7-45da-4dd0-afb7-f386274faea1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Josh Smeaton]

I see - some kind of python type that can be shared amongst consumers might make sense at the application level, but it doesn't at the protocol level. Specific patterns may emerge as application level helpers like Consumer subclasses that mixin specific Groups, and ASGI won't need to have any knowledge of these patterns.

> There's also the idea of adding an easy way to ignore messages you're not interested in if you're using groups as a firehose rather than targeted broadcast

Something like this would probably solve the biggest issue. At an application level I'd imagine it to be something along the lines of try_get_handler(message_type, consumer, ignore_missing=consumer.ignore_missing), but no idea if that's something that could be permitted by the spec/protocol. I'll familiarise myself with the spec and the implementation so I can understand what might and might not be possible at each layer before proposing any specific ideas.

There's some fantastic stuff in channels though, and was fairly easy to hook things together after reading the docs. I'm looking forward to using it more. 

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

[Andrew Godwin]

The code that tries to find a handler is just the dispatch method, so it's actually quite easy to override even at a user level rather than in the core itself: https://github.com/django/channels/blob/master/channels/consumer.py#L61

I would like to see a stronger solution for interfaces and type assertions, but I also think this is something that could be done externally quite easily (in fact we have a library we use at work for this very thing for our redis-based message bus: https://github.com/eventbrite/conformity). The channel layers being just a dumb dictionary transport is quite a nice facade in terms of keeping them low maintenance, and also in letting people solve dependency/versioning/schema upgrade issues in the way that fits them best.

Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b24e2a22-16fd-4507-95e0-80fccadd5fb5%40googlegroups.com.

[Josh Smeaton]

Interesting - I hadn't realised the dispatch method was directly on the consumer. And I think I agree with you that these things can be handled in user code or 3rd party library code quite easily. I guess as channels is used more and more in production applications we'll begin to see a number of helper libraries or higher level abstractions come along to make it easier for users to safely implement their real time apps on top.

Cheers