Can I use Django Channels to solve this architectural prolem?

[psaod...@gmail.com]

Hi there,

I want to develop a Django site where users have the possibility to interact with systemd services in realtime and to see the log output in realtime.

I sketched up a quick draft on my idea how this could work: https://i.imgur.com/pd5uLtl.jpg

One simplified use-case would be:

  1. User visits Django website.
  2. User chooses one of many systemd services. each service would be represented by a single webpage, eg. /services/id/1, /services/id/2...
  3. User gets presented a realtime log, the source being either journald or lastlog-file on the file system.
  4. User clicks on Stop button.
  5. Service gets stopped.
  6. User receives exit codes, etc. in a readable format.
  7. User clicks on start button
  8. User receives exit codes (started successfully/failed/etc) in a readable format.
  9. User leaves Django website.

I wrote a simple Controller (MyController.py) that connects to the systemd interface of the dbus, to controllthe systemd services and listen for property changes, but also to read logfiles from the file system. I inteded MyController to run as root and sit between Django and dbus/systemd.

I would think WebSockets are the best solution to deal with the realtime log display in 3) and with the live interaction in 4) 5) 6) 7) and 8).
I recently read about Django Channels and I believe this would be the solution to Django<----(WebSocket)----->User.

But somehow Django needs to talk to my MyController.py daemon for starting and stopping, as well as getting the systemd exit codes across to the User in realtime.

I believe I cannot use xmlrpc, because that is a pure Request<->Response principle. But a live log is continuous, and if you start a service, that could take ten seconds and I really want to have quick response times. I also thought about background job management, but I prefer WebSockets over this, so I don't have to implement multiple technologies.

My question would be this: Could I somehow use Channels to connect MyController with Django and vice versa, similar to xmlrpc but so that both sides can send?

I am open for all ideas at this point, but I prefer to implement only one kind of technology if that would be possible.

Thanks in advance!

psaodwhatevermynameis.

[Andrew Godwin]

You could use channels (well, ASGI, it's lower-level interface) for the interconnect:

 - Use channels for signals from the web service to the controller daemon (maybe "service.stop" or "service.start" with the service name in the message payload). You can trigger these from simple AJAX or GET views, no need for websocket here.

 - Send the log continuously out to a Group as it's received, one group per service. Then, when someone joins and wishes to tail the log, tie in their socket to the group.

However, this does not supply log playback of lines before a client joins, so you'd have to fill in that gap. In the end, you're probably looking at some sort of datastore in the middle there to store logs and then replay them into a socket as it joins.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/4224be09-059f-42e5-8352-b3aad34bf5b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[psaod...@gmail.com]

Hi Andrew, thanks for the advice. Since I implement websockets to serve my users the live logs, I would just route the service commands (e.g. "service.start") through the websocket, too. That way I also don't have to block other things while waiting for the response whether the command was successful or not.

I read into the documentation and setup a basic web socket site that is working (woohoo). The rest is just more reading and learning and trial and error, but I am confident I get this working.

However, how would I connect my Controller to the Channel layer, setup a consumer and a producer? I got it working for Django and websockets, but from a standalone python app? hmmm..

- psaodicantremembertherest

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

[Andrew Godwin]

On Fri, Nov 18, 2016 at 6:57 AM, <psaod...@gmail.com> wrote:

Hi Andrew, thanks for the advice. Since I implement websockets to serve my users the live logs, I would just route the service commands (e.g. "service.start") through the websocket, too. That way I also don't have to block other things while waiting for the response whether the command was successful or not.

I read into the documentation and setup a basic web socket site that is working (woohoo). The rest is just more reading and learning and trial and error, but I am confident I get this working.

However, how would I connect my Controller to the Channel layer, setup a consumer and a producer? I got it working for Django and websockets, but from a standalone python app? hmmm..

You can use the raw Python ASGI layer directly from outside Django. I need to write some more docs about this use case and how to go with it (right now it's more been in talks I've given) but basically you want to:

* Import projectname.asgi:channel_layer (the same thing you pass to Daphne)

* Call the ASGI functions on that to send and receive messages. They're documented here: http://channels.readthedocs.io/en/stable/asgi.html#specification-details

The normal pattern would be to have a loop that listens on receive_many(["channel1", "channel2"]) for incoming messages and dispatch them to a handler function, and to call send(channel, message) directly from anywhere you want to send a message. Send is always non-blocking, but can raise ChannelFull - the spec linked above should cover the specifics.

Andrew

 

To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/71e13ba0-66dd-49ec-8d78-017eb6a9740f%40googlegroups.com.

[matrix1900]

Hi,

i read the article, because i have the same situation. Its not clear for me how to use the receive_many function outside Django. Its only possible to send message 

Group('sensor').send({'text': 'Hello, world!'}) from outside and receive it on the browser.

But not to receive a message.

Here my code example:

import asyncio
import traceback
import sys
 
from channels import Group
 
from sensor.asgi import channel_layer
 
 
class ChannelsConsumer(object):
    async def start(self, channel_layer, channel_names):
        if isinstance(channel_names, str):
            channel_names = [channel_names]
 
        while True:
            channel, message = channel_layer.receive_many('sensor',block=False)
#            self.stdout.write('RECV' + str(channel) +  str(message))
            Group('sensor').send({'text': 'Recieved' + str(channel) + str(message)})
            if channel:
                print('RECV', channel, message)
                self.stdout.write('RECV' + str(channel) +  str(message))
                #
                # await some_long_async_operation_here()
                #
                Group('sensor').send({'text': 'Hello, world!'})
            else:
                await asyncio.sleep(0.1)
 
           
    def run(self, channel_layer, channel_names):
        loop = asyncio.get_event_loop()
        asyncio.ensure_future(self.start(channel_layer, channel_names))
        try:
            loop.run_forever()
        except Exception as e:
            traceback.print_exc()
        finally:
            loop.close()
 
 
consumer = ChannelsConsumer()
consumer.run(channel_layer, 'sensor')