Django channels with python background tasks

[Sourabh Jaiswal]

Hi,

I am writing a python based application(CLI Back End) which does telnet to some network components and gets some data. It saves the data in sqlite db.

For this application I am writing Django based frond end. Which will start the CLI app and monitor it. For communication between the CLI App and django I am not able to decide what to use. I read somewhere on net that django channels can be used in this problem.

I have no idea about what django channel is and what it does. Can some one please help me in this I need to come up with a tool for this CLI and django communication ASAP.

Thanks in advance!

Regards, Jaiswal.

[Andrew Godwin]

Hi Jaiswal,

I'm afraid that I can't give detailed help about what your best options are or walk you through how to do it - that's something you'll have to research and decide on yourself. Channels allows you to do low-latency communictation between Django back-ends and JavaScript, but anything you can do with it can also be done slower using a polling API connection.

My recommendation would be to start simple - doing it using an API that you poll every few seconds - and then once you have that figured out, look at how you could improve it using WebSockets and channels.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/2d704873-07cf-41d7-a747-0beb9ac85cfa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ryan Nowakowski]

In addition to websockets, channels can be used to run background tasks that could take a long time for example a telnet connection. You can Google Django channels background tasks.

An alternative to channels for background tasks is celery.

[Ryan Nowakowski]

Another idea: create a Django management command that does the telnet stuff and just run that periodically via Cron. You can always move to channels or celery later.

[Sourabh Jaiswal]

Hi Andrew,

Thanks a lot for your reply.

For a fast and easy approach I need the backend to be able to tell its status when asked by Django and be able to act when django sends some activity events.

The back end will be a long running program which will run in an infinite loop once started by django. After that django should have the capability to pause the program or stop the program Such events needs to be sent by django using HTML buttons and the back-end script should be able to act on those even and give ack that its done.

I am really sorry for asking very basic questions. But I started with django this month only so I dont have very good idea about it or how to achieve this.

Can you please guide me a bit. And point me a tutorial of some sort to achieve this.

From your videos about django channels what I understood so far is Django channels can be used to offload the heavy work loads from one app or one server to some small services and apps.

Thanks a lot in advance!

Regards,

Jaiswal.

[Sourabh Jaiswal]

HI Ryan,

The back-end script in this case is not only doing telnet. Its getting data from multiple network elements using telnet and multi-threading. There are multiple threads started when I start the backend script which takes monitoring data from diff-diff network elements at the same time. The communication between the threads is happening using a python Sockets. A TCP server is responsible to pass the messages between threads. So I have to have this background program run to collect data.

The control from django I want is, to be able to start multiple instance of this whole back-end package when user clicks on such cluster in django front end. Passing management IP and other relevant information from django DB. and after starting particular instance django should be able to control it.

What I found out from recent googling and stuff is. I need to write some APIs in back-end program and as Andrew suggested it needs to be written in such a way that it can be polled via django.

So where to start in this regard? can you please guide me for that.

Thanks in advance!

Regards,

Jaiswal.

[Roger Gammans]

I think there is a little bit of confusion in this thread,a s we keep talk about django and the deamon process and ignoring the frontend.

So to clarify; there we are talking about a system with at least 3 execution environments:

1 The Browser

2 Django backend processes

3 The backend daemon/cli -process.

django-channel provides a mechanism which (amongst other uses) can get near-realtime updates from a django environment to the web browser, but if you don't need this you can just store all your status data in the data base . Then when the browser fetches a views it will have the most recent data in the database.

You still however need to communicate between the backend process and the rest of the django environment, the simplest way as Ryan suggesting is to

bring this inside the django environment by implementing it as a management command, from there you have access to the python end of the channels api to send messages to the browsers, or the model objects to save to the database. Management commands can be arbitrarily complex, so that not a problem in itself.

I have a similar system in development and there we do run the backend daemon outside of the dajngo environment, but that makes database access hard as you need a separate copy of the database models, or to use raw SQL. In our system we still have a django management command though which has the responsibility of pumping/handling messages from the deamon to the intended destination.

If you want to keep your backend separate l I can see two reasonable paths:

1. Use a custom set of django views to provide a API your backend connects to over localhost/http to send the it message updates.

2. A django management command as a API interface message handler, which connects to the backend over one of many message channels out there. (ie. zmq, redis, unix-sockets, etc)

Hope this helps a little bit.

[Sourabh Jaiswal]

Hi Roger,

Thanks for your reply.

Yes you are correct. There are actually 2 modules of this project:-

1. Django front end which is totally isolated in itself if I talk about it with back-end reference. And currently it has no mechanism whatsoever to communicate with back-end.

2. python based back-end scripts. which itself is a program. It can be started with some management IPs of Network elements and that program will keep getting the data from those NE synchronously.

    This back-end can not communicate with any external program yet. And uses a global variable file for starting the threads.

Now to link these 2 modules I understand I need to to introduce APIs in module 2. and call those APIs from django.

Which is exactly what you are suggesting in your suggestion 1. Correct me if I am wrong.

Just a little bit more help. Can you please guide me to some tutorial to get this thing started.

Thanks in advance.

Regards,

Jaiswal

[Umar Kambala]

Please when I runserver I gets this error, NameError:name 'template' is not defined. My templates are in this dir mysite/personal/templates/personal. What's my problem?

--
You received this message because you are subscribed to the Google Groups "Django users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/1527411811.5443.61.camel%40gammascience.co.uk.

[Roger Gammans]

Jaiswal,

Either my solution 1 or 2 will solve your problem, or rewriting your scripts as management commands (which is probably the easiest route).

If you want to keep them separate :-

For solution 1; I would look at using the requests library (see https://pypi.org/project/requests/ ) in your backend scripts and adding django-rest-framework to your django project ( http://www.django-rest-framework.org ) to handle to API views.

For solution 2: it is more roll your own; but googling 'zmq' and 'redis' would be a start to get you thinking about how these things can hang together. This is probably the most advanced solution though.

I'm sorry this is not a lot more help, it is all up to understanding what your comfortable with *exactly* what you want to achieve from here.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/59afd3e9-3552-4572-b6a0-0292ebab3a49%40googlegroups.com.