Best practices for calling an external service in a Django app

[Thomas Orozco]

Hi,

Here's my situation: one of my views needs to call an external service (through a socket ; actually the said service is on the same server).

I don't even need to know about the service's response as I'll inquire about it asynchronously in an effort to avoid delay on the user's end.

What are the best practices here ?
Should I open a socket connection each time I want to access the service? Should I keep one open all the time in a separate thread?

Thanks in advance,

Thomas

[Marcin Tustin]

I use a celery task to make this sort of job asynchronous. The task makes a new connection to the "remote" (same server) service each time. This proves to be sufficiently fast.

If you are making a lot of requests, the overhead of making new connections may prove to be significant. I wouldn't anticipate this unless you are making a new request more frequently than perhaps once a second (and even then, I would wait and see what performance was like).

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

--
Marcin Tustin
Tel: 07773 787 105

[Thomas Orozco]

Thanks for the information!

Using celery was my first idea too, but here it is not practical as I'll be transferring sensitive information through the socket and I'd rather avoid having hanging around in the Celery queue.

So I guess opening the socket from inside the view code and then closing it would be OK?

[Marcin Tustin]

The thing is, you will then be doing it synchronously, because python's global interpreter lock means that threads are never really parallel.

To avoid having the sensitive data in your celery queues, have your task retrieve the sensitive data from the database (if that's where its coming from). If it's coming from the frontend, then you are going to have to store it somewhere if you want to do this asynchronously. I doubt that poses such a great additional security risk, as long as it all remains on the same machine, especially if you use a volatile celery backend (e.g. redis with no persistence enabled).

If you are doing it synchronously, then you may want to consider if the client can pass you the data via an ajax call from a web worker, which will improve the user perception of performance.