How to transfer objects between Node.js and Django?

[Cassio Melo]

I'm beginning with websockets and I'm confused with the number
libraries and configuration options. I just want to setup a project
where a node.js server calls a method in python/django and when the
last has finished, it transfers the result back to the node.js
server.

I'm not sure which is the best way of doing that (supposing the method
to be executed in django takes some time to finish). I see two
possibilities:

- node.js send async http requests to django server
- node.js communicates with django using AMQP

Can you give your view about it?

[Micheil Smith]

Hi Cassio,

I would say the best way to do this would be to use either AMPQ or 0MQ. If
using 0MQ, you'd be using a ZMQ_DEALER and ZMQ_ROUTER setup. This
would mean that you can pass many messages along and receive many
messages in a stream.

However, the downside of this, is that you'll need each "request" to have
an ID associated with it, such that you know who to send it to, much like
many RPC protocols. You could possibly even use Thrift, but I don't know
too much about that, so can't speak much on that.

Async HTTP requests would likely be excess on data, but would mean that
you wouldn't need to create your own "http server" interface.

– Micheil

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

[Bika]

I've read about a similar problem here [1], they basically write from node to Redis, and use Redis' pub/sub to broadcast the changes to django.

[1]: http://lincolnloop.com/blog/2012/apr/23/ginger-tech-stack/

[Micheil Smith]

I wouldn't be using redis for this, definitely would use either zmq, rabbitmq,
or thrift. Purely because it's kinda hard to scale redis' pub/sub stuff, and
can be a SPOF.

– Micheil

[Cassio Melo]

Thanks Micheil, Bika,

I succeed in making a simple string transfer using AMQP:


test.py
-------
import pika

connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()

channel.queue_declare(queue='task_queue', durable=True)

print ' [*] Waiting for messages. To exit press CTRL+C'

def callback(ch, method, props, body):
print " [x] Received %r" % (body,)
response = body + " MODIFIED"
#response = get_a_concept()
print " [x] Done"
ch.basic_publish(exchange='',
routing_key=props.reply_to,
properties=pika.BasicProperties(correlation_id =
\

props.correlation_id),
body=str(response))
ch.basic_ack(delivery_tag = method.delivery_tag)

channel.basic_qos(prefetch_count=1)
channel.basic_consume(callback,
queue='task_queue')

channel.start_consuming()


------
app.js
------

var connection = amqp.createConnection({ host: 'localhost' });

connection.addListener('ready', function() {


var exchange = connection.exchange('', {
'type' : 'direct',
durable : false
}, function() {

var queue = connection.queue('incoming', {
durable : false,
exclusive : true }, function() {
queue.subscribe(function(msg) {
console.log("received message: ");
console.log(msg.data.toString());
});

});

exchange.publish('task_queue', "it works!", {
'replyTo' : 'incoming'
});
});

});

---

Still, I'm not sure if this is the best implementation, I'm not even
using queue.bind() method here. The problem arises when I try to pass
a complex object (json or even a simple array). Changing this line

body= (["a","b","c"])#str(response)) causes the following error:

Traceback (most recent call last):
File "test.py", line 56, in <module>
channel.start_consuming()
File "/Library/Python/2.7/site-packages/pika/adapters/
blocking_connection.py", line 293, in start_consuming
(...)
File "/Library/Python/2.7/site-packages/pika/simplebuffer.py", line
62, in write
self.buf.write(data)
TypeError: must be string or read-only character buffer, not list


But that I will check in some python list. If someone has any comment
don't hesitate!

Cheers,
Cassio

[Cassio Melo]

Oh that was easy to fix, just convert it to json:

in test.py :

body= json.dumps(your_object)#str(response))

However, I don't know if this is the best implementation. Suppose the
node.js application makes requests to different methods in python (in
fact the user selects operations in the UI). What would be a nice, (if
possible, elegant) implementation for that using AMQP?

I see some alternatives, but I'm not sure if they are good for this:

pass the method name inside the body of the request

having a channel.basic_consume(callback_method_1, queue='queue_1')
for each method in python.

having an "exchange" for each method

What are your thoughts?