gevent-zeromq req/rep async?

[Wesley]

Hi all,

  I am trying to use asynced zeromq req/rep.

Now I use pyzmq and find gevent_zeromq.

Here is example from gevent site regarding make zeromq req/rep async.

Server.py:

# Note: Remember to ``pip install pyzmq gevent_zeromq``

import gevent

from gevent_zeromq import zmq

# Global Context

context = zmq.Context()

def server():

    server_socket = context.socket(zmq.REQ)

    server_socket.bind("tcp://127.0.0.1:50000")

    for request in range(1,10):

        #import time

        #time.sleep(1)

        print 'server sending No.%s' % request

        server_socket.send("Hello")

        print('Switched to Server for %s' % request)

        # Implicit context switch occurs here

        gevent.sleep(0.1)

        server_socket.recv()

publisher = gevent.spawn(server)

gevent.joinall([publisher])

client.py:

# Note: Remember to ``pip install pyzmq gevent_zeromq``

import gevent

from gevent_zeromq import zmq

# Global Context

context = zmq.Context()

def client():

    client_socket = context.socket(zmq.REP)

    client_socket.connect("tcp://127.0.0.1:50000")

    for request in range(1,10):

        client_socket.recv()

        #print('Switched to Client for %s' % request)

        # Implicit context switch occurs here

        #gevent.sleep(0.1)

        if request==5:

            #import time

            #gevent.sleep(5)

            pass

        client_socket.send("World")

client    = gevent.spawn(client)

gevent.joinall([client])

This problem is , seems it's not asynced...for server site, every send still hangs waiting recv..

Why? then, how to make send/recv not synced in pair?

Thanks in advance.

Wesley

[Elvio Rogelio Toccalino]

I'd venture that the pattern chosen is not the one you want. The REP socket (i think) will not process note than one message at a time.
If you're looking for a burst of requests, you might want to try REQ-DEALER.

[Wesley]

I have just one client and one server, client need to send some requests to server, and receive responses for analysis .

I need:

1. client recv not blocking, that is, I can keep sending even if no response received.

2. For some request, if no response received finally, maybe within give time, do some logic.

 

So, what socket mode I should choose?

Or could you please give a sample snippet?

Thanks.

在 2014年2月28日星期五UTC+8下午9时17分06秒，Elvio Rogelio Toccalino写道：

[Elvio Rogelio Toccalino]

If you want to stick to one-client-one-server, then you'll need for both client and server to be fully async. That means using ROUTER for the client and DEALER for the server.

My recommendation, however, is to try to restructure your code for simplicity. It might be better to have several synchronous clients and/or synchronous servers, and use a proxy [1] to scale the model. That would mean your clients use REQ and your servers REP, which is similar to what you have now.

1 http://api.zeromq.org/3-2:zmq-proxy

--

You received this message because you are subscribed to the Google Groups "gevent-socketio" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gevent-socket...@googlegroups.com.
To post to this group, send email to gevent-...@googlegroups.com.
Visit this group at http://groups.google.com/group/gevent-socketio.
For more options, visit https://groups.google.com/groups/opt_out.

[Wesley]

Hi,

  I am sorry but I'm new to zeromq modes.

So, 

1. you mean ROUTER for client and DEALER for the server, what't the path exactly?

ROUTER ---  DEALER   or    REQ -- ROUTER -- DEALER -- REP ?

2. REQ -- PROXY -- REP could be async? req recv not blocked?

Thanks.

Wesley

在 2014年3月3日星期一UTC+8上午2时41分23秒，Elvio Rogelio Toccalino写道：

[Elvio Rogelio Toccalino]

For the first approach (only one client and only one server) you would be using a ROUTER socket on your client, connecting to a DEALER socket which is bound on the server. The path is simply ROUTER <==> DEALER. This will ensure that the client and the server a both fully async.

The simpler case (which I recommended, and you numbered as "2") is to write a blocking client using REQ socket, and a blocking server using REP socket, and then scale the system by spawning many clients and many servers, and a single proxy (which acts as a brocker). It would look something like this (pardon my bad ascii art):

client1 - REQ == connect ==> .............................................................................. <== connect == REP - server1
client2 - REQ == connect ==> 0== bind == ROUTER - proxy - DEALER == bind ==0 <== connect == REP - server2

clientN - REQ == connect ==> .............................................................................. <== connect == REP - severM

The advantage of the second model is that you end up writing blocking / simpler client and server code, and also that the proxy object is very easy to write with zeromq (lotta examples if you search for "broker").

Hope it helps,
ELvio

[Wesley]

For the first approach, I think I misunderstand async and no-blocking...

Here is my simple code

server.py:

#import gevent

#from gevent_zeromq import zmq

import zmq

# Global Context

context = zmq.Context()

server_socket = context.socket(zmq.DEALER)

server_socket.setsockopt(zmq.IDENTITY,b'A')

server_socket.bind("tcp://127.0.0.1:50000")

def server():

    for request in range(1,10):   

        msg = server_socket.recv()

        print 'client received %s' % msg

        print 'client sending %s' % str(request)

        if request==5:

            continue

        server_socket.send(str(request))  

server()

client.py:

 import zmq

# Global Context

context = zmq.Context()

client_socket = context.socket(zmq.ROUTER)

client_socket.connect("tcp://127.0.0.1:50000")

import time

time.sleep(0.1)

def client():

    for request in range(1,10):

        print 'client sending %s' % str(request)

        client_socket.send_multipart([b'A',str(request)])

        

        ident,msg = client_socket.recv_multipart()

        print 'client received %s' % msg

        

client()

So, both block when request is 5, this is something about blocking, NOT ASYNC...

I think maybe I need to set socket NOBLOCK or just add a timeout.. 

[Elvio Rogelio Toccalino]

Using gevent guarantees that socket.recv() will not block your program, but it will still block your greenlet. I strongly suggest you re-structure your code to use a simpler, better known 0mq pattern.

If you need several requests to leave the client, and process as they come, you might want to use a thread/greenlet per request.

--

[Wesley]

Thanks.

I will test performance on my current code.

If not good, I will change to gevent, and consult with you if hit any problems.

Wesley