Performance vs Clrzmq?

[Stephen Sweeney]

Hi guys,

I've been using clrzmq for a while and had implemented a straightforward PUB/SUB and DEALER/ROUTER generic messaging component that did the job adequately (no brokers/intermediaries, just P2P)

Anyway, whilst porting the code over to use netmq, I'm seeing a significant performance hit (~5x for PUB/SUB, DEALER/ROUTER is okay-ish) even after rewriting to use the NetMQScheduler rather than the prior ZeroMQ frontend/backend sockets approach.

Tried profiling and there's nothing obvious that sticks out, so before I chop and change other stuff at a macro level, I wanted to ask the question:

What is the expectation of NetMQ performance-wise compared to CLRZMQ?

(NetMq = 3.3.0.12-rc1 and CLRZMQ is latest before it fell into non-maintenance mode)

Thanks

Stephen

[Stephen Sweeney]

One of those macro changes did the trick - hard to explain without pasting a lot of code, but essentially I get better performance getting the SUB poller's ReceiveReady call to write to a BlockingCollection/ConcurrentQueue and use a custom pool of threads rather than async Tasks to expose the incoming data (or only one queue reader thread where order of messages is important.) 

Performance for me now is approximately similar - still interested if anyone has examples to the contrary? - i.e. using async tasks and NetMQScheduler for max throughput of <1KB msgs...

Thanks,

S.

[Doron Somech]

Hi Stephen,

From my testing the performance of NetMQ and ZeroMQ on windows is about the same. In some scenarios NetMQ is a little better. Also NetMQ is using IO Completion port so it should scale better in a lot of clients.

Can you upload your test scenario? maybe I can help.

Doron

--
You received this message because you are subscribed to the Google Groups "netmq-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netmq-dev+...@googlegroups.com.
To post to this group, send email to netm...@googlegroups.com.
Visit this group at http://groups.google.com/group/netmq-dev.
For more options, visit https://groups.google.com/d/optout.

[Stephen Sweeney]

Hi Doron,

Thanks for replying so quickly. The new IOCP feature was partly why I went with the 3.3.0.12-rc1 pre-release version.

My issue is effectively resolved by changing how my code interacted with NetMQ. I did a few things to fix, so when I'm done actually shipping my component, I'd like to return to the code and isolate which changes made the difference and then share a sandboxed example of that with you guys if there's any value in it. Specifically I'd like to understand why my specific implementation of the Task/NetMQScheduler didn't do so well when PUB/SUB'ing 1,000,000 very small messages within a unit test (1 PUB, 1 SUB, TCP loopback),

Thanks again,

Stephen