NATS vs NSQ

[Endre Karlson]

Hi, what are the benefits of NATS vs NSQ?

[Žygimantas Stauga]

Both of them has own pros and cons and quite different feature sets. To replace NSQ I think you should look into nats-streaming, because it's more close to NSQ than just NATS.
I'm waiting for nats-streaming clustering to replace NSQ on our stack. As for now we are using NSQ for async communication and NATS for sync communication (request reply).

[Tony Pujals]

I'm curious ... can you elaborate a bit on why you want to replace NSQ with nat-streaming? Any specific deficiency with NSQ or is it just a desire to keep everything under one NATS umbrella?

[Žygimantas Stauga]

1. NSQ message ordering is odd. They don't provide any guaranties, but under high load it's more LIFO than FIFO and this is kinda unfair in our use case. 

2. NSQ may suffer from network partitions. All cluster nodes are independent if your producer can access only node A, consumer can access node B, and node A can access B - messages will queue on node A and consumer will receive nothing until network partition gone. 

3. NSQ stores messages in memory + disk (memory queue size is configurable). On node fail/restart you will lost all messages stored on memory. Small memory limit will nuke our disk. This was true few years ago, maybe this bit is improved now. But small memory limit is usually a bad thing, so you have to risk and find right balance. Not sure if NSQ handles somehow planned restarts with SIGTERM by flushing memory messages to disk. Actually I don't care anymore.

4. Strange go library solutions. They provide higher level consumer where you can write your own logic without worrying about connectivity part. But producer part you must implement connectivity logic by yourself (query nsqlookupd nodes to receive nsqd nodes, renew node list periodically, do loadbalancing, handle errors if node goes away during publish, etc). With 1.0.0 version they made unnecessary response struct change which means we need update all our 100+ services to support this change. This was so frustrating...

But NSQ has one really useful feature - pausing topics/channels. This really helps during critical situations when shit hits the fan.

[Tony Pujals]

Thank you for the analysis / insight. I was mostly concerned for my specific use case with #1, but your points are all important considerations to take into account collectively, and the one useful feature you note isn't particularly compelling for me. Thanks again!