is Cloudfoundry moving away from NATS?

[Camilo Aguilar]

It seems Cloudfoundry is moving away from NATS, I still don't understand the real reasons but I'm trying to decide wether to use NATS or regular HTTP2, using grpc, for communication between my internal services. 

Any feedback is very much appreciated.

Cheers, 

Camilo

[Derek Collison]

My understanding is that the scheduling algorithm that used to be based on NATS is being moved to Diego, based on etcd. I do not believe they are removing all of NATS, but feel free to ask Pivotal directly.

--
You received this message because you are subscribed to the Google Groups "nats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to natsio+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Larry McQueary]

Camilo, I don't understand the real reasons either, re: Cloud Foundry. 

What I do know is that "Everything Old Is New Again"... except me, of course. Apparently I am just old, but with a good memory. ;) What we see here are old ideas being reborn with new names and modern implementations. grpc is CORBA/RMI-IIOP (complete with IDL stubs!), NATS is pub/sub messaging. HTTP/2 is, well, HTTP. 

I'll make the same observations I was making 15-18 years ago, when these ideas were newer: RPC doesn't scale well and requires more complexity than advertised. HTTP point to point is too slow (even with SPDY), too tightly coupled, and doesn't scale well without a lot of additional infrastructure. Using both together inherits the sum of both problems, which seem to be antithetical to the sort of lightweight, decoupled architectures that microservices aim to facilitate.

Obviously we at Apcera believe that if you want maximum simplicity and ease-of-use, maximum decoupling (location transparency), and maximum performance at scale, then lightweight pub-sub is the answer. It's not the answer to every problem, but we believe it is more suitable and a more natural fit to microservice-based architectures that need to scale transparently.

These opinions aside, what's a more natural fit for your architecture? Will you spin up lots of services that may be located on other nodes or even other (sub)networks, but may want to cooperate around a common set of communication "channels"? Or do you have a smaller set of services that only need to communicate point-to-point to one other service/endpoint, and are naturally tightly coupled? Pub/sub is likely a better fit for the former case. RPC over a point-to-point communication channel is likely a good fit for the latter.

(Disclosure: I work for Apcera, the maintainer of NATS, and I have been evangelizing pub/sub messaging for nearly 20 years)

On Fri, Aug 28, 2015 at 3:34 PM, Camilo Aguilar <camilo....@gmail.com> wrote:

[Henrik Johansson]

It sounds like "we know http so let's do that" rather than anything else. I have the feeling that most systems today can benefit from your first case Larry with sets of transparently growing and shrinking producers and consumers. The decoupling that Nats provides is incredibly easy compared to having to coordinate by hand through etcd (or similar) or having networking balancers that proxies calls. Maybe they have other reasons but from that thread it seems weak at best.

[Tyler Treat]

We've been going through the same set of decisions trying to unify our messaging system, i.e. RPC vs. pub/sub. What we've found is that most developers understand RPC, so it's what they usually ask for. I'm personally not a big fan of RPC past a certain point for a number of reasons, but we evaluated things like gRPC, Thrift, etc. Point-to-point certainly has the least performance overhead but, as Larry points out, doesn't scale very well.

So we looked at doing all messaging, RPC and pub/sub, over a single system. RabbitMQ was too slow. Kafka didn't quite fit the problem space. NATS was the only system which was feasible due to how lightweight and low-latency it is.

We really have two messaging needs though: fast/fire-and-forget and reliable/at-least-once. The first is for RPC and ephemeral notifications, and the second is for data-synchronization and audit-able transactions. The latter would also allow offline consumers to "catch up." Both of these need to support server-to-server and client-to-server messaging. Kafka was an obvious fit for the second use case, so we built a system which exposes various transports (WebSocket, TCP, HTTP), robust routing logic, and middleware around Kafka. Besides replayability/durability, another benefit of Kafka is we can hook in new consumers, such as an auditing system. This system is more or less complete and will likely be open sourced in the near future. For the fast/fire-and-forget use case, we've been looking at NATS. We would expose it to web clients through a proxy similar to the Kafka system, which would perform authentication, rate limiting, etc.

Our plan is to then expose these two services through a single "messaging SDK" which would likely provide support for RPC and pub/sub patterns. Due to the nature of distributed systems, I like to encourage async messaging patterns, but there's usually some give and take when working with people trying to ship products.

[Larry McQueary]

Thanks Tyler, this is good validation of what we have been thinking in terms of roadmap in the core project, particularly at-least-once delivery. We are evaluating approaches to this based on other messaging systems we have collectively built as well as the slightly more unique approaches of e.g Kafka and RabbitMQ.

I'm now wondering how far off we will be from your requirements (and perhaps the OP's) once we have that.

Larry McQueary | Sr. Solutions Specialist

la...@apcera.com | 303.819.1700 | @mcqueary

[Camilo Aguilar]

Thanks guys for all the good feedback. I think somebody should gather some of the questions and answers from this mailing list, and create a FAQ in the website. Thanks again!

Cheers, 

Camilo   

You received this message because you are subscribed to a topic in the Google Groups "nats" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/natsio/uS4A8n1D_yI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to natsio+un...@googlegroups.com.

[Larry McQueary]

Camilo... It's in the works. Brian Flannery, our community manager, will make some announcements at the appropriate time, but there will be a revamped website in the near future. Ideally open sourced via GitHub but that's all under discussion. Regardless, we will have an FAQ as you have suggested.

[Brian Flannery]

Camilo,

As Larry indicated, we are in the process of updating some of our web content as we speak. So, your feedback is very timely and I really like your idea.

Also, any other ideas you have please feel free to share those. We want to make sure that we are providing information that is useful to the group, so the more you communicate with us as you have here the better it is for the community as a whole.

Keep the ideas flowing!

B

Camilo   

To unsubscribe from this group and stop receiving emails from it, send an email to natsio+unsubscribe@googlegroups.com.

[Camilo Aguilar]

I finally got a more complete answer to this from CF folks: https://cloudup.com/coqfrjEdXrx

[Derek Collison]

Its unfortunate that they never worked with me or Apcera directly on it. This is the first I have really heard of any concerns. I know they had issues with their own client and switched to ours at some point.

 Many people consider NATS the most stable system and use it happily in production under many different scenarios. We do have a commercial support option as well which could have helped here too, but they decided against that option for some reason.

[Brian Flannery]

Camilo,

Be interested to chat with you briefly (I am the Community Manager at NATS). Please feel free to drop me a quick email - br...@apcera.com

Be good to learn more about what you're trying to accomplish and also discuss some of your feedback - Skype or Google Hangout. Look fwd to syncing up!

[Paulo Pires]

gRPC and NATS are two totally different beasts. I'm using both in our solutions and both deliver awesomely :)