How to send response from Kafka event queue to Kong

[dtra...@gmail.com]

Hi,

we are pretty new to Kong so bear with me. For the sake of simplicity we have 2 microservices, A and B that are connected to Kong. Service A has a REST API that allows clients to interact with, and Service B has it's own REST API too. There is a third case however where a client will make a REST request to Service A which will perform some computation and then Service A will publish an event to a Kafka queue that will get picked up by Service B. Once Service B performs it's computation it will publish a new event to the Kafka queue. How can now the Kafka queue respond back to the client via Kong? How can Kong pick up that event and respond from the originating endpoint to the client?

We are just really unclear how this entire process will work so any help would be much appreciated. 

Thanks

D 

[Benjamin Goering]

I'm pretty sure Kong itself can't handle this use case of orchestrating and waiting for several different microservices to produce a result and then 'tell Kong' to respond to the user.

However this is exactly what another project, Pushpin, does, and you might be interested in taking a look at that. Pushpin and Kong can be layered and used together. There is also a hosted version of Pushpin if you want to try it out quickly.

[Cooper Marcus]

Another architectural option to consider for your case is:

1. Client makes REST request to Service A

2. Service A performs computation, then publishes resulting event to Kafka queue

3. Service B consumes event from Kafka queue

4. Service B performs computation, then publishes resulting event to Kafka queue

5. Service A consumes event from Kafka queue

6. Service A responds back to client that made request in step #1

In this scenario, Kong would sit in between the client and Service A. Kong would not need to perform any business logic, aggregation, or consumption from Kafka.

You mentioned that Service B also has a RESTful API - in the example I give, this API isn't utilized, but assuming you have clients accessing Service B's API, you'd probably want to proxy those requests through Kong too. 

Cheers, Cooper

--
You received this message because you are subscribed to the Google Groups "Kong" group.
To unsubscribe from this group and stop receiving emails from it, send an email to konglayer+unsubscribe@googlegroups.com.
To post to this group, send email to kong...@googlegroups.com.
Visit this group at https://groups.google.com/group/konglayer.
To view this discussion on the web visit https://groups.google.com/d/msgid/konglayer/8d5349c0-2bc1-4775-8403-79e2ec3fd994%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[dtra...@gmail.com]

I think this is the option we started considering while waiting for a response in here. We have already set up Kafka for other purposes so it's only a matter of creating the new events.

The endpoints are essentially becoming smart following this approach which personally I prefer compared to the orchestrator one. 

Thanks

On Friday, 1 September 2017 22:06:32 UTC+1, Cooper Marcus wrote:

Another architectural option to consider for your case is:

1. Client makes REST request to Service A

2. Service A performs computation, then publishes resulting event to Kafka queue

3. Service B consumes event from Kafka queue

4. Service B performs computation, then publishes resulting event to Kafka queue

5. Service A consumes event from Kafka queue

6. Service A responds back to client that made request in step #1

In this scenario, Kong would sit in between the client and Service A. Kong would not need to perform any business logic, aggregation, or consumption from Kafka.

You mentioned that Service B also has a RESTful API - in the example I give, this API isn't utilized, but assuming you have clients accessing Service B's API, you'd probably want to proxy those requests through Kong too. 

Cheers, Cooper

On Fri, Sep 1, 2017 at 1:55 PM, Benjamin Goering <bengo...@gmail.com> wrote:

I'm pretty sure Kong itself can't handle this use case of orchestrating and waiting for several different microservices to produce a result and then 'tell Kong' to respond to the user.

However this is exactly what another project, Pushpin, does, and you might be interested in taking a look at that. Pushpin and Kong can be layered and used together. There is also a hosted version of Pushpin if you want to try it out quickly.

On Thursday, August 31, 2017 at 4:05:15 PM UTC+2, dtra...@gmail.com wrote:

Hi,

we are pretty new to Kong so bear with me. For the sake of simplicity we have 2 microservices, A and B that are connected to Kong. Service A has a REST API that allows clients to interact with, and Service B has it's own REST API too. There is a third case however where a client will make a REST request to Service A which will perform some computation and then Service A will publish an event to a Kafka queue that will get picked up by Service B. Once Service B performs it's computation it will publish a new event to the Kafka queue. How can now the Kafka queue respond back to the client via Kong? How can Kong pick up that event and respond from the originating endpoint to the client?

We are just really unclear how this entire process will work so any help would be much appreciated. 

Thanks

D 

--
You received this message because you are subscribed to the Google Groups "Kong" group.

To unsubscribe from this group and stop receiving emails from it, send an email to konglayer+...@googlegroups.com.

To post to this group, send email to kong...@googlegroups.com.
Visit this group at https://groups.google.com/group/konglayer.