CDAP support for ZDT deployments of apps

[Thribhuvan Krishnamurthy]

Hello CDAP Users,

I have a microservices app developed and deployed on the CDAP. Attached screenshot provides overview of various programs and datasets being in use to address use cases.

1. RequestAPI - a CDAP service that accepts incoming requests and pushes those requests to custom queue implemented using IndexedObjectStore named as "requests"
2. QueueManager - inherited from AbstractDataset, this provides interfaces to push, pull and remove requests.
3. QueueProcessor - a CDAP worker that pulls requests from the custom queue, does processing and stores the results in "results" dataset which is of type KeyValueTable.
4. HAManager - a CDAP worker that monitors for queued requests to make sure all submitted are processed to completion. More details on this below.

To support HA and resiliency, 

• we shall run minimum of 2 instances (we might scale-up more instances on the need basis) of both RequestAPI and QueueProcessor. 
• run HAManager on one instance (instance 0).

Now, the most important part comes. We are also looking at ZDT for this app to meet the defined SLAs. To support, I think we have only option in place wherein we would run two different versions of app simultaneously to ensure there is no downtime. Here are our thoughts.

1. On app version v2
   
   • start RequestAPI immediately after the deployment
   • start QueueProcessor only after it has been shutdown on v1
   • start HAManager only after it has been shutdown on v1
2. On app version v1
   • shutdown RequestAPI only after it is running on v2
   • allow QueueProcessor to gracefully shutdown (expected to take 30s to a minute)
   • shutdown HAManager immediately

Notes:

1. Due to limitations, the consumers don't use the version in the path to call the endpoints on RequestAPI.
2. QueueProcessor can be down from few seconds to a minute.
   
3. HAManager runs on one instance and hence it can be taken down for few seconds (<10s).

Based on the thoughts above, we have few questions unanswered from CDAP perspective and hence help needed.

1. Is it possible to run multiple versions of the same app on CDAP?
2. If yes, then is it possible to route the requests only to newer version of app by cutting down the traffic on the older version? 
3. If not, are there any other means to achieve this?

Regards

Thribhuvan

[Thribhuvan Krishnamurthy]

Hi,

In addition to the above, can someone please let me know how the upgrade works in detail as specified in the link https://cdap.atlassian.net/wiki/spaces/DOCS/pages/477560983/Lifecycle+Microservices#Upgrade-an-Application?

1. Does it shutdown the previous version before starting the newer version?
2. Or is it a rolling upgrade?

What we are looking at is exactly the support for rolling upgrade. That way, we don't have to manage running the RequestAPI on two different app versions on our own.

Regards

Thribhuvan