Canary Deployment for Pub-Sub based micro services

[Ramanjaneyulu Mukkoti]

Team,

we have a micro service and it works based on pub-sub messages. As part of Canary deployment we need to route some traffic to new instance and other traffic to old instances like 10% to new instance and 90% to old instance.

Note: We have a pull and push mechanism approach at subscriber end

Do we have option to split traffic if we have multiple subscribers for same topic?

Could you please help on approach we need to take here?

[Gurkomal (Cloud Platform Support)]

Hello,

The solution should be to use Flow Control [1] as it allows you to control the rate at which the subscriber receives messages.

[1] https://cloud.google.com/pubsub/docs/pull#config

[abou...@google.com]

There would be several ways you could split traffic to have some percentage go to a canary instance. The first two options would require changes on the publish side. First of all, you could use two different topics. The publisher would choose 10% of the traffic to go to the "canary" topic and the remaining 90% could go to the main topic. The second option is to add an attribute to 10% of the messages, e.g., the key "canary" could have a value of "true." Then, you could use two different subscriptions, one for the main tasks and one for the canary tasks and use filtering to ensure each one receives the correct messages. The last option would be to set up the main subscribers to relay 10% of messages to another canary topic. This one has the advantage of not requiring changes on the publisher side. You would have two topics. Your main publishers would publish to the topic as they always do. Your main subscriber would process 90% of the messages itself and for the other 10%, it would publish them to the canary topic so that the canary tasks could process them. 

Kamal

On Friday, October 30, 2020 at 1:58:28 PM UTC-4 ramanjaney...@e.lloydsbanking.com wrote: