Cloud Pub/Sub queries
682 views
Skip to first unread message
Narendra Batra
May 4, 2018, 1:01:15 PM5/4/18
to Google Cloud Pub/Sub Discussions
Have some queries around Cloud Pub/Sub -
a. How do you achieve Blue/Green deployment of Pub/Sub?
b. How to achieve automated testing of complete Pub/Sub setup -
i. Automation for creation of topics and publishing a message on topic
ii. Automation of subscribers pulling a message from pub/sub topic
c. Billing - Is it possible to track usage from a billing perspective back to each topic when multiple topics are being leveraged? How is billing tracked in Pub/Sub - on Topic or on Subscription?
d. Disaster recovery for Pub/Sub
- Is this even a valid requirement? What exactly do we need to save to be able to restore on Disaster recovery?
I assume Pub/Sub is global/multi-regional with SLAs guaranteed by Google (just like any other service)
Appreciate your inputs/comments
Kir Titievsky
May 7, 2018, 9:55:39 AM5/7/18
to Google Cloud Pub/Sub Discussions
Narendra,
Thanks for the interesting questions.
Testing: You can start by using the emulator https://cloud.google.com/pubsub/docs/emulator if you need hermetic tests. Also take a look at how tests are written for the client libraries in your language (https://cloud.google.com/pubsub/docs/reference/libraries).
Billing: At the moment, GCP project is the only dimension available in billing reports for Pub/Sub. We are working to make billing labels available in the coming months.
Disaster recovery: Pub/Sub does a lot to be resilient to disasters. The SLAs apply regardless of disasters. That said, you have to start with your own disaster recovery requirements. Some industries have regulation that defines specific things DR must do. Send me a direct note if you'd like to work through your needs on this.
Kir
Product Manager
Cloud Pub/Sub
Narendra Batra
May 7, 2018, 2:31:06 PM5/7/18
to Kir Titievsky, Google Cloud Pub/Sub Discussions
Kir,
Thanks for your comments!
A few follow up questions/comments -
a. Blue/Green deployments - objective here is to NOT to publish to the same topic while we are deploying and testing Blue/green deployments (so we don't end up mixing live Production messages with Test messages). So how do we achieve it? (Note - Publishers/Subscribers being part of application can be configured to deploy to blue vs green topics)
b. Testing - will take a look at the emulator.
c. Billing - is billing recorded against Topic or Subscriptions?
d. Disaster Recovery - I guess only item to recover there is messages that have not be acknowledged. And I guess Pub/Sub does keep unacknowledged messages for 7 days (where does it keep these messages? In multi-regional GCS buckets or somewhere else?). So as long as our RTO is less than 7 days, we should be good.
Appreciate your thoughts/comments.
Thanks
Naren
--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/359ddeab-c1a7-4a8f-a668-3924e5d04050%40googlegroups.com.
Senior Director, Cloud
eTouch Systems
Kir Titievsky
May 7, 2018, 2:34:50 PM5/7/18
to nba...@etouch.net, Google Cloud Pub/Sub Discussions
Blue/green deployments: seems like your use case should be solved by simply using different projects or topics for testing vs. dev. Just name them different things. You don't have to worry about resource isolation. And individual topics and subscriptions have their own IAM permissions, so you can use those to keep things clean.
Billing: billing is reported at project+product level at the moment. You can't break it down by topic or subscription. The billing model is such that you pay for both publishing and getting the messages.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/359ddeab-c1a7-4a8f-a668-3924e5d04050%40googlegroups.com.
Kir Titievsky | Product Manager | Google Cloud Pub/Sub
Narendra Batra
May 7, 2018, 3:01:09 PM5/7/18
to Kir Titievsky, Google Cloud Pub/Sub Discussions
Thanks for the prompt response!
Any thoughts/insights into DR? I guess Topics/Subscriptions are available across regions, projects, so nothing to recover there. Only thing we need to worry about is messages that have not been acknowledged by subscribers. Where does Pub/Sub keep unacknowledged messages? And as long as we can bring up DR site within a reasonable time frame(less than a day), we should be good.
-Naren
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/359ddeab-c1a7-4a8f-a668-3924e5d04050%40googlegroups.com.
--Kir Titievsky | Product Manager | Google Cloud Pub/Sub
Kir Titievsky
May 7, 2018, 4:54:49 PM5/7/18
to Narendra Batra, Google Cloud Pub/Sub Discussions
Naren,
Correct overall. Something to keep in mind: Pub/Sub keeps any given message in a single region, although, replicated across zones. So you are just fine if any zone becomes unavailable. But in the case where an entire region might become unavailable, the messages stored there are stuck -- acknowledge or not. This is, clearly, a highly unlikely event so you have to decide what degree of DR you need.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/359ddeab-c1a7-4a8f-a668-3924e5d04050%40googlegroups.com.
--Kir Titievsky | Product Manager | Google Cloud Pub/Sub
Reply all
Reply to author
Forward
0 new messages