Workload identity with Kubernetes cronjobs to synch Mongo to Bigquery

19 views
Skip to first unread message

Bruce Mcpherson

unread,
Feb 14, 2024, 6:06:48 AMFeb 14
to Google Apps Script Community
This post is somewhat off-topic but many of you play around with some of these things (especially identity and bigquery) in conjunction with Apps script - so here's the article in case it's of some interest to any of you.https://ramblings.mcpher.com/gcp/workload-identity-bigquery-mongo/

Kubernetes workload identity looks pretty scary when you read about it in the docs, but it really is a better (and simpler) way to give specific permissions to Kubernetes workloads than less secure methods such as using service account keys. I had a specific use case in mind – getting a set of collections from mongodb to bigquery on a regular schedule – and since I’m running Kube in that project anyway, it seemed a reasonable solution to use a Kube cronjob.

Maybe you’re not using kubernetes at all but just want to transfer data from mongo to bigquery – I’ll show you how to run those parts of the article locally too.

Even if that doesn’t match your exact end to end use case, there shoud be something here for anyone who wants to work with any of the topics mentioned in the (long) journey in this article covers.

Here’s a summary of the main topics:

Reply all
Reply to author
Forward
0 new messages