How can I write logs from PubSub to BigTable?

[Ravinder Singh Tanwer]

Hi,

We have requirement to fetch the project logs from stackdriver and write to BigTable for AI/ML processing.

While looking native solution i don't find any template from pubSub to BigTable. 

Can someone please suggest how we can do this?

Thanks

Ravi

[Pierre Oberholzer]

Hi Ravi,

You might want to have a look on my answer using Beam/Dataflow:

https://stackoverflow.com/questions/63035772/streaming-pipeline-in-dataflow-to-bigtable-python/69682742?noredirect=1#comment123383796_69682742

However I'd be interested to get an official template from Google, too.

Best regards, Pierre