deploy apache beam pipeline in GCP

25 views

Skip to first unread message

Thiru Senthilkumar

unread,

Apr 11, 2021, 9:59:20 PM4/11/21

to Google Cloud Pub/Sub Discussions

Dear Experts,

I have developed an apache beam pipeline using python SDK. which actually reads topics from pubsub and generates json files and stores them into a bucket.

Now, the pipeline works well from my local. But, I want to deploy the code in GCP and create DataFlow to stream the pipeline.

I am unable to find any materials on how to deploy .

Please help

Thanks

Thiru

Kir Titievsky

unread,

Apr 12, 2021, 12:41:37 PM4/12/21

to Thiru Senthilkumar, Google Cloud Pub/Sub Discussions

Thiru,

The quickstart instructions for deploying a pipeline on dataflow are here: https://cloud.google.com/dataflow/docs/quickstarts/quickstart-python#run-wordcount-on-the-dataflow-service

The very long version is here: https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline

Let me know where you get stuck when you follow these.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/46c01aa4-ff80-4805-940b-336c3928bf23n%40googlegroups.com.