How to specifu network and subnetwork in GCP dataflow flex-template run command

[Geeta K]

Hi All,

I need to specify network and subnetwork in the below GCP command. But it gives error when I try to give it here. I dont want to use teh default network and subnet and hence need to explicitly mention.

It would be really helpful if I can get some help here.

I'm able to specify network and subnetwork when i try this through GCP console.

gcloud dataflow flex-template run jobname  --template-file-gcs-location \
  --parameters inputFile=\
  --parameters bigtableInstanceId=\
  --parameters bigtableId=\
  --parameters bigQueryDataset=\
  --parameters bigQueryTable=\
  --parameters outputTopic=\
  --parameters  dataflowBucket=\

  --region \

The above command works.

But as soon as I add network and subnetwork like below it gives error.

gcloud dataflow flex-template run jobname  --template-file-gcs-location \
  --parameters inputFile=\
  --parameters bigtableInstanceId=\
  --parameters bigtableId=\
  --parameters bigQueryDataset=\
  --parameters bigQueryTable=\
  --parameters outputTopic=\
  --parameters  dataflowBucket=\

  --region \

 --network  \

 --subnetwork 

[Manpreet Sidhu (Google Cloud Support)]

Can you please give us more details with respect to the error message that you are receiving when you run the gcloud command? This will allow us to better understand what is happening that causes this issue.

[Geeta K]

Hi Manpreet,

Thanks for your response!

When I add

 --network  \

 --subnetwork 

I get error - "Unrecognized argument - network".

I understand that network and subnetwork parameters are not part of this command and hence this error. But what is the other way to specify network and subnetwork?

Thanks

Geetha

[alfio]

Hello,

As specified in this documentation [1]. all pipeline should be passed though the parameter flag. --region does not require the paramater flag as referenced in the documentation aswell. Here is a list of all parameters[2] available to use, --network and --subnetwork are included. Please let us know if you have any further questions.

[1] https://cloud.google.com/sdk/gcloud/reference/dataflow/flex-template/run

[2]https://cloud.google.com/dataflow/docs/guides/specifying-exec-params#setting-other-cloud-dataflow-pipeline-options