[python] Using the cloud-sql-proxy in google dataflow

[Johan Brodin]

Hi,

I am having some issues connecting to Cloud SQL when I am running a Google Dataflow job based on apache beam with the python SDK. 

There is no clear way of starting the cloud-sql-proxy within the Google Dataflow environment and I would love to have something like the socket factory for java to make it work (https://cloud.google.com/sql/docs/mysql/connect-external-app#java).

My current solution is to connect via IP together with username/password, which is far from ideal!

Anyone have similar problems? Anyone solved them? :) 

[David Newgas]

One other option would be to run the Cloud SQL proxy on a separate GCE instance on the same project in the same zone as the Dataflow jobs. Leave the firewall blocking 3306 access from outside your project. This will then allow your dataflow jobs to connect to that instance, and for it to connect to Cloud SQL without any externally open services. This will add some latency and be an extra single point of failure.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/5e49dfbf-f030-4d82-af09-55bc55ae00e9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.