Django, App Engine Flexible and Cloud SQL: OperationalError 2002, Can't connect to local MySQL server through socket

[Tas Sóti]

I am struggling to get my Django app connected to the Cloud SQL database when deployed to the App Engine Flexible environment. I am using a Flexible App Engine instance in europe-west1. Within the same project I have a MySQL Second Generation instance. I can connect to the database from my local computer and everything works fine. However, once I deploy my Django app, it cannot connect to the database. I get the following error:

OperationalError at /api/v01/cron-jobs/process-data-objects/

(2002, "Can't connect to local MySQL server through socket '/cloudsql/cm-platform-01:europe-west1:cm-db-develop' (2)")

Request Method:	GET
Request URL:	http://cm-platform-01.appspot.com/api/v01/cron-jobs/process-data-objects/
Django Version:	1.10.4
Exception Type:	OperationalError
Exception Value:	(2002, "Can't connect to local MySQL server through socket '/cloudsql/cm-platform-01:europe-west1:cm-db-develop' (2)")
Exception Location:	/env/lib/python3.5/site-packages/MySQLdb/connections.py in __init__, line 204
Python Executable:	/env/bin/python3.5
Python Version:	3.5.2
Python Path:	['/home/vmagent/app',  '/env/bin',  '/env/lib/python35.zip',  '/env/lib/python3.5',  '/env/lib/python3.5/plat-linux',  '/env/lib/python3.5/lib-dynload',  '/opt/python3.5/lib/python3.5',  '/opt/python3.5/lib/python3.5/plat-linux',  '/env/lib/python3.5/site-packages']
Server time:	Wed, 12 Apr 2017 12:27:56 +0000

I have been spending my last two days with this, but I could not find a solution. Here is my environment:

Google Cloud SDK version: [150.0.0]

requirements.txt

Django==1.10.4
Pillow==3.4.2
mysqlclient==1.3.10
django-cors-headers==2.0.2
djangorestframework==3.5.4
djangorestframework-jwt==1.9.0
django-crispy-forms==1.6.1
django-filter==1.0.1
gunicorn==19.6
google-api-python-client==1.5.5
google-cloud-storage==0.22.0
python-magic==0.4.12

Django database settings:

'default': {
    'ENGINE': 'django.db.backends.mysql',
    'HOST': '/cloudsql/cm-platform-01:europe-west1:cm-db-develop',
    'NAME': 'cm_develop_01',
    'USER': 'cm-django-app',
    'PASSWORD': '[the-password]'
}

app.yaml

# [START runtime]
runtime: custom
env: flex
entrypoint: gunicorn -b :$PORT common.wsgi

beta_settings:
  cloud_sql_instances: cm-platform-01:europe-west1:cm-db-develop

runtime_config:
  python_version: 3
# [END runtime]

Dockerfile

FROM gcr.io/google_appengine/python
LABEL python_version=python3.5
RUN virtualenv --no-download /env -p python3.5

RUN apt-get update && apt-get install -y libmagic-dev

# Set virtualenv environment variables. This is equivalent to running
# source /env/bin/activate
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH
ADD requirements.txt /app/
RUN pip install -r requirements.txt
ADD . /app/
CMD exec gunicorn -b :$PORT common.wsgi

Apps are authorized (by default):

Any help would be appreciated.

Best,

 Tas

[George (Cloud Platform Support)]

Hello Tas,

To allow your app to connect to your Cloud SQL instance when the app is deployed, add the user, password, database, and instance connection name variables from Cloud SQL to the related environment variables in the app.yaml file:

env_variables:

   # Replace user, password, database, and instance connection name with the values obtained

   # when configuring your Cloud SQL instance.

   SQLALCHEMY_DATABASE_URI: >-

     mysql+pymysql://USER:PASSWORD@/DATABASE?unix_socket=/cloudsql/INSTANCE_CONNECTION_NAME

It may be worthwhile to check in detail this information and the other related steps in the “Using Cloud SQL for MySQL” document. This solution, with relevant detail, is given in the “cloudsql databases with django on 'new' flexible google app engine” question on Stackoverflow.  

The “Running Django in the App Engine Flexible Environment” documentation page might prove quite helpful for related details.

[Tas Sóti]

Hello George,

Thank you for the quick response. I was following the Running Django in the App Engine Flexible Environment tutorial and it does not say anything about the SQLALCHEMY_DATABASE_URI environment variable. Is it not meant for SQLAlchemy only? I am using Django's ORM. I actually have another instance running with the same app.yaml as described above and it is working. The difference is that that project (including both the App Engine and SQL instance) is in the us-central region and I deployed it a couple of months ago, when the App Engine Flexible was still in beta.

So is now the SQLALCHEMY_DATABASE_URI variable meant for using with the Django ORM, too, and the old beta_settings is deprecated?

I'll give it a try, it is just not very logical (at least for me), why the variable name includes "SQLAlchemy" if it is meant for any kind of python<->sql connection.

Thank you in advance,

 Tas

[Ryan Matsumoto]

For Django you should not need to set SQLALCHEMY_DATABASE_URI in app.yaml. This is just used in the Python Google Cloud SQL on Flask code sample to set that environment variable that the sample code uses when deployed. The app.yaml in the Django code sample does not set this environment variable.

I'd recommend following the Django on Flex tutorial end-to-end and see what happens. You'll be able to see if the CloudSQL connection works properly by going to the admin part of the deployed web app at [YOUR_PROJECT_ID].appspot.com/admin, logging in with the superuser you define, and adding a question or two.

Good luck, and let us me know if you have any more issues!

Best,

Ryan Matsumoto

Developer Programs Engineer, Google Cloud (Python)

[Zdenko Hrcek]

Hi Tas,

I made the smae mistake and I spent some time solving it: in app.yaml file, database connection name needs to be in strings like this:

beta_settings:

    cloud_sql_instances: 'cm-platform-01:europe-west1:cm-db-develop' 

I wrote recently blog post about running GAE Flex and Django, maybe some of it will be helpful to you. https://www.the-swamp.info/blog/google-app-engine-flexible/

Zdenko

[Tas Sóti]

Hello Ryan and Zdenko,

Thank you very much for the quick replies!

After some more hours I figured out what the problem was. The Cloud SQL API was not enabled for my project. Although it is nicely stated in the tutorial's Before you begin section to enable the APIs, since I have little experience with the Google Cloud Console, I overlooked the difference between the two:

Google Cloud SQL is enabled by default, so I thought, all was good. Additionally, in the tutorial it is stated that the API is called Cloud SQL Administration. (I know that you can even click the link there which automatically enables the API, but I wanted to go through the console so I see exactly what happens and how to do it myself.)

Another thing what misled me was that all of the APIs were enabled in the Dashboard view, and it took me a while to go to the Library menu on the left in order to find the missing API. I don't know if this happens to others, too. It might be helpful to add a sentence or two in the tutorial regarding this.

I can confirm that for me it did not make any difference if the cloudsql-connection-string was put in quotes or not.

Thanks again and happy Easter! :)

Best,

 Tas