Why is env_variables not transferred to container on instance

[Søren Malling]

I have a app.yaml similar to this

runtime: php
env: flex

runtime_config:
  document_root: Web
  whitelist_functions: exec

env_variables:
  DATABASE_DSN: mysql:unix_socket=/cloudsql/PROJECTNAME:europe-west3:production;dbname=DATABASE
  DATABASE_CREDENTIALS_USER: application
  DATABASE_CREDENTIALS_PASSWORD: 'PASSWORD'

beta_settings:
    cloud_sql_instances: PROJECTNAME:europe-west3:production

after deployment I have following containers running on a instance, that I access via SSH after activating debug

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES39bcf36df191 eu.gcr.io/PROJECTNAME/appengine/default.20170920t100932@sha256:ae8ce69ffe994d200d889c5e30bd48a2d5642e2e1d0632bc351e83a8fd6fef96 "/usr/bin/supervisord" 2 hours ago Up 2 hours 172.17.0.1:8080->8080/tcp gaeapp8c60858d1ef4 gcr.io/google_appengine/cloud-sql-proxy "/cloud_sql_proxy -di" 2 hours ago Up 2 hours gigantic_paref520dd07173a gcr.io/google_appengine/api-proxy "/proxy" 2 hours ago Up 2 hours api2bd7417c099b gcr.io/google_appengine/nginx-proxy "/var/lib/nginx/bin/s" 2 hours ago Up 2 hours 8080/tcp, 8090/tcp, 0.0.0.0:8443->8443/tcp nginx_proxy7526979828d6 gcr.io/google_appengine/fluentd-logger "/opt/google-fluentd/" 2 hours ago Up 2 hours fluentd_logger

When access the container gaeapp and enter printenv I can't see the env_variables, that I'm using for the application to create a connection to the cloud_sql instance. Is there any missing configuration or similar that can explain why the env_variables are not transferred?

[George (Cloud Platform Support)]

Hello Søren,

The recommended way to pass the environment variables for database connection is: 

env_variables:

    # Replace USER, PASSWORD, DATABASE, and CONNECTION_NAME with the

    # values obtained when configuring your Cloud SQL instance.

    MYSQL_USER: USER

    MYSQL_PASSWORD: PASSWORD

    MYSQL_DSN: mysql:dbname=DATABASE;unix_socket=/cloudsql/CONNECTION_NAME

You can check this in more detail on the "Using Cloud SQL for MySQL" documentation page, sub-chapter: "Setting connection strings and adding a library". 

[Søren Malling]

Hi George,

So this is a question of naming and not a issue with the env_variables not being transferred? :-)

Will try it later today and give response here, thanks for the pointer!

[Søren Malling]

Hi George,

So, I tried to change the name of the variables but the vars are still not in the container. Here the env from the container

container_exec gaeapp printenv

Entering gaeapp, pid 3962

TERM=xterm-256color

LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=3

0;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzm

a=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=0

1;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.s

ar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;3

5:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.pn

g=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;

35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.a

sf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:

*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00

;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.

axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

LANG=en_US.UTF-8

SHELL=/bin/bash

LOGNAME=root

USER=root

USERNAME=root

MAIL=/var/mail/root

HOME=/root

SUDO_COMMAND=/usr/local/bin/nsenter -t 3962 -m -u -i -n -p printenv

SUDO_USER=[my user]

SUDO_UID=1002

SUDO_GID=1002

[Søren Malling]

For reference, my app.yml looks like this now

runtime: php
env: flex

runtime_config:
  document_root: Web
  whitelist_functions: exec

env_variables:

  FLOW_CONTEXT: 'Production' #This is a environment for my framework
  MYSQL_DSN: mysql:unix_socket=/cloudsql/[PROJECT]:europe-west3:production;dbname=[DATABASE NAME]
  MYSQL_USER: [MYSQL USER FROM CONSOLE]
  MYSQL_PASSWORD: '[MYSQL PASSWORD FROM CONSOLE]'

beta_settings:
    cloud_sql_instances: [PROJECT]:europe-west3:production

[George (Cloud Platform Support)]

It might well be that your connecting to the container via SSH interferes with your ability to read environment variables. More relevant details, as an example, in the "-link environment variables don't appear in SSH session" posting. 

[Søren Malling]

So not really knowing what you wanted me to test, I tried to inspect the docker container

sudo docker inspect gaeapp

"Config": {
  "Env": [
    "FLOW_CONTEXT=Production",
    "MYSQL_DSN=mysql:unix_socket=/cloudsql/...;dbname=...",
    "MYSQL_PASSWORD=...",
    "MYSQL_USER=...


So, the container seems to get the env passed, but how come they not available inside the container, when the applications has started?

[Takashi Matsuo (Google)]

Hi Søren,

What do you see when you run:

```

sudo docker exec -ti gaeapp printenv

```

?

[Søren Malling]

Then I see the environment variables similar to before, including the MYSQL_* settings

PATH=/opt/php/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

HOSTNAME=f820f74adfc9

APPENGINE.GOOGLEAPIS.INTERNAL_NAME=/gaeapp/appengine.googleapis.internal

FLOW_CONTEXT=Production

GAE_DEPLOYMENT_ID=[DEPLOYMENT_ID]

GAE_MEMORY_MB=614

GAE_SERVICE=default

GAE_VERSION=20170920t233841

GCLOUD_PROJECT=[PROJECT]

GOOGLE_CLOUD_PROJECT=[PROJECT]

MYSQL_DSN=mysql:unix_socket=[SOCKET & DATABASE]

MYSQL_PASSWORD=[PASSWORD]

MYSQL_USER=[USER]

[Takashi Matsuo]

Alright, then the container is getting the env vars.

Do you have actual problems connecting to the db?

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a58411b8-c59c-47b0-a2c4-797c47bd494f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

-- Takashi

[Søren Malling]

Do you have actual problems connecting to the db?

Yes. My application don't running can't read the env and the when running post-deploy-cmd from my composer.json, the FLOW_CONTEXT and MYSQL_* enviroment parameters are not passed to the processes

[Søren Malling]

Just le me try and write that again - the last message was unreadable..

Yes. When trying to run my apllication via the URL, it can't read the env and therefore can't access the database. 

And when running post-deploy-cmd from my composer.json (as described for the Symfony example), the FLOW_CONTEXT and MYSQL_* enviroment parameters are not passed to the processes.

[Takashi Matsuo]

Are you trying to do DB migration with the post-deploy-cmd?

It won't work because we don't run post-deploy-cmd any more, with the new build pipeline.

What gcloud version are you using?

If you're using the old build pipeline (you can do that by setting a flag `gcloud config set app/use_runtime_builders false`), it still invokes post-deploy-cmd.

In that case, post-deploy-cmd should be able to access the envvars.

Can you show me the build logs, or anything that shows how your app is failing?

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/576b9a13-0ff5-451b-8b78-426aeb45a376%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

-- Takashi

[Takashi Matsuo]

Also, in any case, I don't recommend invoking the db migration during the deployment process, because it's dangerous.

What happens if the db migration succeeds, and then the deployment fails? Your app will be in a broken state.

So now the recommended way is to use cloud_sql_proxy locally and do the db migration. Yeah it's cumbersome, but it's all we have right now.

We're planning to provide a tool that can invoke a script with the deployed image. Please stay tuned.

--

-- Takashi

[Søren Malling]

Hi Takashi,

Thanks for that information - it explains why I haven't been able to make it work for 2 days :) Could you be so kind to add a note to the PHP examples, where this is mentioned? 

I'm using Gitlab for CI/CD, so how would you recommend this done? Create a service account for a local cloud_sql_proxy to setup doing deployment of the app? And wait with database migration to the gcloud app deploy is done ? 

[Takashi Matsuo]

Yeah that sounds like a plan to me for now.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/6df7c374-2327-413e-9a70-dba61669b178%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

-- Takashi

[Takashi Matsuo]

And FYI, I filed an issue on the community articles:

https://github.com/GoogleCloudPlatform/community/issues/248

--

-- Takashi

[Søren Malling]

Thanks a lot :-) And thanks for the help!