Variable.get('gcp_project') Issue

[Stefano Giostra]

Hello, 

I wish get the gcp_project env variable, but seems that the dag can't access to this value.

Ok I can set the value in AIRFLOW VARIABLES but there is a manner to get it from environment?

this is the expample of the dag:

``` 

dag_name = 'BB-DagDemo-Basic'
with DAG(dag_id=dag_name, default_args=default_args, schedule_interval="@daily") as ldag:
    project = Variable.get('gcp_project', 'xxx')
    project_id = Variable.get('gcp_project_id', 'yyy')
    gcp_project_id = os.environ.get('GCP_PROJECT_ID', 'example-project')
    logging.info(f"SG - project: {project} - project_id: {project_id} - gcp_project_id: {gcp_project_id}")

```

   

This is the output:

`INFO - SG - project: xxx - project_id: yyy - gcp_project_id: example-project`

[Lucas Esteves Cardozo]

According to the documentation (https://cloud.google.com/composer/docs/how-to/managing/environment-variables), the environment variable which contains the project identifier is GCP_PROJECT, not GCP_PROJECT_ID. You should be able to get it using os.environ.get("GCP_PROJECT").

[Stefano Giostra]

Hi Lucas,

            os.environ.get('GCP_PROJECT')

works fine, thanks.

Now the question is: is  possible to get this info from Variable Airflow object as shows in the GCP Notifications example:https://cloud.google.com/composer/docs/how-to/using/writing-dags

 

Da: cloud-compo...@googlegroups.com <cloud-compo...@googlegroups.com> Per conto di Lucas Esteves Cardozo
Inviato: giovedì 26 novembre 2020 16:20
A: cloud-composer-discuss <cloud-compo...@googlegroups.com>
Oggetto: Re: Variable.get('gcp_project') Issue

--
You received this message because you are subscribed to a topic in the Google Groups "cloud-composer-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cloud-composer-discuss/wKx882dgPMc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cloud-composer-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-composer-discuss/dbb1eba7-43f7-4d86-8b83-59db4af050can%40googlegroups.com.

[Lucas Esteves Cardozo]

I've never tried this myself but, as far as I know, Composer does not come with predefined Variables. I think you'd need to manually set this by yourself. 

[sgus...@butterflynetinc.com]

I am setting this variable myself as part of environment creation.

[Stefano Giostra]

Hi, 

I think that the better solution is that showned by Lucas:  os.environ.get("GCP_PROJECT") 

Infact I introduced it on my code