Confused about passing variables to and from modules - duplication required ?

[Wayne Pascoe]

Hi all,

I'm trying to modularize some terraform code and I'm confused about passing variables (both input and output) and would like to know if there is a cleaner way of doing this with less duplication.

My structure is roughly as follows:

main.tf
-stage
--stage/vpc
--stage/vpn
--dev/vpc
--dev/vpn
--modules
--modules/vpc
--modules/vpn

My questions are:
1. Is there any way to reduce the number of places I have to define variables before I can use them (e.g. top level and have that inherited by any directories lower)
2. Is there any way to avoid duplication of variable passing from my top level main.tf down to the actual module that implements the 'thing'
3. Do I have to output variables at each stage along the way to be able to use them in other modules ?

Variable definitions
I seem to have to define all of the variables in the vars.tf for each section (or I get errors like

Error: resource 'aws_vpc.main' config: unknown variable referenced: 'vpc-cidr'; define it with a 'variable' block

Error: module "vpc": "vpc-cidr" is not a valid argument

Input variables
For input variables, I seem to have to specify them in each module inside main.tf, then again in my environments (dev / stage) and again in my module.

So for example, for the VPC, I'd have something like the following:

main.tf

module "vpc" {
  source = "dev/vpc"
  aws_region = "${var.aws_region}"
  instance_name = "${var.instance_name}"
  env_status = "${var.env_status}"
}

stage/vpc/main.tf

module "vpc" {
  source = "../../modules/vpc"

  aws_region = "${var.aws_region}"
  instance_name  = "${var.instance_name}"
  env_status = "${var.env_status}"

  vpc-cidr = "10.1.0.0/16"
  public-subnet-a-cidr = "10.1.1.0/24"
  private-subnet-a-cidr = "10.1.2.0/24"
}

modules/vpc/main.tf

resource "aws_vpc" "main" {
  cidr_block = "${var.vpc-cidr}"
  tags {
    Name = "${var.instance_name}-vpc"
    env = "${var.env_status}"
  }
}
resource "aws_subnet" "public-a" {
  vpc_id = "${aws_vpc.main.id}"
  map_public_ip_on_launch = true
  availability_zone = "${var.aws_region}a"
  cidr_block = "${var.public-subnet-a-cidr}"
  tags {
    Name = "${var.instance_name}-public-a"
    env = "${var.env_status}"
  }
}

resource "aws_subnet" "private-a" {
  vpc_id = "${aws_vpc.main.id}"
  availability_zone = "${var.aws_region}a"
  cidr_block = "${var.private-subnet-a-cidr}"
  tags {
    Name = "${var.instance_name}-private-a"
    env = "${var.env_status}"
  }
}

Output variables
After creating my VPC, I want to be able to deploy an EC2 node into a specific subnet. In order to do this, I am outputting the subnet id. But at the moment, I have to repeat this in modules/vpc/main.tf and in stage/vpc/main.tf and in dev/vpc/main.tf before I can use it in my top level main.tf to pass it to my vpn module.

Is there any way to avoid this duplication and make it available to the top level just by outputting it at the module level ?

Thanks in advance!

[Andrew Hodgson]

Hi,

I am slightly confused about the layout you have, can you confirm how many state files you end up with at the end of the process? You do have to define the input and outputs at each stage. What I typically end up with is something like this:

/modules
/modules/vpc
/modules/database/
/dev
/test
/prod

In /dev /test and /prod I have the same main.tf, variables.tf and outputs.tf across the three environments. If I change the files in one place, I copy them to the other directories as well. I sometimes have outputs.tf to allow me to get outputs from these environments into other modules, but in most cases I just have main.tf and variables.tf.

So for example here is one main.tf file:

module "vpc" {
source = "../modules/vpc"
tags = "${var.tags}"
vpc_network = "${var.vpc_network}"
}

module "database" {
source = "../modules/database"
vpc_id = "${module.vpc.vpc_id}"
database_password = "${var.database_password}"
database_subnets = ["${module.vpc.database_subnets}"]
tags = "${var.tags}"
}

Here is the variables.tf in my /dev directory:

variable "db_password" {
description = "DB Admin Password"
}

variable "tags" {
description = "Map of default tags to assign to the resources."
type = "map"
}

variable "vpc_network" {
description = "IP address allocation of the VPC for the environment"
}

Here is the variables.tf in my VPC module:

variable "tags" {
description = "Map of default tags to assign to the resources."
type = "map"
}

variable "vpc_network" {
description = "IP address allocation of the VPC for the environment"
}

Here is outputs.tf in the VPC module:
output "vpc_id" {
value = "${aws_vpc.main.id}"
}

output "database_subnets" {
value = ["${aws_subnet.database.*.id}"]
}

I never nest modules more than one deep otherwise you end up in trouble later on with nested outputs etc. If I have instances where I don't want to have a specific module in an environment it just gets left out of main.tf. Variable inputs in the environment modules are usually sources via tfvars files.

Does that help?
Andrew.

________________________________________
From: terrafo...@googlegroups.com [terrafo...@googlegroups.com] on behalf of Wayne Pascoe [anonymou...@gmail.com]
Sent: 04 January 2018 09:59
To: Terraform
Subject: [terraform] Confused about passing variables to and from modules - duplication required ?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-too...@googlegroups.com<mailto:terraform-too...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/37e952a5-e5ba-4987-adcf-adab692132fd%40googlegroups.com<https://groups.google.com/d/msgid/terraform-tool/37e952a5-e5ba-4987-adcf-adab692132fd%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

[Wayne Pascoe]

Hi Andrew,

I end up with 1 state file at the end with all environments in it. I think your structure actually may solve 2 / 3 problems I'm having at the moment, so I'm going to give that a go.

Do you have separate backends for each environment (dev / test / prod ) so that these are totally separate ? Where do you run terraform from in this setup ? / or /dev ?

Thanks!

On Thursday, January 4, 2018 at 4:16:41 PM UTC, Andrew Hodgson wrote:

Hi,

I am slightly confused about the layout you have, can you confirm how many state files you end up with at the end of the process?  You do have to define the input and outputs at each stage.  What I typically end up with is something like this:

/modules
/modules/vpc
/modules/database/
/dev
/test
/prod

...

[Andrew Hodgson]

Hi,

I thought your setup would give you one state file. That isn't recommended in most cases, I have seen some pretty big blow-ups when only one state file is used. I run Terraform from the environment directory like /dev /prod etc, and I have 100% confidence that I am only affecting the stated environment.

Each backend is separate as well either to an S3 bucket or Azure storage account.

Thanks.

Andrew.
________________________________________
From: terrafo...@googlegroups.com [terrafo...@googlegroups.com] on behalf of Wayne Pascoe [anonymou...@gmail.com]

Sent: 04 January 2018 16:22
To: Terraform
Subject: Re: [terraform] Confused about passing variables to and from modules - duplication required ?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-too...@googlegroups.com<mailto:terraform-too...@googlegroups.com>.

To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/7caedacc-3c68-4f5a-ab89-bdd30a14a4fb%40googlegroups.com<https://groups.google.com/d/msgid/terraform-tool/7caedacc-3c68-4f5a-ab89-bdd30a14a4fb%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Wayne Pascoe]

Hi Andrew,

I've moved things to your approach and that solves quite a few things for me - thanks loads !

On Thursday, January 4, 2018 at 5:38:18 PM UTC, Andrew Hodgson wrote:

Hi,