Is it possible to parameterize resource names such as s3 buckets

[Monosij Dutta-Roy]

I was trying to see if I could use a variable name for a bucket and other resources, especially when those resources are name in other resources such as AWS Kinesis as is the following example from Terraform.

Here for the aws_kinesis_firehose_delivery_stream the bucket_arn needs the name of the s3 bucket configured for it.

I tried using the variables file and the local variables in the kinesis.tf (in part shown below).

Neither worked. And I assume this must be possible in the flexibility with modules and such that TF allows.

While I have learnt how to use modules - this particular aspect is for one environ and would eventually be further parameterized by a module name.

#-------------------------------------------------------------#

resource "aws_s3_bucket" "XXXXXX" {

  bucket = "tf-test-bucket"

  acl    = "private"

}

#-------------------------------------------------------------#

resource "aws_kinesis_firehose_delivery_stream" "extended_s3_stream" {

  name        = "terraform-kinesis-firehose-extended-s3-test-stream"

  destination = "extended_s3"

  extended_s3_configuration {

    role_arn   = "${aws_iam_role.firehose_role.arn}"

    bucket_arn = "${aws_s3_bucket.XXXXXX.arn}"

...

}

#-------------------------------------------------------------#

Thanks for your help.

Mono

[Chamila de Alwis]

Last I checked variable interpolation in resource names was not supported. From my experience, trying to do so might indicate you're following a possible anti-pattern to get something done. Would you be able to explain why you need to have variable names inside resource names?

Regards,

Chamila de Alwis

Blog: medium.com/@chamilad

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-too...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/9b7041b0-bb99-43f0-94ce-f47a9fafbf03%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Monosij Dutta-Roy]

Hi Chamila -

Thanks for your email.

The use cases are around creating resource IDs (hopefully this is right term) that have dependencies/ interconnections on other resources.

The example I showed was a Kinesis Firehose stream that needs other resources such as the s3 bucket.

So I could give the stream ID a value from the var file say "stream-1" then create the s3 bucket as $var.stream-1.name."ec2".

Thus: resource would be "aws_kinesis_firehose_delivery_stream" "${var.stream-1}_stream" instead of "extended_s3_stream"

and its related bucket: resource "aws_s3_bucket" "${var.stream-1}_s3" 

That way I could change the stream name (the primary resource) at will and confirm anything connected to this resource will get consistent name and related other attribute values - such as: name = "terraform-kinesis-firehose-extended-s3-test-stream"

It would be a big value in being able to define multiple different VPCs for example. within an environ such as DEV/ PROD - but I believe its value is more for resources such as data related environs, where multiple types of that resource and dependencies can coexist.

From a doc standpoint it would also allow us to look at a var file and see what is defined at a glance.

I guess we can already do is define them as variables in a module as in:

variable "ENV" {}

variable "AWS_REGION" {}

module "main-vpc" {

  source = "terraform-aws-modules/vpc/aws"

  name = "vpc-${var.ENV}"

  cidr = "10.0.0.0/16"

...

}

So I am not sure why I cannot just create such in a file use a variable. I eventually plan to migrate to modules anyway - so i should be ok?

Hope my questions (and answers) are not too confusing.

Please consider that I am a newbie in this coming from a DBMS/ Python background.

Thank you.

Mono

To unsubscribe from this group and stop receiving emails from it, send an email to terrafo...@googlegroups.com.

[Chamila de Alwis]

Hi Monosij,

Sorry, I missed this thread.

IIUC, I think your use case is one that should be achievable through the modules approach (disclaimer: I have not worked with Kinesis services so my assumptions might be wrong)

As I understand, you need to have consistent naming for pairs of resources. As an extension of this requirement, I guess the number of pairs is controllable through a variable (ex: firehose_count). For this, a module (ex: s3backedfirehose) containing implementation for a firehose stream and an accompanying S3 bucket should do. As I see, the name of the Terraform resources isn't the issue here. It is the name of the AWS resources that should change, s3 `bucket` variable and firehose_stream's `name` variable. The module could take the firehose_count as an input variable and have multiple resources created with incrementing names [1] (calling the module with the count arg seems to be a feature to be implemented in the future [2] [3]). So your code could change to the following (untested).

variable "firehose_count" {}

variable "prefix" {}

#-------------------------------------------------------------#

resource "aws_s3_bucket" "firehose_store" {

  count = "${var.firehose_count}"

  bucket = "bucket-${var.prefix}-${[count.index]}"

  acl    = "private"

}

#-------------------------------------------------------------#

resource "aws_kinesis_firehose_delivery_stream" "extended_s3_stream" {

  count = "${var.firehose_count}"

  name        = "terraform-kinesis-firehose-extended-s3-test-stream-${var.prefix}-${[count.index]} "

  destination = "extended_s3"

  extended_s3_configuration {

    role_arn   = "${aws_iam_role.firehose_role.arn}"

    bucket_arn = "${aws_s3_bucket.firehose_store.arn}"

...

}

#-------------------------------------------------------------#

This is assuming I got your requirement right. Also, there could be multiple ways of doing the same thing. Hope this helps.

[1] - https://www.terraform.io/docs/configuration/index.html#example

[2] - https://github.com/hashicorp/terraform/issues/953

[3] - https://www.terraform.io/docs/configuration/modules.html#other-meta-arguments

Regards,

Chamila de Alwis

Blog: medium.com/@chamilad

To unsubscribe from this group and stop receiving emails from it, send an email to terraform-too...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/b0f8128f-b065-49f1-85e1-a974697a876e%40googlegroups.com.

[Monosij Dutta-Roy]

Thanks Chamila. Appreciate the detailed reply.

Will try and see how they work in modules.

BTW I do not mean the name variable to be a variable.

...

I mean the initial declaration as in

resource "aws_s3_bucket" "firehose_store"

TO

 resource "aws_s3_bucket" ${var.SOMETHING} "firehose_store"

...

I do understand the name variable can be a variable or part of.

But the initial declaration seems to be static?

I will try with modules and see how they are applied.

BTW been enjoying your posts on Medium as well.

Thank you for keeping us all updated.

Monosij

To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/CAB2A5_mC8GHPCUedaWc8VeFBgZ%2B9JKqzXFbtnnx8pHm_x2E5DA%40mail.gmail.com.

[chamila]

Monosij,

AFAIK the resource names, as in `resource <type> <name> {}`, are unique to the module call name and the count index. So these would not be static. I still can't see why that would be important to apply changes though. AFAIK resource names are an internal reference for terraform.

BTW been enjoying your posts on Medium as well.

Thanks! Glad they are actually being used! :) 

Regards,

Chamila

Blog: medium.com/@chamilad

To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/CAK89wdEHio3ChJFCrrXjm%2BsiEG--P%2B7N-o2ayuPgGxq0Mi9SXQ%40mail.gmail.com.