Using Terraform to define an S3 bucket and its contents
Nicholas Chammas
Howdy,
I’m new to Terraform. I’m just looking for some high level pointers to get oriented. (Of course, if you are willing to provide additional detail, I’d welcome that too!)
I’m looking to programmatically capture the following in Terraform:
- A requester pays S3 bucket
- Several files that belong in that bucket
The contents of the files in this S3 bucket are determined by some files sitting on a remote server as well as by some files on my local drive.
So first off, is this a good use case for Terraform? This seems like something I could do in Ansible, but I’m interested in exploring Terraform’s declarative-first view of the world.
Second, would I capture the above “infrastructure” in Terraform by using the following data sources?
- s3_bucket: To specify the bucket name and configuration
- s3_bucket_object: To specify the files I want in the bucket
- http: To specify the remote HTTP source of some of the files that go in that S3 bucket
- local file: To specify the local source of some of the files that go in that S3 bucket
Am I on the right track here? Should I be thinking about this differently?
Nick
Andrew Hodgson
Hi,
You would use the _resources_ aws_s3_bucket to create the bucket and aws_s3_bucket_object to upload the data. I don’t know whether the source field in aws_s3_bucket_object can accept a remote URL.
The data sources are for getting the contents of a specific resource in other Terraform resources, they don’t usually create infrastructure.
Andrew.
--
This mailing list is governed under the HashiCorp Community Guidelines -
https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
terraform-too...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/terraform-tool/8fecf9bc-c344-4216-bc24-668203f06a3f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Nicholas Chammas
Ah, I see. A resource to create and manage something I own, and a data source to refer to some data that exists elsewhere. Got it.
So if the aws_s3_bucket_object resource does not accept remote paths for the source of the data, what workaround or alternative would you suggest? For example, is there some way I can connect an aws_s3_bucket_object resource to an http data source via a local file? Does that make sense in Terraform?
Andrew Hodgson
Yes that would be possible using the http data source, then referencing the data source body output in aws_s3_bucket_object. You may need to be careful with encoding on this, and also I would expect that the body of the resource would be written to Terraform state so that may cause other problems.
I would try seeing if you could use a remote path in aws_s3_bucket_object first before going down this route.
Andrew.
________________________________________
From: terrafo...@googlegroups.com [terrafo...@googlegroups.com] on behalf of Nicholas Chammas [nicholas...@gmail.com]
Sent: 14 March 2018 02:09
To: Terraform
Subject: Re: [terraform] Using Terraform to define an S3 bucket and its contents
Ah, I see. A resource to create and manage something I own, and a data source to refer to some data that exists elsewhere. Got it.
So if the aws_s3_bucket_object resource does not accept remote paths for the source of the data, what workaround or alternative would you suggest? For example, is there some way I can connect an aws_s3_bucket_object resource to an http data source via a local file? Does that make sense in Terraform?
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/cc820e81-d0ab-4358-bbb7-3947429086c2%40googlegroups.com<https://groups.google.com/d/msgid/terraform-tool/cc820e81-d0ab-4358-bbb7-3947429086c2%40googlegroups.com?utm_medium=email&utm_source=footer>.
Nicholas Chammas
I would expect that the body of the resource would be written to Terraform state so that may cause other problems.
Oh, you mean the whole contents of the file would be written? That would be a problem, since these are 100MB+ binary artifacts I’m trying to sync from these HTTP sources to S3. I would hope that Terraform would only write a hash of the contents.
In any case, I think I have enough direction to start experimenting myself. Thanks Andrew!