stateless terraform

[Alex Leonhardt]

hi, i posted the same question in gitter but nobody answered....

has anyone had any luck making terraform run in a stateless kind of way ? i tried import and it seems certain details (like asglc user-data) are missing which makes it want to delete my launch-config and update the asg for no reason (asg update as there are other things missing from that import too) ... i know it's been designed with state in mind (for performance reasons, to detect drift, etc.) but I'd like to manage resources (or groups of them rather) as separate entities (and not one global tf state) and determine if I need to do "changes" based on the data I retrieve from (in this case) AWS .. AWS is the source of truth, not my local state file .. basically I'd retrieve the "data" these "entities" need to get things like SGs, Subnets, etc. before running the plan & apply ... any thoughts or suggestions ?

thanks!

alex

[Martin Atkins]

Hi Alex,

Although indeed it's not intended as the primary path, importing a resource into a temporary local state file and then running Terraform against it before discarding the plan should work in principle for most resources.

A couple of exceptions:

• Not all resources support import, of course.
• Some data (e.g. secrets) cannot be read from AWS at all, and so terraform import produces an incomplete state in that case.

Neither of those should apply to launch configurations (assuming that's what you meant by "asglc") so it sounds like you've found a bug there, which I'd invite you to describe in more detail in the AWS provider issue tracker and hopefully we can get that fixed up.

In the long run we plan to make terraform import generate configuration too, which might create some awkwardness for your unusual stateless workflow. However, there's no short-term plan for that so for the time being what you're doing should work, bugs in specific resources notwithstanding.

[Anshu Prateek]

"not one global tf state"..

To clarify - you do not need to maintain one global tf state file. Rather its better to break it down into smaller state files. Your ideas around separate entities/groups are correct, you just need to think state file also as state file for that group rather than one big state file.

The way to do it is to group the entities in either folder wise or use workspaces.

"AWS is the source of truth, not my local state file"

Is your entire infra TF managed or is is TF + manual/other automation. In case of first, you have nothing to worry about. In case of second, it will be difficult.

Also, the state file need not be local. You can/should keep it in a remote state (s3/consul etc) to ensure multiple people can work on your infra (on different groups of infrastructure) at the same time.

TF  refreshes the state from AWS on every plan/apply/refresh but it needs a state file to generate the diff.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-too...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/1b1f6f67-e1a8-47ad-b31e-b6c6562158cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

regards
Anshu Prateek
+91.991.610.2967

[stuart...@jahingo.com]

Wouldn't you still need something very similar to the state file though?

You need some way of mapping a particular AWS ID (e.g. i-3453235) to the config in Terraform, so you know for each import what you are importing to where.

I guess you could store that in a tag, but not all resources support tags and you'd end up with a relatively complex script (find all resources in the account, get tags for each, run terraform import).

So would this actually be workable?

[Alex Leonhardt]

I would like to reply but something keeps deleting them...

[Dominic Tootell]

Hi there,

>>

>> Is your entire infra TF managed or is is TF + manual/other automation.

>>

I'll try to explain the reason behind asking about separate state files (or no state files - and describe first).   

We run our AWS infrastructure as a set of independent microservices (60+).  Each of these microservices is owned by several development teams.  The microservice X, owns the AWS infrastructure it relies upon, for example.   If it requires DDB or SNS, as part of its deployment, it will create and/or configure those AWS components.  It owns that AWS infrastructure.

Microservice X is deployed into the same AWS account, and VPC (and subnets) along side all the other 60+ mircoservices.   The other mircoservices may be using other AWS components, completely separate from this Microservice X.   The state that Microservice X knows about is only it's view of the AWS world; and it shouldn't be concerned with the state of other microservices.

All these independent microservices are deployed into a preexisting VPC/subnet infrastructure (they don't know about the VPC - they don't create it as part of the deployment, they just state : I need to go into a private subnet).

In effect:

- State of the world per microservice (60+ independent views of the world)

- State of the network independent of mircoservices

----

We are looking at "how" to achieve this in a Terraform world.   

thanks

/dom

[Anshu Prateek]

Each of the independent micro services will have their own TF state files by segregating each component into their own folders. And the info that they need to use from others, example: security group or vpc id, will be ideally exported by respective components and the consuming components will use them via remote data .
https://www.terraform.io/docs/providers/terraform/d/remote_state.html

In case the data to be consumed is not TF governed and is not being working upon for being imported, then those can be set as variable in files common across entire setup. We achieve these common files by means of symlinks whereas the source file resides in the root and everyone else just symlinks to that file.

We use the above approach for similar usecase.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-too...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/05bf0ef0-67ed-4358-b4ab-9a789aa796ca%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Alex Leonhardt]

Thanks Anshu!

I think that is something we can work on and give it a go.

Thanks!

Alex