Dependency between resource of one provider and another provider

[ed...@sandpiper.io]

I'll start by saying thanks for making a cool tool!  I recently wrote a plugin for RackSpace's Carina which provides Docker Swarm clusters in the cloud.  This is my first go program so critique is welcome.  There is also a writeup of what I learned while coding the plugin that might be helpful to others working on a plugin.

My goal was to enable creation of Carina clusters and then use the Docker provider to manage the swarm inside the Carina cluster.  There is a problem though.  In the following tf file the Docker provider depends on the carina cluster resource.

provider "carina" { }   resource "carina_cluster" "test" {  name = "test1" }   provider "docker" {  host = "${carina_cluster.test.docker_host}"  cert_path = "${carina_cluster.test.docker_cert_path}" }   resource "docker_network" "mynetwork" {  name = "mynetwork" }

This causes a strait `terraform plan` to fail (because the docker connection information isn't available).  

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but
will not be persisted to local or remote state storage.

Error refreshing state: 1 error(s) occurred:

* Error initializing Docker client: open ${carina_cluster.test.docker_cert_path}/cert.pem: no such file or directory

If I run `terraform graph` the docker provider is seen as a dependency of the carina_cluster resource.  

$ terraform graph

digraph {

       compound = "true"

       newrank = "true"

       subgraph "root" {

               "[root] carina_cluster.test" [label = "carina_cluster.test", shape = "box"]

               "[root] docker_network.mynetwork" [label = "docker_network.mynetwork", shape = "box"]

               "[root] provider.carina" [label = "provider.carina", shape = "diamond"]

               "[root] provider.docker" [label = "provider.docker", shape = "diamond"]

               "[root] carina_cluster.test" -> "[root] provider.carina"

               "[root] docker_network.mynetwork" -> "[root] provider.docker"

               "[root] provider.docker" -> "[root] carina_cluster.test"

       }

}

Why doesn't terraform just respect the dependency graph and ignore the docker provider during plan since the docker_cert_path and docker_host are still in <computed> state?  The dependency literally means that everything in the "docker" provider space doesn't exist until the carina_cluster resource is created.

There is a workaround in which you provide a -target option to terraform so that the carina_cluster is created first.  It's pretty clunky to do this and seems unnecessary.  Is there a better solution that I'm missing?  I poked around in the terraform code but couldn't figure out how to easily force the dependency as the providers seem to get initialized separately from the resources.

This is a more general problem where types of infrastructure depend on the existence of other types (or providers depending on resources) which seems like it might come up a lot as different technologies are mixed together.  This problem seems to have been discussed here and closed.  Not sure why it's not working.

[David Adams]

You may be able to solve it by putting the carina cluster into a module, and then use module outputs to feed into the docker provider config.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/1cb1f043-3df0-4e0f-bc97-4e224f14881d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Martin Atkins]

I think you've encountered the problem that this issue is describing:

https://github.com/hashicorp/terraform/issues/4149

Terraform wants to instantiate the provider to plan, but it can't because the dependent thing hasn't been instantiated yet, and won't be until apply.

The workaround for the moment would be to bootstrap by doing an initial terraform run with -target="carina_cluster.test" . That will instruct Terraform to create that resource in isolation, so it won't need to instantiate the docker provider yet. Once you get past this step, the necessary values to instantiate the docker provider will be known to Terraform and so subsequent runs should proceed as normal.

Long run, the proposed solution to 4149 would effectively make Terraform detect the need for this two-step apply automatically and lead you through it, automatically applying the relevant targeting (or rather, "anti-targeting") to let each run proceed to continue to the next step.