Collecting output of a local-exec provisioner as a data source

[Mark Jaffe]

Hi,

A legacy system we are working hard to automate uses some custom API provided by docker modules, for service-registry and service-discovery we wish to leverage. I've been able to incorporate some resource-provisioning in the "aws_instance" resource, for example:

resource "aws_instance" "ec2" {

  count         = "${var.instance_count}"

  ami           = "${data.aws_ami.coreos_ami.image_id}"

  instance_type   = "${var.instance_type}"

  key_name      = ""

  monitoring    = false

  iam_instance_profile = "${var.iam_profile}"

  subnet_id     = "${element(var.subnet_ids, count.index)}"

  vpc_security_group_ids = [ "${split(",", var.security_groups)}" ]

##  user_data     = <something useful here>

  tags {

    Name        = "${element(data.template_file.name.*.rendered, count.index)}"

    myservice = "${var.service}"

    myenv     = "${var.env}"

  }

  lifecycle {

    ignore_changes = [ "ami", "user_data", "tags" ]

  }

   provisioner "local-exec" {

    command = "docker run --rm docker.repos/datacenter-setup store-server -n ${element(data.template_file.name.*.rendered, count.index)}.${var.data_center}.our-domain -e ${var.env}  -s configserver.${var.data_center}.our-domain"

  }

The above works for me, but when I used the API within the resource to obtain the cloud-init data, it was not available at the correct time. I thought of using a "depends-on" clause, but that did not work either. Extracting the cloud-init into a different module opened up a whole other set of restrictions, and I am stuck trying to understand how to get this to work. I've tried just creating it from a file resource, but I am not able to render the resulting template correctly (it has line-breaks in the wrong places and some special characters like file re-directs become encoded.) 

What I need to be able to do is this:

   provisioner "local-exec" {

    command = "docker run --rm docker.repos/datacenter-setup cloud-config -t aws -n ${element(data.template_file.name.*.rendered, count.index)}.${var.data_center}.our-domain 

}

and just put the output into the user_data for the instance launch.

[David Maze]

On Friday, August 18, 2017 at 6:00:40 PM UTC-4, Mark Jaffe wrote:

What I need to be able to do is this:

   provisioner "local-exec" {

    command = "docker run --rm docker.repos/datacenter-setup cloud-config -t aws -n ${element(data.template_file.name.*.rendered, count.index)}.${var.data_center}.our-domain 

}

and just put the output into the user_data for the instance launch.

It looks like that command doesn't depend on anything other than read-only data, and will produce a consistent output for a set of inputs?  You could try wrapping it in an external data source script; since the protocol is just "get parameters as JSON on stdin and write data as JSON on stdout" it's pretty straightforward, at least, in Python.  That would let you feed the output of the script into the aws_instance user_data.

[Mark Jaffe]

This is considerably more difficult then it appears; it is impossible to debug the interaction between what terraform is sending to my script and what the script is expecting. It just fails to parse the arguments and refuses to provide me any feedback as to what is getting into the program