aws_ecs_service error "Unable to Start a service that is still Draining"

7,116 views
Skip to first unread message

Gerald DeConto

unread,
Sep 17, 2015, 3:54:08 PM9/17/15
to Terraform
Has anyone come across this message while deploying an aws_ecs_service? 

* aws_ecs_service.pipeline-api: InvalidParameterException: Unable to Start a service that is still Draining. 
        status code: 400, request id: [] 

There is no existing aws_ecs_service, elb, aws_instance, etc in AWS because I remove them before redeploying

The aws_ecs_service and other parameters appear to me to be correct and seem consistent with the Terraform documentation and examples.

Any help appreciated as I can't see the problem.  Thanks in advance.

------------------------

provider "aws" {
  region = "${var.aws_region}"
}

resource "aws_iam_instance_profile" "pipeline-api" {
  name = "${var.instance_prefix}"
}

resource "aws_iam_role_policy" "pipeline-api" {
  name = "${var.instance_prefix}"
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:*",
        "ec2:Describe*",
        "s3:*",
        "sns:*",
        "sqs:*",
        "logs:*",
        "ecs:CreateCluster",
        "ecs:DeregisterContainerInstance",
        "ecs:DiscoverPollEndpoint",
        "ecs:Poll",
        "ecs:RegisterContainerInstance",
        "ecs:StartTelemetrySession",
        "ecs:Submit*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF
}

resource "aws_iam_role" "pipeline-api" {
  name = "${var.instance_prefix}"
  path = "/"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
      "Service": "ec2.amazonaws.com"
    },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF
}

resource "aws_instance" "pipeline-api" {
  count = "${var.instance_count}"

  instance_type = "${var.instance_type}"
  ami = "${var.aws_ami}"
  key_name = "${var.key_name}"
  security_groups = ["${split(",", var.aws_security_group)}"]
  subnet_id = "${var.aws_vpc_subnet}"
  iam_instance_profile = "${aws_iam_instance_profile.pipeline-api.name}"
  user_data = "${template_file.userdata_node_provisioner.rendered}"

  tags {
    Name = "${var.instance_prefix}-${var.environment}-${count.index}"
    Index = "${count.index}"
    Service = "pipeline-api"
    Environment = "${var.environment}"
  }
}

resource "template_file" "userdata_node_provisioner" {
  filename = "templates/userdata_node_provision.sh"
  vars {
    instance_prefix = "${var.instance_prefix}"
    environment = "${var.environment}"
    ecs_cluster="${var.instance_prefix}"
  }
}

resource "aws_elb" "pipeline-api" {
  name = "${var.instance_prefix}-${var.aws_group}-${var.environment}"
  security_groups = ["${split(",", var.aws_security_group)}"]
  subnets = ["${split(",", var.aws_vpc_subnets)}"]
  internal = true

  listener {
    instance_port = 80
    instance_protocol = "http"
    lb_port = 80
    lb_protocol = "http"
  }

  health_check {
    healthy_threshold = 2
    unhealthy_threshold = 2
    timeout = 5
    target = "HTTP:80/status"
    interval = 30
  }

  # The LB will point to all nodes
  instances = ["${aws_instance.pipeline-api.*.id}"]
  cross_zone_load_balancing = true
  connection_draining = false
}

resource "aws_route53_record" "pipeline-api" {
  zone_id = "${var.aws_private_dns}"
  name = "${var.instance_prefix}.${var.aws_group}-${var.environment}.moveaws.com"
  type = "CNAME"
  ttl = "300"
  records = ["${aws_elb.pipeline-api.dns_name}"]
}

resource "aws_iam_role_policy" "pipeline-api-ecs" {
  name = "${var.instance_prefix}-ecs"
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:Describe*",
        "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
        "elasticloadbalancing:Describe*",
        "elasticloadbalancing:RegisterInstancesWithLoadBalancer"
      ],
      "Resource": "*"
    }
  ]
}
EOF
}

resource "aws_iam_role" "pipeline-api-ecs" {
  name = "${var.instance_prefix}-ecs"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
      "Service": "ecs.amazonaws.com"
    },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF
}

resource "aws_ecs_cluster" "pipeline-api" {
  name = "${var.instance_prefix}"
}

resource "aws_ecs_task_definition" "pipeline-api" {
  family = "${var.instance_prefix}"
  container_definitions = "${file("task-definitions/pipeline-api.json")}"
}

resource "aws_ecs_service" "pipeline-api" {
  name = "${var.instance_prefix}"
  task_definition = "${aws_ecs_task_definition.pipeline-api.arn}"
  desired_count = "${var.instance_count}"
  iam_role = "${aws_iam_role.pipeline-api-ecs.arn}"

  load_balancer {
    elb_name = "${aws_elb.pipeline-api.id}"
    container_name = "${var.container_name}"
    container_port = "${var.container_port}"
  }
}

output "load balancer dns" {
  value = "\n${aws_elb.pipeline-api.dns_name}"
}

output "instance dns" {
  value = "\n${join("\n", aws_instance.pipeline-api.*.private_dns)}"
}

Jef Statham

unread,
Oct 30, 2015, 12:03:32 PM10/30/15
to Terraform
Renaming my task and deleting the service from .tfstate fixed it for me. Though I wouldn't normally want to rename a task it just happened to be incorrectly named for me at the time I hit that error.
Reply all
Reply to author
Forward
0 new messages