How to delete openshift cluster provisionned

[dwoj...@redhat.com]

/openshift-install destroy cluster

It could be nice to have this command in the HTML presentation page or into the FAQ.

[Derek Carr]

Agreed. We can get that fixed up.

[Just Marvin]

Hi,

    This command seems to hang after a while. It got to this point:

92cce704e10702bf" id=subnet-092cce704e10702bf

INFO Released                                      arn="arn:aws:ec2:us-east-1:844967694736:elastic-ip/eipalloc-06af0cdf704b1b56c" id=eipalloc-06af0cdf704b1b56c

INFO Released                                      arn="arn:aws:ec2:us-east-1:844967694736:elastic-ip/eipalloc-0a20a533002c7dd16" id=eipalloc-0a20a533002c7dd16

INFO Deleted                                       arn="arn:aws:ec2:us-east-1:844967694736:subnet/subnet-0d40262037f773693" id=subnet-0d40262037f773693

INFO Deleted                                       arn="arn:aws:ec2:us-east-1:844967694736:subnet/subnet-054fbe53687951a86" id=subnet-054fbe53687951a86

INFO Released                                      arn="arn:aws:ec2:us-east-1:844967694736:elastic-ip/eipalloc-08c63287b9d3f22cf" id=eipalloc-08c63287b9d3f22cf

INFO Released                                      arn="arn:aws:ec2:us-east-1:844967694736:elastic-ip/eipalloc-058af9157ec1b9146" id=eipalloc-058af9157ec1b9146

INFO Deleted                                       arn="arn:aws:ec2:us-east-1:844967694736:subnet/subnet-0f12cfc43d3ad73c4" id=subnet-0f12cfc43d3ad73c4

INFO Deleted                                       arn="arn:aws:ec2:us-east-1:844967694736:subnet/subnet-051e8f10d3737e75d" id=subnet-051e8f10d3737e75d

   .....and has stayed there for the past 15 mins at least. The ec2 instances have not been deleted.

Regards,

Marvin

[W. Trevor King]

On Sun, Mar 17, 2019, 10:05 Just Marvin  wrote:

    This command seems to hang after a while. It got to this point:

...

INFO Deleted                                       arn="arn:aws:ec2:us-east-1:844967694736:subnet/subnet-051e8f10d3737e75d" id=subnet-051e8f10d3737e75d

   .....and has stayed there for the past 15 mins at least. The ec2 instances have not been deleted.

The debug-level logs in $ASSET_DIR/.openshift_install.log will show what the destroyer is stuck on.

Cheers,

Trevor

[Just Marvin]

Trevor,

time="2019-03-17T12:49:09-04:00" level=debug msg="deleting arn:aws:ec2:us-east-1:84

4967694736:subnet/subnet-0be119054e70bad67: DependencyViolation: The subnet 'subnet

-0be119054e70bad67' has dependencies and cannot be deleted.\n\tstatus code: 400, re

quest id: 79b5995a-31ed-4aa2-91dc-6aeeea39ff16"

time="2019-03-17T12:49:09-04:00" level=debug msg="deleting arn:aws:ec2:us-east-1:84

4967694736:dhcp-options/dopt-00243b36867e35769: DependencyViolation: The dhcpOption

s 'dopt-00243b36867e35769' has dependencies and cannot be deleted.\n\tstatus code: 

400, request id: bc4704b7-098d-4a77-94cf-89068b1abb0a"

time="2019-03-17T12:49:10-04:00" level=debug msg="deleting arn:aws:ec2:us-east-1:84

4967694736:security-group/sg-08dd35c3cd8f439d1: DependencyViolation: resource sg-08

dd35c3cd8f439d1 has a dependent object\n\tstatus code: 400, request id: dc3d1f68-85

33-4783-94a3-d26483c30619"

time="2019-03-17T12:49:11-04:00" level=debug msg="deleting arn:aws:ec2:us-east-1:84

4967694736:vpc/vpc-0c762e4748f7573b5: DependencyViolation: The vpc 'vpc-0c762e4748f

7573b5' has dependencies and cannot be deleted.\n\tstatus code: 400, request id: cd

c70b27-ac51-4b72-b7df-3a116125a3ac"

time="2019-03-17T12:49:11-04:00" level=debug msg="deleting arn:aws:ec2:us-east-1:84

4967694736:volume/vol-02a101c85b27384cb: VolumeInUse: Volume vol-02a101c85b27384cb 

is currently attached to i-0e73c668b8d98be87\n\tstatus code: 400, request id: dc75d

055-4a46-4f9a-96f3-4f9b7b537554"

time="2019-03-17T12:49:11-04:00" level=debug msg="deleting arn:aws:ec2:us-east-1:84

4967694736:security-group/sg-0ce32c25855697df0: DependencyViolation: resource sg-0c

e32c25855697df0 has a dependent object\n\tstatus code: 400, request id: 86b7e0ed-f8

81-4036-b99d-843ace669c2b"

time="2019-03-17T12:49:11-04:00" level=debug msg="deleting arn:aws:ec2:us-east-1:84

4967694736:subnet/subnet-0125e4539cba4e4a1: DependencyViolation: The subnet 'subnet

-0125e4539cba4e4a1' has dependencies and cannot be deleted.\n\tstatus code: 400, re

quest id: 1421aa7d-3e05-489a-804b-7be05e6c0935"

time="2019-03-17T12:49:11-04:00" level=debug msg="search for and delete matching re

sources by tag in us-east-1 matching aws.Filter{\"openshiftClusterID\":\"33cf7a43-2

ec1-44d6-873c-524d0cb106cc\"}"

time="2019-03-17T12:49:11-04:00" level=debug msg="search for IAM roles"

time="2019-03-17T12:49:12-04:00" level=debug msg="search for IAM users"

time="2019-03-17T12:49:12-04:00" level=debug msg="search for and delete matching re

sources by tag in us-east-1 matching aws.Filter{\"kubernetes.io/cluster/gatt-dp5zd\

":\"owned\"}"

    I suspect that this is due to the fact that I created a bastion host in the VPC and gave it an elastic IP address. But after this point, it seems to be going in loop searching for those IAM roles. About 1.5 hours later, I see this on the screen:

INFO RequestError: send request failed

caused by: Post https://elasticloadbalancing.us-east-1.amazonaws.com/: dial tcp: lookup elasticloadbalancing.us-east-1.amazonaws.com on 192.168.122.1:53: server misbehaving  arn="arn:aws:ec2:us-east-1:844967694736:vpc/vpc-0c762e4748f7573b5" id=vpc-0c762e4748f7573b5

INFO get tagged resources: RequestError: send request failed

caused by: Post https://tagging.us-east-1.amazonaws.com/: dial tcp: lookup tagging.us-east-1.amazonaws.com on 192.168.122.1:53: server misbehaving 

    Not sure what thats all about. Can't seem to match those events up to anything in the log.

Regards,

Marvin

[W. Trevor King]

On Sun, Mar 17, 2019, 11:54 Just Marvin wrote:

time="2019-03-17T12:49:11-04:00" level=debug msg="deleting arn:aws:ec2:us-east-1:84

4967694736:volume/vol-02a101c85b27384cb: VolumeInUse: Volume vol-02a101c85b27384cb 

is currently attached to i-0e73c668b8d98be87\n\tstatus code: 400, request id: dc75d

055-4a46-4f9a-96f3-4f9b7b537554"

...

time="2019-03-17T12:49:11-04:00" level=debug msg="deleting arn:aws:ec2:us-east-1:84

4967694736:subnet/subnet-0125e4539cba4e4a1: DependencyViolation: The subnet 'subnet

-0125e4539cba4e4a1' has dependencies and cannot be deleted.\n\tstatus code: 400, re

quest id: 1421aa7d-3e05-489a-804b-7be05e6c0935"

...

time="2019-03-17T12:49:12-04:00" level=debug msg="search for and delete matching re

sources by tag in us-east-1 matching aws.Filter{\"kubernetes.io/cluster/gatt-dp5zd\

":\"owned\"}"

    I suspect that this is due to the fact that I created a bastion host in the VPC and gave it an elastic IP address.

Yup.  You need to reap any dependencies you created yourself, if they'd block the removal of cluster resources.  Or tag your resources with kubernetes.io/cluster/... so the search above finds them, in which case the destroy code will remove them for you.

But after this point, it seems to be going in loop searching for those IAM roles.

The IAM stuff is just because the AWS tag-search API does not support those resource types.  The destroy code is looping between tag searches and IAM iteration hoping to find the resource blocking deletion of the remainder of your dependency tree.

About 1.5 hours later, I see this on the screen:

INFO RequestError: send request failed

caused by: Post https://elasticloadbalancing.us-east-1.amazonaws.com/: dial tcp: lookup elasticloadbalancing.us-east-1.amazonaws.com on 192.168.122.1:53: server misbehaving  arn="arn:aws:ec2:us-east-1:844967694736:vpc/vpc-0c762e4748f7573b5" id=vpc-0c762e4748f7573b5

INFO get tagged resources: RequestError: send request failed

caused by: Post https://tagging.us-east-1.amazonaws.com/: dial tcp: lookup tagging.us-east-1.amazonaws.com on 192.168.122.1:53: server misbehaving 

    Not sure what thats all about.

Yeah, I dunno what that's about either.

Cheers,

Trevor

[Just Marvin]

Trevor,

    I tried the tagging approach. It deleted more stuff, but it still got stuck. In particular, I noted that the code was not handling the delete of the dhcp option sets correctly. I had to go into the VPC and set it manually to "no dhcp option sets" before it got deleted.

Regards,

Marvin