Simple Vault / Consul setup. Cannot get TLS to work

[b...@esteves.me]

I am trying to get a simple Vault setup with Consul as storage, with TLS working, and after scouring the internet for the answers I am asking for help :)

I have managed to get Vault in a non HA setup working with Consul with TLS disabled. 

After adding my CA certs to my config (full chain cert with root, intermediate and cert), removing the TLS disable, and updating the DNS / IP's in my config I am no longer able to unseal the vault and am not sure why exactly.

storage "consul" {

            address = "127.0.0.1:8500"

            advertise_addr = "https://<my.domain>:8200"

            path = "vault"

        }

listener "tcp" {

            address = "0.0.0.0:8200"

            tls_cert_file = "/home/ubuntu/certs/server.pem"

            tls_key_file = "/home/ubuntu/certs/server.key"

        }

Trying the above with "vault unseal" I get the below error. 

Error checking seal status: Get https://127.0.0.1:8200/v1/sys/seal-status: x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs

If i try and put in my public IP / DNS into the listener address field then restart the vault, I get

Error initializing listener of type tcp: listen tcp <my ip>:8200: bind: cannot assign requested address

So I am not sure how to proceed. 

The setup is very simple on an AWS EC2 instance with Consul and the vault running on the same server. No Load balancers or HA. I am sure I am just missing something silly here...

I am also not entirely sure what the command `export VAULT_ADDR=http://0.0.0.0:8200` is for and if that somehow is a cause / solution to my problems. 

Any help would be very welcome. 

[Jeff Mitchell]

Hi Ben,

The error you're getting is because you're trying to connect to Vault via 127.0.0.1 but the certificate you're using doesn't contain that IP address in its subject alternate names. You'd have to see which addresses/host names are contained in the cert and either acquire one that contains 127.0.0.1 in its IP SANs or use a host name.

You can see what's in your cert currently by using `openssl x509 -in <path_to_pem_file> -noout -text`

Best,

Jeff

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Vault" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/eb42c8ce-6f9c-45fb-b316-d3f6492cb855%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[b...@esteves.me]

Hi Jeff,

thanks for the reply. So what I would rather do is use my hostname, which I did try. When I do so though, I get the error that I mentioned in my initial post,

Error initializing listener of type tcp: listen tcp <my ip>:8200: bind: cannot assign requested address

So I am not sure how to get around this. Any ideas?

To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+...@googlegroups.com.

[Jeff Mitchell]

Hi Ben,

Can you share your ip address? If so, can you paste the output of 'ip addr show' and 'netstat -tln'?

Thanks,

Jeff

To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/67c85d2a-5a18-4523-8e52-1dd3566badb6%40googlegroups.com.

[PePe Amengual]

isn't this a simple problem in your host table to point 127.0.0.1 to your my.domain vault address ?

are you using a self sign cert ?

[b...@esteves.me]

Don't see a problem sharing the address as this is just a POC. 

Here is the output from the commands. 

ip addr show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000

    link/ether 06:d4:27:3d:9b:b6 brd ff:ff:ff:ff:ff:ff

    inet 172.31.35.166/20 brd 172.31.47.255 scope global eth0

       valid_lft forever preferred_lft forever

    inet6 fe80::4d4:27ff:fe3d:9bb6/64 scope link

       valid_lft forever preferred_lft forever

netstat -tln

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address           Foreign Address         State

tcp        0      0 127.0.0.1:8500          0.0.0.0:*               LISTEN

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN

tcp        0      0 127.0.0.1:8600          0.0.0.0:*               LISTEN

tcp6       0      0 :::8302                 :::*                    LISTEN

tcp6       0      0 :::22                   :::*                    LISTEN

tcp6       0      0 :::8300                 :::*                    LISTEN

tcp6       0      0 :::8301                 :::*                    LISTEN

and here is what I get when I try and start the vault with the listener address set to my hostname (esteves.me being the hostname)

storage "consul" {

            address = "127.0.0.1:8500"

            advertise_addr = "https://esteves.me:8200"

            path = "vault"

        }

listener "tcp" {

            address = "esteves.me:8200"

            tls_cert_file = "/home/ubuntu/certs/server.pem"

            tls_key_file = "/home/ubuntu/certs/server.key"

        }

sudo vault server -config=/etc/vault.d/config.hcl

Error initializing listener of type tcp: listen tcp 18.195.167.251:8200: bind: cannot assign requested address

I am not seeing my public IP listed in any of the above, so I am guessing that may be an issue, perhaps related to iptables like Pepe Suggested... Its all on AWS, which is a bit new to me, so perhaps I need to enable something on the AWS end?

And no, it's not a self signed cert. 

[PePe Amengual]

advertise_addr = "https://esteves.me:8200" it should not go in the storage section, it should go outside.


your config file should look like this :


{
"disable_mlock": false,
"listener": {
"tcp": {
"address": "0.0.0.0:8200",
"tls_disable": "false",
"tls_cert_file": "/etc/vault/ssl/mycert.crt",
"tls_key_file": "/etc/vault/ssl/mycert.pem"
}
},
"backend": {
"consul": {
"address": "127.0.0.1:8500",
"path": "vault",
"token": "REDACTED"
}
}
}


I hope that helps.

On Friday, 12 January 2018 02:50:29 UTC-8, b...@esteves.me wrote:

[b...@esteves.me]

I tried your above config, but I am still seeing the same error when I try and unseal 

Error checking seal status: Get https://127.0.0.1:8200/v1/sys/seal-status: x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs

esteves.me is my external address though, so while removing it from storage I can understand, should it not go into the address field for the Listener? 

I am sure I am missing something stupid here... 

[PePe Amengual]

Interesting, in our setup we have one more line that I didn't mention :

"api_addr": "https://vault.server.local:443", ( this has to be outside of listener ot backend config sections)

and that is the address vault uses when registering the service and connecting. ( version 0.9.1 I'm using)

The vault.server.local is resolvable by the instance so make sure of adding it to the host table if necessary.

[b...@esteves.me]

I have made a bit of progress, from googling around. I kept my config as above with the changes suggest (minus the recent one with the api_addr) and then instead of vanilla "vault unseal" I specified the address with vault unseal -address "https://esteves.me:8200" and this allowed me to unseal the vault and it seems to be a step in the right direction. 

But now anything I do with regards to the vault requires that I specify the address. Is there a way around this? Why does it default to 127.0.0.1 instead of esteves.me? 

Thanks a lot for the assistance so far! 

[Vishal Nayak]

Hi Ben,

From your previous response, it looks like running a `export
VAULT_ADDR=https://esteves.me:8200` should do the trick.

Regards,
Vishal

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/vault/issues
> IRC: #vault-tool on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Vault" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vault-tool+...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/vault-tool/fda2954d-bfd2-4131-9ac3-9db6cafad2b3%40googlegroups.com.

>
> For more options, visit https://groups.google.com/d/optout.



--

vn

[b...@esteves.me]

Thanks for the suggestion Vishal, I did try this already and it doesn't seem to make any impact. I don't quite understand what that command is supposed to do but it has shown no significant impact anywhere along the process of setting this up. 

I did forget to mention that only when adding esteves.me to my host file under 127.0.0.1, did the above work. Removing this entry causes the unseal to no longer work and I instead get a timeout at the server IP address 

Error checking seal status: Get https://esteves.me:8200/v1/sys/seal-status: dial tcp 18.195.167.251:8200: i/o timeout

[PePe Amengual]

You need to be able to resolve that dns name, there is no way around it otherwise the SSL cert will fail because of the CN not matching.

Anyhow you are not going to interact with vault using the vault cli, your apps will use the API so if you use the API it should work as long as the host sending the request can resolve steves.me and can connect to port 8200.

[b...@esteves.me]

Thats what I don't get. The DNS does resolve. If I do a nslookup / dig on esteves.me it resolves to the IP address of the server. What I don't understand is why the vault insists on trying to connect over 127.0.0.1 when this is not specified anywhere. This seems to be the key to me. 

This whole process is to finally get me to a place where I can connect to this keystore from our Code42 master to store our keys in the vault. The first step in this process is to auth-enable the cert with "vault auth-enable cert" and then it fails again when checking 127.0.0.1, so which I can get the vault unsealed with the above work-around, i am again stuck at this point. 

[PePe Amengual]

in they docs they expect you to export a environment variable so the cli picks up the correct hostname like so :

export VAULT_ADDR="https://steves.me:8200"

then the cli should work if the CERT has the proper CN.

are you sure you exported the variable correctly ? if you run export after setting you see it in the list ?

[b...@esteves.me]

Pepe, I cannot thank you enough. i finally got this to work. There were two problems at work here. One was that I was not using the export command correctly (after starting the vault) and that there was a problem with my security groups in AWS that had blocked port 8200 and so thats why I couldn't unseal on the external hostname. 

Its now working great. Thanks again!

[Jeff Mitchell]

Hi Ben,

You can also use the -address flag with any Vault CLI command to specify a value, but usually using the env var is easier.

Best,

Jeff

To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/59c94c7c-54cb-4095-b6dd-ca66edf16194%40googlegroups.com.

[PePe Amengual]

Glad to hear!!!

Enjoy.

[b...@esteves.me]

Thanks Jeff. I did discover this, and thought it had solved my issues, but as mentioned (I think) it was not an optional flag for the 'vault auth-enable cert' command that I could see.