Vault-ssh-helper setup issues

[Michael Clounie]

Hi there,

Before jumping in I want to verify that the correct setup for OTP ssh is having a vault server with ssh secrets engine enabled etc. and the helper installed on all servers that will be accessed. I have read some posts that make it seem like you need to install a vault server on every host you are looking to access. This isn't the case is it?

All servers are Ubuntu 16.04 hosted in AWS.

I can successfully ssh from the managed server to the vault server and vice versa. All traffic is open on 8200 as well both ways.

Steps Taken ::

Vault Server (already configured/unsealed/TLS is disabled)

vault secrets enable -path ssh ssh

vault write ssh/roles/otp_key_role key_type=otp default_user=ubuntu cidr_list=xx.xx.xxx.xx/32 (IP of server to be accessed)

Managed Server

wget 'https://releases.hashicorp.com/vault-ssh-helper/0.1.4/vault-ssh-helper_0.1.4_linux_386.zip'

unzip vault-ssh-helper_0.1.4_linux_386.zip

I modified all config files as shown below, restarted ssh and sshd, then ran the test command which always times-out :: vault-ssh-helper -config /home/ubuntu/vault-helper-config.hcl -dev -verify-only

Configuration Files on managed server ::

/home/ubuntu/vault-helper-config.hcl

vault_addr = "http://xx.xxx.xx.xx:8200" (Vault server address / listener)

ssh_mount_point = "ssh"

tls_skip_verify = true

allowed_roles = "*"

/etc/pam.d/sshd

# PAM configuration for the Secure Shell service

# Standard Un*x authentication.

#@include common-auth

auth requisite pam_exec.so quiet expose_authtok log=/tmp/vaultssh.log /home/ubuntu/vault-ssh-helper -config=/home/ubuntu/vault-helper-config.hcl -dev

auth optional pam_unix.so not_set_pass use_first_pass nodelay

# Disallow non-root logins when /etc/nologin exists.

account    required     pam_nologin.so

# Uncomment and edit /etc/security/access.conf if you need to set complex

# access limits that are hard to express in sshd_config.

# account  required     pam_access.so

# Standard Un*x authorization.

@include common-account

# SELinux needs to be the first session rule.  This ensures that any

# lingering context has been cleared.  Without this it is possible that a

# module could execute code in the wrong domain.

session [success=ok ignore=ignore module_unknown=ignore default=bad]        pam_selinux.so close

# Set the loginuid process attribute.

session    required     pam_loginuid.so

# Create a new session keyring.

session    optional     pam_keyinit.so force revoke

# Standard Un*x session setup and teardown.

@include common-session

# Print the message of the day upon successful login.

# This includes a dynamically generated part from /run/motd.dynamic

# and a static (admin-editable) part from /etc/motd.

session    optional     pam_motd.so  motd=/run/motd.dynamic

session    optional     pam_motd.so noupdate

# Print the status of the user's mailbox upon successful login.

session    optional     pam_mail.so standard noenv # [1]

# Set up user limits from /etc/security/limits.conf.

session    required     pam_limits.so

# Read environment variables from /etc/environment and

# /etc/security/pam_env.conf.

session    required     pam_env.so # [1]

# In Debian 4.0 (etch), locale-related environment variables were moved to

# /etc/default/locale, so read that as well.

session    required     pam_env.so user_readenv=1 envfile=/etc/default/locale

# SELinux needs to intervene at login time to ensure that the process starts

# in the proper default security context.  Only sessions which are intended

# to run in the user's context should be run after this.

session [success=ok ignore=ignore module_unknown=ignore default=bad]        pam_selinux.so open

# Standard Un*x password updating.

@include common-password

/etc/ssh/sshd_config

# Package generated configuration file

# See the sshd_config(5) manpage for details

# What ports, IPs and protocols we listen for

Port 22

# Use these options to restrict which interfaces/protocols sshd will bind to

#ListenAddress ::

#ListenAddress 0.0.0.0

Protocol 2

# HostKeys for protocol version 2

HostKey /etc/ssh/ssh_host_rsa_key

HostKey /etc/ssh/ssh_host_dsa_key

HostKey /etc/ssh/ssh_host_ecdsa_key

HostKey /etc/ssh/ssh_host_ed25519_key

#Privilege Separation is turned on for security

UsePrivilegeSeparation yes

# Lifetime and size of ephemeral version 1 server key

KeyRegenerationInterval 3600

ServerKeyBits 1024

# Logging

SyslogFacility AUTH

LogLevel INFO

# Authentication:

LoginGraceTime 120

PermitRootLogin yes

StrictModes yes

RSAAuthentication yes

PubkeyAuthentication yes

#AuthorizedKeysFile %h/.ssh/authorized_keys

# Don't read the user's ~/.rhosts and ~/.shosts files

IgnoreRhosts yes

# For this to work you will also need host keys in /etc/ssh_known_hosts

RhostsRSAAuthentication no

# similar for protocol version 2

HostbasedAuthentication no

# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication

#IgnoreUserKnownHosts yes

# To enable empty passwords, change to yes (NOT RECOMMENDED)

PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with

# some PAM modules and threads)

ChallengeResponseAuthentication yes

# Change to no to disable tunnelled clear text passwords

PasswordAuthentication no

# Kerberos options

#KerberosAuthentication no

#KerberosGetAFSToken no

#KerberosOrLocalPasswd yes

#KerberosTicketCleanup yes

# GSSAPI options

#GSSAPIAuthentication no

#GSSAPICleanupCredentials yes

X11Forwarding yes

X11DisplayOffset 10

PrintMotd no

PrintLastLog yes

TCPKeepAlive yes

#UseLogin no

#MaxStartups 10:30:60

#Banner /etc/issue.net

# Allow client to pass locale environment variables

AcceptEnv LANG LC_*

Subsystem sftp /usr/lib/openssh/sftp-server

# Set this to 'yes' to enable PAM authentication, account processing,

# and session processing. If this is enabled, PAM authentication will

# be allowed through the ChallengeResponseAuthentication and

# PasswordAuthentication.  Depending on your PAM configuration,

# PAM authentication via ChallengeResponseAuthentication may bypass

# the setting of "PermitRootLogin without-password".

# If you just want the PAM account and session checks to run without

# PAM authentication, then enable this but set PasswordAuthentication

# and ChallengeResponseAuthentication to 'no'.

UsePAM yes

AuthorizedKeysCommand /opt/authorized_keys_command.sh

AuthorizedKeysCommandUser nobody

 

Thank you for any help! 

[Vishal Nayak]

Hi Michael,

All the managed servers will only have vault-ssh-helper installed and
will not be running Vault servers.

Regards,
Vishal

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/vault/issues
> IRC: #vault-tool on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Vault" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vault-tool+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/vault-tool/51550232-26a7-4f2d-9011-6b3972c4da8d%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
vn

[Michael Clounie]

Ok thank you, thought so. Given the setup above do you see anything wrong that would cause the timeout issue?

[Vishal Nayak]

Hi Michael,

It is possible that vault-ssh-helper in the managed server is not be
able to reach the vault server over the configured "vault_addr".
Are you able to see any helpful information in "/tmp/vaultssh.log"?

Regards,
Vishal

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/vault/issues
> IRC: #vault-tool on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Vault" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vault-tool+...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/vault-tool/47b34819-e6e8-49ba-9ca7-90bd40dd3c7f%40googlegroups.com.

[Michael Clounie]

Currently, there is not vaultssh.log in /tmp on either server.

[Vishal Nayak]

Hi Michael,

The /tmp/vaultssh.log will be populated only when you try to use the
configured PAM settings and attempt a "vault ssh". It won't be there
if you have not done that and if you are just running the
vault-ssh-helper binary from the terminal. I don't see a reason other
than it not being able to reach the vault server from the managed
server. I'd expect some logs in the terminal at least.

You might be able to ssh into the vault server from the managed
server. But, that might not mean that the vault-ssh-helper is able to
reach vault server on the configured vault_addr.
vault-ssh-helper reads the supplied configuration and tries to connect
to vault server. This is the only action that I can think of could be
causing the timeout.

Regards,
Vishal

On Tue, Feb 6, 2018 at 1:15 PM, Michael Clounie
<michael...@nextworld.net> wrote:
> Currently, there is not vaultssh.log in /tmp on either server.
>

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/vault/issues
> IRC: #vault-tool on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Vault" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vault-tool+...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/vault-tool/2f9bf3b5-f2e6-4659-bd7d-6e182999a962%40googlegroups.com.