To clarify some terms:
'vault password': A password used by vault to encrypt/decrypt content.
'passwords stored in vault': A password for something like a user auth or ssh private keys, that is stored in a vault (encrypted with ansible-vault and a vault password)
--vault-password-file is for reading a 'vault password' from a file. ie, a password used by vault
for decrypting/encrypting.
ansible_ssh_pass would need to be populated from a variable in a vault encrypted vars file (or inventory file) for that example.
But that is the wrong approach.
The right approach is:
Use ssh keys and ssh-agent instead of trying to store the password in ansible.
A working key based ssh and ssh-agent setup is a prerequisite for practical ansible use.