If the IP address you want is the hosts primary address you could access it via the ansible_default_ipv4['address'] var.
If you want to make sure the var you are using is the same you are also using to connect via ssh try using the annsible_ssh_host.
Also you should be able to use the SSH_CONNECTION and just extract the IP like so:
my_ip = "{{ ansible_env.ssh_connection.split(' ')[0]}}"
(can't test the syntax right now, but it should work)