In case someone faces this. The current workaround is to specify explicitly
v.ssh_info_public = true
v.base_address = "....."
v.base_mac = "00:00:00:ff:ff:ff"
as suggested by Chris Roberts (HashiCorp). Thanks!
In case multiple vms are used then those values need to be adjusted per vm.