So I found part of the problem.
First the set_fact only seemed to work when using the second format in the documentation
- set_fact:
var2: "something"
when: var2 is not defined
second I found that if the set_fact runs in a role once and you run that role multiple times with different variable the set_fact will hold over until it is run at that level again.
For example:
roles:
- { role: test_role, var1: 'some string', var2: "{{ some_var }}" }
- { role: test_role, var1: 'some string', var2: "{{ some_var }}", var3: "{{ some_var2 }}" }
In the first run var3 gets set to 'something'
on the second run even thought I tell it to set var3 to {{some_var2}} it seems to remember that at that level it was previously set to 'something' and that local setting takes president as if it was set that time around as well (even though it isn't)