Even though there is no chance to a job, as soon as the job builder runs all jobs are updated every time.
I though applying the fix discussed here would help:
https://storyboard.openstack.org/#!/story/2011557
But it is still updating everything on a run making the job builder essentially useless cause config is always reset and you need to click "build now" before seeing "build with parameters"