If you want to decide which scheduler should handle a pod, the most expedient way to do that is to make an admission webhook which decides.
If we permitted that field to change, it would cause races in all existing schedulers, which don't expect pods to be taken away from them, meaning it would be unpredictable which scheduler actually acted on the pod.
Your intention of handing a pod off to another scheduler wouldn't have that problem, but we can't guarantee people would use the ability only for that.
Would this actually be a big deal? IMO, not really, but there's an extremely high bar for changing pods, so at a minimum we'd need to know why the webhook approach doesn't work for you.