I notice Swift Playground now includes hooks to external accessories like Sphero and Mindstorms. Have you every considered this option, Steve? (In all your spare time, of course)
I'm not even sure it would provide much benefit over Scratchx except that it works on the iPad. Perhaps someone else familiar with it could do this via bluetooth commands. I could look at it but I suspect it is beyond my limited programming skills.