Pion is perfect, no need to improve it.
Yeah, this only works on the browser side. IPFS leaves a lot to be desired for programmatic use. And it appears the JS side doesn't speak to the Go side yet. I appreciate the idea of using a DHT for signaling, but I haven't found a good impl that works in the browser and Go. While bittorrent mainline DHT would be reasonable, even webtorrent doesn't seem to write to it properly from the client side.
So I've kinda abandoned using IPFS for signaling until JS IPFS and Go IPFS do better. However I did find
https://www.websocket.in/ which is a wonderful service and one that I would like to see the big guys (Google, Mozilla, CloudFlare, etc) emulate. I am a bit disappointed that my research has shown so few publicly available signaling options. We don't even need ephemeral state, just a pipe that doesn't require sign up...lock it down if you need to, or at first don't even allow any bytes (e.g. encrypted offers/answers) and parse the SDP and put limits on it.
I have been considering taking several signaling approaches and wrapping them in an abstraction so people could have fallbacks. Sadly there aren't that many options available right now so it's fruitless.