Well this is weird.
I have two servers running two different Union Apps.
With one of them, I could put the "<allow-access-from domain="My.IP.Address" to-ports="9110" />
and everything was hunky-dory.
With the other one, the stand-alone flash player would fail to connect with that,
but when I put allow-access-from-domain="*" it was ok.
In both cases, using Union Admin from a browser window was fine (with
just that domain in the allow list).
The two servers are on slightly different OSes: first one is Ubuntu 14.
Second one is its close cousin Debian 8. (The reason for that has to with
a Java-logging-date-time issue, which I have discussed in another thread.)
Other than that, I don't know of any differences between them.
As Chris R said, even without this layer of protection, there is the password,
and at such time as a browser version doesn't work, I'll have to rely on that.