I dont think you did anything wrong.
I think what you are looking for is something like my split-ssh-agent -
This allows you to have multiple keys, allocated as you will between different
agents on the ssh back-end.
From each calling qube, you specify (in policy) what agent should be
called, and this is passed through to the ssh back-end to serve up the
appropriate keys.
You can find it at
https://github.com/unman/qubes-ssh-agent or a
packaged version for easy installation at
https://qubes.3isec.org/tasks.html
If you dont use it, it should give you one idea of how you might go on.