Did you install the Qubes USB Proxy? You need that to use qvm-usb.
Some relevant background knowledge might be due first. For starts, sys-usb in and on itself adds no features, no functionality, it's specifically and purely a self-defense mechanism to protect dom0, nothing more, nothing less. It does however move all your USB to sys-usb, giving you a means to use USB the same way, as if it was used in dom0.
The USB Proxy, however, does add some functionality, and it can be installed in whichever VM you keep your USB Controllers. Be it sys-usb or your wibbly-wobbly-timey-wimey VM, in other words, it doesn't matter where, as long as it is kept safely away from dom0. If you use USB keyboard or USB mouse, however, you need to be careful you don't lock yourself out of your system, especially if sys-usb has automatic start on boot. If USB is the only input you have for keyboard/mouse, then be careful of what you do, or at least make a backup of your system first, just in case you make a mistake.
https://www.qubes-os.org/doc/usb/
Go here, you don't need the full guide. Just scroll (or cftl+f to search) for the headline containing "Qubes-USB-Proxy", it's quite a bit down the page near the bottom.
Once you installed the Qubes Proxy package, you can go the the next headline, which shows you how to use it.
Keep in mind, you need to type this in every time you need to switch it to another VM, or if you stop/start your VM and need it again. This is however far, far easier in Qubes 4, which has a widget that allows for this with 3 small quick clicks of your mouse. So this becomes much easier in Qubes 4, and it's likely not too far from final release now.
You could however make it easy in Qubes 3.2. if you use the same few VM's for the USB. For example you can write a small simple script, and then simply keybind the script with "qvm-run sys-usb bash 'path-to-your-'qvm-usb'-script".
You execute qvm-run in dom0, and you execute qvm-usb in your sys-usb (or whichever VM yo use). To keybind, go to Qubes menu ---> System Tools --> Keyboard settings --> Shortcuts tab --> Click "Add", and type in the qvm-run command.
For example you can pass your Yubi-key to VM-A with Ctrl+Shift+Alt+A or your VM-C with Ctrl+Shift+Alt+K. Whatever you can imagine or desire, the Ctrl+Shift+Alt is nice because it's easy to just hold all 3 keys down without worring about which one to holddown, while also not causing many keybind conflicts.