Make sure template has these packages installed:
qubes-usb-proxy bluez blueman pulseaudio-module-bluetooth
Start the VM. Connect the dongle to the VM. Put the speaker/headphones in pairing mode. Open Bluetooth Devices and click search. Let the search finish. Do NOT connect to it yet.Run these commands:
pulseaudio -k
pulseaudio -D
Now connect the speaker/headphones. It should pair and connect successfully. Run rhythmbox, sound should be routed to the headphones.