I'm not sure how you would do that, it's likely easy to tunnel kvm with some extra qemu parameters but Packer also uses qemu-img which I think only works with local files.
On positive side is that packer is just a statically linked binary so you can easily create a script that you execute over ssh that uploads any source files, downloads packer, executes it, and export any artifact, and finally cleans up.
// Rickard
PS. If you in detail could describe how to execute qemu towards a remote kvm host it might be worth opening a feature request and someone might implement it.