in the reference pdb just put the atoms you need.
So, if I have the pdb of the target (full system, same number of atoms that you have in the MD) I grep out CA and redirect to a pdb
for example:
grep CA my_target_pdb_with_all_atoms.pdb >target.pdb
and then I make sure that the last two fields of beta and occupancy contain 1.00 1.00 (this means that you will use the atoms to align and measure the rmsd. In plumed you can separate the two sets).
and your line in plumed will be
RMSD REFERENCE=target.pdb TYPE=OPTIMAL
Note that dumping out the atoms from a vmd selection will not work since VMD is renumbering the atoms while plumed takes the index of the atom to match the target coordinates with the right atom in the MD system.
Cheers
D