It's relative to the Moment Reference Position, Zref.
I also checked VSPAERO by setting ground height to 2.0 and the CG height to zero. With a default wing placed at Z = 0, the CL is about 0.83 for 10 degree alpha and Mach 0.1 / Re 1e6. If I then move the wing to Z = 5 and run again without changing the ground height or CG height, the CL drops to about 0.78. If I then change the CG height to 5 (wing Z and CG Z delta is back to zero) and run again, I get the original 0.83 CL.
So the ground effect is defined relative to the CG_ref and will apply it appropriately to all geometries and their relative distances to the CG. I did some more digging into the code and found where VSPAERO applies ground effect height. The calculation is made to change the Z position of every vortex plate in the system. So when Dave warns in the code not to rotate your geometry into the ground, he expects you to know how that could happen.