I am using the digital clock as well and also noticed that it had changed
its font recently. However I could change it back to my preferred font without
problem, simply via right click -> Properties -> Font.
Maybe just try and change it manually by editing the config in
~/.config/xfce4/xfconf/xfce-perchannel-xml/xfce4-panel.xml?
It might be necessary to do so when logged out of xfce, or it might
overwrite the config again.
cheers
Holger