The following command starts GNU Chess in engine mode using the xboard protocol, but it will not start the XBoard program itself (the graphical user interface):
gnuchess --xboard
If you want to start XBoard and make it use GNU Chess as chess engine, you have to run this command:
xboard -fcp 'gnuchess --xboard'
The manual of GNU Chess explains it:
Antonio Ceballos
GNU Chess Team