Typically how this is done is to replace the Svc/GroundInterface component's implementation with one that uses your packet format. We are working to make this simpler in the future, but haven't gotten there yet. I will agree, we need a document to describe this process. I'll take that action.
F´ uses an custom format between the FSW and ground in order to be simple and easy to use for testing. Each project's requirements for operational communication are different so we leave it up to the project to implement their packeting scheme as the choose. This usually involves re-implementing the ground interface and communication components (SocketIpDriver for example) for their specific setup.
-Michael