The following may be hints to guide you, and are also questions to help troubleshoot:
Have you set up TeraTerm to have any Transmit Delay? With CP/M, this may be needed to get reliable communications (same as any other system based on G.Searles version of the CP/M CBIOS).
Does transferring files using
DOWNLOAD.COM work reliably for you?
When trying KERMIT, it is required to use two separate serial ports; one port for command and control, and one port for the data transfer. As a side-note, I have not tested the version of kermit that is included in TeraTerm. I have to give that a test to give you proper answers there, but that will have to wait until after January 1st.
With XModem, I have used lrzsz under Minicom extensively, and tested the TeraTerm XModem. I do experience the occasional issue, but nothing like what you describe.
By the way, I am sorry about the very late response. I try my best to not leave questions to the group go unanswered. These last couple of weeks have been rather busy :)