New version of my Xmodem program

[Martin Eberhard]

I've just released version 2.9 of my Xmodem file transfer program for CP/M. This is a minor tweak on version 2.8: the receiver is now flushed of any garbage characters before receiving a file. (Previous versions would get hung up due to leftover characters in the UART's buffers if you had aborted a file transfer before trying to transfer a new file.)

You can find the new version in my directory that's hosted on Mike Douglas's website, here:

https://deramp.com/downloads/mfe_archive/005-Documentation%20and%20Code%20by%20Martin/010%20Code%20by%20Martin/CPM%20Code/XMODEM/

There's also a handy little program for confirming the functionality of your serial link, called XMTERM, here:

https://deramp.com/downloads/mfe_archive/005-Documentation%20and%20Code%20by%20Martin/010%20Code%20by%20Martin/CPM%20Code/XMTERM/

XMTERM is a simple terminal program that uses the same configuration file (XMODEM.CFG) as does the Xmodem program, so it tests the serial link as well as the config file setup.

Cheers,
Martin

[Phillip Stevens]

> I've just released version 2.9 of my Xmodem file transfer program for CP/M. This is a minor tweak on version 2.8: the receiver is now flushed of any garbage characters before receiving a file. (Previous versions would get hung up due to leftover characters in the UART's buffers if you had aborted a file transfer before trying to transfer a new file.)

Hi Martin,

I've been using your new version 2.9 on my CP/M machine.

I assemble it with DRI MAC, and load with MLOAD.
Very simple build. Works great.

Because my CP/M BIOS serial handling doesn't touch BC or DE, I've removed that protection from GOBIOS (which takes quite a few cycles out of the transmit and receive loops).

What I'm finding is that the 2.9 version pauses and waits while sending to CP/M much more often than the 2.7 version. The actual throughput is quite a bit lower.

There are some notes in your 2.8 release related to this, and I wonder if by "speeding up" the GOBIOS routine I've in-fact broken the carefully choreographed timing of buffer read and write? Would that be the case?

Cheers, Phillip

[Phillip Stevens]

> > I've just released version 2.9 of my Xmodem file transfer program for CP/M. This is a minor tweak on version 2.8: the receiver is now flushed of any garbage characters before receiving a file. (Previous versions would get hung up due to leftover characters in the UART's buffers if you had aborted a file transfer before trying to transfer a new file.)

> I assemble it with DRI MAC, and load with MLOAD. Very simple build. Works great.

> Because my CP/M BIOS serial handling doesn't touch BC or DE, I've removed that protection from GOBIOS (which takes quite a few cycles out of the transmit and receive loops). What I'm finding is that the 2.9 version pauses and waits while sending to CP/M much more often than the 2.7 version. The actual throughput is quite a bit lower.

> There are some notes in your 2.8 release related to this, and I wonder if by "speeding up" the GOBIOS routine I've in-fact broken the carefully choreographed timing of buffer read and write? Would that be the case?

Hi Martin,

I've done some experiments with 2.9, by calculating the correct timings for CRTIME (from my bios) and adjusting for the revised GOBIOS cycles in CONTO (& RDRTO). I've tried with different CPUMHZ values. I've set WBPERX to 252. No matter what I do, I'm still getting the "Retry 0: NAK on sector" message from my PC end (moserial 3.0.12) very often, and substantially more often than with 2.7.

Previously (2.7) the CP/M receive direction would fill RAM (approx 54kB) without showing a NAK and then write to disk and repeat, but now (2.9) it only does a few kB before showing repeated NAKs and is very choppy and slow. I replicate this on the same machine receiving the same file, so the only change is xmodem.

Any thoughts on what has changed which might cause this different behaviour?

Cheers, Phillip

[Tony Nicholson]

On Thursday, March 17, 2022 at 3:36:28 PM UTC+11, Phillip wrote:

> Previously (2.7) the CP/M receive direction would fill RAM (approx 54kB) without showing a NAK and then write to disk and repeat, but now (2.9) it only does a few kB before showing repeated NAKs and is very choppy and slow. I replicate this on the same machine receiving the same file, so the only change is xmodem.
>
> Any thoughts on what has changed which might cause this different behaviour?

Phillip,

I found that XMODEM sometimes does a NAK if the CP/M file writing takes
too long (the timeout settings are hard to get right on faster Z80/Z280
systems). The workaround for me was to reduce the amount of in-memory
buffering.

Version 2.9 as a new setting /K that allows you to select this. I put /K20
in my XMODEM.CFG file to limit it to 20Kbytes on my problematic
system.

Tony

[Phillip Stevens]

> > Phillip wrote:
> > Previously (2.7) the CP/M receive direction would fill RAM (approx 54kB) without showing a NAK and then write to disk and repeat, but now (2.9) it only does a few kB before showing repeated NAKs and is very choppy and slow. I replicate this on the same machine receiving the same file, so the only change is xmodem.
> > Any thoughts on what has changed which might cause this different behaviour?

>Tony Nicholson wrote:
> I found that XMODEM sometimes does a NAK if the CP/M file writing takes too long (the timeout settings are hard to get right on faster Z80/Z280 systems). The workaround for me was to reduce the amount of in-memory buffering. Version 2.9 as a new setting /K that allows you to select this. I put /K20 in my XMODEM.CFG file to limit it to 20Kbytes on my problematic system.

Hi Tony, thanks for the hints, but it seems that I've got a different issue.

Firstly, I've isolated it down to just z80/SIO/v2.9. Both z80/ACIA and 8085/ACIA are working as expected. And with the Z80/SIO the v2.7 is working perfectly too.

I've got the code from both v2.7 and v2.9 side by side, and I'm trying to decipher the new code flow. There have been many substantial changes.

I've also spent some time with the logic analyser and traces. The failing process begins when is a block is received and rather than being ACKed(0x06) it is NAKed (0x15). Perhaps noise, but I doubt that. It will be something programmatic causing the CRC to fail (or something). The sender then repeats the block but the receiver then goes into a long time out, before the receiver sends NAK again. At this point the sender then repeats the block a second time the receiver ACKs it, and things continue normally.

So, I'm thinking to try to reduce the timeout before sending that second NAK, to speed up reinitialisation of transmission. But that is a work around.
What I need to do is to find out what is causing the block errors in the first place (given the triage above).

Interesting problem...

Cheers, Phillip

[Phillip Stevens]

> Phillip Stevens wrote:

> I've also spent some time with the logic analyser and traces. The failing process begins when is a block is received and rather than being ACKed(0x06) it is NAKed (0x15). Perhaps noise, but I doubt that. It will be something programmatic causing the CRC to fail (or something). The sender then repeats the block but the receiver then goes into a long time out, before the receiver sends NAK again. At this point the sender then repeats the block a second time the receiver ACKs it, and things continue normally.

> So, I'm thinking to try to reduce the timeout before sending that second NAK, to speed up reinitialisation of transmission. But that is a work around. What I need to do is to find out what is causing the block errors in the first place (given the triage above).

> Interesting problem...

Well, by adjusting some timings the problem seems to have resolved itself. I'm not sure that these fixes make any substantial difference, but anyway there's nothing to see here.

Left a gist of the timing changes here.
https://gist.github.com/feilipu/73f84745d9019a20ee1ef15d1fee948c