Bit-bang serial at 115200 baud

[Steve Cousins]

I'm considering increasing the baud rate of the bit-bang serial port on SC114, SC516 and SC527 to 115200 baud. If successful I'll produce new versions of SCM and CP/M.

Like the 9600 baud code it relies on the RTS signal from the bit-bang port reliably controlling the flow of data from the terminal. My main concern is the timing involved in detecting the start bit and controlling the RTS signal. This is quite tight at 115200 baud. I think different computers will vary in the timing here and this may trip up my code.

I've attached two test programs. If anyone would care to try them I'd appreciate knowing the results.

Both programs are HEX files that can be 'sent' to SCM. They are then started with the command "G E000". Note the start address is E000 rather than the usual 8000.

ECHO.HEX
This simply echos any character from the terminal, so just press keys and see if the correct character is displayed by the terminal.

PATCH.HEX
This patches SCM with code to support the bit-bang serial port at 115200 baud. After executing the code with "G E000", issue the command "CONSOLE 5" to switch to 115200 baud. All the usual SCM features should work, including downloading other SCM Apps.

Steve

[Bill Shen]

I tried both patch.hex and echo.hex on S114.  echo is sorta working.  If I type fast, it would drop characters.  patch.hex worked similarly, it worked most of the time, but sometimes input characters are dropped.  I'm running both without hardware handshake.  If I turn on hardware shake in TeraTerm, echo.hex nor patch.hex would respond to keyboard input.

I have TeraTerm in Windows environment.  My USB-serial adapter is CP2102.

What version of SCM for S114 should I be using?  I'm using SCM-S2-2022-02-27-monitor -v130-BIOS-v130-SC114.hex
  Bill

[Steve Cousins]

Hi Bill,

Thanks for trying the code. The results are very helpful. It has shown that there is sufficient variation in serial ports that I need to take a different approach to the one I used for 9600 baud.

The attached revised code has grown somewhat in order to account for a wider range of timings. This new approach has also meant having to deal with a serial port that sends a second character immediately following the first, even after RTS is raised. I've left very wide timing windows for things to happen so hopefully it will work everywhere. 

Assuming this new code works it would be nice to optimise it but I really don't know what the limits need to be. Currently it allows a lot of time for the terminal to respond to RTS. This means it will slow down software like MSBASIC that regularly looks for keys, such as ctrl/C.

The attached code works the same as the earlier version. See instructions below.

Performance measured by sending a HEX file to SCM: Transfer speed approx. 7k bytes per second, roughly equivalent to a hardware serial port at 70000 baud. So about 60% the speed of a 115200 baud hardware serial port. That's more than seven times faster than the 9600 baud bit-bang serial currently used in SCM.

If you or anyone else could try it that would be very helpful. I need to know the code is reliable with a wide range of computers.

Steve

Instructions:

Both programs are HEX files that can be 'sent' to SCM. They are then started with the command "G E000". Note the start address is E000 rather than the usual 8000.

ECHO2.HEX

This simply echos any character from the terminal, so just press keys and see if the correct character is displayed by the terminal.

PATCH2.HEX

This patches SCM with code to support the bit-bang serial port at 115200 baud. After executing the code with "G E000", issue the command "CONSOLE 5" to switch to 115200 baud. All the usual SCM features should work, including downloading other SCM Apps.

RTS/CTS hardware flow control is required. No delays need to be set in the terminal software. I believe the program should work with all versions of SCM.

[Bill Shen]

Steve,
Echo2 is more reliable, but not completely.  Once in a while I'll see garbage characters echo back.

Patch2 will accept 115200 bps and respond to keyboard command input.  However, when I repeatedly typed "asdf" (I can type these rapidly), which should get illegal command response every time, but it'll hang after lesser than 10 such tries.

I wonder if it is because I have a different USB-adapter.  Mine is 6-pin CP2102 like this:
https://www.ebay.com/itm/134453404131  

This is my setup:
* SC114 pc board, v1.0
* I downloaded the latest SC114 from your site, SCM-S2-2022-10-16-Monitor-v131-BIOS-v131-SC114.hex.  
* I use W27C512 instead of OTP so I can reprogram it many time.  Because A15 is high for ROM, I do have add 0x8000 offset to W27C512 when programming it.
* My SC114 ran at 14.7MHz, but I changed it back to 7.37MHz for your echo/patch tests.
* USB-serial is CP2102 with CP2102's CTS connected to SC114's RTS.  I have used CP2102 for many years and I know its hardware handshake works well with TeraTerm.
* My terminal is TeraTerm.

I've noticed a couple things about my built of SC114 (this is true of both v130 and latest v131 software)
1. It won't communicate with TeraTerm if I enabled TeraTerm's hardware handshake flow control.
2. When file load in SCMonitor, I need to set transmit delay to 1ms per character and 1ms per line.

Are 1 and 2 unexpected in a typical SC114?  

Many months ago I have patched SCM v130 to fix the two problems above and forgot about the patch until I tried SCM v131.  I didn't want to bother you with issues 1 and 2 because I was given these blank SC114 boards and did not purchase SC114 from you so they are not your problems to solve.  They may contribute to why patch/echo not working consistently.
  Bill

[Steve Cousins]

Hi Bill,

Thanks for the detailed info.

I'm puzzled by your experience that SC114 and SC130 don't work with hardware handshaking enabled. Also, the need for a 1ms delay is not normal, but I assume that is related to the first issue. I use TeraTerm on Windows but with a different USB adapter. As far as I know the symptoms you describe are not generally experienced by others. Most people who buy my kits also buy the USB adapter featured in my photos. However, in the case of SC114's bit-bang port it could be that not having had reported problems may not be significant as I suspect most people add a hardware serial port and thus don't make much use of the bit-bang port. As you have two systems with the handshaking issue I can only assume it is due to the use of a different USB adapter design. This is very odd.

Given that you have this handshaking problem I'm not surprised my 115200 baud code is unreliable. My code relies on the hardware handshaking to ensure characters are only sent from the PC when the Z80 is looking for them.

Steve

[Alan Cox]

> Given that you have this handshaking problem I'm not surprised my 115200 baud code is unreliable. My code relies on the hardware handshaking to ensure characters are only sent from the PC when the Z80 is looking for them.

There are two things that will bite you
- The handshaking response time on some USB dongles is horrible (I've
not looked to see if it's done on the host so has the USB latency)
- Some devices (including some PC real serial ports) turn on bigger
FIFOs at higher speed and turn them off at low speed. This would be
fine except that they often respond to hardware flow control several
characters late. That's acceptable for real serial where there is a
latency implicit in the flow control (even more so in XON/XOFF).

It's been the bane of wiring bitbangers like the ZX Spectrum to PC
hardware for a long time.

You may also be able to do real 9600 baud bidirectional bitbanging at
8MHz. There was scary code to do it at lower speeds on some other
machines like the Microbee.

Alan

[Bill Shen]

Steve,
Thanks for confirming SC114 should work with hardware handshake enabled; furthermore, it does not need 1ms delay.  I've placed order for FT232RL serial-to-USB adapter to see if the problem is with the adapter.  I think Alan is right about different USB-serial adapter implement hardware handshake differently.  I've patched SC114 firmware to work with CP2102 USB-serial adapter without handshake and without 1ms delay, but the patch doesn't work in CP/M environment, specifically XMODEM doesn't work; I believe hardware handshake is required for XMODEM file transfer.
  Bill

[Fred Weigel]

Bill... XMODEM uses 128 byte packets and a wee bit of overhead for 132 byte packet. After sending

a packet, the sender WAITS. The sender does NOT begin sending until receiving an ACK or NAK.

XMODEM does require 8 bit clean communications -- CP/M 2.2 only offers 7 bit (by specification).

Hardware handshake is not needed. I use 38400 bps with a 2Mhz 8080 without issue. 4000 characters

per second. (altair-duino).

[Fred Weigel]

And... XMODEM the protocol... I use PCPUT.COM and PCGET.COM patched for my hardware. But... have CP/NET 1.2 running, so big transfers go through that... (which is also packet based, with a slightly larger packet, but the same rules apply -- it is "client driven". The server waits for a complete command packet, executes and sends reply packet). Both XMODEM and my CP/NET protocol are "half duplex".  Now, if

interrupts are introduced, things do change... and not for the better.

[Bill McMullen]

Bill - A different perspective on the adapter issue:
By definition, XMODEM is half duplex and has built-in flow control; a maximum of 132 bytes transmitted or received before a line turnaround to wait for an ACK or NAK.  The CP2102 has a 640 byte Tx buffer and a 576 byte Rx buffer which should never overflow during a valid XMODEM exchange.

My $.02 is that at 115200 baud, hardware flow control (i.e. RTS/CTS) should NOT be required with the CP2102 so long as the receiving processor can receive a serial byte and store it in ~87 us.  For a Z80 that's up to roughly 22 instructions at just 1 MHz!  Obviously a lot fewer instructions if using relatively slow CALL/RETs to Tx/Rx subroutines.

The FT232R has a 256 byte Tx buffer and a 128 byte Rx buffer.  Theoretically it would be possible to have a Rx overrun when using XMODEM on USB 2.0 full speed (12 Mbit) to 115200 baud RS232 without flow control.

If you're looking for maximum XMODEM throughput, one thing you may want to check is your USB settings for "latency".  The single byte ACK/NAK will force a latency timeout before sending the USB packet.

[Alan Cox]

On Wed, 12 Apr 2023 at 14:38, Fred Weigel
<fridtjof.ma...@gmail.com> wrote:
>
> And... XMODEM the protocol... I use PCPUT.COM and PCGET.COM patched for my hardware. But... have CP/NET 1.2 running, so big transfers go through that... (which is also packet based, with a slightly larger packet, but the same rules apply -- it is "client driven". The server waits for a complete command packet, executes and sends reply packet). Both XMODEM and my CP/NET protocol are "half duplex". Now, if
> interrupts are introduced, things do change... and not for the better.

There is a trick to interrupts. The SIO has a 3 byte buffer and the
16x50 more. So if you send a 3 byte packet to kick the other end, and
which requires a response before you blast bits at it you can use
interrupts nicely. This is actually how Apple Localtalk is
implemented.


Alan

[Bill Shen]

While waiting for FT232RL to show up, I looked into why CP2102's handshake doesn't work with SC114's bit-bang receiver.  What I found out is CP2102's CTS is sluggish to respond to the short period of enabling/disabling CTS in the bit-bang algorithm.  The 2nd problem is my PC may continue to send data up to 5 microsecond after the negation of CTS.  The 2nd problem is a generic problem because there will always be some latency between handshake negated and host stops sending data, so it is necessary to check incoming data for a while after handshake is negated.

SC114's @RX routine and the identical code in SC114's CPM BIOS takes 50 bytes of code space.  This is the original listing:

                    0102  ; Receive byte
                    0103  ;   On entry: No parameters required
                    0104  ;   On exit:  A = Byte received via bit-bang serial port
                    0105  ;             NZ flagged if character input
                    0106  ;             BC DE HL IX IY I AF' BC' DE' HL' preserved
                    0107  ; The receive input must be on bit 7 of the port kRxPrt
2004: C5            0108  @Rx:        PUSH BC             ;Preserve BC
2005: AF            0109              XOR  A              ;Enable RTS line so
2006: D3 20         0110              OUT  (kRtsPrt), A   ;  terminal can send a character
2008: 06 0A         0111              LD   B, 10
200A: 10 FE         0112  @RxDelay:   DJNZ @RxDelay       ;Wait a while
200C: 3C            0113              INC  A              ;Disable RTS line so
200D: D3 20         0114              OUT  (kRtsPrt), A   ;  terminal will not send a character
200F: DB 28         0115  @RxWait:    IN   A, (kRxPrt)    ;Read receive port [11]
2011: E6 80         0116              AND  0x80           ;Test receive input [7]
2013: 28 05         0117              JR   Z, @RxBegin    ;Abort if no start bit [12/7]
2015: 10 F8         0118              DJNZ @RxWait        ;Timeout?
2017: C1            0119              POP  BC             ;Restore BC
2018: AF            0120              XOR  A              ;Return Z as no character received
2019: C9            0121              RET
201A: D5            0122  @RxBegin:   PUSH DE             ;Preserve DE
201B: 1E 08         0123              LD   E, 8           ;Prepare bit counter
201D: 06 52         0124              LD   B, 82          ;Delay 1.5 bits [7]
201F: 10 FE         0125  @RxBit:     DJNZ @RxBit         ;Loop until end of delay [13/8]
2021: DB 28         0126              IN   A, (kRxPrt)    ;Read receive port [11]
2023: E6 80         0127              AND  0x80           ;Mask data bit and clear carry [7]
2025: CB 19         0128              RR   C              ;Rotate result byte right [8]
2027: B1            0129              OR   C              ;OR input bit with result byte [4]
2028: 4F            0130              LD   C, A           ;Store current result byte [4]
2029: 06 37         0131              LD   B, 55          ;Delay 1 bit [7]
202B: 1D            0132              DEC  E              ;Decrement bit counter [4]
202C: 20 F1         0133              JR   NZ, @RxBit     ;Repeat until zero [12/7]
202E: 10 FE         0134  @RxStop:    DJNZ @RxStop        ;Wait for stop bit
2030: F6 FF         0135              OR   0xFF           ;Return NZ as character received
2032: 79            0136              LD   A, C           ;Return byte received
2033: D1            0137              POP  DE             ;Restore DE
2034: C1            0138              POP  BC             ;Restore BC
2035: C9            0139              RET

I re-write the bit-bang receive to give CP2102 more time to assert CTS and also time to check for incoming data after negation of CTS.  The program is 49 bytes so it fits within the original @RX.  I also patch the modification to "CPM-E_SCM_PutSysPlus_128MB_code8000.hex" so CP/M BIOS will have the same modification.

;4/14/23 alternative approach
;patch SCM SC114 to work with CP2102.  Patch code needs to be 50 bytes or less
;Same routine is used in SC114 implementation of CP/M BIOS
;CP2102 USB-serial adapter needs more time to assert CTS
;CP2102 USB-serial adapter may continue to transmit data up to 5uS after negation of handshake
;
; org 2004h ;SCMonitor origin for the bit-bang receive, @RX
org 0aa11h ;location of bit-bang receive in SC114 CP/M BIOS is 0xEA11
; patching the PutSys routine to be relocated to 0xEA11
AtRX:
push bc
ld bc,2002h ;c is pass count flag, c=2 for the pass with RTS enabled,
; c=1 for pass where RTS is disabled
xor a
out (20h),a ;enable RTS so terminal can send char
AtRxWait:
in a,(28h) ;read receive data [11]
rlca ;shift data bit into carry [4]
  jr nc,AtRxBegin ;start bit-bang receive if start bit detected [12/7]
  djnz AtRxWait ;loop for a while looking for start bit [13/8]
  ld a,1 ;disable RTS
  out (20h),a
ld b,10h ;wait period for checking on receiver after RTS negated
dec c
jr nz,AtRxWait
;Z flag set at this point
  pop bc
  ret ;time out, no character received, return Z
AtRxBegin:
ld b,8 ;loop count [7]
ld a,71 ;wait 1.5 bit time [7]
; first pass has 40 clocks of uncertainty,
;    1.5 bit time is 7+7+11+1127 (156uS) +5uS -0uS to read data
; next 8 passes are 4+8+7+13+727=103uS, slightly shorter than 104uS to compensate for
;   jitter in the 1.5 bit time
; no need to wait for stop bit, return the data after 8 data bits received to give program
;   more time to process data
AtRxBit:
dec a ;delay loop [4]
jr nz,AtRxBit ;[12/7], 70*(12+4)+7=1127
        IN   A,(28h)     ;Read receive port [11]
rlca ;data in bit 7, rotate into carry [4]
rr c ;regC accumulates result shifted in from carry [8]
ld a,46 ;wait 1 bit time [7], 45*(12+4)+7=727
djnz AtRxBit ;[13/8]
ld a,1
out (20h),a ;disable RTS
or a ;Return NZ as character received
        ld a,c           ;Return byte received
        pop bc             ;Restore BC
        ret

With this patch to CP/M's BIOS and hardware handshake enabled, XMODEM can now transfer some files, but something is still not right; XMODEM can transfer program up to 30K, but hang for bigger program.  Another problem, XMODEM can only transfer ASCII data; binary data causes it to hang immediately.  

I like to know whether XMODEM worked flawlessly using FT232 USB-serial adapter?
  Bill

[Steve Cousins]

Hi Bill,

Thanks for all the work you are putting in.

Using my serial adapter with my 9600 baud bit-bang code for CP/M, I get the same results with XMODEM as you do. With large text files the transfer hangs at about 30k bytes and binary files seem to hang almost immediately. 

I don't normally use XMODEM with my plain CP/M systems so I wasn't aware of this problem. I do use DOWNLOAD.COM successfully with the bit-bang code. Everything I normally use seems to work fine with the bit-bang code. 

I can also confirm that my CP/M BIOS for the 68B50 ACIA works fine for large (>40k) XMODEM transfers in text and binary formats. This BIOS uses the same framework as the bit-bang code, so only the very lowest level functions are different. I believe the binary file transfer with the bit-bang code fails due to this console status code, which is unique to the bit-bang BIOS:

            LD   A,(BBRxByte)   ;Get character waiting
            OR   A              ;Is there a character waiting?
            JR   NZ,@YesRx      ;Yes, so skip...
            CALL Rx             ;Go test for a serial receive
            JR   Z,@NoRx        ;NZ flagged if received, A = Char
            LD   (BBRxByte),A   ;Store character received
@YesRx:     LD   A,0xFF         ;Return status = Char available
            RET
@NoRx:      LD   A,0            ;Return status = Char not available
            RET

If the character waiting in BBRxByte is 0x00 then it is ignored and a new character received instead. Stupid mistake! I need to fix it. Text files do not contain 0x00 bytes so they are fine.

I don't know why text files hang at about 30k. Perhaps it is when data is written to disk. I'll have to look again some other time.

Steve

[Bill Shen]

OK, the mystery of binary transfer solved!

You are right that download.com also works on my SC114 with BIOS patch for CP2102, so there is an working alternative to xmodem.com.  Since you are working on a faster bit-bang routine, up to 115200 bps, I think the 68B50 ACIA board may not be necessary.  I know SC114 has lots of design margin so 14.7MHz operation is reliable.  SC114 at 115200bps plus a CF board that everyone is making and 14.7MHz CPU clock is an economical and capable CP/M computer.
  Bill