Help with xmodem for CP/M

[James Moxham]

I'm wondering if some kind person would have a copy of a "CP/M" version of xmodem? 

By way of background, I'll explain 1) what this is, 2) why it might be useful, 3) why it may not have ever been written 4) solutions, and 5) specific edits needed to existing xmodem source code.

1) "CP/M" xmodem is a program that would run on any CP/M machine. It would use CP/M BDOS calls rather than direct I/O.

2)  Traditionally, xmodem would be patched for a specific modem or ports on a particular computer. I've done this a number of times, for a Z80 board, for the Propeller CP/M emulation, for various FPGA boards and now need to do it again for an Arduino emulation. For the latter, I decided to fire up my old IDE and cross compilers, only to find that half these programs no longer work on a modern PC. Lots of errors about 64bit vs 32bit compatibility. I've got a partial workaround with Virtual Box running XP but it is not a great solution and took half a day to set up and only works if you have an original XP CD.

Imagine if there was a version of xmodem that ran on any CP/M machine!

3) This may not have ever been written because CP/M strips away the high bit so that only ascii characters 0 to 127 are allowed. This is something at the heart of the CP/M code and changing it would involve changing every CP/M program out there, so it seems we are stuck with this. I think the same issue applies to the RDR and PUN devices too. So in practice, anything using the BDOS calls can only send text files and not binary files. Does this matter? Well, ANSI and VT100 etc terminals found workarounds for printing the extended characters from 128 to 255 - either by using an alternative font, or with special codes to do these characters. There must be workarounds to this problem!

4) A simple workaround is to send files as Intel Hex. These only use ascii characters under 127. There are programs to turn binary to hex "UNLOAD.COM" and hex to binary "LOAD.COM". There are some limitations - eg all files need to be offset by 256 bytes as LOAD cannot handle the first 256 bytes of a program, as it was designed to create a file that would be loaded into a Z80 at address 256. Hex files also have a disadvantage in that you are sending 2 bytes for every byte, plus the line number, the checksum, and the number of bytes in the line. So a program takes maybe 2.3 times longer to transfer. Another option is Base64, which is the workhorse of email and internet pictures. This takes three bytes (24 bits) and turns them into 4 bytes of 6 bits each, using only certain ascii characters, eg Hello World becomes SGVsbG8gV29ybGQ=   So it is possible to send binary files if you can encode and decode them into to text files. 

5) The patch to xmodem should be fairly simple. Looking at https://www.seasip.info/Cpm/bdos.html , replace all OUT calls with a call to BDOS function 2. For input, I think BDOS function 6 might do the trick, use E = 0FEh to scan and see if there is a character, then E = 0FFh  (or BDOS function 1) to get the character. 

Any help or advice would be most appreciated.

Cheers, James Moxham, Adelaide, Australia

[phillip.stevens]

I'm wondering if some kind person would have a copy of a "CP/M" version of xmodem? 

By way of background, I'll explain 1) what this is, 2) why it might be useful, 3) why it may not have ever been written 4) solutions, and 5) specific edits needed to existing xmodem source code.

1) "CP/M" xmodem is a program that would run on any CP/M machine. It would use CP/M BDOS calls rather than direct I/O.

It is not "XMODEM" but "KERMIT" runs XMODEM, and can be patched to CP/M. I've put a version in the `SYS.CPM` drive here.

The cpm-tools usage instructions for these FATFS drives are found here.

The instructions to do the patching are found here.

Works as advertised.

Enjoy.

Regards, Phillip

[phillip.stevens]

4) A simple workaround is to send files as Intel Hex. These only use ascii characters under 127. There are programs to turn binary to hex "UNLOAD.COM" and hex to binary "LOAD.COM". There are some limitations - eg all files need to be offset by 256 bytes as LOAD cannot handle the first 256 bytes of a program, as it was designed to create a file that would be loaded into a Z80 at address 256.

 

The tool to use is "MLOAD.COM", which allows different offsets, file concatenation, merging, etc.

I use it and Microshell "SH.COM" all the time.

Both from the same place.

[StephenA]

Hi James,

Xmodem protocol relies on an 8 bit checksum.

Sending an Xmodem data stream out via a BDOS call would (as you point out) strip the 8th bit and in doing so break the protocol. And that's just the start. You also have to dodge CP/M's interpretation of "special" control characters. Very unlikely it would get past the first packet. So the changes would not be trivial. I imaging you would need to build a "wrapper" along the lines uuencode/decode that sits between the Xmodem packet layer and the wire. A major rewrite which would in reality produce an entirely new protocol.

I believe some CP/M 2.2 re-implementations may have modified the behavior of some BDOS calls so that they do not strip the 8th bit. Very helpful but very non-standard.

If your just after an already patched version of CPM Xmodem for the RC2014 I believe you will find it here:-

http://web1.foxhollow.ca/cpm/

Grant Seales "package" format (DOWNLOAD.COM) looks to solve much the same problem. But only for getting software into a RC2014.

[StephenA]

It occurred to me that someone may have written native CPM version of uuencode/decode.

Appears that there is:-

ftp://www.zimmers.net/pub/cpm/ALLFILES.html

Look for uucode.lbr

This may use native BDOS calls I guess? You still have the problem of dodging the control character interpretations.

Might be worth a look.

[James Moxham]

Lots of great ideas, thanks!!

will check out mload and kermit. 

Re bdos calls, looking at sample BIOS from gaby.de, the code the user writes is supposed to set the 7th bit low. The CP/M ccp for instance, printing a character, just calls that bdos code the user added. So if you call your own bdos code with a jump to 0005, I don't think CP/M gets involved in that?

Any bdos the user has written should send and receive control characters ok. So all characters from 0 to 127 are valid, and that includes ACK and NAK. What won't go through are any binary numbers from 128 to 255, but what if you never use these numbers? A hex file, for instance uses : and numbers 0 to 9 and letters A to F and CR and LF and ^Z at the end. So all those should go through calls to BDOS? (ie a call to 0005H, and then CP/M doesn't even know you did that call)

[phillip.stevens]

Lots of great ideas, thanks!!

will check out mload and kermit.

For me it comes down to the "minimum viable workflow". Since I'm working with z88dk most of the time (in either C or asm), then I'll typically build to the CP/M target, and then use the simple slowprint.py program to send IntelHEX to the RC2014. On the RC2014 I'll use PIP to receive the IntelHEX, and then convert it to binary using MLOAD. It is pretty quickly repeatable, and doesn't require jumping through hoops to iterate. I also use "sh" to make the selection of drives unnecessary; it is just comfortable.

On the host PC

; Assemble with z88dk for RC2014 CP/M

; zcc +cpm -clib=sdcc_iy -v -m --list test.asm -o test
; appmake +glue --ihex --clean -b test -c test
; python slowprint.py > /dev/ttyUSB0 < test__.ihx

On the RC2014

B> a:sh
       MicroShell - Version 2.0
Copyright(c)1982 New Generation Systems
B0% pip test.hex=con:
; CTRL Z to finish downloading
B0% mload test.hex
B0% test
; test program runs (or doesn't run).

 

Re bdos calls, looking at sample BIOS from gaby.de, the code the user writes is supposed to set the 7th bit low. The CP/M ccp for instance, printing a character, just calls that bdos code the user added. So if you call your own bdos code with a jump to 0005, I don't think CP/M gets involved in that?

That's correct. The CCP (command and control processor) is not involved in user programs and can, if needed for space, be overwritten by the user program. All good CP/M user programs are supposed to use the BDOS calls, only.

 

Any bdos the user has written should send and receive control characters ok. So all characters from 0 to 127 are valid, and that includes ACK and NAK. What won't go through are any binary numbers from 128 to 255, but what if you never use these numbers? A hex file, for instance uses : and numbers 0 to 9 and letters A to F and CR and LF and ^Z at the end. So all those should go through calls to BDOS? (ie a call to 0005H, and then CP/M doesn't even know you did that call)

Yes, except the user doesn't write the BDOS. The BDOS was provided by DRI and is usually not changed. The hardware builder should write the BIOS which provides the underlying routines specific to hardware. And in my implementations for the RC2014 I don't modify bytes sent to the BDOS call CONOUT. But I DO strip the sign bit from incoming characters with the CONIN call, because that is the "standard" thing to do.

P.

[Richard Deane]

DR CP/NET would allow easy copy of files between cpm and host, just needs implementing over fast link. E.g. rc2014 WiFi card. Good version available for kaypro, as example, on www.durgadas.com

Z80pack from Udo Munk also has some stuff on CP/NET.

[James Moxham]

By way of background, I've got a terminal working now on a touchscreen. It has an SD card on the back of the display and I figured that could be another disk drive for any CP/M machine. The particular attraction is that the files on that SD card are FAT. So potentially, with a few taps on a touchscreen, you could transfer files to and from any CP/M machine. But it needs a universal xmodem. There are other solutions but they may not work with fast baud rates as there is no flow control (ie Grant's Download program, or using PIP). By the time you write a program with xon/xoff flow control, I figure you may as well use xmodem which is a standard and which is on pretty much all terminal programs. 

I very much agree with the sentiment that all CP/M programs should use BIOS calls rather than patching in low level IN/OUT code for every different hardware configuration. Even though, theoretically, you could not strip out the 7th bit on your own bios, I agree that the convention is that you do.

Back to coding :)

[Mark Bramwell]

as stated already, there is xmodem already compiled and ready to go for the RC2014.  It uses stdin for I/O, it does not use an alternate comport.  That means if your terminal emulator is currently connected and you have the CPM prompt, it can use that communications path to perform an xmodem transfer.  I am using it at 115,200 baud.

http://web1.foxhollow.ca/cpm/xmodem/xmodem.zip

[Bill Shen]

CP/M was new to me 12-18 months ago so when I went through the BIOS for the first time I saw the "and 7Fh' statement in Console In routine and thought that was unnecessary and commented it out.  I did this for both CP/M-68K as well as CP/M2.2 for Z280 and didn't have any issues.  I use XMODEM for CP/M2.2 and KERMIT for CP/M-68K both using the console I/O for file transfer.  They both work well.
  Bill

[Alan Cox]

On 21 July 2018 at 16:52, Bill Shen <coinst...@gmail.com> wrote:
> CP/M was new to me 12-18 months ago so when I went through the BIOS for the
> first time I saw the "and 7Fh' statement in Console In routine and thought
> that was unnecessary and commented it out. I did this for both CP/M-68K as
> well as CP/M2.2 for Z280 and didn't have any issues. I use XMODEM for
> CP/M2.2 and KERMIT for CP/M-68K both using the console I/O for file
> transfer. They both work well.

None of my CP/M or MP/M BIOSen do the masking. Never had a problem as a result.

I think it was in the reference BIOS because at the time 7bit + parity
was the default.

Alan

[phillip.stevens]

> CP/M was new to me 12-18 months ago so when I went through the BIOS for the
> first time I saw the "and 7Fh' statement in Console In routine and thought
> that was unnecessary and commented it out.  I did this for both CP/M-68K as
> well as CP/M2.2 for Z280 and didn't have any issues.  I use XMODEM for
> CP/M2.2 and KERMIT for CP/M-68K both using the console I/O for file
> transfer.  They both work well.

None of my CP/M or MP/M BIOSen do the masking. Never had a problem as a result.

I think it was in the reference BIOS because at the time 7bit + parity
was the default.

Removed the strip sign bit from the CP/M-IDE for ACIA serial, Spencer's SIO, and SMB SIO.

Now Mark's XMODEM patches work as expected too.

P.

[James Moxham]

Rummaging around in some files, I found a working xmodem written in C.

It originally came with the disk images for BDS C and this is available at https://www.bdsoft.com/resources/bdsc.html

The files are CMODEM.C, CMODEM2.C and CMODEM.H  and the instructions on how to compile and link are in the CMODEM.C  program.

The data for the modem ports are in a file HARDWARE.H which can be modified for a particular machine.

It seems to compile and run ok.

The variant of C is not quite the same as on a modern Arduino but it is very similar. Personally, I found it easier to follow than the assembly version. 

Could be useful!

[Tom Szolyga]

Hi James,

Would you provide a link to CMODEM.

Thanks,

Tom

[Richard Deane]

the files are in the examples.lbr file included with the bds distribution in the link below. I opened the lbr file under mac osx with "the unarchiver". I guess modern zip programs would open it on a windows system.

Richard

[James Moxham]

Phillip Stevens suggested Kermit earlier in this post. Very interesting suggestion! I found a 'generic' version of Kermit that uses CP/M calls so doesn't need to be recompiled, and some instructions http://altairclone.com/downloads/cpm/CPM%202.2/Kermit/ReadMe.pdf    Ran STAT and told it to link the RDR to UR2 and the PUN to UP2. Then ran Kermit and the command SET PORT UR2 and set it to send a file (the sender outputs the first packet) and a packet of data appeared on the punch device port. Very nice!!

Kermit is interesting because you can change the packet size. On an arduino the serial buffer is 64 bytes, so if the packet size is smaller than this, there will never be buffer overruns. 

So this looks like a generic solution that should run on any CP/M board. Back to coding... :)

[phillip.stevens]

On Monday, 6 August 2018 17:20:13 UTC+10, James Moxham wrote:

Phillip Stevens suggested Kermit earlier in this post. Very interesting suggestion! I found a 'generic' version of Kermit that uses CP/M calls so doesn't need to be recompiled, and some instructions http://altairclone.com/downloads/cpm/CPM%202.2/Kermit/ReadMe.pdf    Ran STAT and told it to link the RDR to UR2 and the PUN to UP2. Then ran Kermit and the command SET PORT UR2 and set it to send a file (the sender outputs the first packet) and a packet of data appeared on the punch device port. Very nice!!

Kermit is interesting because you can change the packet size. On an arduino the serial buffer is 64 bytes, so if the packet size is smaller than this, there will never be buffer overruns.

Unfortunately, it looks like the original website z80.eu is down, and I don't know whether it is permanent situation or not. The Wayback Machine has z80.eu under this link though.

To get the original Kermit HEX and assembly, then the Columbia Uni Kermit page has that. The instructions on the z80.eu webpage are still valid.

On a CP/M machine use the MLOAD tool to link together the main Kermit HEX file CPSKER.HEX and the customisation HEX file for generic CP/M CPVGEN.HEX.

The MLOAD command looks exactly like specified.

a>  MLOAD KERM411=CPSKER,CPVGEN

Note that MLOAD command doesn't need the default extensions of KERM411.COM nor the CPSKER.HEX and CPVGEN.HEX

The attached cpm80.zip file contains all the Kermit HEX file options, and source and Intel HEX for MLOAD.COM too. It is the same file as found on the Columbia Uni Kermit CP/M webpage, attached here in case the Internet forgets.

Enjoy, Phillip

[phillip.stevens]

For interest, an historical Kermit postscript, from Columbia University.

The Kermit protocol was largely influenced by limitations of the DEC-20. The DEC-20, with its PDP-11/40 front end, was designed on the assumption that terminal input comes directly from people sitting at keyboards typing with their fingers at a relatively slow rate — maybe 10 characters per second, tops — whereas large amounts of sustained output can be sent from the computer to the screen. RSX20F, the front end's operating system, therefore allocates small buffers for input and large ones for output. We learned about this the hard way, when we bought our first terminals that included a "transmit screen" function (HDS Concept-100s). As soon as someone tried it, the front end crashed. Similar phenomena were observed with autorepeat keys (as when one of our programmers fell asleep on the keyboard)(5), and again when DEC first released its VT100 terminal: when doing smooth scrolling at 9600 bps the terminal overwhelmed the poor front end with XOFFs and XONs. Later releases of RSX20F coped with these problems in draconian fashion — if input buffers could not be allocated fast enough, the front end would set the line's speed to zero for a second or two! The lesson? Don't send sustained bursts of terminal data into the DEC-20 — it's like trying to make sparrow eat a meatball hero. Kermit's normal packets are therefore quite short, 96 characters maximum — seeds, insects, and worms that a sparrow can digest.

Another peculiarity of the DEC-20 is its sensitivity to control characters. During normal terminal dialog, 17 of the 33 ASCII control characters are used for special purposes — editing, program interruption, flow control, status reporting, signalling end of file, etc. — rather than being accepted as data. Even though a DEC-20 program can open the terminal in "binary mode" to bypass the special processing of these characters, it is not necessarily desirable to do so, because some of these functions could be useful during data transfer. The lesson here is don't send control characters "bare" when transferring data. In fact, the Kermit protocol (in its most basic configuration) sends packets that are strictly short lines of text.

The IBM mainframe (by this time, the 360/91 had been replaced by a 4341 running the VM/CMS operating system) had its own set of peculiarities. Unlike the DEC-20, it used half duplex communication, and used 7 data bits with parity when communicating with the outside ASCII world. This meant that our file transfer protocol would have to be half duplex too, and would require a special mechanism for transmitting 8-bit binary data through a 7-bit communication link. Furthermore, as all communication was through either a 3705 front end (linemode) or an IBM Series/1 (or equivalent, e.g. 7171 or 4994) 3270 protocol converter, both of which treated many of the control characters as commands to be executed immediately, the prohibition on bare control characters in data was reinforced. Reducing the protocol to the lowest common denominator made it work in all cases, but at the expense of efficiency and elegance. Some of the resulting shortcomings were addressed in later years by the addition of long packets and full-duplex sliding-window packet transport to the protocol, as well as a control-character "unprefixing" option.

By happy coincidence, the combined quirks of the DEC-20, the IBM mainframe, and the CP/M microcomputer resulted in a design that would prove adaptable to practically any computer capable of asynchronous communication. A file was first transferred with Kermit protocol on April 29, 1981, by two instances of Kermit-20 running on a single DEC-20, using two serial ports interconnected by a null modem cable.

[James Moxham]

Perfect timing, many thanks for those links as that is just what I am looking for. There is an excellent description of the problem on this site http://www.quicktrip.co.nz/jaqblog/home/69-z80part4   Essentially, calls to the RDR device will hang CP/M, so the workaround is to use IOBYTE and let Kermit direct where things go. Then a few tweaks inside the BIOS to direct output to various UARTs depending on IOBYTE. 

Very interesting regarding buffers. On an arduino mega, I can gobble up the bytes fast enough at 9600 baud but not any faster. But 9600 is fine at the moment and can look at optimising code down the track. 

It is great fun getting inside the Kermit protocol and getting the first few packets working. I started with a SEND and decoded the first packet to make sure I understood the protocol. Then change it to receiving, and I know the packet I send is the right protocol, then see what comes back. It is a very logical system. There is a workaround for the 7 bit problem for instance and that gets sorted in the first packet. Ultimately, I'd like Kermit to become a generic FAT disk drive for any CP/M system as I think that could be quite useful.

[James Moxham]

Reading through the Kermit site http://www.columbia.edu/kermit/archive.html  there is this rather delightful quote:

"Kermit 80 was one of the original Kermit programs, first written in 1981 for the Intertec Superbrain and then made portable to many other CP/M systems, all mutually incompatible: [insert long list of computers]"

Indeed, all "mutually incompatible". There must be a generic CP/M version in all that list! I think what I am looking for here is the source code for CPVGEN.HEX  In particular, before modifying the BIOS (which I have already done, then modified it back again), how exactly is Kermit using IOBYTE. For instance, is it using BIOS calls to functions 7 and 8 (the proper way) or is it talking directly to ram location 3 (is that the correct location?). And does it define IOBYTE lower two bits 01 as Terminal and 00 as Serial Port, or the other way round?

If it is possible to work out what Kermit is doing, then it ought to be possible to modify the CP/M BIOS such that any call to write a byte will then check the iobyte status and direct the byte to a particular physical serial port. 

All good fun trying to hack all this old software into doing new tricks :)

[phillip.stevens]

On Tuesday, 7 August 2018 22:11:57 UTC+10, James Moxham wrote:

Perfect timing, many thanks for those links as that is just what I am looking for.

You're welcome. I find it interesting to read computer history, because it is all so recent and has developed so quickly.

There is an excellent description of the problem on this site http://www.quicktrip.co.nz/jaqblog/home/69-z80part4  Essentially, calls to the RDR device will hang CP/M, so the workaround is to use IOBYTE and let Kermit direct where things go. Then a few tweaks inside the BIOS to direct output to various UARTs depending on IOBYTE.

Yes, I read quite a bit about the IOBYTE, and I think that Grant's implementation is quite correct. I copied most of it for CP/M-IDE, amongst other things, but made the redirections blindingly obvious by writing them out in binary form. Works for me.

 

It is great fun getting inside the Kermit protocol and getting the first few packets working. I started with a SEND and decoded the first packet to make sure I understood the protocol. Then change it to receiving, and I know the packet I send is the right protocol, then see what comes back. It is a very logical system. There is a workaround for the 7 bit problem for instance and that gets sorted in the first packet. Ultimately, I'd like Kermit to become a generic FAT disk drive for any CP/M system as I think that could be quite useful.

Funny that you say Kermit as FAT disk drive. My initial interest was somewhat similar. I have an interest in HP RPN calculators, having grown up on them (literally as my father brought a HP-35 home when I was in single digits), and the HP-48GX supports Kermit protocol. My plan was to use my Arduino clone (with integrated SD card), and multiple serial ports to serve as a Kermit file server for my HP-48GX. I could also use the Arduino clone as a data sensor (ADC), or controller (with DC capable DAC). In the end I was just at the cusp of boredom with AVR, and decided to build a Z80 machine

And now here we are again, back with Kermit.

[phillip.stevens]

On Tuesday, 7 August 2018 23:52:05 UTC+10, James Moxham wrote:

Reading through the Kermit site http://www.columbia.edu/kermit/archive.html  there is this rather delightful quote:

"Kermit 80 was one of the original Kermit programs, first written in 1981 for the Intertec Superbrain and then made portable to many other CP/M systems, all mutually incompatible: [insert long list of computers]"

Indeed, all "mutually incompatible". There must be a generic CP/M version in all that list! I think what I am looking for here is the source code for CPVGEN.HEX.

There are at least two pieces of code you need. In the zip file above the system specifics are contained in cpxsys.asm, and the more generic code in cpsker.asm. You should be able to get a good idea of what is going on from these two files, although there are others to understand too.

Good luck with the 8080 mnemonics. :-P

The cpsker.asm contains this table.

; This is the header for the system-independent portion of KERMIT, which
; consists of the following files (in this order):
;
; CPSKER.ASM - this file
; CPSDEF.ASM - definitions for both KERMIT and KERSYS
; CPSMIT.ASM - initialization, main loop, miscellaneous commands
; CPSCOM.ASM - (BYE, EXIT, LOG, SET, SHOW, and STATUS) (Part1 of 2)
; CPSPK1.ASM - the KERMIT protocol handler (SEND, RECEIVE, LOGOUT,
; CPSPK2.ASM - and FINISH commands) (In two parts)
; CPSREM.ASM - REMOTE commands etc
; CPSSER.ASM - SERVER commands etc (Empty as yet)
; CPSTT.ASM  - the transparent commands (TRANSMIT, CONNECT)
; CPSCPM.ASM - CP/M commands (DIR, ERA)
; CPSWLD.ASM - the wildcard handler
; CPSCMD.ASM - the command parser
; CPSUTL.ASM - utility routines
; CPSDAT.ASM - Data space and the overlay link space

[phillip.stevens]

On Tuesday, 7 August 2018 23:52:05 UTC+10, James Moxham wrote:

Reading through the Kermit site http://www.columbia.edu/kermit/archive.html  there is this rather delightful quote:

"Kermit 80 was one of the original Kermit programs, first written in 1981 for the Intertec Superbrain and then made portable to many other CP/M systems, all mutually incompatible: [insert long list of computers]"

Indeed, all "mutually incompatible". There must be a generic CP/M version in all that list! I think what I am looking for here is the source code for CPVGEN.HEX.

There are at least two pieces of code you need. In the zip file above the system specifics are contained in cpxsys.asm, and the more generic code in cpsker.asm. You should be able to get a good idea of what is going on from these two files, although there are others to understand too.

Good luck with the 8080 mnemonics. :-P

Actually, despite the fact I should be asleep, it is the cpxcon.asm file that is relevant. It contains the routines to transfer the BIOS jump points into the Kermit internal standard calls. The code looks like this.

;
;       System-dependent initialization
;       Called once at program start.
sysinit:
;
; [13]  Had to move this call to here, as the prtstr routine needs this
; before the config message is sent out.  It has only been moved.
;
 call iniadr ;Initialize the BIOS addresses

and then

;       This one is hopefully the last "improvement" in view of GENERIC
; Kermit. It uses for Character-I/O the BIOS-routines ( instead of the
; "normal" BDOS routines. What does it give us (hopefully) : More speed,
; higher chance of success ( I/O byte implemented in BIOS [if at all]),
; but no "extra" device handling - that's done by BDOS.
;
;       How do we "get" the call-adresses?  Location 0 has a JMP Warm-Boot
; in CP/M which points into the second location of the BIOS JMP-Vector.  The
; next three locations of the JMP-Vector point to the CONSTAT,CONIN,CONOUT
; BIOS-routines.  CONOUT wants the character in C.
;
; - Bernie Eiben

iniadr: lhld 1  ;get BIOS Warmstart-address
 lxi d,3        ;next adress is CONSTAT in BIOS
 dad d
 shld bconst+1  ;stuff it into the call-instruction
 lxi d,3        ;next adress is CONIN in BIOS
 dad d
 shld bconin+1  ;
 lxi d,3        ;next adress is CONOUT in BIOS
 dad d
 shld bcnout+1
 lxi d,3        ;next address is LIST in BIOS
 dad d
 shld blsout+1
 lxi d,10*3     ;get printer status routine
 dad d
 shld bprtst
 ret ;And return

bconst: jmp $-$ ;Call BIOS directly (filled in by iniadr)
bconin: jmp $-$ ;Call BIOS directly (filled in by iniadr)
bcnout: jmp $-$ ;Call BIOS directly (filled in by iniadr)
blsout: jmp $-$ ;....
bprtst: jmp $-$ ;Call BIOS directly for printer status

So in the generic version it is using the BIOS routines directly, rather than going via the BDOS. Luckily, I think that all RC2014 CP/M BIOS versions handle the IOBYTE correctly, so this is not a problem.

[James Moxham]

Many thanks Phillip. That code looks very promising and makes sense to go straight to the BIOS. Looks a bit self-modifying too!

As an aside, I see there are a number of serial UART boards for the RC2014, are these using IOBYTE or are programs being patched to talk to specific ports?

Time for a few tweaks to the CP/M bios. I am hoping that by keeping the kermit packet size under the buffer size of the arduino, I can then push the baud rate up on the data transfer serial port. 

Sorry about keeping you awake thinking about this!

[phillip.stevens]

James Moxham wrote:

As an aside, I see there are a number of serial UART boards for the RC2014, are these using IOBYTE or are programs being patched to talk to specific ports?

AFAIK, every RC2014 CP/M implementation properly supports the IOBYTE mechanism.

The three that I know of are:

1. Grant Searle's version, for the SIO, with the IOBYTE handling here - supports CF cards in 8 bit mode, requires 64kB/32kB RAM/ROM
   
2. ROMWBW for the SIO, with the IOBYTE handling here - supports every conceivable option, requires 512kB/512kB RAM/ROM.
   
3. CP/M-IDE for the SIO, with the IOBYTE handling here - supports FATFS formatted IDE and CF drives in 16 bit mode, requires 64kB/32kB RAM/ROM.

For the ACIA, there is typically only one serial interface per RC2014, so the IOBYTE handling is also done, but it is a mute point.

 

Time for a few tweaks to the CP/M bios. I am hoping that by keeping the kermit packet size under the buffer size of the arduino, I can then push the baud rate up on the data transfer serial port.

 Changing the software serial buffering on the Arduino side is well documented. The file to modify is here.

On the RC2014 side, serial buffer size depends on the implementation

1. Grant Searle's version, for the SIO, 60 Byte Rx, 0 Byte Tx, configured here.
   
2. ROMWBW for the SIO, 32 Byte Rx, 0 Byte Tx, configured here.
   
3. CP/M-IDE for the SIO, 255 Byte Rx, 15 Byte Tx, configured from within z88dk

AFAIK, only CP/M-IDE does Tx buffering, which is useful if you're sending a lot characters in one go. I've set it up so that the Tx buffer can be any 2^n size, of which 2^n-1 is available. 16 Bytes of Tx buffer seems to be a good balance between performance and RAM consumed. I don't suggest it is worth increasing much, as the Tx will simply wait if the buffer if full, exactly like all the solutions without Tx buffers.

Realistically, Tx buffering is only really useful when you are reading a pre-assembled series of Bytes into the CP/M BIOS routines, like a VT100 colour code, for example. Otherwise, there is sufficient overhead in CP/M to make buffering fairly unnecessary.

 

Sorry about keeping you awake thinking about this!

No problem. I think we're on the same TZ anyway.

Cheers, Phillip 

[James Moxham]

Same time zone, ok, so I'm the one staying up late now!!

Those links are very interesting. Good to compare RC2014 and the N8VEM.

There are many variables so I am trying to nail them down. 

1) Have I got the correct version of Kermit? Well, it boots up with "Kermit-80 v4.11 configured for Generic CP/M-80 with Generic (Dumb) CRT Terminal type selected" and the command "set port" gives

Kermit-80   0A:>set port ?

CRT device

PTR device

TTY device

UC1 device

UR1 device

UR2 device

so I think that is listing all the devices. 

2) What command do you issue to Kermit with the port? This site "http://www.quicktrip.co.nz/jaqblog/home/69-z80part4" suggests "SET PORT CRT" but that does not make sense, as that ought to send both the commands and the actual data to the same port (which indeed it seems to do, I see a packet of data come back if I ask Kermit to send". I have seen other sites suggest "SET PORT UR2" but I don't know exactly what that does. I suspect you can set the port to any of the above list, then map these inside the bios to various physical ports. It probably doesn't matter which you do, but the logical thing is to try to copy some sort of standard that someone else has worked out. 

3) Inside the N8VEM code as linked in that last post are things like this:

const:
	LD A,(iobyte)
	AND 00001011b ; Mask off console and high bit of reader
	CP 00001010b ; redirected to reader on UR1/2 (Serial A)
	JR Z,constA
	CP 00000010b ; redirected to reader on TTY/RDR (Serial B)
	JR Z,constB
	AND $03 ; remove the reader from the mask - only console bits then remain
	CP $01
	JR NZ,constB

This makes a lot of sense. You read the iobyte value, then direct things as needed to different areas. 

Andrew Lynch's code says this:

; MAP LOGICAL CHARACTER DEVICES TO PHYSICAL CHARACTER DEVICES
	;
	; IOBYTE (0003H)
	; ==============
	;
	; Device LST: PUN: RDR: CON:
	; Bit positions 7 6 5 4 3 2 1 0
	;
	; Dec Binary
	;
	; 0 00 TTY: TTY: TTY: TTY:
	; 1 01 CRT: PTP: PTR: CRT:
	; 2 10 LPT: UP1: UR1: BAT:
	; 3 11 UL1: UP2: UR2: UC1:
	;
	; TTY: Teletype device (slow speed console)
	; CRT: Cathode ray tube device (high speed console)
	; BAT: Batch processing (input from RDR:, output to LST:)
	; UC1: User-defined console
	; PTR: Paper tape reader (high speed reader)
	; UR1: User-defined reader #1
	; UR2: User-defined reader #2
	; PTP: Paper tape punch (high speed punch)
	; UP1: User-defined punch #1
	; UP2: User-defined punch #2
	; LPT: Line printer
	; UL1: User-defined list device #1

And again that does seem to make some sense. In simple terms, if the hardware had four UARTs , the IOBYTE would allow you to map things to each of these ports. 

4) What doesn't work is using the BIOS RDR routine. Looking at CP/M bios calls

	JMP	BOOT	;-3: Cold start routine
	JMP	WBOOT	; 0: Warm boot - reload command processor
	JMP	CONST	; 3: Console status
	JMP	CONIN	; 6: Console input
	JMP	CONOUT	; 9: Console output
	JMP	LIST	;12: Printer output
	JMP	PUNCH	;15: Paper tape punch output
        JMP     READER	;18: Paper tape reader input

the important one there is call 3, console status. There is no equivalent for reader status, so call 18 really should not be used as it will hang the computer if there are no bytes. The iobyte system is far more logical, everything goes through const, conin and conout.

I'm not quite sure how to pull all this together. Assuming a simple board with two uarts, one is for the keyboard/display and one is for communications. The bios can be altered and it can redirect const, conin and conout to either of these 2 ports (and, theoretically, to another 2 as well as iobyte has 2 bits per 'device'). 

I very much want to preserve whatever standards are out there, particularly the N8VEM and RC2014 and Grant Searle's designs.

Still working out where to go next. Possibly hacking the bios so that it also prints out the iobyte status so can truly see what (if any) changes kermit makes as it starts a data transfer.

Cheers, James Moxham

[Alan Cox]

> AFAIK, only CP/M-IDE does Tx buffering, which is useful if you're sending a
> lot characters in one go. I've set it up so that the Tx buffer can be any
> 2^n size, of which 2^n-1 is available. 16 Bytes of Tx buffer seems to be a
> good balance between performance and RAM consumed. I don't suggest it is
> worth increasing much, as the Tx will simply wait if the buffer if full,
> exactly like all the solutions without Tx buffers.

Have you actually measured this out of curiosity. I know when we
measured the Fuzix routines at 115,200 baud the cost of the interrupt
and tx service was actually no different to a simple polling transmit.

Alan

[James Moxham]

oops, google groups has deleted the second half of my post. That will learn me for writing too much at once! After midnight so a bit late to type it all in. Essentially, next step is hacking the bios to print out the iobyte status so can view it as kermit runs and see what it is changed to with various SET PORT commands. Then modify the bios accordingly. But try to copy the N8VEM/RC2014/Grant Searle standards already out there.

Cheers, James Moxham

[phillip.stevens]

Yes, I measured buffer usage with a logic analyser on both RC2014 and YAZ180, when I was experimenting. It also helped substantially with the assembly mandelbrot shark-jumping (mentioned elsewhere).

I'm using a cut through for the ACIA/SIO Tx, which means that the Tx interrupt doesn't trigger if the Tx Byte can be poked directly into the USART (because it was idle). So, the measure of buffer usage is just to count the number of interrupts which occur in series.

The short of the story is that the RC2014 (@7.3728MHz) gets about 5 to 10 bytes into the Tx buffer (based on limited testing), but the YAZ180 (@36.864MHz) can fill any reasonable sized buffer (because losing characters was a bug exposed because I was not 'busy waiting' on buffer exhaustion).

In practice, the Tx buffer only really helps when a bunch of characters need to be "read" into the output from RAM/ROM, such as a VT100 colour code or a printing a string. If the characters are being calculated, then they are generated too slowly and the buffer remains empty.

I would guess that Fuzix has quite a few levels of indirection between the application and the USART Tx register, and that diving through the indirection causes sufficient delay to make a Tx buffer irrelevant.

Phillip

[James Moxham]

Huge amount of progress in the last 24h thanks to Andrew Quinn. Now have Kermit file transfers working between a CP/M board and a terminal program on a PC.

This is using a generic kermit program that did not need recompiling (Kermit-80 v4.11 configured for Generic CP/M-80 with Generic CRT Terminal), and which should work for all CP/M systems. The secret is to modify the CP/M bios (which is in the spirit of CP/M - you supply a modified bios for your particular hardware). Search for any line of code in the bios that does a OUT or IN to the terminal serial port. Replace this with a read of iobyte (which is in ram location 0003) and then if this value is zero, send or receive the bytes as normal, and if the value is 85, send it to the communications serial port. 85 is binary 01010101 and I think this is because it is redirecting to one of four ports 00, 01, 10 or 11. 

Then after running kermit, use SET PORT CRT and this will send the data packets to the comms port, and type the progress on the screen (eg Sending 1 2 3 4 5 Completed).

I used Teraterm on the PC side to test the transfers. Works brilliantly!