SCZ180 supported by z88dk

[Phillip Stevens]

Been fairly productive over the past week, and have added the SC130 (and partial SC126) to the z88dk platform list as scz180.

They are supported by the new library.

Support will eventually cover four startup models.

• rom - which implies that the code relies on no other code, and is usually used to build a bios or monitor.
• hbios - which will be a native app model that will call the ROMWBW HBIOS entry points.
• cpm - which is the only useful model currently, with applications running on either of ROMWBW CP/M or Z-System booted from ROM.
• none - which is no drivers.

The only useful model currently is the cpm startup model. Using this model allows you access to all the z88dk C libraries written for the z180. Including a pretty efficient set of math libraries, optimised for z180 (even if I have to say so myself).

I've finished a working (but still a bit rough and not properly tested) version of the SD FATFS support. Therefore, you can boot from ROM into CP/M, and then access FATFS formatted SD cards completely normally (as if you were on a PC). The application can be stored in the ROMWBW formatted drive, or uploaded using xmodem, and then it can use the ChaN FatFs system calls to access files stored on a FAT partition.

To use this you can...

Install the ff library from z88dk-libs, into your z88dk repository.

z88dk-lib +scz180 -f ff

Then your application can be compiled with this incantation...

zcc +scz180 -subtype=cpm -clib=sdcc_iy -llib/scz180/ff -v -m -SO3 --math32_z180 --max-allocs-per-node80000 diskio_check.c -o diskchk

The resulting diskchk_CODE.bin can be then uploaded from the CP/M or Z-System command line using Xmodem (for example).

CP/M-80 v2.2, 54.0K TPA

B>a:

A>b:xm r diskchk.com

XMODEM v12.5 - 07/13/86
RBC, 28-Aug-2019 [WBW], ASCI

Receiving: A0:DISKCHK.COM
376k available for uploads
File open - ready to receive
To cancel: Ctrl-X, pause, Ctrl-X
CKCK

A>diskchk

test_diskio(0, 3, 0x2C7C, 0x1000)
**** Test cycle 1 of 3 start ****
 disk_initalize(0) - ok.
**** Get drive serial number ****
 disk_ioctl(0, ATA_GET_SN, 0x2C7C) - failed 4.
Serial number of the drive 0 is .
**** Get drive size ****
 disk_ioctl(0, GET_SECTOR_COUNT, 0xD7D1) - ok.
Number of sectors on the drive 0 is 7837696.
**** Get sector size ****
 disk_ioctl(0, GET_SECTOR_SIZE, 0xD7D7) - ok.
 Size of sector is 512 bytes.
**** Get block size ****
 disk_ioctl(0, GET_BLOCK_SIZE, 0xD7D5) - ok.
 Size of the erase block is 160 sectors.
**** Single sector write test ****
 disk_write(0, 0x2C7C, 0, 1) - ok.
 disk_ioctl(0, CTRL_SYNC, NULL) - ok.
 disk_read(0, 0x2C7C, 0, 1) - ok.
 Data matched.
**** Multiple sector write test ****
 disk_write(0, 0x2C7C, 1, 4) - ok.
 disk_ioctl(0, CTRL_SYNC, NULL) - ok.
 disk_read(0, 0x2C7C, 1, 4) - ok.
 Data matched.
**** Single sector write test (misaligned address) ****
 disk_write(0, 0x2C7F, 5, 1) - ok.
 disk_ioctl(0, CTRL_SYNC, NULL) - ok.
 disk_read(0, 0x2C81, 5, 1) - ok.
 Data matched.
**** 4GB barrier test ****
 disk_write(0, 0x2C7C, 6, 1) - ok.
 disk_write(0, 0x2C7C, 5, 1) - ok.
 disk_ioctl(0, CTRL_SYNC, NULL) - ok.
 disk_read(0, 0x2C7C, 6, 1) - ok.
 disk_read(0, 0x2C7C, 5, 1) - ok.
 Data matched.
**** Test cycle 1 of 3 completed ****

--- snip ---

Congratulations! The disk driver works well.

A>

The below screenshot is two iterations of the above test cycle.

And then following screenshots capture the raw transmit and receive rates for the CSIO SPI implementation.

Nearly line rate.

 

Next step is to support an application startup model.

I've been trying to understand enough of ROMWBW to see if there is an API (like BDOS), but I can't discern a clear path.

The other option would be to use the SCM, but it isn't currently available for the SC130. I hope that will be done soon too.

Cheers, Phillip

[Steve Cousins]

Hi Phillip,

You have been busy. Very impressive.

SCM configuration S6 works with SC130. It is the same firmware as used in SC126.

RomWBW does have an API. It is documented in Doc\RomWBW Architecture.pdf in the RomWBW download.

Steve

On Wednesday, 6 November 2019 13:31:10 UTC, Phillip Stevens wrote:

Been fairly productive over the past week, and have added the SC130 (and partial SC126) to the z88dk platform list as scz180.

They are supported by the new library.

Support will eventually cover four startup models.

• rom - which implies that the code relies on no other code, and is usually used to build a bios or monitor.

• app - which will be a native app model that (if it existed) would call the SCM entry points.

• cpm - which is the only useful model currently, with applications running on either of ROMWBW CP/M or Z-System booted from ROM.
• none - which is no drivers.

The only useful model currently is the cpm startup model. Using this model allows you access to all the z88dk C libraries written for the z180.

Including a pretty effective set of math libraries (even if I have to say so myself).

I've finished a working (but still a bit rough and not properly tested) version of the SD FATFS support. Therefore, you can boot from ROM into CP/M, and then access FATFS formatted SD cards completely normally (as if you were on a PC).

To use this... you can... install the ff library from z88dk-libs, into your z88dk repository.

z88dk-lib +scz180 -f ff

Then your applications can be compiled with this incantation,...

zcc +scz180 -subtype=cpm -clib=sdcc_iy -llib/scz180/ff -v -m -SO3 --math32_z180 --max-allocs-per-node80000 @diskio_check.c -o diskchk

[Phillip Stevens]

Steve Cousins wrote:

Hi Phillip,

RomWBW does have an API. It is documented in Doc\RomWBW Architecture.pdf in the RomWBW download.

Thanks for restraining yourself from writing "RTFM" in your comment. ;-)

It is all there. I just need to read the document (rather than the code).

Cheers, Phillip

[Phillip Stevens]

Phillip Stevens wrote:

Been fairly productive over the past week, and have added the SC130 (and partial SC126) to the z88dk platform list as scz180.

Support will eventually cover four startup models.

• rom - which implies that the code relies on no other code, and is usually used to build a bios or monitor.
• hbios - which will be a native app model that will call the ROMWBW HBIOS entry points.
• cpm - which is the only useful model currently, with applications running on either of ROMWBW CP/M or Z-System booted from ROM.
• none - which is no drivers.

The only useful model currently is the cpm startup model. Using this model allows you access to all the z88dk C libraries written for the z180. Including a pretty efficient set of math libraries, optimised for z180 (even if I have to say so myself).

Next step is to support an application startup model.

Ok, well the next step is now well underway. I've committed the hbios subtype, which now has direct connectivity to the HBIOS serial drivers for the SC126 and SC130.

ROMWBW has a debug monitor dbgmon that can load Intel Hex, and it has two versions of CP/M and other programs also in ROM.

This subtype utilises the ROMWBW HBIOS system calls (call 0xFFF0) to provide two terminals stdin, stdout, stderr, and ttyin, ttyout, and ttyerr, on asci0 and asci1 respectively.

A resulting application example_CODE.bin (or example.bin) can be uploaded using the CP/M xmodem function and then run. If it doesn't overwrite the CP/M BDOS/BIOS when the data and bss sections are expanded and with its heap and stack usage, then when the application exits it will return to CP/M, otherwise the hardware will be restarted, and reboots into dgbmon. Alternatively an Intel Hex file (example.ihx) can be uploaded by the dgbmon directly using the L function.

The application is loaded at 0x0100 and initially places its stack at 0xF000, which is the future base of the dbgmon. Wayne is planning to move the dbgmon from 0xC000 currently to 0xF000 to provide more uninterrupted space for the z88dk app subtype from 0x0100 through to 0xF000.

TO DO (in later smaller commits).

 - connect the sys/time.h functions, pending PR #54 at ROMWBW.

 - connect the hbios disk functions to FatFs, so long as z88dk newlib doesn't have file acccess functions.

 - bank switching so app can expand beyond one page.

[Marten Feldtmann]

That's nice stuff ! Thank you very much. What are the differences between the support for SC126 and SC130 ? Limitations ? Missing features ?

Marten

[Phillip Stevens]

Marten Feldtmann wrote:

That's nice stuff ! Thank you very much. What are the differences between the support for SC126 and SC130 ?

Basically just my ability to test. I have a SC130, so I can only test hardware that I have.

That means that the second SPI port isn't tested, for example. Ideally, the SPI CS control functions should be extended. I was lazy.

And the LED & GPIO aren't tested. But that's not a big deal.

 

Limitations ? Missing features ?

The z88dk new library doesn't have a C file interface yet. Its on the long term to do.

So I'll need to hook the ChaN FatFs code into the HBIOS disk interface. I assume that this won't be too hard,.. but it isn't done yet.

Might be a few weeks out though, as I'll be AFK for a few weekends.

Cheers, Phillip

[Phillip Stevens]

Phillip Stevens wrote:

Been fairly productive over the past week, and have added the SC130 (and partial SC126) to the z88dk platform list as scz180.

 

Ok, well the next step is now well underway. I've committed the app subtype, which now has direct connectivity to the HBIOS serial drivers for the SC126 and SC130.

This subtype utilises the ROMWBW HBIOS system calls (call 0xFFF0) to provide two terminals stdin, stdout, stderr, and ttyin, ttyout, and ttyerr, on asci0 and asci1 respectively.

A resulting application example_CODE.bin (or example.bin) can be uploaded using the CP/M xmodem function and then run. If it doesn't overwrite the CP/M BDOS/BIOS when the data and bss sections are expanded and with its heap and stack usage, when the application exits it will return to CP/M, otherwise the hardware will be restarted, and reboots into dgbmon.

 

Alternatively an Intel Hex file can be uploaded by the dgbmon directly (although this is much slower in practice).

The application is loaded at 0x0100 and initially places its stack at 0xF000, which is the base of the dbgmon. Wayne has moved the dbgmon to 0xF000 to provide uninterrupted space for the z88dk app subtype from 0x0100 through to 0xF000.

TO DO (in later smaller commits).

 - connect the sys/time.h functions, pending PR #54 at ROMWBW.

Now the 'nix like <sys/time.h> functions have been completed. This means that a standard 'nix time stamp can be loaded into a 32 bit time_t register, and real time will be maintained. There is a 20ms resolution on the timer, which is reported in nanoseconds as per the standard. The Y2K epoch is offset from the 'nix epoch by the UNIX_OFFSET of 946684800 seconds.

How does this work?

Start with this program...

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <time.h>
#include <sys/time.h>
#include <arch/scz180/system_time.h>

#pragma printf = "%li %lu %s %u" // enables printf format conversion for %li %lu %s, %u only, to save space


#define UNIX_OFFSET 946684800UL

void main(void)
{
 struct timespec startTime, endTime, resTime;
 
 resTime.tv_sec = 1573555318 - UNIX_OFFSET; // it was about this time when I tested. to get timestamp: date +%s
 
// clock_settime(CLOCK_REALTIME,&resTime); // set the real time once, and it will be maintained through reboots

 clock_gettime(CLOCK_REALTIME,&startTime);
 printf("Start: %li sec %lu msec\n", startTime.tv_sec, startTime.tv_nsec/1000000 );

 fprintf(stdout,"Hello World R\nHello World C\nHello World 2\nHello World 0\nHello World 1\nHello World 4\n");

 for (uint16_t i= 0; i < 512; ++i) {
 
 clock_gettime(CLOCK_REALTIME,&endTime);
 printf("End: %li sec %lu msec\n", endTime.tv_sec, endTime.tv_nsec/1000000 );
 
 timersub(&endTime, &startTime, &resTime);
 printf("Elapsed: %li sec %lu msec\n", resTime.tv_sec, resTime.tv_nsec/1000000 );
 }
}

Use this command line

zcc +scz180 -subtype=app -SO3 -clib=sdcc_iy -v -m --list --math32_z180 --c-code-in-asm --max-allocs-per-node40000 test.c -o test -create-app

This will generate test_CODE.bin which can be uploaded to ROMWBW CP/M using XM , from drive a:

A> b:xm r test.com

and send it using xmodem, and run it in the usual way.

Enjoy, Phillip

Breaking down the command line for reference.

zcc <- the z88dk preprocessor. this calls everything
+scz180 <- the target in this case the SC130, but it will also work for SC126 (limited hardware support though)
-subtype=app <- the specific goal for the compilation, whether app for HBIOS or cpm for CPM or rom for raw hardware
-SO3 <- optimise strongly, always do this, unless you suspect a compiler bug
-clib=sdcc_iy <- use the zsdcc (sdcc) compiler, together with the new library in iy variant vs. -clib=new for sccz80 and new libarary
-v <- verbose
-m <- provide a map file, needed to create the app
--list <- provide listings
--math32_z180 <- use the 32 bit IEEE z180 specific library
--c-code-in-asm <- provide C comments in assembly (easier to read listings), but breaks some optimisation for sccz80
--max-allocs-per-node40000 <- optimise register allocation for sdcc reasonably strongly 1000000 to 10000 are sensible values
test.c <- the input file, can also be a number of files if proceeded by @. i.e. @test.lst
-o test <- output file root name
-create-app <- build an intel hex application

[Phillip Stevens]

Phillip Stevens wrote:

Added the SC130 (and partial SC126) to the z88dk platform list as scz180.

I've committed the hbios subtype, which now has direct connectivity to the HBIOS serial drivers for the SC126 and SC130.

This subtype utilises the ROMWBW HBIOS system calls (call 0xFFF0) to provide two terminals stdin, stdout, stderr, and ttyin, ttyout, and ttyerr, on asci0 and asci1 respectively.

Now the 'nix like <sys/time.h> functions have been completed. This means that a standard 'nix time stamp can be loaded into a 32 bit time_t register, and real time will be maintained. There is a 20ms resolution on the timer, which is reported in nanoseconds as per the standard. The Y2K epoch is offset from the 'nix epoch by the UNIX_OFFSET of 946684800 seconds.

Just added the Michael Duane Rice code for esoteric time calculations, so now it is easy to get nice things like fully broken down and formatted time in applications.

Tue Nov 12 10:41:58 2019

with a few extra lines in the code.

#include <lib/scz180/time.h>

void main(void)
{
 time_t theTime;
 struct tm CurrTimeDate; // set up an array for the time info.
 char timeStore[26];

// snip

 time(&theTime);
 localtime_r(&theTime, &CurrTimeDate);
 asctime_r(&CurrTimeDate, timeStore);
 fprintf(stdout, "%s\n", timeStore);

// snip

 fprintf(ttyout, "SYSTEM Time: %s\r\n", ctime( (time_t *)&theTime)); // optionally all in one line, on the second serial port

}

Just substitute SCZ180 for YAZ180 in the readme instructions. The libraries are already compiled, they just need to be installed with the tool.

> z88dk-lib +scz180 -f time

And add the library to the command line.

> zcc +scz180 -subtype=app -SO3 -v -m -clib=sdcc_iy -llib/scz180/time --math32_z180 --max-allocs-per-node40000 test.c -o test

The library also includes a set of functions related the sun and moon, as well as sidereal time functions.

Admittedly, I've never built a sidereal time clock using these functions.

Cheers, Phillip

[Phillip Stevens]

Phillip Stevens wrote:

I have added the SC130 (and partial SC126) to the z88dk platform list as scz180.

Support will eventually cover four startup models.

• rom - which implies that the code relies on no other code, and is usually used to build a bios or monitor.

• hbios - which is a native app model that will call the ROMWBW HBIOS entry points.
• cpm - which is for applications running on either of ROMWBW CP/M or Z-System booted from ROM.

• none - which is no drivers.

The most useful models currently are the hbios and  cpm startup models. Using these models allows you access to all the z88dk C libraries written for the z180. Including a pretty efficient set of math libraries, optimised for z180 (even if I have to say so myself).

I've committed the hbios subtype, which now has direct connectivity to the HBIOS serial drivers for the SC126 and SC130.

ROMWBW has a debug monitor dbgmon that can load Intel Hex from serial, and it has two versions of CP/M and other programs also in ROM.

This subtype utilises the ROMWBW HBIOS system calls (call 0xFFF0) to provide two terminals stdin, stdout, stderr, and ttyin, ttyout, and ttyerr, on asci0 and asci1 respectively.

A resulting application example.bin can be uploaded using the CP/M xmodem function and then run as EXAMPLE.COM. If it doesn't overwrite the CP/M BDOS/BIOS when the data and bss sections are expanded and with its heap and stack usage, then when the application exits it will return to CP/M, otherwise the hardware will be restarted, and reboots into dgbmon. 

 

Alternatively an Intel Hex file (example.ihx) can be uploaded by the dgbmon directly using the L function. So reboot and then M L to load a new version of your program. I use cat on linux to simply upload the hex file. Now that dbgmon has been optimised a little, I find it can keep up with 115200 baud cat with the scz180 platform.

The application is loaded at 0x0100 and initially places its stack at 0xF000, which is the base of the dbgmon. Wayne has moved the dbgmon to 0xF000 to provide more uninterrupted space for the z88dk hbios subtype.

TO DO (in later smaller commits).

 - connect the sys/time.h functions, pending PR #54 at ROMWBW. - DONE

 - connect the hbios disk functions to FatFs, so long as z88dk newlib doesn't have file acccess functions. - DONE

 - bank switching so app can expand beyond one page.

Now the final piece of  support has been added to the ROMWBW HBIOS to allow the use of applications that need to have FAT32 formatted SD cards.

This -subtype=hbios model uses the drivers present in the HBIOS to provide the largest possible application space, and direct access to all HBIOS functions.

The hbios subtype assumes you will run code from 0x0100, and doesn't overwrite the monitor at 0xF000. Although really, all that is needed to be preserved is the hbios stubs in the upper few bytes from 0xFE00, if you're pushed for space.

Using these functions and libraries for the z88dk scz180 target (also rc2014 target, but that's another story), it is possible to read and write SD cards formatted with FAT32 (and smaller FATs too but why bother), as well as using all the other HBIOS API calls. So a GIF image can be read from a FAT SD card, decompressed and output to a screen, for example.

The new thing comes in three parts.

1. writing the C interfaces to call HBIOS functions, and
2. interfacing ChaN FatFS to provide diskio functions that call the HBIOS, and
3. providing the standard ChaN FatFS layer, and writing applications.

1. The z88dk now supports a variety of functions that call the HBIOS. The HBIOS API is not orthogonal, so writing clean functions is a bit tricky, but I've got a basic set that allow most things to be done. Because each function is generic (can be used to call any HBIOS function) they are simply named after the registers they provide and return. So just read the ROMWBW API and see which calling style function you need to use.

// return DEHL, function BC
uint32_t hbios(uint16_t func);

// return A, function BC
uint8_t hbios_a(uint16_t func);

// return A, function BC, arg DE
uint8_t hbios_a_de(uint16_t func, uint16_t arg);

// return A, function BC, arg DEHL
uint8_t hbios_a_dehl(uint16_t func, uint32_t arg);

// return A, function BC, arg DE, void * HL
uint8_t hbios_a_de_hl(uint16_t func, uint16_t arg, void * buffer);

// return E, function BC
extern uint8_t hbios_e(uint16_t func);

// return E, function BC, arg DE
extern uint8_t hbios_e_de(uint16_t func, uint16_t arg);

// return E, function BC, arg DEHL
uint8_t hbios_e_dehl(uint16_t func, uint32_t arg);

// return E, function BC, arg DE, void * HL
uint8_t hbios_e_de_hl(uint16_t func, uint16_t arg, void * buffer);

2. Using these HBIOS functions for the required diskio layer for ChaN FatFS looks like below example.

Note that the BC registers are usually used by HBIOS to carry the function and parameter. Further parameters are passed in DE, HL, or DEHL.

Depending on whether you want to use the result returned in A or E, you can choose the function that produces the correct outcome.

/*-----------------------------------------------------------------------*/
/* Write Sector(s)                                                       */
/*-----------------------------------------------------------------------*/

DRESULT disk_write (
    BYTE pdrv,              /* Physical drive number */
    const BYTE *buff,       /* Pointer to the data to be written */
    LBA_t sector,           /* Start sector number (LBA) */
    UINT count              /* Sector count (1..128) */
)
{
    uint8_t wattempt = WRITE_ATTEMPTS;      /* Write attempts */
    uint8_t resp = 0;

    if (count == 0 ) return RES_PARERR;     /* sector count can't be zero */

    do {
        if ( hbios_a_dehl( BF_DIOSEEK<<8|pdrv, sector|LBA_ADDRESS ) == 0) {
            resp = hbios_e_de_hl( BF_DIOWRITE<<8|pdrv, count, buff );
            if ( resp == count ) resp = RES_OK;
        }
    } while  ((--wattempt != 0) && (resp != RES_OK) );

    return resp;
}

3. The ChaN FatFS library is ostensibly the easiest component, as it usually just works (once you've got diskio working).

To use the FatFS library, you need to install both the time library and the ff library for scz180, as the FAT file system will update the file information with the real-time counter in the scz180.

Then finally, an example program looks like this.

/*----------------------------------------------------------------------*/
/* Foolproof FatFs sample project for Z80           (C)feilipu, 2019    */
/*----------------------------------------------------------------------*/

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include "ffconf.h"            /* FatFs configuration. Use same file as you used to compile library */

#if __YAZ180
#include <arch/yaz180.h>
#include <lib/yaz180/ff.h>     /* Declarations of FatFs API */
#elif __SCZ180
#include <arch/scz180.h>
#include <lib/scz180/ff.h>     /* Declarations of FatFs API */
#elif __RC2014
#include <arch/rc2014.h>
#include <lib/rc2014/ff.h>     /* Declarations of FatFs API */
#else
#warning - no FatFs library available
#endif


#pragma printf = "%s %u"      /* Reduce program size by removing unneeded format converters */

#pragma output CRT_ORG_BSS = 0x9000   /* move bss origin to address 0x9000 (check map for overlap between data and bss) */

// zcc +rc2014 -subtype=hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/hbios/diskio_hbios -llib/rc2014/ff ffmain.c -o ffmain -create-app

// zcc +scz180 -subtype=hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/scz180/time -llib/scz180/diskio_sd -llib/scz180/ff ffmain.c -o ffmain -create-app

// zcc +scz180 -subtype=hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/scz180/time -llib/hbios/diskio_hbios -llib/hbios/ff ffmain.c -o ffmain -create-app

// zcc +hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/hbios/time -llib/hbios/diskio_hbios -llib/hbios/ff ffmain.c -o ffmain -create-app

// cat > /dev/ttyUSB0 < fftest.ihx

static FATFS FatFs;         /* FatFs work area needed for each volume */
static FIL Fil;             /* File object needed for each open file */

static char buffer[128];    /* must be in bss (which is above 0x8000) for romwbw hbios */

void main (void)
{
    UINT bw;
    UINT br;
    FRESULT res;

    res = f_mount(&FatFs, "2:", 1);                 /* Give a work area to the SD drive */
    if(res != FR_OK) printf("\nf_mount error #%u\r\n", res);


    if (f_open(&Fil, "2:bfile.txt", FA_CREATE_ALWAYS | FA_WRITE | FA_READ ) == FR_OK)
    {    /* Create a file if needed */

        res = f_write(&Fil, "It works!\r\n", 11, &bw);  /* Write data to the file */
        if(res != FR_OK) printf("\nf_write error #%u\r\n", res);
       
        br = 0;
        res = f_read(&Fil, buffer, 128, &br);            /* Read data from the file */
        if(res != FR_OK) printf("\f_read error #%u\r\n", res);

        res = f_close(&Fil);                             /* Close the file */
        if(res != FR_OK) printf("\nf_close error #%u\r\n", res);        

        printf("\r\n%u bytes: %s\r\n", br, buffer);
        if(br > 0) printf("It works!\r\n");
    }
    else
    {
        printf("\nf_open error #%u\r\n", res);
    }


    f_mount(0, "2:", 0);                                /* Unmount and free work area */
}

Let me know if you get this working (or conversely don't).

An interesting challenge would be to load a C monitor that could read and load applications from the SD card, and reload itself (via a small stub) when the application was done.

Cheers, Phillip

[Wayne Warthen]

I am very pleased to see this integration of RomWBW HBIOS into z88dk.

Thanks!

Wayne

On Monday, December 9, 2019 at 3:56:07 AM UTC-7, Phillip Stevens wrote:

Phillip Stevens wrote:

I have added the SC130 (and partial SC126) to the z88dk platform list as scz180.

Support will eventually cover four startup models.

• rom - which implies that the code relies on no other code, and is usually used to build a bios or monitor.
• hbios - which is a native app model that will call the ROMWBW HBIOS entry points.
• cpm - which is for applications running on either of ROMWBW CP/M or Z-System booted from ROM.
• none - which is no drivers.

The most useful models currently are the hbios and  cpm startup models. Using these models allows you access to all the z88dk C libraries written for the z180. Including a pretty efficient set of math libraries, optimised for z180 (even if I have to say so myself).

I've committed the hbios subtype, which now has direct connectivity to the HBIOS serial drivers for the SC126 and SC130.

ROMWBW has a debug monitor dbgmon that can load Intel Hex from serial, and it has two versions of CP/M and other programs also in ROM.

This subtype utilises the ROMWBW HBIOS system calls (call 0xFFF0) to provide two terminals stdin, stdout, stderr, and ttyin, ttyout, and ttyerr, on asci0 and asci1 respectively.

A resulting application example.bin can be uploaded using the CP/M xmodem function and then run as EXAMPLE.COM. If it doesn't overwrite the CP/M BDOS/BIOS when the data and bss sections are expanded and with its heap and stack usage, then when the application exits it will return to CP/M, otherwise the hardware will be restarted, and reboots into dgbmon. 

 

Alternatively an Intel Hex file (example.ihx) can be uploaded by the dgbmon directly using the L function. So reboot and then M L to load a new version of your program. I use cat on linux to simply upload the hex file. Now that dbgmon has been optimised a little, I find it can keep up with 115200 baud cat with the scz180 platform.

The application is loaded at 0x0100 and initially places its stack at 0xF000, which is the base of the dbgmon. Wayne has moved the dbgmon to 0xF000 to provide more uninterrupted space for the z88dk hbios subtype.

TO DO (in later smaller commits).

 - connect the sys/time.h functions, pending PR #54 at ROMWBW. - DONE

 - connect the hbios disk functions to FatFs, so long as z88dk newlib doesn't have file acccess functions. - DONE

 - bank switching so app can expand beyond one page.

Now the final piece of  support has been added to the ROMWBW HBIOS to allow the use of applications that need to have FAT32 formatted SD cards.

This -subtype=hbios model uses the drivers present in the HBIOS to provide the largest possible application space, and direct access to all HBIOS functions.

The hbios subtype assumes you will run code from 0x0100, and doesn't overwrite the monitor at 0xF000. Although really, all that is needed to be preserved is the hbios stubs in the upper few bytes, if you're pushed for space.

// zcc +scz180 -subtype=hbios -v -m -SO3 -clib=sdcc_iy -llib/scz180/time -llib/scz180/ff --math32_z180 --list --max-allocs-per-node100000 fftest.c -o fftest -create-app

// cat > /dev/ttyUSB0 < fftest.ihx

static FATFS FatFs;         /* FatFs work area needed for each volume */
static FIL Fil;             /* File object needed for each open file */

static char buffer[128];    /* must be in bss (which is above 0x8000) for romwbw hbios */

void main (void)
{
    UINT bw;
    UINT br;
    FRESULT res;

    res = f_mount(&FatFs, "2:", 1);                 /* Give a work area to the SD drive */
    if(res != FR_OK) printf("\nf_mount error #%u\r\n", res);

    if (f_open(&Fil, "2:bfile.txt", FA_OPEN_ALWAYS | FA_WRITE | FA_READ ) == FR_OK)

    {    /* Create a file if needed */

        res = f_write(&Fil, "It works!\r\n", 11, &bw);  /* Write data to the file */
        if(res != FR_OK) printf("\nf_write error #%u\r\n", res);
       
        br = 0;
        res = f_read(&Fil, buffer, 128, &br);            /* Read data from the file */
        if(res != FR_OK) printf("\f_read error #%u\r\n", res);

        res = f_close(&Fil);                             /* Close the file */
        if(res != FR_OK) printf("\nf_close error #%u\r\n", res);        

        printf("\r\n%u bytes: %s\r\n", br, buffer);
        if(br > 0) printf("It works!\r\n");
    }
    else
    {
        printf("\nf_open error #%u\r\n", res);
    }


    f_mount(0, "2:", 0);                                /* Unmount and free work area */
}

[Phillip Stevens]

Wayne Warthen wrote:

I am very pleased to see this integration of RomWBW HBIOS into z88dk.

My pleasure. It helps to integrate different code sets, and it certainly adds to the z88dk platform options too.

Also, having ChaN FAT FS available to the z88dk newlib is pretty useful as FAT drivers are not included in the library as yet.

Long post on the latest below.

Phillip Stevens wrote:

I have added the SC130 (and partial SC126) to the z88dk platform list as scz180.

Support will eventually cover four startup models.

• rom - which implies that the code relies on no other code, and is usually used to build a bios or monitor.
• hbios - which is a native app model that will call the ROMWBW HBIOS entry points.
• cpm - which is for applications running on either of ROMWBW CP/M or Z-System booted from ROM.
• none - which is no drivers.

The most useful models currently are the hbios and cpm startup models. Using these models allows you access to all the z88dk C libraries written for the z180.

Its been quite a few weeks of moving code around, fixing issues, and general festivities, but now this is pretty much all finished and working.

For the first time, I can now do accurate analysis of the performance of IDE SDC and CF vs SD Cards, and have a single model for driving all of the SCZ180 (sc126, sc130), RC2014, and all the other HBIOS enabled platforms with ROMWBW.

Part of the work was separating the ChaN FAT File System from the underlying diskio drivers on each platform, and then making it possible to add different parts of the libraries depending on which drivers are required.

In summary, the SCZ180 platform just has CSIO SD drivers included in z88dk. An additional library to provide the diskio layer for the CSIO SD Card has been created. In parallel a library to provide diskio off the back of the HBIOS API has also been created. When building applications, either CSIO SD or HBIOS SD (or any other hardware) interfaces can then be chosen at build time, by linking from the command line.

The RC2014 platform has an 82C55 IDE based diskio platform included in z88dk. But if needed any another diskio layer can be linked from the command line, and this will override the integrated version.

Any other HBIOS equipped platform can use the hbios target, together with the diskio layer using the HBIOS API, to access the ChaN FAT file system library code.

So what does the user code look like? Glad you asked... this is to copy one file "random1.txt" to another file "random2.txt".

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <time.h>
#include <sys/time.h>


#if __YAZ180
#include <arch/yaz180.h>
#include <lib/yaz180/time.h>        /* Declaration of system time */
#elif __SCZ180
#include <arch/scz180.h>
#include <lib/scz180/time.h>        /* Declaration of system time */
#elif __HBIOS
#include <lib/scz180/time.h>        /* Declaration of system time */
#elif __RC2014

#include <lib/rc2014/time.h>        /* Declaration of system time */
#warning No time measurement possible.
#else
#error Do you have time?
#endif


#include "ffconf.h"


#if __YAZ180

#include <lib/yaz180/ff.h>          /* Declarations of FatFs API */

#include <arch/yaz180/diskio.h>     /* Declarations of diskio & IDE functions */
//#elif __RC2014
//#include <lib/rc2014/ff.h>        /* Declarations of FatFs API */
//#include <arch/rc2014/diskio.h>   /* Declarations of diskio & IDE functions */
#elif __RC2014
#include <lib/hbios/ff.h>           /* Declarations of FatFs API */
#include <lib/hbios/diskio_hbios.h> /* Declarations of diskio functions */
#include <arch/hbios.h>             /* Declarations of HBIOS functions */
#elif __SCZ180

#include <lib/scz180/ff.h>         /* Declarations of FatFs API */

#include <lib/scz180/diskio_sd.h>  /* Declarations of diskio functions */
#include <arch/scz180.h>           /* Declarations of SD functions */
//#elif __SCZ180
//#include <lib/hbios/ff.h>           /* Declarations of FatFs API */
//#include <lib/hbios/diskio_hbios.h> /* Declarations of diskio functions */
//#include <arch/hbios.h>             /* Declarations of HBIOS functions */
#elif __HBIOS
#include <lib/hbios/ff.h>           /* Declarations of FatFs API */
#include <lib/hbios/diskio_hbios.h> /* Declarations of diskio functions */
#include <arch/hbios.h>             /* Declarations of HBIOS functions */

#else
#warning - no FatFs library available
#endif

#pragma output CRT_ORG_BSS = 0x9000     // move bss origin to address 0x9000 (check map to confirm there is no overlap between data and bss sections)
#pragma printf = "%s %c %u %li %lu"     // enables %s, %c, %u, %li, %lu only

// zcc +yaz180 -subtype=app -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/yaz180/time -llib/yaz180/ff fileCopyTest.c -o fileCopyTest -create-app

// zcc +rc2014 -subtype=cpm -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/rc2014/ff fileCopyTest.c -o fileCopyTest -create-app
// zcc +rc2014 -subtype=hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/hbios/time -llib/hbios/diskio_hbios -llib/hbios/ff fileCopyTest.c -o fileCopyTest -create-app

// zcc +scz180 -subtype=hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/scz180/time -llib/scz180/diskio_sd -llib/scz180/ff fileCopyTest.c -o fileCopyTest -create-app
// zcc +scz180 -subtype=hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/hbios/time -llib/hbios/diskio_hbios -llib/hbios/ff fileCopyTest.c -o fileCopyTest -create-app

// zcc +hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/hbios/time -llib/hbios/diskio_hbios -llib/hbios/ff fileCopyTest.c -o fileCopyTest -create-app


static FILINFO Finfo;

static FATFS FatFs;                 /* FatFs work area needed for each volume */

static FIL FileIn, FileOut;         /* File object needed for each open file */

static BYTE buffer[4096];           /* 4kB working buffer */

int main (void)
{
    UINT bw;
    UINT br;
    DWORD bwt = 0;

    FRESULT res;

    struct timespec startTime, endTime, resTime;

    startTime.tv_sec = 1577836800 - UNIX_OFFSET;

    clock_settime(CLOCK_REALTIME, &startTime);                  /* Set the time of day, y2k epoch */

    if ((res = f_mount(&FatFs, "2:", 1)) == FR_OK) {            /* Give a work area to the 0th or 2nd drive */

   
        printf("\r\n\nFatFs.fs_type %u\nFatFs.fsize %lu\n", FatFs.fs_type, FatFs.fsize);
        printf("\r\nOpening random1.txt");

        if ((res = f_open(&FileIn, "2:random1.txt", FA_READ)) == FR_OK) { /* "0:random1.txt" for SD drivers */

            printf(" - Opened");

            if ((res = f_lseek(&FileIn, 0)) == FR_OK) {

                printf("\r\nCreating random2.txt");

                if ((res = f_open(&FileOut, "2:random2.txt", FA_CREATE_ALWAYS | FA_WRITE)) == FR_OK) { /* "0:random2.txt" for SD drivers */

                    printf(" - Created\r\n\nCopying...");

                    clock_gettime(CLOCK_REALTIME,&startTime);

                    for (;;) {
                        res = f_read(&FileIn, buffer, sizeof(buffer), &br);
                        if (res != FR_OK || br == 0) break;     // eof or error
                        res = f_write(&FileOut, buffer, br, &bw);
                        bwt += bw;
                        if (res != FR_OK || bw < br) break;     // error or disk full
                    }

                    clock_gettime(CLOCK_REALTIME,&endTime);

                    f_close(&FileOut);

                    if ((res = f_unlink("2:random2.txt")) != FR_OK) {
                        printf("\r\nCouldn't delete random2.txt - f_unlink error #%u\r\n", res);
                    }
                } else {
                    printf("\r\nCouldn't open random2.txt - f_open error #%u\r\n", res);                
                }
             } else {
                printf("\r\nCouldn't seek random1.txt - f_lseek error #%u\r\n", res);
            }
            f_close(&FileIn);
        } else {
            printf("\r\nCouldn't open random1.txt - f_open error #%u\r\n", res);
        }

        timersub(&endTime, &startTime, &resTime);

        printf("\r\nCopied %lu bytes", bwt );

        printf(", the time taken was %li.%.4lu seconds\n", resTime.tv_sec, resTime.tv_nsec/100000 );

        f_mount(0, "2:", 0);                                    /* Free work area */
    } else {
        printf("\r\nCouldn't mount drive - f_mount error #%u\r\n", res);
    }
    return 0;
}

Once the program has been prepared, it can be loaded into the ROMWBW dgbmon, as demonstrated below.

RetroBrew HBIOS v2.9.2-pre.22, 2019-11-30

SC130 Z8S180-N @ 36.864MHz IO=0xC0
1 MEM W/S, 2 I/O W/S, INT MODE 2
512KB ROM, 512KB RAM

ASCI0: IO=0xC0 ASCI W/BRG MODE=115200,8,N,1
ASCI1: IO=0xC1 ASCI W/BRG MODE=115200,8,N,1
DSRTC: MODE=STD IO=0x0C NOT PRESENT
MD: UNITS=2 ROMDISK=384KB RAMDISK=384KB
PPIDE: IO=0x20 NOT PRESENT
SD: MODE=SC OPR=0x0C CNTR=0xCA TRDR=0xCB DEVICES=1
SD0: SDHC NAME=NCard BLOCKS=0x00779800 SIZE=3827MB

Unit        Device      Type              Capacity/Mode
----------  ----------  ----------------  --------------------
Char 0      ASCI0:      RS-232            115200,8,N,1
Char 1      ASCI1:      RS-232            115200,8,N,1
Disk 0      MD1:        RAM Disk          384KB,LBA
Disk 1      MD0:        ROM Disk          384KB,LBA
Disk 2      SD0:        SD Card           3827MB,LBA

SC130 Boot Loader

ROM: (M)onitor (C)P/M (Z)-System (F)orth (B)ASIC (T)-BASIC  
Disk: (0)MD1 (1)MD0 (2)SD0


Boot Selection? M

Loading Monitor...

Monitor Ready (H for Help)
>L

Loaded
>R100

FatFs.fs_type 3
FatFs.fsize 7640

Opening random1.txt - Opened
Creating random2.txt - Created

Copying...
Copied 1048576 bytes, the time taken was 18.3800 seconds

Monitor Ready (H for Help)
>L

Loaded
>R100

FatFs.fs_type 3
FatFs.fsize 7640

Opening random1.txt - Opened
Creating random2.txt - Created

Copying...
Copied 1048576 bytes, the time taken was 44.4400 seconds

Monitor Ready (H for Help)
>

The above is an example of two separate builds. Firstly using the CSIO SD Card drivers, and secondly using the HBIOS API.

The extra time taken by HBIOS is because of the bank switching and other overheads associated with the API.

Tthe question of how fast is an SD Card relative to a 82C55 Compact Flash IDE Drive can be answered by adding the following run.

YAZ180 - yabios - CRT
> :? :-)

> loadh 3
::####################

Done!
Loaded Bank: 3

> mkb 3
Initialised Bank: 3

> initb 3

FatFs.fs_type 3
FatFs.fsize 1816

Opening random1.txt - Opened
Creating random2.txt - Created

Copying...
Copied 1048576 bytes, the time taken was 7.0429 seconds

>

And the question of how fast is an SD Card relative to a 82C55 SSD IDE Drive can be answered by this final run.

YAZ180 - yabios - CRT
> :? :-)

> loadh 3
::############################

Done!
Loaded Bank: 3

> mkb 3
Initialised Bank: 3

> initb 3

FatFs.fs_type 3
FatFs.fsize 15161

Opening random1.txt - Opened
Creating random2.txt - Created

Copying...
Copied 1048576 bytes, the time taken was 5.0625 seconds

>

At the C level for user code I get the following results:

• SD Card (CSIO) can produce about 55kB/s throughput. (EDITED)
• CF IDE Drive (82C55) can produce about 295kB/s throughput.
• SSD IDE Drive (82C55) can produce about 410kB/s throughput.

The TL;DR, the SD Card performance is not that bad, and is only about half as fast as the parallel interface storage.

I was expecting it to be about one tenth the performance.

(EDIT - Actually it is about one sixth, more in line with expectation).

Happy New Year,

Phillip

[Bill Shen]

Very nice work.  Two questions/comments:
1.  The CPU clock strongly affects the file transfer speed.  You used SCZ180 for the benchmark, but I also saw a comment about 36.8MHz clock.  So was the SCZ180's clock doubled for this benchmark?
2.  My experience with CF disks is that fastest CF disk can be almost as fast as DiskOnModule which probably not as fast as SSD.  The slowest CF disk can be 1/3 of the fastest CF disk.  These are limited hand-on experiences with older generation of 32M-512M CF disks. 
  Bill

In summary, the SCZ180 platform just has CISO SD drivers included in z88dk. An additional library to provide the diskio layer for the CISO SD Card has been created. In parallel a library to provide diskio off the back of the HBIOS API has also been created. When building applications, either CSIO SD or HBIOS SD (or any other hardware) interfaces can then be chosen at build time, by linking from the command line.

The RC2014 platform has an 82C55 IDE based diskio platform included in z88dk. But if needed any another diskio layer can be linked from the command line, and this will override the integrated version.

Any other HBIOS equipped platform can use the hbios target, together with the diskio layer using the HBIOS API, to access the ChaN FAT file system library code.

So what does the user code look like? Glad you asked... this is to copy one file "random1.txt" to another file "random2.txt".

#include <stdlib.h>
#include <stdio.h>
#include <string.h>


#include <time.h>
#include <sys/time.h>


#if __YAZ180
#include <arch/yaz180.h>
#include <lib/yaz180/time.h>        /* Declaration of system time */
#elif __SCZ180
#include <arch/scz180.h>
#include <lib/scz180/time.h>        /* Declaration of system time */
#elif __HBIOS
#include <lib/scz180/time.h>        /* Declaration of system time */
#elif __RC2014

#warning No time possible

#include "ffconf.h"

#pragma output CRT_ORG_BSS = 0x9000     // move bss origin to address 0x9000 (check map to confirm there is no overlap between data and bss sections)
#pragma printf = "%s %c %u %li %lu"     // enables %s, %c, %u, %li, %lu only

// zcc +yaz180 -subtype=app -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/yaz180/time -llib/yaz180/ff fileCopyTest.c -o fileCopyTest -create-app

// zcc +rc2014 -subtype=cpm -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/rc2014/ff fileCopyTest.c -o fileCopyTest -create-app

// zcc +rc2014 -subtype=hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/hbios/diskio_hbios -llib/rc2014/ff fileCopyTest.c -o fileCopyTest -create-app

// zcc +scz180 -subtype=hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/scz180/time -llib/scz180/diskio_sd -llib/scz180/ff fileCopyTest.c -o fileCopyTest -create-app

// zcc +scz180 -subtype=hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/scz180/time -llib/hbios/diskio_hbios -llib/hbios/ff fileCopyTest.c -o fileCopyTest -create-app

// zcc +hbios -clib=sdcc_iy -SO3 -v -m --list --max-allocs-per-node100000 -llib/scz180/time -llib/hbios/diskio_hbios -llib/hbios/ff fileCopyTest.c -o fileCopyTest -create-app

        printf("\r\nCopied %lu bytes, the time taken was %li.%.4lu seconds\n", bwt, resTime.tv_sec, resTime.tv_nsec/100000 );

        f_mount(0, "2:", 0);                                    /* Free work area */
    } else {

        printf("Couldn't mount drive - f_mount error #%u\r\n", res);
    }
    return 0;
}

The above is an example of two separate builds. Firstly using the CISO SD Card drivers, and secondly using the HBIOS API.

• SD Card (CISO) can produce about 155kB/s throughput.

• CF IDE Drive (82C55) can produce about 295kB/s throughput.
• SSD IDE Drive (82C55) can produce about 410kB/s throughput.

The TL;DR, the SD Card performance is not that bad, and is only about half as fast as the parallel interface storage.

I was expecting it to be about one tenth the performance.

Happy New Year,

Phillip

[Phillip Stevens]

Bill Shen wrote:

Very nice work.  Two questions/comments:
1.  The CPU clock strongly affects the file transfer speed.  You used SCZ180 for the benchmark, but I also saw a comment about 36.8MHz clock.  So was the SCZ180's clock doubled for this benchmark?

Thanks. :-)

Yes, the SC130 I was using was configured with X2 clock, so that it was running at 36.864MHz, with one memory wait state and two (plus one) I/O wait states. This is the same configuration that I use with the YAZ180.

The idea of the test below (not the overall work) was to answer the question, "What is the best user level (in C) performance I can expect from mass storage?"

To answer the question, and get the best case scenario for each storage option, I had to remove all complexity from the testing and use the same hardware to the greatest extent possible.

Hence HBIOS API diskio is not being used, although HBIOS API serial is being used along with the ROMWBW dbgmon.

Another test that I'm planning on doing is, "Using the HBIOS API, what is the best user level (in C) performance I can expect from mass storage?" 

With the SC130, I can use both CSIO SD and 82C55 IDE interfaces, and the z88dk HBIOS driver supports all the disk options on the platform.

The only hardware I don't have for the RC2014 bus is a 8-Bit CF card interface, but that doesn't stop someone else plugging this info when the framework is there.

Once I've sorted out a few wrinkles, I'll be able compare RAM disk, with SD, and IDE interfaces for performance, across the same HBIOS platform.

2.  My experience with CF disks is that fastest CF disk can be almost as fast as DiskOnModule which probably not as fast as SSD.  The slowest CF disk can be 1/3 of the fastest CF disk.  These are limited hand-on experiences with older generation of 32M-512M CF disks.

The SD card I used was a Sandisk 4GB generic, the CF card was the Verbatim 1GB drive (from Officeworks), and the SSD was a Kingspec 64GB drive (from Amazon).

So, my experiences are also quite limited, but more to commercially (available new) available storage.

Phillip

Phillip Stevens wrote:

For the first time, I can now do accurate analysis of the performance of IDE SDC and CF vs SD Cards, and have a single model for driving all of the SCZ180 (sc126, sc130), RC2014, and all the other HBIOS enabled platforms with ROMWBW.

Part of the work was separating the ChaN FAT File System from the underlying diskio drivers on each platform, and then making it possible to add different parts of the libraries depending on which drivers are required.

In summary, the SCZ180 platform just has CSIO SD drivers included in z88dk. An additional library to provide the diskio layer for the CSIO SD Card has been created. In parallel a library to provide diskio off the back of the HBIOS API has also been created. When building applications, either CSIO SD or HBIOS SD (or any other hardware) interfaces can then be chosen at build time, by linking from the command line.

The RC2014 platform has an 82C55 IDE based diskio platform included in z88dk. But if needed any another diskio layer can be linked from the command line, and this will override the integrated version.

Any other HBIOS equipped platform can use the hbios target, together with the diskio layer using the HBIOS API, to access the ChaN FAT file system library code.

At the C level for user code I get the following results:

• SD Card (CSIO) can produce about 55kB/s throughput (EDITED).

• CF IDE Drive (82C55) can produce about 295kB/s throughput.
• SSD IDE Drive (82C55) can produce about 410kB/s throughput.

[Phillip Stevens]

Phillip Stevens wrote:

The SC130 I was using was configured with X2 clock, so that it was running at 36.864MHz, with one memory wait state and two (plus one) I/O wait states. This is the same configuration that I use with the YAZ180.

The idea of the test below (not the overall work) was to answer the question, "What is the best user level (in C) performance I can expect from mass storage?"

To answer the question, and get the best case scenario for each storage option, I had to remove all complexity from the testing and use the same hardware to the greatest extent possible.

Hence HBIOS API diskio is not being used, although HBIOS API serial is being used along with the ROMWBW dbgmon.

Another test that I'm planning on doing is, "Using the HBIOS API, what is the best user level (in C) performance I can expect from mass storage?" 

With the SC130, I can use both CSIO SD and 82C55 IDE interfaces, and the z88dk HBIOS driver supports all the disk options on the platform.

The only hardware I don't have for the RC2014 bus is a 8-Bit CF card interface, but that doesn't stop someone else plugging this info when the framework is there.

 "What is the best user level (in C) performance I can expect from mass storage?"

 

At the C level for user code I get the following results:

• SD Card (CSIO) SanDisk Generic can produce about 55kB/s throughput (EDITED).

• CF IDE Drive (82C55) can produce about 295kB/s throughput.
• SSD IDE Drive (82C55) can produce about 410kB/s throughput.

The TL;DR, the SD Card performance is not that bad, and is only about half as fast as the parallel interface storage.

I was expecting it to be about one tenth the performance.

"Using the HBIOS API, what is the best user level (in C) performance I can expect from mass storage?" 

At the C level for user code I get the following results:

• SD Card (CSIO) SanDisk Generic can produce about 23.3kB/s throughput.
• SD Card (CSIO) SanDisk Ultra can produce about 27.4kB/s throughput.
• CF IDE Drive (82C55) can produce about 165kB/s throughput.
• SSD IDE Drive (82C55) can produce about 235kB/s throughput.

The difference in performance is explained by the fact that the HBIOS has to do bank switching for each call, and makes function calls via several levels of indirection (tables).

For the user, the ability to address multiple disk hardware types simply by changing drive number is very useful, and in reality the performance difference doesn't matter.

Oh, and I learned that the ChaN FatFS system requires a pointer to a FATFS structure to mount properly, and just passing an address of a structure won't work.

That simple mistake cost me a few nights sleep.

#pragma output CRT_ORG_BSS = 0x9000 /* move bss origin to address 0x9000 (check map to confirm there is no overlap between data and bss sections) */

static FIL FileIn, FileOut;         /* File object needed for each open file */

#define BUFFER_SIZE 4096            /* size of working buffer (on heap) */
static BYTE * buffer;               /* working buffer */
                                    /* must be in bss (which is above 0x8000) for romwbw hbios buffer */


static FATFS * FatFs;               /* FatFs work area needed for each volume */
                                    /* Pointer to the filesystem object (on heap) */

int main (void)
{
    UINT bw;
    UINT br;
    DWORD bwt = 0;
    FRESULT res;

    struct timespec startTime, endTime, resTime;

    FatFs = (FATFS *)malloc(sizeof(FATFS));                     /* Get work area for the volume */
    buffer = (BYTE *)malloc(sizeof(BYTE)*BUFFER_SIZE);          /* Get working buffer space */

    startTime.tv_sec = 1577836800 - UNIX_OFFSET;
    clock_settime(CLOCK_REALTIME, &startTime);                  /* Set the time of day, y2k epoch */

    if ((res = f_mount(FatFs, "2:", 1)) == FR_OK) {             /* Give a work area to the default drive */
   
        printf("\r\n\nFatFs->fs_type %u\nFatFs->fsize %lu\n", FatFs->fs_type, FatFs->fsize);
        printf("\r\nOpening 2:random1.txt");

        if ((res = f_open(&FileIn, "2:random1.txt", FA_READ)) == FR_OK) {

            printf(" - Opened");

            if ((res = f_lseek(&FileIn, 0)) == FR_OK) {

                printf("\r\nCreating 2:random2.txt");

                if ((res = f_open(&FileOut, "2:random2.txt", FA_CREATE_ALWAYS | FA_WRITE)) == FR_OK) {

                    printf(" - Created\r\n\nCopying...");

                    clock_gettime(CLOCK_REALTIME,&startTime);

                    for (;;) {

                        res = f_read(&FileIn, buffer, sizeof(BYTE)*BUFFER_SIZE, &br);
                        if (res != FR_OK || br == 0) break;     // error or eof
                        res = f_write(&FileOut, buffer, br, &bw);
                        bwt += bw;
                        if (res != FR_OK || bw != br) break;     // error or disk full

                    }

                    clock_gettime(CLOCK_REALTIME,&endTime);

                    f_close(&FileOut);

                    if ((res = f_unlink("2:random2.txt")) != FR_OK) {

                        printf("\r\nCouldn't delete 2:random2.txt - f_unlink error #%u\r\n", res);
                    }
                } else {
                    printf("\r\nCouldn't open 2:random2.txt - f_open error #%u\r\n", res);                
                }
             } else {
                printf("\r\nCouldn't seek 2:random1.txt - f_lseek error #%u\r\n", res);
            }
            f_close(&FileIn);
        } else {
            printf("\r\nCouldn't open 2:random1.txt - f_open error #%u\r\n", res);
        }

        timersub(&endTime, &startTime, &resTime);

        printf("\r\nCopied %lu bytes", bwt );
        printf(", the time taken was %li.%.4lu seconds\n", resTime.tv_sec, resTime.tv_nsec/100000 );

        f_mount(0, "2:", 0);                                    /* Free work area */
    } else {

        printf("\r\nCouldn't mount drive - f_mount error #%u\r\n", res);
    }
    // Perform any shutdown/cleanup.
    free(buffer);
    free(FatFs);
    return 0;
}

Cheers, Phillip

[Wayne Warthen]

Good stuff Phillip.

The performance degradation when using HBIOS is expected, but was more then I would have thought.  Question:  when using HBIOS, where was the disk buffer located?  If the buffer was located in the lower 32K, then HBIOS needs to double buffer all disk I/O.  This would certainly have an affect on the measurements.

-Wayne

On Monday, January 6, 2020 at 3:59:48 AM UTC-8, Phillip Stevens wrote:

Phillip Stevens wrote:

The SC130 I was using was configured with X2 clock, so that it was running at 36.864MHz, with one memory wait state and two (plus one) I/O wait states. This is the same configuration that I use with the YAZ180.

The idea of the test below (not the overall work) was to answer the question, "What is the best user level (in C) performance I can expect from mass storage?"

To answer the question, and get the best case scenario for each storage option, I had to remove all complexity from the testing and use the same hardware to the greatest extent possible.

Hence HBIOS API diskio is not being used, although HBIOS API serial is being used along with the ROMWBW dbgmon.

Another test that I'm planning on doing is, "Using the HBIOS API, what is the best user level (in C) performance I can expect from mass storage?" 

With the SC130, I can use both CISO SD and 82C55 IDE interfaces, and the z88dk HBIOS driver supports all the disk options on the platform.

The only hardware I don't have for the RC2014 bus is a 8-Bit CF card interface, but that doesn't stop someone else plugging this info when the framework is there.

 "What is the best user level (in C) performance I can expect from mass storage?"

 

At the C level for user code I get the following results:

• SD Card (CISO) can produce about 155kB/s throughput.

• CF IDE Drive (82C55) can produce about 295kB/s throughput.
• SSD IDE Drive (82C55) can produce about 410kB/s throughput.

The TL;DR, the SD Card performance is not that bad, and is only about half as fast as the parallel interface storage.

I was expecting it to be about one tenth the performance.

"Using the HBIOS API, what is the best user level (in C) performance I can expect from mass storage?" 

At the C level for user code I get the following results:

• SD Card (CISO) SanDisk Generic can produce about 23.3kB/s throughput.
• SD Card (CISO) SanDisk Ultra can produce about 27.4kB/s throughput.

[Phillip Stevens]

Wayne Warthen wrote:

The performance degradation when using HBIOS is expected, but was more then I would have thought.  Question:  when using HBIOS, where was the disk buffer located?

For the hbios target and subtargets (off rc2014 and scz180), I've made a simple assumption that the bss area starts at 0x8000. If the buffers are not initialised then they would automatically be put into the bss area, and are in the right place.

Similarly heap allocated buffers are between bss and the stack. The stack is located from 0xF000 so that dgbmon is not overwritten (but a user could move it up to 0xFE00 if they needed more space).

I did have an issue with collisions, and it is a bit tedious to check the map file all the time. So I've asked for a warning to be provided in an enhancement issue. For the application here I needed to relocate the bss area to start at 0x9000 to prevent a collision.

 

If the buffer was located in the lower 32K, then HBIOS needs to double buffer all disk I/O.  This would certainly have an affect on the measurements.

I'm pretty sure that double buffering is not the issue with the SD interface performance, though it is surprisingly impacted.

And, PPIDE performance is inline with the expectation.

I've got to look at SD code again, so I can see whether there are easy optimizations possible.

Cheers Phillip

 

#pragma output CRT_ORG_BSS = 0x9000 /* move bss origin to address 0x9000 (check map to confirm there is no overlap between data and bss sections) */

#define BUFFER_SIZE 4096            /* size of working buffer (on heap) */
static BYTE * buffer;               /* working buffer */

                                    /* buffer must be in bss (which is above 0x8000) for romwbw hbios */

[Wayne Warthen]

On Monday, January 6, 2020 at 2:29:38 PM UTC-8, Phillip Stevens wrote:

Wayne Warthen wrote:

The performance degradation when using HBIOS is expected, but was more then I would have thought.  Question:  when using HBIOS, where was the disk buffer located?

I've got to look at SD code again, so I can see whether there are easy optimizations possible.

Cheers Phillip

Just glancing at the SD driver in RomWBW HBIOS, it is pretty obvious that the reading and writing blocks of data could be significantly optimized.  Every byte I/O is done via a separate routine that is called and that routine further calls a routine to do the bit inversion.  It works nicely, but it far from optimal.  Sigh.

[Phillip Stevens]

Wayne Warthen wrote:

Just glancing at the SD driver in RomWBW HBIOS, it is pretty obvious that the reading and writing blocks of data could be significantly optimized.  Every byte I/O is done via a separate routine that is called and that routine further calls a routine to do the bit inversion.  It works nicely, but it far from optimal.  Sigh.

For z88dk I found an efficient byte mirroring routine. I think I found it in FUZIX, and copied from there. HBIOS mirrors bytes with either a slow iteration, or a fast table.

I'll try with the fast table option tonight, but I don't imagine it will make a significant difference.

But, if it does then it might be worth ditching the slow iteration and using the z88dk/FUZIX version by default?

Otherwise, I've found from looking at the logic traces that the block read and block write drivers in z88dk can push the CSIO more than twice as fast as HBIOS.

This is mostly because the bit mirroring routine is timed to be done whilst waiting for the next byte to be processed (received or transmitted respectively) by the CSIO hardware.

It looks like the SD_PUT and SD_GET routines in HBIOS do this less optimally, because they have to deal with quite a few different hardware types.

I don't know whether it will be possible to do the mirroring and the reception/transmission processes in parallel, while still maintaining support for all the hardware options?

But that will be the hot-spot that will produce the most improvement, imho.

Phillip

[Phillip Stevens]

On Tuesday, 7 January 2020 10:42:14 UTC+11, Phillip Stevens wrote:

Wayne Warthen wrote:

Just glancing at the SD driver in RomWBW HBIOS, it is pretty obvious that the reading and writing blocks of data could be significantly optimized.  Every byte I/O is done via a separate routine that is called and that routine further calls a routine to do the bit inversion.  It works nicely, but it far from optimal.  Sigh.

For z88dk I found an efficient byte mirroring routine. I think I found it in FUZIX, and copied from there. HBIOS mirrors bytes with either a slow iteration, or a fast table.

I'll try with the fast table option tonight, but I don't imagine it will make a significant difference.

But, if it does then it might be worth ditching the slow iteration and using the z88dk/FUZIX version by default?

Using the efficient mirroring routine, from http://www.retroprogramming.com/2014/01/fast-z80-bit-reversal.html

• SD Card (CSIO) SanDisk Generic - old 23.3kB/s  - new 30kB/s.
• SD Card (CSIO) SanDisk Ultra - old 27.4kB/s - new 35kB/s.

An easy improvement. I'll do a PR.

P.

[Wayne Warthen]

Thank you Phillip.  The PR will be merged shortly.

-Wayne

On Tuesday, January 7, 2020 at 12:30:36 AM UTC-8, Phillip Stevens wrote:

On Tuesday, 7 January 2020 10:42:14 UTC+11, Phillip Stevens wrote:

Wayne Warthen wrote:

Just glancing at the SD driver in RomWBW HBIOS, it is pretty obvious that the reading and writing blocks of data could be significantly optimized.  Every byte I/O is done via a separate routine that is called and that routine further calls a routine to do the bit inversion.  It works nicely, but it far from optimal.  Sigh.

For z88dk I found an efficient byte mirroring routine. I think I found it in FUZIX, and copied from there. HBIOS mirrors bytes with either a slow iteration, or a fast table.

I'll try with the fast table option tonight, but I don't imagine it will make a significant difference.

But, if it does then it might be worth ditching the slow iteration and using the z88dk/FUZIX version by default?

Using the efficient mirroring routine, from http://www.retroprogramming.com/2014/01/fast-z80-bit-reversal.html

• SD Card (CISO) SanDisk Generic - old 23.3kB/s  - new 29.3kB/s.
• SD Card (CISO) SanDisk Ultra - old 27.4kB/s - new 34.8kB/s.

An easy improvement. I'll do a PR.

P.

[Phillip Stevens]

Wayne Warthen wrote:

Just glancing at the SD driver in RomWBW HBIOS, it is pretty obvious that the reading and writing blocks of data could be significantly optimized.  Every byte I/O is done via a separate routine that is called and that routine further calls a routine to do the bit inversion.  It works nicely, but it far from optimal.  Sigh.

Using the efficient mirroring routine, from http://www.retroprogramming.com/2014/01/fast-z80-bit-reversal.html

• SD Card (CSIO) SanDisk Generic - old 23.3kB/s  - new 30kB/s.
• SD Card (CSIO) SanDisk Ultra - old 27.4kB/s - new 35kB/s.

Something didn't feel right about my first tests with the SD Card.

I saw about about a 2x difference in CSIO drive, but that doesn't equate to a 6x difference in performance.

Should have done the calculation, and realised that the theoretical maximum off a 36.864MHZ CSIO is only 115.2kB/s.

So I went back and did them again tonight. And indeed, I found an error with my calculation. I'll call it a typo.

• SD Card (CSIO) SanDisk Generic - z88dk SD drivers 55kB/s (EDITED).
• SD Card (CSIO) SanDisk Ultra - z88dk SD drivers 70kB/s.

But at least now the relative differences for CSIO SD and PPIDE are aligned.

And there's no real reason to "fix" the HBIOS SD driver, as it is behaving as it should do.

Sorry about the mis-direction.

Phillip