VT132: Display Controls mode

[Tony Nicholson]

I've built two VT132s - the RC2014 form factor one, and a standalone.  They're great!

One of the "features" I used back-in-the-day on a real VT220 terminal was "Display Controls" - where each of the received characters (including the control ones) were rendered and displayed on screen in a single character cell (like a small CR and LF were displayed instead of performing a return and new line).  This made debugging faulty escape sequences and serial communications easy - because I could "see" the raw data being sent to the terminal.

Would others find this a worthwhile enhancement to the firmware?

Maybe there's room in the ESP32 firmware to implement this!

Tony

[TheHighNibble]

Good question/feature request. I thought about this during development, but did not see any references to it in the VT100 documentation so I didn't pursue it.

I think I have the characters available in the original set of character glyphs. I'll just have to research (easily done) how it was activated and implemented on the VT220 so it at least follows an original DEC implementation.

I'll provide feedback later on the weekend.

[TheHighNibble]

Got it mostly working!

As the Quick Menu follows VT510 conventions the feature is on the Quick Menu as Display -> [X]Show control characters  the [X] representing a check box (as per https://vt100.net/docs/vt510-rm/chapter2.html#S2.5.15 )

The feature is implemented as per the VT220 since the font is from the VT220 ROMS and the VT510 specification for this feature is slightly different.

(see https://vt100.net/docs/vt220-rm/chapter2.html#S2.7 )

It isn't fully tested, but 

• I have got it working with the the scroll history 
• and it made me complete the 132 column DEC MCS font to include all the missing glyphs in addition to the glyphs for the control characters, so it works in 132 column mode too
• if you have selected PC Code Page 437, then you just see the CP437 characters, not the DEC mnemonic  glyphs for the control characters

I still need to find a way to test the XON/XOFF (DC1, DC3) behaviour when in this mode to make sure that is following the spec.

If I don't find any knock-on problems I might release a Beta in the next few days.

[Tony Nicholson]

Wow - that was quick!  Great news Dave.

Looking forward to trying it!

Tony

On Saturday, March 27, 2021 at 8:26:38 PM UTC+11 TheHighNibble wrote:

Got it mostly working!

As the Quick Menu follows VT510 conventions the feature is on the Quick Menu as Display -> [X]Show control characters  the [X] representing a check box (as per https://vt100.net/docs/vt510-rm/chapter2.html#S2.5.15 )

The feature is implemented as per the VT220 since the font is from the VT220 ROMS and the VT510 specification for this feature is slightly different.

(see https://vt100.net/docs/vt220-rm/chapter2.html#S2.7 )

It isn't fully tested, but 

• I have got it working with the the scroll history 
• and it made me complete the 132 column DEC MCS font to include all the missing glyphs in addition to the glyphs for the control characters, so it works in 132 column mode too
• if you have selected PC Code Page 437, then you just see the CP437 characters, not the DEC mnemonic  glyphs for the control characters

I still need to find a way to test the XON/XOFF (DC1, DC3) behaviour when in this mode to make sure that is following the spec.

If I don't find any knock-on problems I might release a Beta in the next few days.

[snip]

[TheHighNibble]

Yep, that was the good news. And so far so good. But this might have to go out as a Beta with a known issue:

• someone let me know in the past week that OTA updates from GitHub are broken again
• it was working the day I released V1.2.1 because on that day I have confirmed people did the upgrade
• since then GitHub and Amazon S3 have gone and changed their CA Root certificates again which I have fixed
• but there has also been a change to how Amazon S3 serves files and it is not compatible with Espressif esp-idf V4.0 that I am using
• looks like it might be fixed in esp-idf V4.2 which is in production but I will have to port to and then do a lot of testing, I guess that's what my upcoming 2 week holiday is for?
• meanwhile you would have to download any update/beta to a local web server for OTA or use the esptool.py to update over a USB-FTDI adapter

[TheHighNibble]

This feature (along with a fix to the OTA problem and a few other new features) has been release in V1.3.0 on GitHub. 

See the release notes at https://github.com/thehighnibble/vt132/releases.

Thanks for the suggestion Tony.

[Mark Cummings]

I like the sound of that as it would help me with character checks from different systems. I'm using something like that to mark up some character tables I'm researching for APL-Z, and by typing a small font with CR, LF etc in my table makes it easy to tell it's a control character of some kind rather than the actual glyph. If I could see that on the screen instead it would tell me exactly what code it received rather than me trying to second guess. It would also be handy if I could see all characters instead of just control characters, but any control characters is still a help.

Mark

[Tony Nicholson]

Mark,

All the characters (byte values 0..255) are displayed in this mode.

I’ve attached a photo of my VT132 display.  The Display Controls mode

outputs a new line when it encounters a line feed.  I’ve just noticed that

a TAB is expanded to spaces too (shouldn’t it display as a small HT Dave?)

Tony

[Tony Nicholson]

On Tuesday, April 20, 2021 at 7:10:51 PM UTC+10 Tony Nicholson

All the characters (byte values 0..255) are displayed in this mode.

I’ve attached a photo of my VT132 display.  The Display Controls mode

outputs a new line when it encounters a line feed.

and I meant to say it also does this for a vertical tab, and form-feed too.

Tony

[TheHighNibble]

From the VT220 programmers manual: https://vt100.net/docs/vt220-rm/chapter2.html#S2.7

• When you select the "Display Controls" field, the terminal displays all control functions and prevents most from executing. There are only two exceptions. LF, FF, and VT cause a new line (CR LF), and XOFF (DC3) and XON (DC1) maintain flow control if enabled. LF, FF, and VT are displayed before CRLF is executed, and DC1 and DC3 are displayed after execution.

So LF, FF, VT are working as expected. But you are right about <TAB> (ctrl-I) it should not tab across, but should show an HT glyph.

I'll look at getting a fix for this in the next release, or issue a patch release sooner if possible.

[Mark Cummings]

hi Tony,

Thanks for the feedback. I am as-yet unable to test your findings on a VT132 as I only just finished building my VT132 a few days ago and unfortunately it it is not running due to a fault, possibly the ESP32 internal flash memory. We haven't gotten to the bottom of that yet, although I suspect I will be having to replace the ESP32, not an easy task.

Eventually I will get to try your suggestion. I assume that's running a version of CP/M BASIC or something on the IMSAI 8080 replica with your VT132 connected to a serial ports ? if not can you please clarify.

Mark.

[Tony Nicholson]

On Tuesday, April 20, 2021 at 10:51:04 PM UTC+10 Mark Cummings wrote:

hi Tony,

Thanks for the feedback. I am as-yet unable to test your findings on a VT132 as I only just finished building my VT132 a few days ago and unfortunately it it is not running due to a fault, possibly the ESP32 internal flash memory. We haven't gotten to the bottom of that yet, although I suspect I will be having to replace the ESP32, not an easy task.

Eventually I will get to try your suggestion. I assume that's running a version of CP/M BASIC or something on the IMSAI 8080 replica with your VT132 connected to a serial ports ? if not can you please clarify.

Mark.

[snip]

Mark,

Sad to hear of your issues getting the VT132 up - I had similar issues on my RC2014 board version. This was my second surface mount soldering attempt (the first went fine with the IMSAI8080esp).

On mine it booted up OK but a simple loopback (connecting Rx to Tx) didn't work.  My over jealous solder sucking to check for shorts around pins 14 to 16 of the ESP32 caused solder to wick under the the metal shell of the ESP32.  Dave (The HighNibble) came to the rescue offering to desolder and replace the ESP32 for me and it's been good ever since!  Lucky we are "local" Australians!

Yes I did "cobble up" a MBASIC program on my IMSAI8080esp to do the screen-shot.  The program on the screen pumps out 64 bytes at a time to avoid MBASIC doing its own line wrapping (I'm a bit rusty with BASIC and couldn't remember if it had a modulo function or logical and operation!).  I could have whipped up a version in C or Z80 assembler quicker!

When you get around to trying the APL/Z characters - if you need them, I could dig out the downloadable APL font from the VAX APL implementation and send this to you.

Tony

[Mark Cummings]

hi Tony,

just so you can see why I want to use the VT132 terminal to do further testing, I ran the same BASIC program as you demonstrated above except using GSXBASIC (which is all I could find at the time). I tried using the TTY: in the GUI and then again using a PuTTY: session directly from Linux over a ttyUSB0 serial connection. I've attached a screen shot of each. In the TTY: there is no option to change the font, but in PuTTY: I selected APL2741 as the font. I've also tried other fonts (supposedly with APL glyphs) in PuTTY: with no obvious differences.

Note the second half of the character set looks like the first half of the set which leads me to believe that something is truncating the hi bit, although that could just be the font itself or some setting I haven't found yet.

Mark

[TheHighNibble]

When you get around to trying the APL/Z characters - if you need them, I could dig out the downloadable APL font from the VAX APL implementation and send this to you.

Yes please. Can you please get the  VT220 compatible VAX APL font and send me the file. I am in the process of implementing VT220 soft fonts (DRCS/DECDLD) on the VT132 and I am in need of original DEC font files to test the implementation.

Thanks,
Dave.

[Mark Cummings]

I'm also interested in a copy of the character set to look at the mapping of characters and font format. From my research so far, the attached file shows the character set map as far as I know. I created this map from the VAX APL manual.

Mark

[TheHighNibble]

OK. I've checked the code for the VT132, stuck in some debug statements etc... and played around with a number of scenarios and believe that the VT132 is behaving correctly.

What I've found is that CP/M (for example) never outputs the ASCII <tab> (0x09) character, but instead does it's own padding to spaces. The CP/M 2.2 documentation for TYPE and for PIP confirm this. They interpret an ASCII <tab> (0x09) on the fly and keep count of that current column and pad a <tab> with spaces based on an 8 character tab width.

So I suspect that this is what MBASIC is doing too. I don't think its the CP/M BDOS/BIOS routines where this happens (but I could be wrong) I think it is each application itself that implements this? But someone with a deeper understanding of CP/M internals might be able to confirm (Udo?) 

[Tony Nicholson]

On Thursday, April 22, 2021 at 7:39:41 AM UTC+10 TheHighNibble wrote:

OK. I've checked the code for the VT132, stuck in some debug statements etc... and played around with a number of scenarios and believe that the VT132 is behaving correctly.

What I've found is that CP/M (for example) never outputs the ASCII <tab> (0x09) character, but instead does it's own padding to spaces. The CP/M 2.2 documentation for TYPE and for PIP confirm this. They interpret an ASCII <tab> (0x09) on the fly and keep count of that current column and pad a <tab> with spaces based on an 8 character tab width.

So I suspect that this is what MBASIC is doing too. I don't think its the CP/M BDOS/BIOS routines where this happens (but I could be wrong) I think it is each application itself that implements this? But someone with a deeper understanding of CP/M internals might be able to confirm (Udo?) 

[snip]

 

I can confirm it is CP/M MBASIC doing the tab expansion.

I tried the same with BASIC running under RSX280 (a clone of PDP-11 BASIC-11 running on a Z280 CPU), and it definitely shows the HT glyph in Display Controls mode. (Ignore that it replaces Form-feed with Line-feeds though).

Display controls does indeed do the right thing!

Tony

[Tony Nicholson]

For those following along at home...

The CP/M Plus Programmers Guide (from http://www.cpm.z80.de/manuals/cpm3-pgr.pdf ) page 3-3 describes the BDOS console output function 2 as

"

BDOS FUNCTION 2: CONSOLE OUTPUT

Entry Parameters:

Registers C: 02H

E: ASCII Character

The Console Output function sends the ASCII character from register E to the logical console device, CONOUT:. When the Console Mode is in the default state (see Section 2.2.1), Function 2 expands tab characters, CTRL-I, in columns of 8 characters, checks for stop scroll, CTRL-S, start scroll, CTRL-Q, and echoes characters to the logical list device, LST:, if printer echo, CTRL-P, has been invoked.

"

Clearly there is some TAB expansion from CP/M BDOS calls.

Also, I re-did my "output all bytes program" in HI-TECH C - this time using the BIOS console output routine.  Here it is -

#include <cpm.h>
void main()
{
  int i;
  for (i = 0; i < 256; i++) {
    bios(_CONOUT, i);
    if ( ((i+1) & 63) == 0 ) {
      bios(_CONOUT, 13); /* CR LF */
      bios(_CONOUT, 10);
      }
    }
}

Compile with HI-TECH C using "c -o allchars.c" and run with Display Controls enabled to see it all works!

Tony

[udo....@freenet.de]

TheHighNibble schrieb am Mittwoch, 21. April 2021 um 23:39:41 UTC+2:

So I suspect that this is what MBASIC is doing too. I don't think its the CP/M BDOS/BIOS routines where this happens (but I could be wrong) I think it is each application itself that implements this? But someone with a deeper understanding of CP/M internals might be able to confirm (Udo?) 

CP/M BDOS expands tab to spaces, same as most OS's do. Early terminals, teletypes, video cards would ignore a tab, so this needs

to be addressed in the OS's to get the same result on any output device.

BASIC implementations running on various different platforms, like MBASIC, XYBASIC, probably also expand tab self, because

this then is a bit less dependent on the platform. If you really need to know better try it and see what happens, there was no

common solution way back then.