ydb TTY IO needs cleanup.

[Kevin Toppenberg]

I've spent quite a bit of time stepping through the opening, using, setting resetting, closing of TTY IO devices.  Here are some take away points.

• IO is intrinsically a complicated topic.  One wants code as general as possible to handle many different kinds of devices that may be opened.  But then there are the exceptions that need to be handled.  So hats off to the creators of this code that has chugged along for many decades.
  
  
• The part that needs cleaning up is the STATE SYSTEM of tty io devices.  The underlying TTY system (part of the unix operating system) maintains its own state.  ydb can query this state with getattr() and set it with setattr().  It often does this, and saves a copy of this state in tt_ptr->ttio_struct.  THE PROBLEM IS, that this is not the ONLY place where state is kept.  For example, tt_ptr->canonical maintains what ydb thinks the 'canonical' state SHOULD be.  But it often is out of sync with what the TTY system is actually doing.  So for this setting alone there are THREE locations that reflect the state.  And it is not clear in the code which is the "gold standard."  Another example is that the tty ECHO setting is not saved in tt_ptr->  in the same way that canonical is.  Instead it is stored in a tt_ptr->term_ctrl.  But then this also is not vigorously synched with the actual TTY system.  Sometimes in resetterm, tt_ptr->ttio_struct is the starting point, and then it is tweaked and used for setattr().  But this was not properly initialized, see below.
  
  
• Initialization of the state is nebulous.  For example, when ydb starts up, the TTY subsystem is typically already in canonical mode (where TTY does basic line editing).  But ydb initializes itself such that tt_ptr->canonical is false.  But although it sets this value, it doesn't transmit those changes to the tty system.  There are many back and forths in the code, so sometimes it eventually gets in sync.  But I have some sample code that demonstrates it not working. 
  
  
• What to do about this?  Well, ydb has been going great for a long time.  And it's just me that seems to be bothered by this.  My primary gripe has been an ECHO/NOECHO mode that doesn't work because of the way it is coded, and not being able to read BACKSPACE, which involves the CANONICAL mode settings.  So I am trying to come up with a solution.  But it is very slow going.  I have to go through it over and over, and then walk away from the computer for awhile to sort out what I was seeing and come up with ideas to try. 
  
  
• As a small gripe, I find that c  programmers seem to LOVE short (and thereby cryptic) variable names.  Want to pass a pointer refers to a structure containing information about device parameters?  Yeah, let's call that "pp".  Modern editors now often provide name completion, so long descriptive variable names are usable without having to type the entire thing out.  But I think ydb was written before those days.  It makes for difficulty trying to figure out what various parts are doing.   Even if one uses short variable names, it would be so helpful if there was a brief explanation of its use at its point of declaration.  I always try to write my code with the idea that I won't remember what "pp" means when I review the code years later.
  
  
• I can't help but compare this c code for ydb to the pascal code for CPRS.  Both are large code bases with lots of code written many decades back. I know that c is supposed to be faster because the programmer has to think at a lower level. But I can't help but feel that I am walking on eggshells because all those manual malloc() calls and pointer arithmetic have to be treated like loose cannon.  I think pascal code could be equally fast, and much safer.  But pascal's days are sadly past, so that ship has sailed.  :-(
  

So no real questions. Just thought I would share my progress. 

Kevin

[K.S. Bhaskar]

Kevin, thanks for your comments. You're pretty much spot on, and hats off to you for diving into murky waters.

Regards

– Bhaskar

[Kevin Toppenberg]

I am trying to think through how the TTY should work in different circumstances. 

Terminology:

  Canonical -- if (+) --> means that TTY manages line editing, and after NL (or sometimes CR) is pressed, return bolus of user input.

                        if (-) --> means that TTY returns each letter as entered, and lets ydb manage line editing. 

  Echo -- means that the TTY system sends a character, e.g. 'A' to the screen when the types.  If enabled, the user can see progress on the screen as they type, though under TTY control.

In addition to above TTY echo settings, YDB can also write out user input to the screen.  This functionality is controlled by tt_ptr->term_ctrl, which is a collection of bit flags.  The YDB-defined flag TRM_NOECHO controls if YDB should write a copy of what it reads to the screen.   "ECHO"/"NOECHO" is also a USE parameter, e.g. USE $P:NOECHO.  It is my understanding that if a user specifies NOECHO, that they do not want anything to appear on the screen.  Thus ydb should not, itself, write out anything to the screen to compensate for TTY no echo.

In standard ydb, canonical and echo mode were bound together.  One of my goals has been to allow separation. 

I am going consider the various combinations.

                 CANONICAL(+)                        CANONICAL(-)

        +---------------------------------+--------------------------------------+

        | User typing is shown on screen  | As user types, each keystroke        |

ECHO(+) | during entry.  After completion | is returned to ydb, one at a         |

        | ydb should NOT show, so needs   | time.  Because TTY has echo          |

        | TRM_NOECHO should enabled (+)   | enabled, ydb should NOT echo,        |

        |                                 | so TRM_NOECHO should be enabled(+).  |

        +---------------------------------+--------------------------------------+

        | User typing should NOT be shown | As user types, each keystroke is     |

ECHO(-) | on screen (e.g. password input) | returned to ydb. TTY has not put to  |

        | After completion, ydb also      | screen, but since echo(-), ydb should|

        | should NOT show, so TRM_NOECHO  | NOT write out.                       | 

        | should be enabled (+)           | Thus TRM_NOECHO should be enabled(+) |

        +---------------------------------+--------------------------------------+

NOTE that in every case above, TRM_NOECHO should be (+).  

QUESTION: Under what circumstances should TRM_NOECHO be false-- i.e. we DO want ydb do write out to screen?  I guess it would be if user wanted to turn OFF TTY echo, but still want something to appear on screen(?).  But how would such a circumstance be specified with use parameters??

Anyone have input to this?

Kevin

[Kevin Toppenberg]

NOTE: I found this in the programmer guide here: https://docs.yottadb.com/ProgrammersGuide/ioproc.html#echo

ECHO

[NO]ECHO Applies to: TRM

Enables or disables the echo of terminal input. If you disable ECHO, the EDITING functions will be disabled and any input is not available for later recall.

By default, terminal input ECHOes.

Example:

use $principal:noecho

This example disables the echo of terminal input.

So

                 CANONICAL(+)                        CANONICAL(-)
        +---------------------------------+--------------------------------------+
        | User typing is shown on screen  | As user types, each keystroke        |
ECHO(+) | during entry.  After completion | is returned to ydb, one at a         |
        | ydb should NOT show, so needs   | time.  Because TTY has echo          |
        | TRM_NOECHO should enabled (+)   | enabled, ydb should NOT echo,        |
        |                                 | so TRM_NOECHO should be enabled(+).  |
        +---------------------------------+--------------------------------------+
        | User typing should NOT be shown | As user types, each keystroke is     |
ECHO(-) | on screen (e.g. password input) | returned to ydb. TTY has not put to  |
        | After completion, ydb also      | screen, but since echo(-), ydb should|
        | should NOT show, so TRM_NOECHO  | NOT write out.                       |
        | should be enabled (+)           | Thus TRM_NOECHO should be enabled(+) |
        +---------------------------------+--------------------------------------+

NOTE that in every case above, TRM_NOECHO should be (+). 

[Kevin Toppenberg]

When I try to change fonts, the post button is too close!  Posted before I was ready.  I'll try again.

NOTE: I found this in the programmer guide here: https://docs.yottadb.com/ProgrammersGuide/ioproc.html#echo

ECHO

[NO]ECHO Applies to: TRM

Enables or disables the echo of terminal input. If you disable ECHO, the EDITING functions will be disabled and any input is not available for later recall.

By default, terminal input ECHOes.

Example:

use $principal:noecho

This example disables the echo of terminal input.


So

                 CANONICAL(+)                        CANONICAL(-)
        +---------------------------------+--------------------------------------+
        | Editing is managed by TTY       | Editing managed by ydb               |
ECHO(+) |                                 |                                      |
        |                                 |                                      |
        +---------------------------------+--------------------------------------+
        | Editing, albeit invisible,  is  | "EDITING functions will be disabled  |
ECHO(-) | managed by TTY                  | and any input is not available for    |
        |                                 | later recall"                        |
        +---------------------------------+--------------------------------------+


KT

[Kevin Toppenberg]

One last idea to consider is that perhaps ydb always plans for TTY echo to be false, and then uses the ECHO/NOECHO device parameter to if ydb writes or doesn't write output to screen.  That is to say, that is would be used to control the TRM_NOECHO flag.

KT

[Maury Pepper]

KT wrote: "QUESTION: Under what circumstances should TRM_NOECHO be false-- i.e. we DO want ydb do write out to screen?   I guess it would be if user wanted to turn OFF TTY echo, but still want something to appear on screen(?).  But how would such a circumstance be specified with use parameters??"

Either way, I would say, NOECHO should never write user input to the device (which, btw, is not always a "screen".) From the user's point of view, ECHO/NOECHO is a simple concept. If the user wants to write to the screen based on the input, there's nothing stopping it. It will be output from the user's routine not some reflex action by the interface. (Maybe there's a reason why CANONICAL and ECHO are not separate.)

[Kevin Toppenberg]

Maury,

Thanks for thinking about this.  Clearly, ydb has been going strong for decades working well with CANONICAL and ECHO linked.  There are some edge cases (e.g. not being able to read a BACKSPACE key) that triggered all this.

We'll see if I am able to find a solution or not.  :-)

Kevin

[David Whitten]

Not to be a wrench-thrower, but doesn’t half-duplex and full-duplex I/o enter into this as well ?

Dave

713-87-3834

--
You received this message because you are subscribed to the Google Groups "Everything MUMPS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to everythingmumps+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/everythingmumps/8de4f248-c738-497e-81e5-c64ca5e03154n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kevin Toppenberg]

Dave,

That is an interesting thought.  But I haven't seen any discussion of variables for setting that with the getattr() and setattr() functions that are available in C to interact with the TTY system.  So I don't know how that would come into play.

Kevin

To unsubscribe from this group and stop receiving emails from it, send an email to everythingmum...@googlegroups.com.

[Maury Pepper]

Nice curve ball Dave, but no, it's not relevant in this situation. From a practical point of view, you could say that the definition of the device type (e.g. TTY) implies full duplex. From a technical point of view, handling half duplex takes place at a lower level of communication than what the user controls. Switching flow direction is handled by data communication equipment (e.g. modem) so it can be transparent to the user. If it isn't, then we need to define a new device type.

[Kevin Toppenberg]

I want to consider further possible configurations:

What if the YDB "ECHO" device parameter does NOT directly drive the TTY echo state, but instead just focuses on output result?  I will call this "YDB ECHO" vs "TTY ECHO"

In table below, I will consider all combinations of these 4 variables, given 16 combination.

As user types "hello<return>, what happens in each case???    

NOTE: In table below, "H e l l o" is used to show that characters are returned one at a time, NOT that space is added between.

    YDB ECHO  CANONICAL   TTY ECHO  TRM_NOECHO      USER TYPES         TTY OUTPUTS   YDB OUTPUTS   NET OUTPUT       COMMENT
 1    (+)       (+)         (+)       (+)           Hello<enter>         Hello         ""              Hello         OK
 2    (+)       (+)         (+)       (-)           Hello<enter>         Hello        Hello         Hello Hello      BAD
 3    (+)       (+)         (-)       (+)           Hello<enter>          ""           ""                ""          violates YDB ECHO (+)
 4    (+)       (+)         (-)       (-)           Hello<enter>          ""          Hello            Hello         OK, but output after <enter>
 5    (-)       (+)         (+)       (+)           Hello<enter>         Hello        ""               Hello         violates  YDB ECHO (-)
 6    (-)       (+)         (+)       (-)           Hello<enter>         Hello        ""               Hello         violates  YDB ECHO (-)
 7    (-)       (+)         (-)       (+)           Hello<enter>          ""          ""               ""            OK            
 8    (-)       (+)         (-)       (-)           Hello<enter>          ""          Hello            Hello         violates YDB ECHO (-)          
 9    (+)       (-)         (+)       (+)           Hello<enter>        H e l l o      ""              Hello         OK
10    (+)       (-)         (+)       (-)           Hello<enter>        H e l l o    H e l l o        HHeelloo       BAD
11    (+)       (-)         (-)       (+)           Hello<enter>           ""          ""              ""            violates YDB Echo (+)
12    (+)       (-)         (-)       (-)           Hello<enter>           ""        H e l l o         Hello         OK
13    (-)       (-)         (+)       (+)           Hello<enter>        H e l l o      ""              Hello         volates YDB Echo (-)
14    (-)       (-)         (+)       (-)           Hello<enter>        H e l l o    H e l l o        HHeelloo       volates YDB Echo (-)
15    (-)       (-)         (-)       (+)           Hello<enter>           ""          ""               ""           OK
16    (-)       (-)         (-)       (-)           Hello<enter>           ""        H e l l o         Hello         violates YDB Echo (-)
       
SUMMARY OF ACCEPTABLE CONFIGURATIONS

    YDB ECHO  CANONICAL   TTY ECHO  TRM_NOECHO      USER TYPES         TTY OUTPUTS   YDB OUTPUTS   NET OUTPUT       COMMENT
 1    (+)       (+)         (+)       (+)           Hello<enter>         Hello         ""              Hello         OK
 9    (+)       (-)         (+)       (+)           Hello<enter>        H e l l o      ""              Hello         OK
12    (+)       (-)         (-)       (-)           Hello<enter>          ""         H e l l o         Hello         OK
 7    (-)       (+)         (-)       (+)           Hello<enter>          ""           ""                ""          OK            
15    (-)       (-)         (-)       (+)           Hello<enter>          ""           ""                ""          OK
                                      

The above shows that if the user DOES want output [YDB ECHO (+)], and CANONICAL is OFF, (items 9 & 12) then this can be achieve two different ways:

 9 -- TTY (+) and TRM_NOECHO (+)   <--- This lets TTY IO take care of output

12 -- TTY (-) and TRM_NOECHO (-)   <--- This lets YDB take care of output  (preferred)

I think I like this approach. 

Kevin

[Kevin Toppenberg]

 SUMMARY OF ACCEPTABLE CONFIGURATIONS

    YDB ECHO  CANONICAL   TTY ECHO  TRM_NOECHO      USER TYPES         TTY OUTPUTS   YDB OUTPUTS   NET OUTPUT       COMMENT
 1    (+)       (+)         (+)       (+)           Hello<enter>         Hello         ""              Hello         OK
 9    (+)       (-)         (+)       (+)           Hello<enter>        H e l l o      ""              Hello         OK
12    (+)       (-)         (-)       (-)           Hello<enter>          ""         H e l l o         Hello         OK
 7    (-)       (+)         (-)       (+)           Hello<enter>          ""           ""                ""          OK            
15    (-)       (-)         (-)       (+)           Hello<enter>          ""           ""                ""          OK
                                      

The above shows that if the user DOES want output [YDB ECHO (+)], and CANONICAL is OFF, (items 9 & 12) then this can be achieve two different ways:

 9 -- TTY (+) and TRM_NOECHO (+)   <--- This lets TTY IO take care of output

12 -- TTY (-) and TRM_NOECHO (-)   <--- This lets YDB take care of output  (preferred)

I am going to exclude #9, and use #12 instead.  I will therefore have this logic

      Device parameters  -->  Internal ydb & TTY IO state settings 

    YDB ECHO  CANONICAL  -->  TTY ECHO  TRM_NOECHO  TTY CANONICAL   
 1    (+)       (+)             (+)       (+)           (+)
12    (+)       (-)             (-)       (-)           (-)
 7    (-)       (+)             (-)       (+)           (+)
15    (-)       (-)             (-)       (+)           (-)

Kevin