Need help with READ command and BACKSPACE key

[Kevin Toppenberg]

In code below, I run it and press [BACKSPACE] repeatedly, but it is never read.

yottadb>for i=1:1:5 read *temp:0 write temp,! hang 0.5

-1
-1
-1
-1
-1

In the code below, the [BACKSPACE] is read.

yottadb>for i=1:1:5 read *temp:1 write temp,! hang 0.5
127
127
127
127
127

The only difference between these lines of code is the timeout of 0 (immediate return) vs 1 second timeout.  Does anyone know why this would be?  Here are the device settings

yottadb>zshow "d"

/dev/pts/0 OPEN TERMINAL NOPAST NOESCA NOREADS TYPE WIDTH=213 LENG=47

yottadb>w $ZVER
GT.M V6.3-008 Linux x86_64
yottadb>

Thanks

Kevin

[Sam Habiel]

:0 will only read what's already in the OS buffer. You have nothing there at that point (you have to press backspace). That part is quite complicated as it differs by device type.

--Sam

--
You received this message because you are subscribed to the Google Groups "Everything MUMPS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to everythingmum...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/everythingmumps/416bca87-e535-43ce-8f7d-d2600c726c27n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kevin Toppenberg]

Where did the backspace keystrokes go then?  Maybe the buffer will be empty when a read:0 comes in, but it should be there on the next cycle, right?  During the hang 0.5, there is plenty of time for me to put more keys into the buffer.

I am trying to set up a message loop where I need non-blocking IO reads.  Is there a different way to achieve this?  Some way to tell YottaDB to just pass everything through with no filtering?

Thanks

KT

[Sam Habiel]

I don't know. Sorry. I know others around here are much better than me at this.

--Sam

To view this discussion visit https://groups.google.com/d/msgid/everythingmumps/e49520fd-abb3-4701-a631-ed1a587a6b27n%40googlegroups.com.

[Kevin Toppenberg]

OK, thanks for thinking about this.  I'll see if anyone else has any ideas.  I may ask on Hardhats also

KT

[K.S. Bhaskar]

I have experimented with the various deviceparameters, and don't have a way to make the two timeouts behave the same. However, $test is different. In the following, I didn't hit the backspace key sometimes:

YDB>for i=1:1:5 read *temp:0 write $test," ",temp,! hang 0.5
0 -1
0 -1
0 -1
0 -1
0 -1

YDB>for i=1:1:5 read *temp:1 write $test," ",temp,! hang 0.5
1 127
1 127
0 -1
0 -1
0 -1

YDB>

So I don't have an explanation. As this anomaly is not likely to make it up on our list of priorities to address, my suggestion in your application is to check $test if that helps.

Regards

– Bhaskar

[Kevin Toppenberg]

Bhaskar,

Thank you for taking the time to look into this.  I see that $test varies between the two states, but this would not be sufficient to know that a [backspace] key had been struck.

I suspect what is happening is that Yottadb provides for input editing, such that a READ X and then the user typing 'A[backspace]B' will result in X='B'.

I understand that this issue may not rise in priority to point of investigation.

I am considering trying to figure this out myself.  I'll make a separate post about compiling the source code.

Kevin

[Kevin Toppenberg]

Here is an interesting twist.  Instead of having a timeout of 0, which never returns the backspace, or timeout of 1 second, which ruins the idea of a non-blocking check for user input, I tried using fractional timeouts.  And it works!!!  See below. 

yottadb>f i=1:1:5 read *x:0 w x,! hang 0.5   <---- same as before.  Never reads backspace

-1
-1
-1
-1
-1

yottadb>f i=1:1:5 read *x:0.01 w x,! hang 0.5   <--- sometimes reads backspace. 
-1
127
-1
127
-1

yottadb>f i=1:1:5 read *x:0.001 w x,! hang 0.5  <--- seems to miss all

-1
-1
-1
-1
-1

yottadb>f i=1:1:5 read *x:0.01 w x,! hang 0.5   <--- worked with trial above, but this time didn't.  I pressed backspace, but it wasn't read.

-1
-1
-1
-1
-1

yottadb>f i=1:1:5 read *x:0.1 w x,! hang 0.5  <--- seems to read fairly consistently.
-1
127
127
127
127

yottadb>f i=1:1:50000 read *x:0.001 w:x>0 x,!   <---- WORKS EVERY TIME!!  Every backspace keystroke results in a '127' read.  Hurrah!
127
127
127
127
a97
b98
c99
d100
e101
f102
g103

127
127
127
127
127

%YDB-I-CTRLC, CTRL_C encountered

[Kevin Toppenberg]

Ahhh, not so fast....  There appears to be some race condition for processing input buffer characters.  If the loop is really fast, then it always gets the 127 (backspace) key.  But if there is a delay in the loop, then some other process or action appears to remove the 127 from the buffer.  I have several examples below, each with increasing HANG times.  In the last example below, I only got 1 char for 127 even though I struck the key many times.

So if my key-polling message loop is really fast, i.e. does no other substantial work, then I can get 127's back.  Otherwise something gobbles them up.

Sigh.

KT

Examples below:

Yottadb>F I=1:1:5000 READ *X:0.001 W:X>0 X,!   <--- baseline, no HANG
127
127
127
a97
a97
a97
%YDB-I-CTRLC, CTRL_C encountered

yottadb>F I=1:1:5000 READ *X:0.001 W:X>0 X,! HANG 0.001  <--- smallest HANG.  All backspace keystrokes are returned. 
127
127
127
aa97
aa97
aa97
%YDB-I-CTRLC, CTRL_C encountered

yottadb>F I=1:1:5000 READ *X:0.001 W:X>0 X,! HANG 0.01  <--- 10x longer hang.  Most backspace keys still returned. 
a97
a97
127
127
127
127
^C%YDB-I-CTRLC, CTRL_C encountered

yottadb>F I=1:1:5000 READ *X:0.001 W:X>0 X,! HANG 0.1  <--- Another 10x longer hang.  Backspace key struck many times, but only 1 returned
aa97
aa97
127    <---- only 1 backspace returned. 
^C%YDB-I-CTRLC, CTRL_C encountered

[Mark Sires]

Kevin,

I think this is actually a Linux issue, not a YottaDB issue.  If you type any character except DEL (ascii 127), it is captured with a R *X:0 loop.  I suspect that Linux is processing the DEL different than all the other characters.  If you use BKSP (ascii 8 or CTRL-H), it captures it as well.

Mark

[Maury Pepper]

Mark, Kevin, Bhaskar, Sam ...

Please correct me if I'm too far out of the loop, but according to the M Standard and the YottaDB Programmers Guide, the time for a HANG is in whole seconds so H .5 is the same as H 0. On the other hand, in YottaDB, the timeout for a READ is accurate to 3 decimal places. If that's the case then some of the trials here would have finished execution before anything could be typed. The examples I've seen here seem to support this explanation. I suggest trying:  F i=1:1:20 H .5  and see if it takes 10 seconds to terminate. (If there's a Win10 version of YDB, I'd love to run it.)
          -maury-

--

You received this message because you are subscribed to the Google Groups "Everything MUMPS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to everythingmum...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/everythingmumps/317086e4-55a7-4a62-9cbd-8cae7a41d2ecn%40googlegroups.com.

[Kevin Toppenberg]

Hmmm.... very curious.  I see that the backspace key is generating #127.  This is true for iTerm on Mac and PuTTY on Windows.  ASCII table says #127 is DEL.  When I press the DEL key on my keyboard, it generates an escape sequence.  So there is a bit of mixup there that is not a YottaDB issue.

I agree that the ASCII backspace (BS), #8 is always captured. 

To investigate if this is a linux issue, I did some research and came up with this command:

 strace -o debug.txt -f -ff -e trace=read,write ~/runAV

runAV is the script that normally launches our YottaDB.  The strace command will show system calls to and from a program. 

After YottaDB launched, I pasted this code in:

F I=1:1:5000 READ *X:0.01 W:X>0 X,! HANG 0.1

I then typed 'a','b','c' and struck the backspace key 10 times.  I think I was able to capture a 127 two times.

I then halted and examined the debug files.

...

read(0, "\r", 1)                        = 1   <---- here I pressed enter to start executing code line
write(0, "\33[?1l\33>", 7)              = 7
write(0, "\n", 1)                       = 1

read(0, "a", 1)                         = 1  <-- here I typed 'a'
write(0, "a", 1)                        = 1
write(0, "97\n", 3)                     = 3

read(0, "b", 1)                         = 1  <-- here I typed 'b'
write(0, "b", 1)                        = 1
write(0, "98\n", 3)                     = 3

read(0, "c", 1)                         = 1 <-- here I typed 'c'
write(0, "c", 1)                        = 1
write(0, "99\n", 3)                     = 3

read(0, "\177", 1)                      = 1  <--- here is where I started pressing BACKSPACE key 10 times. 
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4   <--- notice that only 2 instances of 127 are read

--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---    <--- here I typed CTRL-C to break the loop
write(0, "%YDB-I-CTRLC, CTRL_C encountered"..., 33) = 33
write(0, "\n", 1)                       = 1
write(0, "\r", 1)                       = 1
write(0, "yottadb>", 8)                 = 8
write(0, "\33[?1h\33=", 7)              = 7

read(0, "h", 1)                         = 1   <-- starting to type halt
write(0, "h", 1)                        = 1
write(0, "\10", 1)                      = 1
write(0, "\33[C", 3)                    = 3
read(0, "a", 1)                         = 1
write(0, "a", 1)                        = 1
write(0, "\10", 1)                      = 1
write(0, "\33[C", 3)                    = 3
read(0, "l", 1)                         = 1
write(0, "l", 1)                        = 1
write(0, "\10", 1)                      = 1
write(0, "\33[C", 3)                    = 3
read(0, "t", 1)                         = 1
write(0, "t", 1)                        = 1
write(0, "\10", 1)                      = 1
write(0, "\33[C", 3)                    = 3
read(0, "\r", 1)                        = 1
write(0, "\33[?1l\33>", 7)              = 7
write(0, "\n", 1)                       = 1
+++ exited with 0 +++

Next, I repeated the test with the shorter hang time:

F I=1:1:5000 READ *X:0.01 W:X>0 X,! HANG 0.001

and here it the debug log from that:

read(0, "\r", 1)                        = 1 <-- return to start code
write(0, "\33[?1l\33>", 7)              = 7
write(0, "\n", 1)                       = 1
read(0, "a", 1)                         = 1  <-- a
write(0, "a", 1)                        = 1
write(0, "97\n", 3)                     = 3
read(0, "b", 1)                         = 1  <-- b
write(0, "b", 1)                        = 1
write(0, "98\n", 3)                     = 3
read(0, "c", 1)                         = 1 <-- c
write(0, "c", 1)                        = 1
write(0, "99\n", 3)                     = 3
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4  <-- reads 127
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4  <-- reads 127
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4  <-- reads 127
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4  <-- reads 127
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4  <-- reads 127
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4  <-- reads 127
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4  <-- reads 127
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4  <-- reads 127
read(0, "\177", 1)                      = 1
write(0, "\177", 1)                     = 1
write(0, "127\n", 4)                    = 4  <-- reads 127
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
write(0, "%YDB-I-CTRLC, CTRL_C encountered"..., 33) = 33

Thus Mark may be on to the real problem here.  It seems that YottaDB never is able to read the 127 with the longer hang time.

I'll next research how to modify the way linux is handling this.

Thanks

KT

[Kevin Toppenberg]

Maury,

The YottaDB entry for hang says this (https://docs.yottadb.com/ProgrammersGuide/commands.html#hang):

The numeric expression specifies the time in seconds to elapse before resuming execution; actual elapsed time may vary slightly from the specified time. If the numeric expression is negative, HANG has no effect. Portability requirements for YottaDB only guarantee accuracy to the nearest second. However, more accuracy can be found on different UNIX systems.

I performed this test to find the actual length of hang 0.001 on my linux system:

yottadb>f n=100,1000,5000,7000,8000,10000 w !,"STARTING ",n," CYCLES..." s start=$tr($h,",",".") f i=1:1:n h 0.001 if i=n s
end=$tr($h,",","."),delta=(end-start)*100000 w !,start,!,end,!,"elapsed=",delta," or ",delta/n," per cycle",!
STARTING 100 CYCLES...
67178.57377
67178.57378
elapsed=1 or .01 per cycle

STARTING 1000 CYCLES...
67178.57378
67178.57390
elapsed=12 or .012 per cycle

STARTING 5000 CYCLES...
67178.57390
67178.57414
elapsed=24 or .0048 per cycle

STARTING 7000 CYCLES...
67178.57414
67178.57458
elapsed=44 or .00628571428571428571 per cycle

STARTING 8000 CYCLES...
67178.57458
67178.57493
elapsed=35 or .004375 per cycle

STARTING 10000 CYCLES...
67178.57493
67178.57525
elapsed=32 or .0032 per cycle

Kevin

[Kevin Toppenberg]

With the help of chatGPT, I have confirmed that this is a Linux issue, specifically in the TTY system. 

If I type "stty raw" at the command line before launching YottaDB, then all backspace chars are returned.  After leaving YottaDB, I can type "stty sane" and it returns normal function, and repeating the test shows only a few backspaces getting through.

If I type: stty erase ''  <-- two singe quotes

then I also can capture the backspace chars in YottaDB, and again "stty sane" will restore function.    Apparently `stty erase '^?'` will also restore function (though I didn't test this).

It is possible for a c program, such as YottaDB, to directly tell TTY to disable BACKSPACE filtering.  Suggested code as follows:

Include the Necessary Headers:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <termios.h>

C code that will modify TTY behavior.

struct termios orig_termios, new_termios;

// Function to restore original terminal settings
void reset_terminal_mode() {
    tcsetattr(STDIN_FILENO, TCSANOW, &orig_termios);
}

// Function to set raw mode
void set_raw_mode() {
    tcgetattr(STDIN_FILENO, &orig_termios);
    atexit(reset_terminal_mode); // Ensure settings are restored on exit

    new_termios = orig_termios;
    new_termios.c_lflag &= ~(ICANON | ECHO); // Disable canonical mode and echo
    new_termios.c_cc[VERASE] = _POSIX_VDISABLE; // Disable erase character

    tcsetattr(STDIN_FILENO, TCSANOW, &new_termios);
}

int main() {
    set_raw_mode();

    // Your program logic here
    // ...

    return 0;
}

The above code seems to turn raw mode on, which is more than I would need, but it might be a nice extension for YottaDB to allow for an IO channel to be in raw mode.

Kevin

[K.S. Bhaskar]

Maury, to run YottaDB on Win 10, try a Docker container (https://yottadb.com/product/get-started/) or WSL 2 (https://learn.microsoft.com/en-us/windows/wsl/install). The odds of a native Windows port in the near future are vanishingly low.

Regards

– Bhaskar

[Chris U]

Kevin,

127 is the decimal representation of the ASCII code for delete.

\177 interestingly is the octal representation of the same ASCII code.

Take a look at "stty erase" and see if you can force the session to behave a certain way.

To view this discussion visit https://groups.google.com/d/msgid/everythingmumps/816e7db9-0f21-48d5-aec7-66cec622a6a6n%40googlegroups.com.

[Kevin Toppenberg]

Chris,

Yes, you are correct, that stty erase ''  will make this work correctly.   But it is not reasonable (or at least it is impractical and will have secondary consequences) to have to tweak stty *prior* to running YottaDB.

I previously thought that this meant that this was not a YottaDB issue.  But now I am reconsidering.  YottaDB actively sends messages to the tty system to affect its management.  For example, imaging that YottaDB was a word processor.  It would need to have full control over the backspace key.  So it would have to send signals to tty telling it to not process the backspace itself.

There is a termios code module in c, and as I have looked through the source code, I see that YottaDB is using this to control how tty works. 

So now I think it is not controlling it properly.  But I have to prove this, and that is going to be tricky.

:-)

KT