Any way to avoid mixing quickly entered text with output?

[yon...@yahoo.com]

UNIX and Linux terminals allow text input even when the currently running program is still outputing text. For example, inside certain applications, such as "perl -d", where you send many lines of "print 'a';", or in Oracle's sqlplus or MySQL's mysql, where you send many "select 1 from dual;", the input and output are garbled. Some programs do a better job, such as bash, python.

This is an age old question so it's not surprising people have already discussed. See
https://unix.stackexchange.com/questions/210628/why-does-the-terminal-echo-keystrokes-when-commands-are-running
and a quote of Dennis M. Ritchie's words at
https://unix.stackexchange.com/questions/178989/is-it-safe-to-type-another-command-into-stdin-while-the-previous-command-is-writ/179002#179002

So we know UNIX terminals are designed this way, so users can see what they type immediately. But it would be nice to give users an option. Windows command console strictly serializes input and output based on the time of the input and output. It doesn't suit all needs but is the effect sometimes I want when using UNIX/Linux. None of the discussions I read mentions modern terminal capabilities that might offer some help. Note: this is not about turning off local echo. Lowering baud rate with stty has no effect either.

While not directly related to a discussion about terminals, since some programs (bash, python) do a better job, how do they achieve that?

[Scott Dorsey]

<yon...@yahoo.com> wrote:
>UNIX and Linux terminals allow text input even when the currently running p=
>rogram is still outputing text. For example, inside certain applications, s=
>uch as "perl -d", where you send many lines of "print 'a';", or in Oracle's=
> sqlplus or MySQL's mysql, where you send many "select 1 from dual;", the i=
>nput and output are garbled. Some programs do a better job, such as bash, p=
>ython.

This is a feature, not a bug.
Redirect your output elsewhere so it doesn't look confusing. Or turn
off echoing with stty -echo and be done with it.

>So we know UNIX terminals are designed this way, so users can see what they=
> type immediately. But it would be nice to give users an option. Windows co=
>mmand console strictly serializes input and output based on the time of the=
> input and output. It doesn't suit all needs but is the effect sometimes I =
>want when using UNIX/Linux. None of the discussions I read mentions modern =
>terminal capabilities that might offer some help. Note: this is not about t=

>urning off local echo. Lowering baud rate with stty has no effect either.

Terminals are intended for human beings, where this is a feature. I cannot
imagine an application where it would be a problem other than if you have a
machine talking to another machine, in which case shutting off echo is a
definite win.

>While not directly related to a discussion about terminals, since some prog=

>rams (bash, python) do a better job, how do they achieve that?

In what way do they do a better job? They're just using cooked mode like
everything else.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."

[yon...@yahoo.com]

> >While not directly related to a discussion about terminals, since some prog=
> >rams (bash, python) do a better job, how do they achieve that?
>
> In what way do they do a better job? They're just using cooked mode like
> everything else.

In case I was not clear, suppose you copy these lines
date
date
date
and paste them all at once to bash. The terminal screen will neatly show one line of shell prompt and the "date" command, followed by the result of the command (date and time) on the next line, and prompt plus commnad, and command result, ... one line at a time.

But if you copy all these lines
print 'X';
print 'X';
print 'X';
and paste them all at once to the Perl debugging shell given by "perl -d" followed by ^D, i.e. at the __DB<1> prompt, the lines you pasted will be mixed with the 3 lines of 'X' character.

I hope there's a way to set some terminal option so that each "print 'X';" line is clearly printed *before* the X character it prints. If you have Perl installed on your Windows box, do the same. You'll get

DB<1> print 'X';
X
DB<2> print 'X';
X
DB<3> print 'X';
X

That's what I want.

[Scott Dorsey]

<yon...@yahoo.com> wrote:
>
>In case I was not clear, suppose you copy these lines
>date
>date
>date

>and paste them all at once to bash. The terminal screen will neatly show on=
>e line of shell prompt and the "date" command, followed by the result of th=
>e command (date and time) on the next line, and prompt plus commnad, and co=

>mmand result, ... one line at a time.

That's because bash is doing the echoing rather than having the OS doing
the echoing. I forget if bash uses cbreak mode or raw.
I know that normal sh does not do this.

>But if you copy all these lines
>print 'X';
>print 'X';
>print 'X';

>and paste them all at once to the Perl debugging shell given by "perl -d" f=
>ollowed by ^D, i.e. at the __DB<1> prompt, the lines you pasted will be mix=

>ed with the 3 lines of 'X' character.

This is how normal cooked I/O operates.

>I hope there's a way to set some terminal option so that each "print 'X';" =
>line is clearly printed *before* the X character it prints. If you have Per=

>l installed on your Windows box, do the same. You'll get

No... Unix was not designed with the intention that people were going to
paste big blocks of text into a terminal. Put it in a script, run the
script.

[yon...@yahoo.com]

> No... Unix was not designed with the intention that people were going to
> paste big blocks of text into a terminal. Put it in a script, run the
> script.

I know it was not designed with that intention. But things change, don't they? UNIX has been around for 50 years. If some people have the need (as someone in my first quoted link does, or me), why not change the tty driver to give us the option? If UNIX has to stick to the UNIX standard to be called UNIX, Linux is not bound by the restriction, so I hope someday this can be implemented on Linux.

I know putting in a script won't have the problem. That's what I've been doing over the years. Another option is to run those programs through Expect and let it catch the input and echo it after the result of the previous command has come out.

Saying bash runs in cooked mode by default is probably better than saying it runs in raw or cbreak. But raw or cooked is just a shorthand for a bunch of specific terminal characteristics.

[Kaz Kylheku]

On 2020-04-08, yon...@yahoo.com <yon...@yahoo.com> wrote:
> UNIX and Linux terminals allow text input even when the currently
> running program is still outputing text. For example, inside certain
> applications, such as "perl -d", where you send many lines of "print
> 'a';", or in Oracle's sqlplus or MySQL's mysql, where you send many
> "select 1 from dual;", the input and output are garbled. Some programs
> do a better job, such as bash, python.

This is because the TTY line discipline is in a mode where echo (stty echo)
is turned on.

Regardless of whether echo is turned on, the TTY device receives
characters and buffers them at all times, even when the application
isn't reading. (If the buffer gets full, flow control has to kick in,
failing which, data will be dropped.)

If echo is turned on, then the low level code in the TTY device driver
spits back whatever it receives.

Interactive bash does a "better job" because it puts the tty driver in a
raw mode with disabled echo, in order to have more fully featured
editing. When executing a command, bash puts itself into the background
and restores the "cooked mode" TTY settings.

Echo is very important for the sake of rudimentary editing. Most input
processing programs do not have interactive editing features, yet
sometime you have to interact with them.

For intance, cat:

$ cat
abc_

cat just calls read() and that's it. When you hit Enter to generate
a newline, the read() returns an entire line.

The reason you can see what you type, and use backspace to make
corrections: that's all done in the TTY driver, invisibly to cat.

And the TTY driver, ont he ther hand, does its job without visibility as
to what the cat process is doing. The editing works without regard for
whether some process is calling read() on the file descriptor.

Implementation-wise, it happens in software levels below where file
descriptor reads are handled. There some very low level driver that
could be a serial device or console or the master side of a pseudo-tty.
That low level driver typically spontaneously sends characters to a
higher level, at interrupt time. In that higher level is an mediating
module called the "line discipline" which implements the echo,
backspace, word erase and whatnot. That module, in turn, sends data to a
higher layer still where file descriptor interaction takes place: like
blocking a read() on the availability of data, putting
data in the buffr and unblocking it.

--
TXR Programming Lanuage: http://nongnu.org/txr
Music DIY Mailing List: http://www.kylheku.com/diy
ADA MP-1 Mailing List: http://www.kylheku.com/mp1

[Kaz Kylheku]

On 2020-04-09, yon...@yahoo.com <yon...@yahoo.com> wrote:
>> >While not directly related to a discussion about terminals, since some prog=
>> >rams (bash, python) do a better job, how do they achieve that?
>>
>> In what way do they do a better job? They're just using cooked mode like
>> everything else.
>
> In case I was not clear, suppose you copy these lines
> date
> date
> date
> and paste them all at once to bash. The terminal screen will neatly show one line of shell prompt and the "date" command, followed by the result of the command (date and time) on the next line, and prompt plus commnad, and command result, ... one line at a time.
>
> But if you copy all these lines
> print 'X';
> print 'X';
> print 'X';
> and paste them all at once to the Perl debugging shell given by "perl -d" followed by ^D, i.e. at the __DB<1> prompt, the lines you pasted will be mixed with the 3 lines of 'X' character.

That's simply because in the span of 35 years, the Perl people haven't
managed to implement a line editing mode in perl -d.

If you want, you can try patching Perl to use a line editing library
like linenoise, libedit. (Not sure if GNU readline would be comopatible
with the Perl license; it can only be integrated into programs
with GPL-compatible licenses.)

Note, however, also that programming languages with "plain" interactive
debugging modes like "perl -d" can be integrated with editors or IDE
environments.

That could be why the Perl maintainers have not bothered
with better TTY support.

So that is to say, maybe people are using "perl -d" as a coprocess
out of their Emacs or whatever.

When a debugger is used as a coprocess, its TTY support is
irrelevant. It cannot be used, because the standard input and output
are not a TTY; they are pipes!

Also, even without a full blown editor, it's possible to have
a line editing interactive mode as a front-end for a program
that doesn't have one.

Take a look at rlwrap:

https://github.com/hanslub42/rlwrap

"rlwrap runs the specified command, intercepting user input in order to
provide readline's line editing, persistent history and completion."

If you run perl -d under rlwrap, you should get the behavior
you're looking for.

On Ubuntu, it's "sudo apt-get install rlwrap".

It seems there is more than one available program for this
in Ubuntu, because when you install it, you see this:

Setting up rlwrap (0.43-1) ...
update-alternatives: using /usr/bin/rlwrap to provide /usr/bin/readline-editor (readline-editor) in auto mode
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

I.e. the abstract interface is /usr/bin/readline-editor, which
will run whatever program is installed as the provider,
like rlwrap.