[BUG] Confusing behavior of the char_avail() function breaks titlestring

[Jason Franklin]

Greetings:

I discovered a bug a while ago that can be observed in xterm, gnome-terminal,

and konsole (on KDE).

To reproduce the bug, with the latest build...

1. cd src/

2. VIMRUNTIME=../runtime ./vim -u NONE --cmd 'set nocp lz title'

3. note that the title string is not drawn, also, possibly, the cursor is not

   in its proper place

Further, this works if "set nocp lz title" is done in a special vimrc file

instead of on the command line.

I have investigated this issue, and I observed that, when maketitle() is

called, it calls the redrawing() function to see if it needs to do its

business to update the title.  The redrawing() function returns false on the

first call, and the reason is that char_avail() returns TRUE.  I can't

figure out why char_avail() would return true when Vim is just starging up

and no keys have been pressed.

Currently, the redrawing() function is a bit difficult to read, the current

version is reproduced below:

/*

 * Return TRUE if redrawing should currently be done.

 */

    int

redrawing(void)

{

#ifdef FEAT_EVAL

    if (disable_redraw_for_testing)

return 0;

    else

#endif

return ((!RedrawingDisabled

#ifdef FEAT_EVAL

    || ignore_redraw_flag_for_testing

#endif

) && !(p_lz && char_avail() && !KeyTyped && !do_redraw));

}

I have refactored this function for readability.  The new function is here:

/*

 * Return TRUE if redrawing should currently be done.

 */

    int

redrawing(void)

{

#ifdef FEAT_EVAL

    if (disable_redraw_for_testing)

return FALSE;

#endif

    if (RedrawingDisabled

#ifdef FEAT_EVAL

    && !ignore_redraw_flag_for_testing

#endif

       )

return FALSE;

    return !p_lz || !char_avail() || KeyTyped || do_redraw;

}

Using early return here makes the function easier to understand, in my

opinion.

Anyway, the debugger tells me that the reason redrawing() returns FALSE on

the first call to maketitle() is because char_avail() returns TRUE.  Why

would this be the case with my "./vim" command above?

Thanks,

Jason Franklin

[Bram Moolenaar]

Possibly Vim asks the terminal for the version, and the response can be
read from the input. You could make t_RV empty to avoid that.

--
hundred-and-one symptoms of being an internet addict:
7. You finally do take that vacation, but only after buying a USB modem
and a laptop.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Jason Franklin]

On Wednesday, January 29, 2020 at 3:28:06 PM UTC-5, Bram Moolenaar wrote:

Possibly Vim asks the terminal for the version, and the response can be
read from the input.  You could make t_RV empty to avoid that.

I don't think this is the right fix.  NeoVim doesn't have this problem for two reasons that I can tell:

1. maketitle() is called twice in the NeoVim startup process, once in Vim

2. in NeoVim, char_avail() is zero even with the same options as above (-u NONE --cmd 'set nocp lz title')

I'm not sure why char_avail() returns a different response here.

I'm not sure how to go about setting t_RV to an empty string. I tried the following:

  VIMRUNTIME=../runtime ./vim -u NONE --cmd 'set nocp t_RV= lz title'

with no success.

-- 

Jason Franklin

[Christian Brabandt]

On Mi, 29 Jan 2020, Jason Franklin wrote:

> 1. maketitle() is called twice in the NeoVim startup process, once in Vim
> 2. in NeoVim, char_avail() is zero even with the same options as above (-u
> NONE --cmd 'set nocp lz title')
>
> I'm not sure why char_avail() returns a different response here.
>
> I'm not sure how to go about setting t_RV to an empty string. I tried the
> following:
>
> VIMRUNTIME=../runtime ./vim -u NONE --cmd 'set nocp t_RV= lz title'

Yes, that should be correct. There are a couple of other terminal
properties, that Vim queries on startup, like terminal version (t_TV),
cursor position (t_u7), terminal foreground/background color (t_RF,
t_RB), terminal cursor blinking (t_RC) and cursor style (t_RS), possibly
more, that I couldn't find out easily.

Have you tried with a dumb terminal, so Vim won't try to figure all this
out, e.g. TERM=dumb vim .... (you won't have fun with this one, as it
doesn't show colors)

or alternatively with one of the builtin terminals:

VIMRUNTIME=../runtime ./vim -u NONE -N --cmd ':set term=builtin_ansi'

Best,
Christian
--
Wie man sein Kind nicht nennen sollte:
Immo Bilie

[Franklin, Jason]

On 1/30/20 8:02 AM, Christian Brabandt wrote:
> Have you tried with a dumb terminal, so Vim won't try to figure all this
> out, e.g. TERM=dumb vim .... (you won't have fun with this one, as it
> doesn't show colors)
>
> or alternatively with one of the builtin terminals:
>
> VIMRUNTIME=../runtime ./vim -u NONE -N --cmd ':set term=builtin_ansi'

I gave this a shot, and it definitely didn't solve the problem.

I think the real solution here is to make sure that maketitle() can do
its work at start up regardless of the result of char_avail(). Is that
reasonable? More work may be necessary to really get it right, though
(see the following note).

N.B.:

I need to note here that the title string is not the only user interface
element that is broken by this problem. Try the following:

VIMRUNTIME=../runtime ./vim -u NONE \
--cmd 'set nocp lz title laststatus=2'

You will see that even the status line is blocked at startup. The
screen will redraw when a key is pressed (e.g., "l"). However, this
looks really bad and happens with any basic vimrc file that sets these
options.

Again, this happens in: xterm, konsole, and gnome-terminal.

--
Jason Franklin

[Bram Moolenaar]

I don't see any problem. Window title is updated and status line shows
properly.

--
When a fly lands on the ceiling, does it do a half roll or
a half loop?

[Franklin, Jason]

On 1/30/20 10:14 AM, Bram Moolenaar wrote:
> I don't see any problem. Window title is updated and status line shows
> properly.
>

I have attached a screenshot that shows what I see after running this
command:

VIMRUNTIME=../runtime ./vim -u NONE \
--cmd 'set nocp lz title laststatus=2'

When you say there is no problem, do you mean that the rendering happens
as expected, or after you press a key? I need to press a key to see the
screen update.

--
Jason Franklin

[Bram Moolenaar]

I don't need to press a key.

You could try to set $TERM to different values.

--
hundred-and-one symptoms of being an internet addict:

23. You can't call your mother... she doesn't have VOIP

[Franklin, Jason]

On 1/30/20 1:53 PM, Bram Moolenaar wrote:
>> When you say there is no problem, do you mean that the rendering happens
>> as expected, or after you press a key? I need to press a key to see the
>> screen update.
>
> I don't need to press a key.

Have you tried using *specifically* xterm? I'm using xterm installed
from the xterm package on Debian 10.1.

> You could try to set $TERM to different values.

I'm not sure what value might work???

I've tried several, with the default being TERM=xterm. None change the
behavior. Screenshots of xterm before the command, after the command,
and after a keypress are attached.

The command is:

VIMRUNTIME=../runtime ./vim -u NONE --cmd 'set nocp lz title'

I have confirmed this on multiple machines already.

Can someone else please confirm this?

--
Jason Franklin

[Bram Moolenaar]

Jason Franklin wrote:

> On 1/30/20 1:53 PM, Bram Moolenaar wrote:
> >> When you say there is no problem, do you mean that the rendering happens
> >> as expected, or after you press a key? I need to press a key to see the
> >> screen update.
> >
> > I don't need to press a key.
>
> Have you tried using *specifically* xterm? I'm using xterm installed
> from the xterm package on Debian 10.1.

I'm using xterm.

> > You could try to set $TERM to different values.
>
> I'm not sure what value might work???
>
> I've tried several, with the default being TERM=xterm. None change the
> behavior. Screenshots of xterm before the command, after the command,
> and after a keypress are attached.
>
> The command is:
>
> VIMRUNTIME=../runtime ./vim -u NONE --cmd 'set nocp lz title'
>
> I have confirmed this on multiple machines already.
>
> Can someone else please confirm this?

I tried a few more times and finally noticed it once. Must be a race
condition.

--
Computers are useless. They can only give you answers.
-- Pablo Picasso

[Franklin, Jason]

On 1/31/20 3:15 PM, Bram Moolenaar wrote:
>> The command is:
>>
>> VIMRUNTIME=../runtime ./vim -u NONE --cmd 'set nocp lz title'
>>
>> I have confirmed this on multiple machines already.
>>
>> Can someone else please confirm this?
>
> I tried a few more times and finally noticed it once. Must be a race
> condition.
>

Thank goodness! I was beginning to think I was crazy.

It's weird that I can reliably produce the bug on every invocation. I
don't have any problem reproducing from one try to the next.

Maybe the build configuration is different? See below...

VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Jan 29 2020 10:46:03)
Included patches: 1-172
Compiled by jfrankli@qipb7lnca
Huge version without GUI. Features included (+) or not (-):
+acl -farsi -mouse_sysmouse -tag_old_static
+arabic +file_in_path +mouse_urxvt -tag_any_white
+autocmd +find_in_path +mouse_xterm -tcl
+autochdir +float +multi_byte +termguicolors
-autoservername +folding +multi_lang +terminal
-balloon_eval -footer -mzscheme +terminfo
+balloon_eval_term +fork() +netbeans_intg +termresponse
-browse +gettext +num64 +textobjects
++builtin_terms -hangul_input +packages +textprop
+byte_offset +iconv +path_extra +timers
+channel +insert_expand -perl +title
+cindent +job +persistent_undo -toolbar
-clientserver +jumplist +popupwin +user_commands
-clipboard +keymap +postscript +vartabs
+cmdline_compl +lambda +printer +vertsplit
+cmdline_hist +langmap +profile +virtualedit
+cmdline_info +libcall -python +visual
+comments +linebreak -python3 +visualextra
+conceal +lispindent +quickfix +viminfo
+cryptv +listcmds +reltime +vreplace
+cscope +localmap +rightleft +wildignore
+cursorbind -lua -ruby +wildmenu
+cursorshape +menu +scrollbind +windows
+dialog_con +mksession +signs +writebackup
+diff +modify_fname +smartindent -X11
+digraphs +mouse -sound -xfontset
-dnd -mouseshape +spell -xim
-ebcdic +mouse_dec +startuptime -xpm
+emacs_tags -mouse_gpm +statusline -xsmp
+eval -mouse_jsbterm -sun_workshop -xterm_clipboard
+ex_extra +mouse_netterm +syntax -xterm_save
+extra_search +mouse_sgr +tag_binary
system vimrc file: "$VIM/vimrc"
user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
defaults file: "$VIMRUNTIME/defaults.vim"
fall-back for $VIM: "/usr/local/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -Og -ggdb -g3
Linking: gcc -L/usr/local/lib -Wl,--as-needed -o vim -lm
-ltinfo -lnsl -ldl

--
Jason Franklin

[Jason Franklin]

I've come up with a small patch that fixes this issue for me very reliably for

me.  It breaks no tests, and it fixes both the lagging status line and title string

on my machine:

diff --git a/src/main.c b/src/main.c

index 68a419eaa..4bd0219c0 100644

--- a/src/main.c

+++ b/src/main.c

@@ -893,11 +893,11 @@ vim_main2(void)

     }

 #endif

 

+    do_redraw = TRUE;

+

     TIME_MSG("before starting main loop");

 

-    /*

-     * Call the main command loop.  This never returns.

-     */

+    // Call the main command loop.  This never returns.

     main_loop(FALSE, FALSE);

 

 #endif // NO_VIM_MAIN

The key is that stuff_empty() should be TRUE when Vim starts up (I think).  So, we have

the condition 

else if (do_redraw || stuff_empty())

in main.c on line 1268.  Since this evaluates to TRUE, we proceed with the redrawing.  Finally,

we set do_redraw to FALSE later in this block.  Thus, we do all of the drawing on start up and we

set do_redraw to FALSE as if it was never set before.  The fact that main_loop() doesn't return

means that we don't break anything.

This seems to fix my issue without breaking anything else.  However, I can't see all of the

other possible problems.

Bram, is this reasonable?

-- 

Jason Franklin

[Bram Moolenaar]

Yes, that makes sense. It only happens once, after most of the startup
has been done. It should happen on a next pass through main_loop(), but
apparently some typehead is consumed at a lower level without returning
to the main loop.

--
How To Keep A Healthy Level Of Insanity:
11. Specify that your drive-through order is "to go".

[Franklin, Jason]

On 2/3/20 3:30 PM, Bram Moolenaar wrote:
> Yes, that makes sense. It only happens once, after most of the startup
> has been done. It should happen on a next pass through main_loop(), but
> apparently some typehead is consumed at a lower level without returning
> to the main loop.

Thanks, Bram.

I looked at the NeoVim code to see how they avoid this UI problem. Their
solution was to do two maketitle() calls (I think, ultimately, two
redraws after startup), which I think is not correct.

So, it might be that the char_avail() code should be reviewed to
actually fix the problem. I just don't know exactly how it "should"
work, if it's even broken at all.

--
Jason Franklin

[Bram Moolenaar]

Jason Franklin wrote:

> On 2/3/20 3:30 PM, Bram Moolenaar wrote:
> > Yes, that makes sense. It only happens once, after most of the startup
> > has been done. It should happen on a next pass through main_loop(), but
> > apparently some typehead is consumed at a lower level without returning
> > to the main loop.
>
> Thanks, Bram.
>
> I looked at the NeoVim code to see how they avoid this UI problem. Their
> solution was to do two maketitle() calls (I think, ultimately, two
> redraws after startup), which I think is not correct.

It's an easy way out to just redraw more often, which doesn't show any
problems but will make Vim slower.

> So, it might be that the char_avail() code should be reviewed to
> actually fix the problem. I just don't know exactly how it "should"
> work, if it's even broken at all.

It's probably not broken at all. But when a code is parsed that doesn't
have any effect to the user, especially a terminal reply for number of
colors, default background, etc., it may loop at a low level and not go
back when the buffer becomes empty, and the redraw only happens at a
higher level. It does return K_IGNORE in some cases, perhaps it should
more often.

--
How To Keep A Healthy Level Of Insanity:

15. Five days in advance, tell your friends you can't attend their
party because you're not in the mood.