Looking for suggestions for how to debug 'flaky' Vim test

[mss...@gmail.com]

I've built a version of Vim, v8.2.3321, with huge features and no gui on a debian-based system and see intermittent failures in the regression test Test_conceal_two_windows.  The specific test that fails is when option concealcursor is set to "v" and visual mode is entered (test_conceal.vim, line 88).  The only reason it fails is because the cursor position is incorrect.  In particular, the cursor position is where it would be if the string |hidden| were not concealed.  By the way, this places the cursor beyond the end-of-line. 

Furthermore, this error only occurs in an xterm, i.e., when the X server is running; it does not occur in a console window without X. 

To investigate this problem I decided to use gdb to attach to the instance of Vim that the test creates in a terminal window.  However, the "attach" fails with the diagnostic "ptrace: Operation not permitted".  If I understand this correctly, it means that I, as an unprivileged user, or gdb lack the CAP_SYS_PTRACE capability.

So I'm looking for answers/advice/guidance as follows:

1)  If using gdb makes sense, how do I acquire that capability?

2)  To the developers of these tests, how would you investigate the failure?

Thanks in advance,

  -mike

[Bram Moolenaar]

There is no single way, every test can be flaky in different ways.

Using gdb generally only helps when a test fails. With flaky tests
often timing matters and putting a breakpoint somewhere completely
changes that.

It helps a lot if you can make the test nearly always fail. For example
by using valgrind, which makes everything slower.

Another way is to add logging calls. In script start with ch_logfile()
at the top of the test function that fails and then put ch_log() calls
in various places. The log includes a timestamp which can indicate slow
operations. You can also add calls in the code. This requires
recompiling, but it's required to pinpoint the part in the command that
matters. Call ch_log(NULL, "text").

It can also help to add some ":sleep 10m" commands or ":redraw" in some
places. This requires guessing what is happening.

--
Did Adam and Eve have navels?

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Dominique Pellé]

mss...@gmail.com <mss...@gmail.com> wrote:

> I've built a version of Vim, v8.2.3321, with huge features and
> no gui on a debian-based system and see intermittent failures
> in the regression test Test_conceal_two_windows.

Probably the most useful things to do are:

- post here the logs when the test fails
- post the output of "vim --version"
- try with the latest version of Vim from git
Your version is from Aug 9, 2021, which is quite
recent though.
- try with valgrind (uncomment the line
"# VALGRIND = ..." in vim/src/testdir/Makefile)
- try with a asan (address sanitizer) build i.e.
uncomment the line SANITIZER_CFLAGS = ...
in vim/src/Makefile, clean and rebuild.

Regards
Dominique

[mss...@gmail.com]

Bram,

Thanks for taking the time to respond.  It is helpful to have the possibilities elaborated.  

I don't think that this is a Vim problem, mostly because it isn't flaky without X.  I can use your suggestions to see if it is timing related.  If it is not, my expectation is that the source of the  problem is somewhere in the bowels of X.  That's what I hope gdb will reveal but I also understand it will be a mighty tough row to hoe!  Regardless, I'm should  learn something new and perhaps it will be useful. 

-mike

[mss...@gmail.com]

Dominique,

Thanks for your suggestions.  I'm running a light version of linux so I'll have to install valgrind but that's not hard.  I'll rebuild Vim following your two suggestions and see what turns up. 

By the way, I didn't provide the log files because the information was limited to asserting a byte-difference in the dump files.  I had to use "od" to know it was a cursor position issue. 

Much obliged!