RFC - External debugger API for debugging vimscript

[Ben Jackson]

Hi Bram,

When we were discussing vim9script, I mentioned that an interface for external/graphical debuggers for vimscript would be useful. Your response at the time was:

> > >> * provide a interface to a debug adapter to allow debugging of vimscript 
> > >> executing in a Vim instance (i.e. Vim Debug Adapter) 
> > > 
> > > Sure. 
> 
> > Woo \o/. Debugging vimscript with vimspector would be very kickass. 
> > I’m more than willing to put in hard graft to make this happen. 

> It's not really related to Vim9 script, could be an independend project. 

So, challenge accepted. Over the last few months I've been working on this and have a mostly working prototype. I have even recently used it to debug a real problem in a real vim plugin.

Here's a demo: https://files.gitter.im/vimspector/Lobby/qnjv/vimspector-vimscript-demo.gif

What you can see in the demo:

- I open a .vim file, set a breakpoint and launch vim in a terminal window (using vimsector)

- Vim (in the terminal window) hits the breakpoint and vimspector jumps to the current line in my code

- Using vimspector I can step throught the vimscript, inspect local, script, window, etc. variables and view the _full_ execution stack (including source's etc.)

Very briefly, the way this works is the vim-under-debug delegates the debug command loop (in debug.c) to a vimscript function, implemented by my "vim-debug-adapter" plugin. This callback's job is to provide a single command to execute (such as "next" or "step" or an ex command, like 'break add'). It does this by having a channel, connected to a debug adapter in a "request one command" loop. Meanwhile the debug adapter can issue other requests, implemented in the channel's callback.

In order to make all of this work, a fairly large amount of change is required in Vim, such as:

- delegating command reading from the debugger loop (https://github.com/puremourning/vim/blob/debugger/src/debugger.c#L166-L216)

- completing the implementation of estack to hold data for sourced files, ufuncs, auto commands, etc. - this allows a new function (debug_getstack) to get the full stack trace shown in the demo. Normally stack traces only include the latest run of ufunc calls.

- adding some vimscript commands like `debug_getvariables( scope )`, `debug_eval( in_scope, expr )`, debug_getstack (https://github.com/puremourning/vim/blob/debugger/src/evalfunc.c#L58-L60)

- adding a way to evaluate a command in a script context (as well as a function context) that isn't the top of the stack (essentially use the _full_ execution stack for debug_backtrace_level)

- changes to breakpoints so that you can set a file-line breakpoint _within_ a function such that it fires when the function executes, rather than only when it is defined (this is quite hairy at the moment)

- probably a bunch of equivalent changes for vim9 (haven't tried this yet)

The very-work-in-progress vim changes are here : https://github.com/puremourning/vim/compare/master...puremourning:debugger

The runtime code that provides the interface to DAP is here: https://github.com/puremourning/vim-debug-adapter/blob/master/runtime/nub.vim (and the debug adapter itself: https://github.com/puremourning/vim-debug-adapter)

So the purpose of this RFC is to see whether I should progress further down this route:

- share the demo/prototype for visibility

- get your appetite for merging a patch like this (I would push the changes in smaller tested pieces of course)

- get your general thoughts on the approach above

- gauge community reaction, thoughts, comments, insults etc.

Thanks for everything. If the general reaction is positive, I'll make a proper plan and send some more detailed RFCs for the various aspects.

Kind regards,

Ben

[Bram Moolenaar]

Ben Jackson wrote:

> When we were discussing vim9script, I mentioned that an interface for
> external/graphical debuggers for vimscript would be useful. Your response
> at the time was:
>
> > > >> * provide a interface to a debug adapter to allow debugging of
> vimscript
> > > >> executing in a Vim instance (i.e. Vim Debug Adapter)
> > > >
> > > > Sure.
> >
> > > Woo \o/. Debugging vimscript with vimspector would be very kickass.
> > > I’m more than willing to put in hard graft to make this happen.
>
> > It's not really related to Vim9 script, could be an independend project.
>
> So, challenge accepted. Over the last few months I've been working on this
> and have a mostly working prototype. I have even recently used it to debug
> a real problem in a real vim plugin.
>
> Here's a
> demo: https://files.gitter.im/vimspector/Lobby/qnjv/vimspector-vimscript-demo.gif

Looks very interesting!

> What you can see in the demo:
>
> - I open a .vim file, set a breakpoint and launch vim in a terminal window
> (using vimsector)
> - Vim (in the terminal window) hits the breakpoint and vimspector jumps to
> the current line in my code
> - Using vimspector I can step throught the vimscript, inspect local,
> script, window, etc. variables and view the _full_ execution stack
> (including source's etc.)

Do I get it right that vimspector is a Vim plugin?

> Very briefly, the way this works is the vim-under-debug delegates the debug
> command loop (in debug.c) to a vimscript function, implemented by my
> "vim-debug-adapter" plugin. This callback's job is to provide a single
> command to execute (such as "next" or "step" or an ex command, like 'break
> add'). It does this by having a channel, connected to a debug adapter in a
> "request one command" loop. Meanwhile the debug adapter can issue other
> requests, implemented in the channel's callback.

That sounds like a nice mechanism. So it's mostly the same as the
existing debug code, but instead of prompting the user in the
Vim-under-debug itself the prompt goes over the channel.

I haven't dug into this, but I assume the output of a command also goes
over the channel, thus the Vim-under-debug doesn't show any debug
output.

> In order to make all of this work, a fairly large amount of change is
> required in Vim, such as:
>
> - delegating command reading from the debugger loop
> (https://github.com/puremourning/vim/blob/debugger/src/debugger.c#L166-L216)

This can be improved. Instead of hard coding the function name it could
be specified with a command:
:debugfunc MyDebugger
:debugfunc NONE

> - completing the implementation of estack to hold data for sourced files,
> ufuncs, auto commands, etc. - this allows a new function (debug_getstack)
> to get the full stack trace shown in the demo. Normally stack traces only
> include the latest run of ufunc calls.

This can most likely be a separate change. I was working towards this
with the "exestack" stuff. Which was far from complete. Should also be
used for error messages and exceptions.

> - adding some vimscript commands like `debug_getvariables( scope )`,
> `debug_eval( in_scope, expr )`, debug_getstack
> (https://github.com/puremourning/vim/blob/debugger/src/evalfunc.c#L58-L60)
> - adding a way to evaluate a command in a script context (as well as a
> function context) that isn't the top of the stack (essentially use the
> _full_ execution stack for debug_backtrace_level)

This sounds similar to something as the "frame" command in gdb, to
change the debugger scope.

> - changes to breakpoints so that you can set a file-line breakpoint
> _within_ a function such that it fires when the function executes, rather
> than only when it is defined (this is quite hairy at the moment)
> - probably a bunch of equivalent changes for vim9 (haven't tried this yet)
>
> The very-work-in-progress vim changes are here
> : https://github.com/puremourning/vim/compare/master...puremourning:debugger
> The runtime code that provides the interface to DAP is
> here: https://github.com/puremourning/vim-debug-adapter/blob/master/runtime/nub.vim
> (and the debug adapter
> itself: https://github.com/puremourning/vim-debug-adapter)
>
> So the purpose of this RFC is to see whether I should progress further down
> this route:
>
> - share the demo/prototype for visibility
> - get your appetite for merging a patch like this (I would push the changes
> in smaller tested pieces of course)
> - get your general thoughts on the approach above
> - gauge community reaction, thoughts, comments, insults etc.
>
> Thanks for everything. If the general reaction is positive, I'll make a
> proper plan and send some more detailed RFCs for the various aspects.

I won't have much time to dig into the inners of this, but generally it
sounds like a good way to go. The target audience will be plugin
writers, thus I would suggest to first get some feedback from them.
That probably requires making many things work, but it appears you
already have that.

How about writing a "Intro in Vim debugging" that plugin writers can
follow, a hands-on kind of training.

The implementation details can still change, I think it's important to
first check the boundaries of what can be done with the current
mechanism, what is needed for the debugging.

--
SECOND SOLDIER: It could be carried by an African swallow!
FIRST SOLDIER: Oh yes! An African swallow maybe ... but not a European
swallow. that's my point.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Ben Jackson]

Thanks, Bram.

Yep it’s a Debug Adapter Protocol _client_ in Vim - supports graphical debugging of any language that there’s an adapter for (i.e. anything VScode supports). DAP is like the debugging equivalent of LSP.

All possible thanks to your incredible work on vim 8.

>
>> Very briefly, the way this works is the vim-under-debug delegates the debug
>> command loop (in debug.c) to a vimscript function, implemented by my
>> "vim-debug-adapter" plugin. This callback's job is to provide a single
>> command to execute (such as "next" or "step" or an ex command, like 'break
>> add'). It does this by having a channel, connected to a debug adapter in a
>> "request one command" loop. Meanwhile the debug adapter can issue other
>> requests, implemented in the channel's callback.
>
> That sounds like a nice mechanism. So it's mostly the same as the
> existing debug code, but instead of prompting the user in the
> Vim-under-debug itself the prompt goes over the channel.

Yes, that’s exactly it.

>
> I haven't dug into this, but I assume the output of a command also goes
> over the channel, thus the Vim-under-debug doesn't show any debug
> output.

Sort of, the usual debug output is mostly suppressed in vim-under-debug (it’s still printing the ‘breakpoint hit at file:line’), but that’s mostly just work-in-progress stuff.

Regarding the result of evaluations, if the command run is, say, echom or redraw, it will be echo’d/redrawn in vim-under-debug, but the _return_ value of the evaluation is sent to the debugger over the channel.

>
>> In order to make all of this work, a fairly large amount of change is
>> required in Vim, such as:
>>
>> - delegating command reading from the debugger loop
>> (https://github.com/puremourning/vim/blob/debugger/src/debugger.c#L166-L216)
>
> This can be improved. Instead of hard coding the function name it could
> be specified with a command:
> :debugfunc MyDebugger
> :debugfunc NONE

Yes, absolutely, I was actually planning to make it an option (e.g. :set debughandler=FunctionName). This is just in the “still discovering what to solve next” phase. The option seemed easy to add so I left it for later.

>
>> - completing the implementation of estack to hold data for sourced files,
>> ufuncs, auto commands, etc. - this allows a new function (debug_getstack)
>> to get the full stack trace shown in the demo. Normally stack traces only
>> include the latest run of ufunc calls.
>
> This can most likely be a separate change. I was working towards this
> with the "exestack" stuff. Which was far from complete. Should also be
> used for error messages and exceptions.

Yes, that was the plan - pull this part of the change out (and maybe improve general backtraces too for day-to-day operations not involving the debugger) as the first step.

>
>> - adding some vimscript commands like `debug_getvariables( scope )`,
>> `debug_eval( in_scope, expr )`, debug_getstack
>> (https://github.com/puremourning/vim/blob/debugger/src/evalfunc.c#L58-L60)
>> - adding a way to evaluate a command in a script context (as well as a
>> function context) that isn't the top of the stack (essentially use the
>> _full_ execution stack for debug_backtrace_level)
>
> This sounds similar to something as the "frame" command in gdb, to
> change the debugger scope.

It is, and I initially actually added debug_setframe, but in practice it was safer/easier in my testing to add a function to evaluate an expression in a specific frame (reason was that if the expression threw an exception it was hard to un-set this frame without things getting messed up). If you prefer the more general approach I’ll spend more time on that.

>
>> - changes to breakpoints so that you can set a file-line breakpoint
>> _within_ a function such that it fires when the function executes, rather
>> than only when it is defined (this is quite hairy at the moment)
>> - probably a bunch of equivalent changes for vim9 (haven't tried this yet)
>>
>> The very-work-in-progress vim changes are here
>> : https://github.com/puremourning/vim/compare/master...puremourning:debugger
>> The runtime code that provides the interface to DAP is
>> here: https://github.com/puremourning/vim-debug-adapter/blob/master/runtime/nub.vim
>> (and the debug adapter
>> itself: https://github.com/puremourning/vim-debug-adapter)
>>
>> So the purpose of this RFC is to see whether I should progress further down
>> this route:
>>
>> - share the demo/prototype for visibility
>> - get your appetite for merging a patch like this (I would push the changes
>> in smaller tested pieces of course)
>> - get your general thoughts on the approach above
>> - gauge community reaction, thoughts, comments, insults etc.
>>
>> Thanks for everything. If the general reaction is positive, I'll make a
>> proper plan and send some more detailed RFCs for the various aspects.
>
> I won't have much time to dig into the inners of this, but generally it
> sounds like a good way to go.

THanks ;)

> The target audience will be plugin
> writers, thus I would suggest to first get some feedback from them.
> That probably requires making many things work, but it appears you
> already have that.

Sounds like a good idea. I did ask in #vim and a number of people were very interested, though overwhelmingly there was support for better backtraces anyway, so I’ll certainly complete that portion.

>
> How about writing a "Intro in Vim debugging" that plugin writers can
> follow, a hands-on kind of training.

Good plan!

[Bram Moolenaar]

Ben Jackson wrote:

[...]

> > I haven't dug into this, but I assume the output of a command also goes
> > over the channel, thus the Vim-under-debug doesn't show any debug
> > output.
>
> Sort of, the usual debug output is mostly suppressed in
> vim-under-debug (it’s still printing the ‘breakpoint hit at
> file:line’), but that’s mostly just work-in-progress stuff.
>
> Regarding the result of evaluations, if the command run is, say, echom
> or redraw, it will be echo’d/redrawn in vim-under-debug, but the
> _return_ value of the evaluation is sent to the debugger over the
> channel.

Sounds like the message redirection should allow for redirecting to a
function, which would then send it over a channel. Currently it's
possible to redirect to a variable, it should be possible to redirect to
a callback function.

> >> In order to make all of this work, a fairly large amount of change is
> >> required in Vim, such as:
> >>
> >> - delegating command reading from the debugger loop
> >> (https://github.com/puremourning/vim/blob/debugger/src/debugger.c#L166-L216)
> >
> > This can be improved. Instead of hard coding the function name it could
> > be specified with a command:
> > :debugfunc MyDebugger
> > :debugfunc NONE
>
> Yes, absolutely, I was actually planning to make it an option (e.g.
> :set debughandler=FunctionName). This is just in the “still
> discovering what to solve next” phase. The option seemed easy to add
> so I left it for later.

Both would work. I was thinking of the symmetry with ":redir", it's
kind of redirecting the input.

> >> - adding some vimscript commands like `debug_getvariables( scope )`,
> >> `debug_eval( in_scope, expr )`, debug_getstack
> >> (https://github.com/puremourning/vim/blob/debugger/src/evalfunc.c#L58-L60)
> >> - adding a way to evaluate a command in a script context (as well as a
> >> function context) that isn't the top of the stack (essentially use the
> >> _full_ execution stack for debug_backtrace_level)
> >
> > This sounds similar to something as the "frame" command in gdb, to
> > change the debugger scope.
>
> It is, and I initially actually added debug_setframe, but in practice
> it was safer/easier in my testing to add a function to evaluate an
> expression in a specific frame (reason was that if the expression
> threw an exception it was hard to un-set this frame without things
> getting messed up). If you prefer the more general approach I’ll spend
> more time on that.

I suppose setting the frame is something that exists in the remote
debugger. The API can include the frame in the evaluation command,
without changing the frame for following commands.


--
./configure
Checking whether build environment is sane ...
build environment is grinning and holding a spatula. Guess not.

[puremo...@gmail.com]

Waking up this thread, just to say that this is still work in progress. I actually use it quite often for my own debugging, but I'm holding off on patches for now while vim9script develops. I think it makes logical sense for this all to happen after vim9script is mature.

But as a teaser, I noticed that Bram added support for debugging vim9script statements yesterday, and so I've hooked that in to my prototype. So here's a demo of Vimspector debugging a simple vim9 script:

https://asciinema.org/a/25SLL99WepcubhweLoQblz2Cg

Lots more to do on this, but now the framework is in place.

[Bram Moolenaar]

> Waking up this thread, just to say that this is still work in progress. I
> actually use it quite often for my own debugging, but I'm holding off on
> patches for now while vim9script develops. I think it makes logical sense
> for this all to happen after vim9script is mature.
>
> But as a teaser, I noticed that Bram added support for debugging vim9script
> statements yesterday, and so I've hooked that in to my prototype. So here's
> a demo of Vimspector debugging a simple vim9 script:
>
> https://asciinema.org/a/25SLL99WepcubhweLoQblz2Cg
>
> Lots more to do on this, but now the framework is in place.

I'm glad the Vim9 debugger features work for you. I plan to add the
possibility to inspect function arguments next. Not sure if I add more
now, it should be sufficient for basic debugging. Perhaps we need
to be able setting a breakpoint?

--
You can test a person's importance in the organization by asking how much RAM
his computer has. Anybody who knows the answer to that question is not a
decision-maker.
(Scott Adams - The Dilbert principle)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// \\\

\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///

[Ben Jackson]

Yeah, thanks!

If eval_variable (via lookup_debug_var) or similar would return function args for def functions, then that would be pretty neat and we can watch and inspect them.

For my side I would need to add a list of locals and script vars for vim9 script contexts (for the scopes window) and fix up the stack trace (which contains the <SNR> literally at the moment).

For breakpoints, I haven’t looked at that yet, but it think that would be the next thing. For legacy vim script I modified the way breakpoints are triggered such that a line breakpoint within the body of a function triggers both when defining the function _and_ when executing it the function (this allows setting line breakpoints within function bodies); for this I use dbg_find_breakpoint_in_func (here: https://github.com/puremourning/vim/blob/debugger/src/debugger.c#L954).

I’d probably look to do the same for def functions. Something like calling dbg_find_breakpoint_in_func from the ISN_DEBUG handler, though I only looked at it really really briefly.

The remaining thing would be whether or not we can support arbitrary execution of def functions while debugging, e.g. to print the return of some def function call.