Channel based plugins and scheduling

[Paul Jolly]

Hi all,

Within the channel-based plugin govim
(https://github.com/govim/govim), the bulk of functionality is
triggered by user actions in Vim.

However, certain events are triggered entirely independently of Vim,
for example a list of diagnostics arriving from an LSP server, the LSP
server asking govim to show an error message. There are others.

When handling these non-Vim triggered events, any call from govim to
Vim is effectively racing (i.e. in a race condition) with whatever Vim
might be doing at the time. For example, we might happen to call from
govim -> Vim while Vim is in the middle of handling a listener_add
callback that itself calls ch_evalexpr which results in a call Vim ->
govim.

One thing we're not clear on is whether we should be "scheduling"
certain things in Vim. The channel docs
(https://github.com/vim/vim/blob/master/runtime/doc/channel.txt)
highlight that we should be checking what mode the user is in, but I
don't think this handles the case described.

What do I mean by scheduling? As I understand it, if when handling a
non-Vim triggered event we instead made a call into Vim that did
something like:

timer_start(0, { -> dostuff() })

then dostuff() will be "scheduled" by Vim "at a convenient time".

Would this approach work?

We're seeing some tests randomly failing and this "scheduling" problem
is our current hypothesis. But our understanding here is vague, so any
suggestions/guidance appreciated.

Thanks,


Paul

[Bram Moolenaar]

Vim will try to handle messages as soon as possible, but in a "safe"
situation. That is usually when waiting for a user to type. But it
might still be halfway a command, such as when editing the command line,
in Insert mode or after an operator such as "d".

Some things should be safe, such as using popup_notification() to show a
message. But something like ":normal" or "feedkeys()" will interfere
with what the user is doing. It's up to the plugin to detect this and
schedule doing the work later.

Using a timer won't help much, since you do not know when the state has
changed. You can use autocommands, e.g.:
if mode() =~ [iR]
au InsertLeave * call DoTheWork()
endif

There is no generic autocommand event for changing mode, perhaps that is
want you are looking for? Or a "NormalMode" event for when nothing is
pending? That may take a while though.

--
hundred-and-one symptoms of being an internet addict:
232. You start conversations with, "Have you gotten an ISDN line?"

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Paul Jolly]

> Using a timer won't help much, since you do not know when the state has
> changed. You can use autocommands, e.g.:
> if mode() =~ [iR]
> au InsertLeave * call DoTheWork()
> endif
>
> There is no generic autocommand event for changing mode, perhaps that is
> want you are looking for? Or a "NormalMode" event for when nothing is
> pending? That may take a while though.

Thanks, Bram. This gives us food for thought; we'll continue
investigating the test failures to see if we can work out where things
are going wrong.

[Andy Massimino]

> For example, we might happen to call from govim -> Vim while Vim is in the middle of handling a listener_add callback that itself calls ch_evalexpr which results in a call Vim ->govim.

Can this actually happen?  Is this considered a "safe" situation then?

I assume adding a zero tick timer wouldn't help there because vim could just schedule the timer callback within the listener callback.  

--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/CACoUkn5RPm5GrJA1aXDiz32T4dvm3wJ-hSZ24VrSEe0yNwaVog%40mail.gmail.com.

[Paul Jolly]

> > For example, we might happen to call from govim -> Vim while Vim is in the middle of handling a listener_add callback that itself calls ch_evalexpr which results in a call Vim ->govim.
>
> Can this actually happen? Is this considered a "safe" situation then?

Yes I think it can. See below for an example.

> I assume adding a zero tick timer wouldn't help there because vim could just schedule the timer callback within the listener callback.

I think you're right. The timer-based solution was something of a stab
in the dark. However, trying it and this discussion has, I think,
brought me to the "right" answer.

Consider the following example (requires Go to be installed):
https://gist.github.com/myitcv/8a55f106a604584164d50e82a31d28ef

vi -u slow_function.vim

Then:

:call MyFunc()

The sequence here (which you can see from Vim screen,
/tmp/vim_channel.log and /tmp/slow_job.log) is as follows:

* the channel is established between Vim and slow_job.go
* we call MyFunc()
* MyFunc calls ch_evalexpr which is blocking; MyFunc does not, at this
point, return therefore
* slow_job.go receives the message from Vim's ch_evalexpr call:

{'Comment':'hello from Vim', 'Delay': '10s', 'Calls':[['ex', 'echom
"Hello from govim"']]}

* slow_job.go does some logging, then it makes the calls requested in
the message
* slow_job.go calls back into Vim with the channel command: ['ex',
'echom "Hello from govim"']
* Vim shows this message, but note MyFunc is still blocked on
ch_evalexpr (and hence so is the user)
* slow_job.go then sleeps for the requested 10s
* slow_job.go then returns, which allows ch_evalexpr to return, which
allows MyFunc() to return, at which point the user can continue

The above example is slightly contrived, because the ex call from
slow_job.go into Vim happens whilst Vim is blocked executing the
remote MyJob() function.

Things can go wrong when the call from govim (i.e. in place of
slow_job.go) to Vim happens at any random time (and they do happen at
random times). Because sometimes it's the case that Vim is in the
middle of something where we then get a "cannot do this here" error in
response to the call from govim -> Vim. I think the canonical example
here is when Vim is in the middle of a listener_add callback, but a) I
might be wrong on this and b) there could be other cases. We see the
classic race condition of "govim starts to call Vim just as Vim starts
to call govim" regularly.

The answer here is I think to keep track on the Vim side (in the govim
VimScript shim) of the number of active ch_evalexpr calls. Instead of
scheduling with a timer_start as I previously suggested, we could do
the following:

* if a "sensitive" call from govim -> Vim comes in and there are no
active ch_evalexpr calls from Vim -> govim, then handle immediately
* if, however, there are active ch_evalexpr calls, add the request to
the end of a pending queue
* when the number of active ch_evalexpr calls reaches zero, process
the pending queue in order

We can keep track of all of this from within the govim VimScript shim.

I think we also have a handle on the calls that are "sensitive".

Any thoughts on the above analysis/suggestion?

Thanks

[Bram Moolenaar]

I think that is only true if the callback does something involving a
blocking wait. Such as that ch_evalexpr(). So the problem is not
invoking the listener callback itself, but what happens inside the
callback. There are many callbacks these days, thus this can happen in
many places and there is not much point for Vim to do something for
that. We can document where a blocking wait happens, which is where
other callbacks can be invoked, possibly leading to recursive calls.

> might be wrong on this and b) there could be other cases. We see the
> classic race condition of "govim starts to call Vim just as Vim starts
> to call govim" regularly.
>
> The answer here is I think to keep track on the Vim side (in the govim
> VimScript shim) of the number of active ch_evalexpr calls. Instead of
> scheduling with a timer_start as I previously suggested, we could do
> the following:
>
> * if a "sensitive" call from govim -> Vim comes in and there are no
> active ch_evalexpr calls from Vim -> govim, then handle immediately
> * if, however, there are active ch_evalexpr calls, add the request to
> the end of a pending queue
> * when the number of active ch_evalexpr calls reaches zero, process
> the pending queue in order
>
> We can keep track of all of this from within the govim VimScript shim.
>
> I think we also have a handle on the calls that are "sensitive".
>
> Any thoughts on the above analysis/suggestion?

I think you are correct that you need to look at places with a blocking
wait where a callback would cause trouble. Then adding the event to a
queue, instead of doing the work immediately, and check that queue for
work after the blocking wait would indeed be the best solution.

If this gets too complicated, my suggestion to add an autocommand event
when Vim is in a "safe" state would help. E.g. when waiting for the
start of a command in Normal mode (without a register, count, pending
operator, etc.). Perhaps even when not halfway a mapping, thus actually
waiting for they user to type.

This would be too long when in Insert mode or editing the command line,
thus perhaps we need events for these as well.

--
You can tune a file system, but you can't tuna fish
-- man tunefs

[Paul Jolly]

Thanks, Bram.

> > Things can go wrong <snip>

>
> I think that is only true if the callback does something involving a
> blocking wait. Such as that ch_evalexpr(). So the problem is not
> invoking the listener callback itself, but what happens inside the
> callback. There are many callbacks these days, thus this can happen in
> many places and there is not much point for Vim to do something for
> that. We can document where a blocking wait happens, which is where
> other callbacks can be invoked, possibly leading to recursive calls.

Absolutely. I was just using this as an example. We do indeed see this
in many different situations.

> > Any thoughts on the above analysis/suggestion?
>
> I think you are correct that you need to look at places with a blocking
> wait where a callback would cause trouble. Then adding the event to a
> queue, instead of doing the work immediately, and check that queue for
> work after the blocking wait would indeed be the best solution.
>
> If this gets too complicated, my suggestion to add an autocommand event
> when Vim is in a "safe" state would help. E.g. when waiting for the
> start of a command in Normal mode (without a register, count, pending
> operator, etc.). Perhaps even when not halfway a mapping, thus actually
> waiting for they user to type.

https://github.com/govim/govim/pull/523 has a WIP version of the
approach I described and it appears to be working well (it's currently
pending a few fixes from gopls which is why the build is broken). We
will need to give it some more exposure to CI, but for now things are
working well on my local machine.

One question about the "safe" state autocommand you're proposing:
would this only be triggered when there are no other autocommands in
progress? i.e. the "safe" autocommand itself will be the only
autocommand at the time when called? If so, that would work. Would
there also be an autocommand for when you are leaving this "safe"
state?

> This would be too long when in Insert mode or editing the command line,
> thus perhaps we need events for these as well.

I think we would also need a way to ask "are there any current
autocommands?". The logic would become:

1. if a "sensitive" call from govim -> Vim comes in and there are no
active ch_evalexpr calls from Vim -> govim, no current autocommands
and we are in the "safe" state, then handle immediately
2. if, however, there are either active ch_evalexpr calls, or current
autocommands or we are not in the "safe" state, add the request to the

end of a pending queue

3. when the number of active ch_evalexpr calls reaches zero, there are
no current autocommands and we again enter a "safe" state, process the
pending queue in order

That way, the edges that move us from >0 to 0 active ch_evalexpr
calls, and the "safe" autocommand, would be the triggers for point 3
and processing the pending queue.

For now we try the original scheme I described and report back with
any findings. But very happy to continue discussions on better
approaches people might have.

Thanks,


Paul

[Bram Moolenaar]

> One question about the "safe" state autocommand you're proposing:
> would this only be triggered when there are no other autocommands in
> progress? i.e. the "safe" autocommand itself will be the only
> autocommand at the time when called? If so, that would work. Would
> there also be an autocommand for when you are leaving this "safe"
> state?

It would only be triggered when Vim is not halfway a mapping, ":normal",
feedkeys(), autocommand, etc.

There will be no "leave", because the user then already typed something
and I don't see what you can do. Perhaps what you want is a way to
obtain the status, thus after triggering the event Vim would keep this
"safe" state until input is found (other then processing messages).
Hmm, when invoking a callback you would be temporarily outside of this
"safe" state, since you are executing a sequence of commands, but that's
where you would check... That's a catch 22.

> > This would be too long when in Insert mode or editing the command line,
> > thus perhaps we need events for these as well.
>
> I think we would also need a way to ask "are there any current
> autocommands?". The logic would become:

Why autocommands? We also have callbacks that might be busy.

> 1. if a "sensitive" call from govim -> Vim comes in and there are no
> active ch_evalexpr calls from Vim -> govim, no current autocommands
> and we are in the "safe" state, then handle immediately
> 2. if, however, there are either active ch_evalexpr calls, or current
> autocommands or we are not in the "safe" state, add the request to the
> end of a pending queue
> 3. when the number of active ch_evalexpr calls reaches zero, there are
> no current autocommands and we again enter a "safe" state, process the
> pending queue in order
>
> That way, the edges that move us from >0 to 0 active ch_evalexpr
> calls, and the "safe" autocommand, would be the triggers for point 3
> and processing the pending queue.
>
> For now we try the original scheme I described and report back with
> any findings. But very happy to continue discussions on better
> approaches people might have.

What if another plugin is doing a ch_evalexpr()? I suppose that doesn't
interfere with what your plugin is doing.

--
Eagles may soar, but weasels don't get sucked into jet engines.

[Paul Jolly]

> It would only be triggered when Vim is not halfway a mapping, ":normal",
> feedkeys(), autocommand, etc.

Perfect.

> There will be no "leave", because the user then already typed something
> and I don't see what you can do. Perhaps what you want is a way to
> obtain the status, thus after triggering the event Vim would keep this
> "safe" state until input is found (other then processing messages).
> Hmm, when invoking a callback you would be temporarily outside of this
> "safe" state, since you are executing a sequence of commands, but that's
> where you would check... That's a catch 22.

Yeh this is where I think it gets tricky. But I think we could treat
the "obtaining of the status" as an optimisation that we don't
necessarily need. Reason being, if the handling of a message from
govim is itself treated as "unsafe", then we know we will get a "safe"
autocommand callback soon after. So the command/whatever we were asked
to "schedule" by govim will simply happen a tiny bit later than if
were were able to query the "safe" state. Would that work?

> > I think we would also need a way to ask "are there any current
> > autocommands?". The logic would become:
>
> Why autocommands? We also have callbacks that might be busy.

You're quite right; for some reason I unnecessarily constrained my
thinking to autocommands.

> What if another plugin is doing a ch_evalexpr()? I suppose that doesn't
> interfere with what your plugin is doing.

It could/would interfere... and it's a problem with the approach I've
described, that's for sure because all I'm doing is:

let s:activeGovimCalls += 1
let l:resp = ch_evalexpr(s:channel, a:args)
let s:activeGovimCalls -= 1

Clearly I can't "control" ch_evalexpr calls in any other plugins.

So generally speaking, I think we need some help from Vim itself here:
either to keep track of ch_evalexpr calls (and anything else?) or by
fleshing out the "safe" autocommand above. Or maybe both?

Thanks

[Bram Moolenaar]

So roughly:

Add a safeState() function, which returns true when:
- not halfway a mapping or stuffed command
- no operator pending
- not executing autocommand
- no autocomplete active perhaps?
- not blocked on waiting, e.g. ch_evalexpr()
- not in a nested callback (if that is even possible)

Trigger autocommand when entering "safe" state:
- NormalSafe
- InsertSafe
- CmdlineSafe

So that a plugin can:
- When callback is invoked and safeState() returns false, add to work queue
- When *Safe autocommand event triggers, process work queue

Perhaps safeState() should have an argument about what is considered
safe. E.g. Command-line mode might not be safe. Although mode() could
be used for that. In fact, mode() contains more detail about the state,
what we don't have from there is whether a mapping is active or
something else that doesn't require user interaction.

Or the return value could be a list of what "unsafe" things are
currently happening. Although that might make it difficult to decide
when to trigger the autocommand.

An alternative is a state() function that returns flags for what is
active:
m inside mapping
x executing command (something in stuff buffer)
a autocommand busy
w waiting in channel function, such as ch_evalexpr()
c in callback, repeated when nesting

Then trigger a StateChange autocommand when this value changes? Might
be triggered too often. Well, could use the pattern to define what to
wait for. More like StateLeave then, e.g.:
au StateLeave w call WaitEnded()

--
If you're sending someone Styrofoam, what do you pack it in?

[Paul Jolly]

So roughly:

Thanks - I'll give this some thought later today and reply with any questions/thoughts.

[Paul Jolly]

Hi Bram,

Thanks for taking the time to put together the proposal.

> Add a safeState() function, which returns true when:
> - not halfway a mapping or stuffed command
> - no operator pending
> - not executing autocommand
> - no autocomplete active perhaps?
> - not blocked on waiting, e.g. ch_evalexpr()
> - not in a nested callback (if that is even possible)

Just to confirm: this function would return true in a channel callback
handler (assuming there is nothing else going on). Because in a
channel callback handler, as you mention below, is the place that we'd
want to call this function and want to benefit from being able to
immediately callback to govim if it was safe to do so.

> Trigger autocommand when entering "safe" state:
> - NormalSafe
> - InsertSafe
> - CmdlineSafe

To keep things simple, do we need the three separate autocommands?
Would one "safe" not be enough?

Or are you doing this so that the plugin doesn't additionally need to
check "which mode are we in?"

> So that a plugin can:
> - When callback is invoked and safeState() returns false, add to work queue
> - When *Safe autocommand event triggers, process work queue

Modulo the question above about the separate autocommands, sounds good.

> Perhaps safeState() should have an argument about what is considered
> safe. E.g. Command-line mode might not be safe. Although mode() could
> be used for that. In fact, mode() contains more detail about the state,
> what we don't have from there is whether a mapping is active or
> something else that doesn't require user interaction.
>
> Or the return value could be a list of what "unsafe" things are
> currently happening. Although that might make it difficult to decide
> when to trigger the autocommand.

I'm all for keeping this simple for now :) We won't be using the
distinction between the three separate autocommands and for now, we're
simply looking for a "safe/not safe". But it's highly likely we're not
thinking as far ahead as you :)

> An alternative is a state() function that returns flags for what is
> active:
> m inside mapping
> x executing command (something in stuff buffer)
> a autocommand busy
> w waiting in channel function, such as ch_evalexpr()
> c in callback, repeated when nesting
>
> Then trigger a StateChange autocommand when this value changes? Might
> be triggered too often. Well, could use the pattern to define what to
> wait for. More like StateLeave then, e.g.:
> au StateLeave w call WaitEnded()

Per above, I think all we (govim) are currently after is to know
whether, when we call sateState() or whatever, there are currently any
blocked ch_evalexpr calls, because our current approach only works for
the ch_evalexpr calls we (govim) have made. We'd be making this call
from within the channel callback handler. So the only possibility (at
least as far as I understand it) is that things like autocommands,
functions, commands etc could only, at this point, be blocked if they
are using ch_evalexpr. Or is that not the case?

It's quite possible there are other blocking calls I'm not aware of,
so please only consider my comments here as seeking to clarify what we
are trying to solve for: you have a far better and wider grasp of the
problems involved and I understand you're trying to solve for other
situations too!

Thanks,


Paul

[Bram Moolenaar]

Paul Jolly wrote:

> Thanks for taking the time to put together the proposal.
>
> > Add a safeState() function, which returns true when:
> > - not halfway a mapping or stuffed command
> > - no operator pending
> > - not executing autocommand
> > - no autocomplete active perhaps?
> > - not blocked on waiting, e.g. ch_evalexpr()
> > - not in a nested callback (if that is even possible)
>
> Just to confirm: this function would return true in a channel callback
> handler (assuming there is nothing else going on). Because in a
> channel callback handler, as you mention below, is the place that we'd
> want to call this function and want to benefit from being able to
> immediately callback to govim if it was safe to do so.

I assume there may be situations where a function wants to know where it
was called from. It could be interactive, the user typed something, or
from a callback. Therefore just true/false is not going to be
sufficient. Thus I tend to go for a state() function, so you can filter
out things you don't care about, e.g.:

if state()->substitute('c', '', 'g') == ''
" nothing relevant busy

> > Trigger autocommand when entering "safe" state:
> > - NormalSafe
> > - InsertSafe
> > - CmdlineSafe
>
> To keep things simple, do we need the three separate autocommands?
> Would one "safe" not be enough?
>
> Or are you doing this so that the plugin doesn't additionally need to
> check "which mode are we in?"

Well, some things might not work well when typing text. But it's true
that we could trigger SafeState and let the autocommand use mode() to
find out where it is.

I found that ch_read() and ch_readraw() can get to the same point of
handling nested messages, if they read JSON.

Inside :sleep messages are also handled, thus if a callback uses :sleep
to wait a bit, it might cause other callbacks to be invoked.

The listener callback can also do something that waits, but I suppose it
is not specific for the listener callback, a timer callback is similar.
Well, the listener callback is invoked when changing text, thus if
somehow a nested callback tries to change text we have a problem.
This more a detection of a state that Vim is busy with a command, then
changing text is not OK.

Another thing that plugins might want to check for: When info is
avaialble and a popup is to be displayed, this probably should not be
done if the screen is shifted up to show messages.

> It's quite possible there are other blocking calls I'm not aware of,
> so please only consider my comments here as seeking to clarify what we
> are trying to solve for: you have a far better and wider grasp of the
> problems involved and I understand you're trying to solve for other
> situations too!

I'm trying to avoid adding a safeState() function that we need to keep
tuning, since what is "safe" is hard to define. Safe for what? Thus
letting that up to the plugin writer is probably best, just make it
possible to check the state, more accurately than what mode() does.

Hmm, I wonder if Visual selection is safe for you? Should SafeState
trigger when in Visual mode? we could use the pattern of SafeState to
select when to trigger, or just trigger more often and let the
autocommand use mode() to skip in same states.

--
hundred-and-one symptoms of being an internet addict:

266. You hear most of your jokes via e-mail instead of in person.

[Bram Moolenaar]

I was wondering where to trigger SafeState. I first thought it would be
sufficient to do this higher up, where we are going to wait for the user
to type a character and nothing is pending. Then it's safe to deal with
any callbacks.

However, think of this scenario:
1. In a safe state, SafeState is triggered.
2. Waiting for a character, receive a message.
3. Invoke callback for the message, it uses ch_evalexpr().
4. While in ch_evalexpr() another message arrives. Since it's
not safe now, work is postponed.
5. ch_evalexpr() finishes
6. go back to waiting for a character

Now, so long as the user doesn't type anything, the work won't be
executed. That may take long time. Or the user is actually waiting for
the work to be done, which is a kind of deadlock.

Thus it seems that we need to trigger SafeState also before 6. Under
the condition we started waiting for a character in a safe state.
And it was not safe at some moment (to avoid it being triggered too
often).

It does mean that the display won't be updated, thus we would need to do
that, like when a timer callback was invoked.

Does this sound OK?

--
This sentence is not sure that it exists, but if it does, it will
certainly consider the possibility that other sentences exist.

[Bram Moolenaar]

I wrote:

> I was wondering where to trigger SafeState. I first thought it would be
> sufficient to do this higher up, where we are going to wait for the user
> to type a character and nothing is pending. Then it's safe to deal with
> any callbacks.
>
> However, think of this scenario:
> 1. In a safe state, SafeState is triggered.
> 2. Waiting for a character, receive a message.
> 3. Invoke callback for the message, it uses ch_evalexpr().
> 4. While in ch_evalexpr() another message arrives. Since it's
> not safe now, work is postponed.
> 5. ch_evalexpr() finishes
> 6. go back to waiting for a character
>
> Now, so long as the user doesn't type anything, the work won't be
> executed. That may take long time. Or the user is actually waiting for
> the work to be done, which is a kind of deadlock.
>
> Thus it seems that we need to trigger SafeState also before 6. Under
> the condition we started waiting for a character in a safe state.
> And it was not safe at some moment (to avoid it being triggered too
> often).
>
> It does mean that the display won't be updated, thus we would need to do
> that, like when a timer callback was invoked.
>
> Does this sound OK?

I decided to give it a go and did an implementation of SafeState.
It triggers in Normal, Insert and Command-line mode when nothing is
pending. And when a blocking channel call has been made, where the
state was safe before blocking, so that when something is postponed it
can be handled afterwards.

It may be that we need to check more conditions whether it is actually
"safe". And some things depend on what the requirements for "safe" are.
E.g. currently it is not checking for Visual mode, you can use mode()
for that. And it doesn't check whether the screen is scrolled (not sure
how you would check for that, might need the state() function).

When SafeState is triggered after the blocking channel call, we are in
the input loop, thus no screen update will take place. You can use
:redraw, but in Command-line mode that might not work well.

Anyway, please try it out, so we can decide whether we can keep
SafeState, modify it, or perhaps drop it.

--
We're knights of the round table
We dance whene'er we're able
We do routines and chorus scenes
With footwork impeccable.
We dine well here in Camelot
We eat ham and jam and spam a lot.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

[Paul Jolly]

> > Does this sound OK?
>
> I decided to give it a go and did an implementation of SafeState.
> It triggers in Normal, Insert and Command-line mode when nothing is
> pending. And when a blocking channel call has been made, where the
> state was safe before blocking, so that when something is postponed it
> can be handled afterwards.
>
> It may be that we need to check more conditions whether it is actually
> "safe". And some things depend on what the requirements for "safe" are.
> E.g. currently it is not checking for Visual mode, you can use mode()
> for that. And it doesn't check whether the screen is scrolled (not sure
> how you would check for that, might need the state() function).
>
> When SafeState is triggered after the blocking channel call, we are in
> the input loop, thus no screen update will take place. You can use
> :redraw, but in Command-line mode that might not work well.
>
> Anyway, please try it out, so we can decide whether we can keep
> SafeState, modify it, or perhaps drop it.

Thanks, Bram. Sorry, not been able to keep up with the threads and progress!

Very keen to give this a try, thanks.

Two questions.

Firstly, is there still a plan for the safeState/state/whatever
function? Because without it, if a message arrives at a "safe" time we
need something unsafe to happen before it will be run?

Secondly, in the following situation, when does SateState get triggered?

1. user makes change in insert mode
2. listener_add callback fires and make ch_evalexpr call
3. Vim receives and handles a channel-based message; let's just assume
we put this work onto a pending queue (in the absence of safeState())
4. ch_evalexpr returns
5. the listener_add callback returns

I'm assuming the answer is "after step 5" - but is it immediately
after the listener_add callback returns?

Thanks,


Paul

[Bram Moolenaar]

Paul Jolly wrote:

> > > Does this sound OK?
> >
> > I decided to give it a go and did an implementation of SafeState.
> > It triggers in Normal, Insert and Command-line mode when nothing is
> > pending. And when a blocking channel call has been made, where the
> > state was safe before blocking, so that when something is postponed it
> > can be handled afterwards.
> >
> > It may be that we need to check more conditions whether it is actually
> > "safe". And some things depend on what the requirements for "safe" are.
> > E.g. currently it is not checking for Visual mode, you can use mode()
> > for that. And it doesn't check whether the screen is scrolled (not sure
> > how you would check for that, might need the state() function).
> >
> > When SafeState is triggered after the blocking channel call, we are in
> > the input loop, thus no screen update will take place. You can use
> > :redraw, but in Command-line mode that might not work well.
> >
> > Anyway, please try it out, so we can decide whether we can keep
> > SafeState, modify it, or perhaps drop it.
>
> Thanks, Bram. Sorry, not been able to keep up with the threads and progress!
>
> Very keen to give this a try, thanks.
>
> Two questions.
>
> Firstly, is there still a plan for the safeState/state/whatever
> function? Because without it, if a message arrives at a "safe" time we
> need something unsafe to happen before it will be run?

Yes, adding the state() function would be needed. It's a bit of work,
not sure when I get to it.

The SafeState event is always triggered when going through the main
loop and nothing is pending, thus every time after the user finishes
some action. That can be after every typed letter.

> Secondly, in the following situation, when does SateState get triggered?
>
> 1. user makes change in insert mode
> 2. listener_add callback fires and make ch_evalexpr call
> 3. Vim receives and handles a channel-based message; let's just assume
> we put this work onto a pending queue (in the absence of safeState())
> 4. ch_evalexpr returns
> 5. the listener_add callback returns
>
> I'm assuming the answer is "after step 5" - but is it immediately
> after the listener_add callback returns?

It would be at 4. when ch_evalexpr() is done and leaves the blocking
loop. Then listener_add() is still active then, thus it might be too
early to run the work.

Thinking about this, we should probably trigger SafeState later. So we
would set a flag in ch_evalexpr() that we did block, and check for that
flag when back in the main loop, after the callback finished.

Hmm, perhaps we don't need that flag, we could always trigger SafeState,
and you need to make sure you added the autocommand only when you
actually have some work pending. That's tricky, if you are not careful
the event is triggered very often (looping may only wait for 20 msec
before trying again, e.g. checking for a job does polling).

Another way would be that the plugin explicitly asks for triggering the
event. Thus if you add something to the work queue you call some
function and the event is triggered once as soon as the safe state is
reached in the waiting loop. For the higher level
(Normal/Insert/Commandline mode) it would still always be triggered.

I guess that just relying on adding the autocommand when it's needed is
the simplest, it does not require a new mechanism. It's up to the
plugin then to clean up the autocommand when it's not needed.

Also, perhaps we should have a different event for when it is triggered
from the waiting loop, instead of from the toplevel, since Vim is in a
different state.

--
We're knights of the Round Table
Our shows are formidable
But many times
We're given rhymes
That are quite unsingable
We're opera mad in Camelot
We sing from the diaphragm a lot.

[Paul Jolly]

> > Firstly, is there still a plan for the safeState/state/whatever
> > function? Because without it, if a message arrives at a "safe" time we
> > need something unsafe to happen before it will be run?
>
> Yes, adding the state() function would be needed. It's a bit of work,
> not sure when I get to it.

Understood. From our side I think we would need this in place before
being able to switch over because something like this would I think be
noticeable.

Would a simple first cut of this be to expose the number of blocked
calls, e.g. ch_evalexpr and friends? We could then grow/add to this
function as we learn/experiment more?

<snip>

> Thinking about this, we should probably trigger SafeState later. So we
> would set a flag in ch_evalexpr() that we did block, and check for that
> flag when back in the main loop, after the callback finished.
>
> Hmm, perhaps we don't need that flag, we could always trigger SafeState,
> and you need to make sure you added the autocommand only when you
> actually have some work pending. That's tricky, if you are not careful
> the event is triggered very often (looping may only wait for 20 msec
> before trying again, e.g. checking for a job does polling).

I'm not familiar with Vim's internals: please can you explain the
different loops you're referring to here?

> Another way would be that the plugin explicitly asks for triggering the
> event. Thus if you add something to the work queue you call some
> function and the event is triggered once as soon as the safe state is
> reached in the waiting loop. For the higher level
> (Normal/Insert/Commandline mode) it would still always be triggered.
>
> I guess that just relying on adding the autocommand when it's needed is
> the simplest, it does not require a new mechanism. It's up to the
> plugin then to clean up the autocommand when it's not needed.

Assuming the adding/removing of an autocommand is not too expensive,
that sounds like a reasonable first approach.

> Also, perhaps we should have a different event for when it is triggered
> from the waiting loop, instead of from the toplevel, since Vim is in a
> different state.

Per above, I'd be grateful if you could explain the significance of
these different loops.

Many thanks,


Paul

[Bram Moolenaar]

> > > Firstly, is there still a plan for the safeState/state/whatever
> > > function? Because without it, if a message arrives at a "safe" time we
> > > need something unsafe to happen before it will be run?
> >
> > Yes, adding the state() function would be needed. It's a bit of work,
> > not sure when I get to it.
>
> Understood. From our side I think we would need this in place before
> being able to switch over because something like this would I think be
> noticeable.
>
> Would a simple first cut of this be to expose the number of blocked
> calls, e.g. ch_evalexpr and friends? We could then grow/add to this
> function as we learn/experiment more?
>
> <snip>
>
> > Thinking about this, we should probably trigger SafeState later. So we
> > would set a flag in ch_evalexpr() that we did block, and check for that
> > flag when back in the main loop, after the callback finished.
> >
> > Hmm, perhaps we don't need that flag, we could always trigger SafeState,
> > and you need to make sure you added the autocommand only when you
> > actually have some work pending. That's tricky, if you are not careful
> > the event is triggered very often (looping may only wait for 20 msec
> > before trying again, e.g. checking for a job does polling).
>
> I'm not familiar with Vim's internals: please can you explain the
> different loops you're referring to here?

Most callbacks are invoked from parse_queued_messages(). This is used
when waiting for a key and when sleeping. Only timer callbacks are
called elsewhere (and differently when using the GUI).

> > Another way would be that the plugin explicitly asks for triggering the
> > event. Thus if you add something to the work queue you call some
> > function and the event is triggered once as soon as the safe state is
> > reached in the waiting loop. For the higher level
> > (Normal/Insert/Commandline mode) it would still always be triggered.
> >
> > I guess that just relying on adding the autocommand when it's needed is
> > the simplest, it does not require a new mechanism. It's up to the
> > plugin then to clean up the autocommand when it's not needed.
>
> Assuming the adding/removing of an autocommand is not too expensive,
> that sounds like a reasonable first approach.
>
> > Also, perhaps we should have a different event for when it is triggered
> > from the waiting loop, instead of from the toplevel, since Vim is in a
> > different state.
>
> Per above, I'd be grateful if you could explain the significance of
> these different loops.

It's hard to explain, and it differs per system and whether the GUI is
used. The general idea is "waiting for the user to type a character".
While doing that we trigger timers, read channels and process messages.

--
MAN: Fetchez la vache!
GUARD: Quoi?
MAN: Fetchez la vache!

[Paul Jolly]

> > I'm not familiar with Vim's internals: please can you explain the
> > different loops you're referring to here?
>
> Most callbacks are invoked from parse_queued_messages(). This is used
> when waiting for a key and when sleeping. Only timer callbacks are
> called elsewhere (and differently when using the GUI).

<snip>

> > Per above, I'd be grateful if you could explain the significance of
> > these different loops.
>
> It's hard to explain, and it differs per system and whether the GUI is
> used. The general idea is "waiting for the user to type a character".
> While doing that we trigger timers, read channels and process messages.

Thanks. My understanding this fully is probably not significant to us
reaching a solution here so if you'll excuse the odd silly question :)

Did you see my suggestion about a bare-bones state() function?

[Bram Moolenaar]

> > > I'm not familiar with Vim's internals: please can you explain the
> > > different loops you're referring to here?
> >
> > Most callbacks are invoked from parse_queued_messages(). This is used
> > when waiting for a key and when sleeping. Only timer callbacks are
> > called elsewhere (and differently when using the GUI).
>
> <snip>
>
> > > Per above, I'd be grateful if you could explain the significance of
> > > these different loops.
> >
> > It's hard to explain, and it differs per system and whether the GUI is
> > used. The general idea is "waiting for the user to type a character".
> > While doing that we trigger timers, read channels and process messages.
>
> Thanks. My understanding this fully is probably not significant to us
> reaching a solution here so if you'll excuse the odd silly question :)
>
> Did you see my suggestion about a bare-bones state() function?

I did a first implementation of state(). No tests yet.

--
In his lifetime van Gogh painted 486 oil paintings. Oddly enough, 8975
of them are to be found in the United States.

[Paul Jolly]

This is great, thanks. I'm using state() and SafeStateAgain
(adding/removing the autocommand as required)

I've got a WIP PR with the changes up at https://github.com/govim/govim/pull/533

However one issue I'm seeing is that autocommands are not fired when
doing work in response to SafeStateAgain. The scenario looks like
this:

1. govim calls Vim with work to schedule
2. Vim reports that it is not safe and so I add the work to a queue
and a callback to SafeStateAgain
3. when safe, Vim fires the SafeStateAgain autocommand
4. as part of this work, Vim calls back to govim with ch_evalexpr
5. govim receives the message and calls back to Vim with a command to
open a file
6. ... but not autocommands fire (whereas they did previously)
7. the call from govim to Vim (5) returns
8. the ch_evalexpr call from Vim to govim (4) returns

Any ideas what might be going on here?

[Christian Brabandt]

On Di, 17 Sep 2019, Paul Jolly wrote:

> This is great, thanks. I'm using state() and SafeStateAgain
> (adding/removing the autocommand as required)
>
> I've got a WIP PR with the changes up at https://github.com/govim/govim/pull/533
>
> However one issue I'm seeing is that autocommands are not fired when
> doing work in response to SafeStateAgain. The scenario looks like
> this:
>
> 1. govim calls Vim with work to schedule
> 2. Vim reports that it is not safe and so I add the work to a queue
> and a callback to SafeStateAgain
> 3. when safe, Vim fires the SafeStateAgain autocommand
> 4. as part of this work, Vim calls back to govim with ch_evalexpr
> 5. govim receives the message and calls back to Vim with a command to
> open a file
> 6. ... but not autocommands fire (whereas they did previously)
> 7. the call from govim to Vim (5) returns
> 8. the ch_evalexpr call from Vim to govim (4) returns
>
> Any ideas what might be going on here?

Sounds like you need a nested somewhere. Have a look at
`:h autocmd-nested`

Best,
Christian
--
Jeder Mensch kann irren. Im Irrtum verharren wird jedoch nur der Tor.
-- Marcus Tullius Cicero (106-43 v.Chr.)

[Paul Jolly]

> > Any ideas what might be going on here?
>
> Sounds like you need a nested somewhere. Have a look at
> `:h autocmd-nested`

That was exactly it, thanks very much.

I'm seeing some some instances of not getting callbacks via
SafeStateAgain... but I'll have to investigate that later.

[Paul Jolly]

> I'm seeing some some instances of not getting callbacks via
> SafeStateAgain... but I'll have to investigate that later.

This was actually my mistake; state() returns 'c' when in a channel
callback handler. If that is the only state, then it is safe for me to
immediately schedule the work. But for now I've switched to evaluating
state('w') to be more specific to waiting state.

However I am seeing a situation where neither SafeState nor
SafeStateAgain are being triggered.

The sequence looks like this:

1. mouse move triggers call of function that calls ch_evalexpr into govim
2. govim calls back into Vim to evaluate an expression; this returns
3. govim calls Vim to schedule some work; it's not safe at this point
because the ch_evalexpr in 1 is still blocked; so this work is queued
and we register an autocommand for SafeState,SafeStateAgain
4. govim calls back into Vim (as part of same call from 1) to
create_popup; this returns
5. govim calls back into Vim to redraw; this returns
6. the ch_evalexpr from 1 returns, and the function returns

However at this point, we do not receive a SafeState or SafeStateAgain
autocommand event. So everything blocks (because my govim test is
waiting)

Any ideas, Bram?

[Bram Moolenaar]

How does the mouse move trigger the sequence? It's when returning from
this that SafeStateAgain should be triggered. But that currently only
happens when checking channels for messages. That should be
encountered, since you have channels, but perhaps the safe flag got
reset without anything causing Vim to return to the main loop?

Let me add some log statements so you can see what happens to the state.

--
Error:015 - Unable to exit Windows. Try the door.

[Paul Jolly]

> How does the mouse move trigger the sequence?

Sorry, I should have covered that before. This is a hover test, so I
use test_setmouse followed by:

feedkeys("\<MouseMove>\<Ignore>", "xt")

It's then the balloonexpr function that triggers the ch_evalexpr in
point 1 above.

I've attached the channel log file - that's probably more useful than
me attaching snippets.

You'll see that govim calls into Vim to schedule work (s:schedule
call); when it is safe to run that work, Vim calls back to govim
("schedule"). All of the "s:schedule" calls (with their respective ID)
are matched by "schedule" calls, apart from the last one:

0.700121 RECV on 0(sock): '[0,[47,"call","s:schedule",4]]

which is why we see things lock up.

Thanks

[Bram Moolenaar]

> > How does the mouse move trigger the sequence?
>
> Sorry, I should have covered that before. This is a hover test, so I
> use test_setmouse followed by:
>
> feedkeys("\<MouseMove>\<Ignore>", "xt")
>
> It's then the balloonexpr function that triggers the ch_evalexpr in
> point 1 above.

OK, so the whole sequence is triggered by the balloon eval
functionality. This happens in check_due_timer(). That's a fairly
normal way a callback is invoked, in this case evaluating 'balloonexpr'.

> I've attached the channel log file - that's probably more useful than
> me attaching snippets.
>
> You'll see that govim calls into Vim to schedule work (s:schedule
> call); when it is safe to run that work, Vim calls back to govim
> ("schedule"). All of the "s:schedule" calls (with their respective ID)
> are matched by "schedule" calls, apart from the last one:
>
> 0.700121 RECV on 0(sock): '[0,[47,"call","s:schedule",4]]
>
> which is why we see things lock up.

You can see in line 409:
0.181045 : safe state reset

That's why it stops triggering SafeStateAgain. This is after a call to
s:define(), that is that doing? Somehow it doesn't go back to the main
loop to check the state.

--
Facepalm statement #6: "Estland is a fantasy place, just like Middle Earth and
Madagaskar"

[Paul Jolly]

> OK, so the whole sequence is triggered by the balloon eval
> functionality. This happens in check_due_timer(). That's a fairly
> normal way a callback is invoked, in this case evaluating 'balloonexpr'.

Ok, thanks.

> You can see in line 409:
> 0.181045 : safe state reset
>
> That's why it stops triggering SafeStateAgain. This is after a call to
> s:define(), that is that doing? Somehow it doesn't go back to the main
> loop to check the state.

All callbacks we make from Vim -> govim use ch_evalexpr, for
autocommands, commands, functions, listener_add callbacks etc. That is
they all block. This makes reasoning about things much easier. If
anything needs to happen async we make that decision within govim.

Similarly for calls from govim -> Vim we block on the govim side; we
then use ch_sendexpr to reply from Vim with an [error(, result)]
array:

https://github.com/govim/govim/blob/e8054412bdbc5c2fd232a56fdc79ca4fc6082312/plugin/govim.vim#L221

So lines 408-410:

0.180945 on 0: Invoking channel callback <SNR>17_define
0.181045 : safe state reset
0.181146 SEND on 0(sock): '[49,["callback",45,[""]]]

is the "response" to govim from the call from line 404:

0.180915 RECV on 0(sock): '[0,[45,"ex","call
feedkeys(\"\\\u003cMouseMove\u003e\\\u003cIgnore\u003e\", \"xt\")"]]

(you can see we also have an ID in there)

Hopefully that help to clarify things?

[Bram Moolenaar]

OK, so the feedkeys() causes the safe state to be reset. That is
correct, since Vim is then busy with processing input, which isn't
considered safe.

The problem is that with the "xt" argument to feedkeys() the input is
processed right away, and we don't return to the main loop. I guess
that we should check for a safe state at the start of feedkeys(), and if
it's true and the input has been processed, check if it's still safe.

--
Facepalm statement #7: "Last week I almost got pregnant!"

[Paul Jolly]

> OK, so the feedkeys() causes the safe state to be reset. That is
> correct, since Vim is then busy with processing input, which isn't
> considered safe.
>
> The problem is that with the "xt" argument to feedkeys() the input is
> processed right away, and we don't return to the main loop. I guess
> that we should check for a safe state at the start of feedkeys(), and if
> it's true and the input has been processed, check if it's still safe.

Honestly, with feedkeys I'm never 100% sure I'm doing the "right"
thing, so please tell me if this is _not_ the right thing to be doing
:)

[Bram Moolenaar]

It's better to avoid feedkeys(). ":normal" is a bit better (it saves
and restores the typeahead buffer).

Please try with patch 8.1.2053.

--
Microsoft's definition of a boolean: TRUE, FALSE, MAYBE
"Embrace and extend"...?

[Paul Jolly]

> It's better to avoid feedkeys(). ":normal" is a bit better (it saves
> and restores the typeahead buffer).

Noted, thank you. I've raised
https://github.com/govim/govim/issues/535 on the govim side to make
the switch.

> Please try with patch 8.1.2053

Just tested with 8.1.2056; all looks great, thanks. Log attached for reference

We'll now give this a bit more exposure in our CI tests and I'll run
it locally as my setup and report back with any further
questions/issues.

Thanks again, Bram.


Paul

[Bram Moolenaar]

Thanks for checking. I'll write tests for state() and the autocommand
events.

--
GALAHAD turns back. We see from his POV the lovely ZOOT standing by him
smiling enchantingly and a number of equally delectable GIRLIES draped
around in the seductively poulticed room. They look at him smilingly and
wave.

"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

[Paul Jolly]

Hi Bram,

> Thanks for checking. I'll write tests for state() and the autocommand
> events.

The only issue we're seeing is that things "lock up" with GVim.

Taking a simple example test, and running the same govim test in Vim
and then GVim:

* Vim reports that the state is 'c'
* GVim reports that the state is 'oSc'

Whether this is significant or not I'm not sure. But on the basis of
the above state we immediately schedule work. In Vim everything
proceeds just fine, but in GVim things lockup.

The log file sequences are here:

https://gist.github.com/myitcv/05c226d2388a9877317082398f1d9904

Any thoughts on what's going on here?

Thanks,


Paul

[Bram Moolenaar]

Paul Jolly wrote:

> > Thanks for checking. I'll write tests for state() and the autocommand
> > events.
>
> The only issue we're seeing is that things "lock up" with GVim.
>
> Taking a simple example test, and running the same govim test in Vim
> and then GVim:
>
> * Vim reports that the state is 'c'
> * GVim reports that the state is 'oSc'
>
> Whether this is significant or not I'm not sure. But on the basis of
> the above state we immediately schedule work. In Vim everything
> proceeds just fine, but in GVim things lockup.

The "o" indicates operator-pending. To find out what happens, you could
add an extra ch_log() call in op_pending() to see which of the
conditions is causing this. Because of this the "S" would also appear,
and then the SafeState and SafeStateAgain autocommands won't be
triggered.

> The log file sequences are here:
>
> https://gist.github.com/myitcv/05c226d2388a9877317082398f1d9904
>
> Any thoughts on what's going on here?

Not really. The wait-for-character loop is a bit different, but it
still calls parse_queued_messages().

--
LAUNCELOT leaps into SHOT with a mighty cry and runs the GUARD through and
hacks him to the floor. Blood. Swashbuckling music (perhaps).
LAUNCELOT races through into the castle screaming.
SECOND SENTRY: Hey!

[Paul Jolly]

> The "o" indicates operator-pending. To find out what happens, you could
> add an extra ch_log() call in op_pending() to see which of the
> conditions is causing this. Because of this the "S" would also appear,
> and then the SafeState and SafeStateAgain autocommands won't be
> triggered.

Added some logging and bizarrely, op_pending is not being called when
I run this test with GVim (at least GVim hangs first).

I'm nowhere near close enough to the code to work out what's going on
here so hopefully this gives you a clue :)

> > Any thoughts on what's going on here?
>
> Not really. The wait-for-character loop is a bit different, but it
> still calls parse_queued_messages().

Adding logging in the VimScript shim for govim, I can see GVim is
blocked executing this line:

https://github.com/govim/govim/blob/dd90b98557f38dca3785573eab235592b89003c3/plugin/govim.vim#L238

(i.e. a log line added just before appears, a log line after does not)

So I don't think this has anything to do with being blocked on
receiving/sending messages per se.

Thanks

[Bram Moolenaar]

Paul Jolly wrote:

> > The "o" indicates operator-pending. To find out what happens, you could
> > add an extra ch_log() call in op_pending() to see which of the
> > conditions is causing this. Because of this the "S" would also appear,
> > and then the SafeState and SafeStateAgain autocommands won't be
> > triggered.
>
> Added some logging and bizarrely, op_pending is not being called when
> I run this test with GVim (at least GVim hangs first).
>
> I'm nowhere near close enough to the code to work out what's going on
> here so hopefully this gives you a clue :)

You must be doing something wrong. f_state() always calls op_pending().
Is the log not started when you call state()?

> > > Any thoughts on what's going on here?
> >
> > Not really. The wait-for-character loop is a bit different, but it
> > still calls parse_queued_messages().
>
> Adding logging in the VimScript shim for govim, I can see GVim is
> blocked executing this line:
>
> https://github.com/govim/govim/blob/dd90b98557f38dca3785573eab235592b89003c3/plugin/govim.vim#L238
>
> (i.e. a log line added just before appears, a log line after does not)
>
> So I don't think this has anything to do with being blocked on
> receiving/sending messages per se.

So what is the value of l:expr when that happens?

--
Overflow on /dev/null, please empty the bit bucket.

[Paul Jolly]

> > Added some logging and bizarrely, op_pending is not being called when
> > I run this test with GVim (at least GVim hangs first).
> >
> > I'm nowhere near close enough to the code to work out what's going on
> > here so hopefully this gives you a clue :)
>
> You must be doing something wrong. f_state() always calls op_pending().
> Is the log not started when you call state()?

The log is definitely started.

Actually, when using GVim I'm not getting any ch_log messages that
originate from vim itself, e.g.

SafeState: back to waiting, triggering SafeStateAgain

whereas when using Vim I do.

So that's probably why I'm not seeing the ch_log lines I just added?

> > So I don't think this has anything to do with being blocked on
> > receiving/sending messages per se.
>
> So what is the value of l:expr when that happens?

l:expr is:

'echom "Hello from BufRead main.go"'

[Bram Moolenaar]

Paul Jolly wrote:

> > > Added some logging and bizarrely, op_pending is not being called when
> > > I run this test with GVim (at least GVim hangs first).
> > >
> > > I'm nowhere near close enough to the code to work out what's going on
> > > here so hopefully this gives you a clue :)
> >
> > You must be doing something wrong. f_state() always calls op_pending().
> > Is the log not started when you call state()?
>
> The log is definitely started.
>
> Actually, when using GVim I'm not getting any ch_log messages that
> originate from vim itself, e.g.
>
> SafeState: back to waiting, triggering SafeStateAgain
>
> whereas when using Vim I do.

Weird. I just tried and it works fine for me. Is this with GTK?

I do notice the log is filled with these SafeState lines, we should
probably reduce that.

> So that's probably why I'm not seeing the ch_log lines I just added?
>
> > > So I don't think this has anything to do with being blocked on
> > > receiving/sending messages per se.
> >
> > So what is the value of l:expr when that happens?
>
> l:expr is:
>
> 'echom "Hello from BufRead main.go"'

Hmm, perhaps it hangs in a "hit enter" prompt, but doesn't redraw the
screen? You can use a debugger to find out where it is hanging. Or use
ch_log() if that works.

--
** Hello and Welcome to the Psychiatric Hotline **
If you are obsessive-compulsive, please press 1 repeatedly.
If you are co-dependent, please ask someone to press 2.
If you have multiple personalities, please press 3, 4, 5 and 6.
If you are paranoid-delusional, we know who you are and what you want
- just stay on the line so we can trace the call.
If you are schizophrenic, listen carefully and a little voice will
tell you which number to press next.
If you are manic-depressive, it doesn't matter which number you press
- no one will answer.
If you suffer from panic attacks, push every button you can find.
If you are sane, please hold on - we have the rest of humanity on the
other line and they desparately want to ask you a few questions.

[Paul Jolly]

> Weird. I just tried and it works fine for me. Is this with GTK?

Yes, it's with GTK.

> I do notice the log is filled with these SafeState lines, we should
> probably reduce that.

Yep, I agree (or perhaps put behind some sort of build flag in case we
need to re-enable?)

> > l:expr is:
> >
> > 'echom "Hello from BufRead main.go"'
>
> Hmm, perhaps it hangs in a "hit enter" prompt, but doesn't redraw the
> screen? You can use a debugger to find out where it is hanging. Or use
> ch_log() if that works.

I think you might be onto something here. Because the tests actually
pass when run on CI (which is using Docker).

So aside from Vim-based ch_log not appearing to output anything with
GVim (GTK)... I think we're "done" here.

Thanks for helping to work through this.


Paul

[Paul Jolly]

Hi Bram,

> Thanks for helping to work through this.

Just a quick status update: our govim builds have been very solid
since these changes. Test failures we've seen have been gopls related,
so at least as far as we've been able to tell so far the SafeState*
changes are working well.

Thanks again,


Paul

[Bram Moolenaar]

Paul Jolly wrote:

> Just a quick status update: our govim builds have been very solid
> since these changes. Test failures we've seen have been gopls related,
> so at least as far as we've been able to tell so far the SafeState*
> changes are working well.

Glad to hear this solution works. I hope a few others try it out as
well.


--
f y cn rd ths thn y cn hv grt jb n cmptr prgrmmng