let timeout_id = settimeout(700, 'echo("hello")')
To cancel the timeout before it's fired:
canceltimeout(timeout_id)
setinterval() also returns an id that can be used with canceltimeout.
The reason for this patch is simple: asynchronous functionality is needed to implement real-time collaborative editing in Vim. This is one of the most voted-for features (see http://www.vim.org/sponsor/vote_results.php).
Along with Matt Kaniaris, I founded Floobits to build real-time collaboration into every editor. We wrote a plugin for Vim, but we had to use hacks to get async behavior (abusing feedkeys or client-server). These methods had side-effects such as breaking leaderkeys or other shortcuts. After a lot of experimenting, we decided to try patching Vim.
Since Vim is character-driven, we had to munge some low-level input functions to get the desired behavior. We changed gui_wait_for_chars() and mch_inchar() so that call_timeouts() is run every ticktime milliseconds. The default ticktime is 100ms.
This patch isn't finished yet, but it works on unix-based OSes. If the reaction is positive, our intention is to change mch_inchar() (or something similar) in other OS-specific files. That will get async functions working for everyone.
Even if our patch isn't the best approach, we'd love to help get async functions in Vim. Doing so will open the door to a lot of cool plugins.
Oh, and this is the first time either myself or Matt have submitted a patch to Vim, so please be gentle.
Sincerely,
Geoff Greer
Thanks for the feedback. I've attached a new patch with some basic comments and non-ANSI C function definitions.
I'd really like to get feedback on the changes to gui_wait_for_chars() and mch_inchar(). That's where the meat of this patch is.
Sincerely,
Geoff
I live this patch. Bram, we discused about this similar topic at several times. And you didn't answer whether vim will include this feature clearly.
However, I think this feature is extendable way which vim get modern software designs. Let's include this feature!
Thanks.
- Yasuhiro Matsumoto
Forgive me if I'm wrong, I didn't spend more than a few minutes looking over the patch.
It looks like this is not really "asynchronous" behavior, but rather adding a timer-like hook into the main loop. Pro: no worries about thread safety, race conditions, etc. Con: if the function called by the timer event takes a long time, Vim hangs until it's done.
If I'm correct, and this gets included, the help will need to be very explicit about making sure to limit the time taken by the function called! And there should definitely be a way to interrupt the action.
Ben, why should asynchronous receive any type of restriction not already placed on code that runs some milliseconds after a key press (an existing hook). Cancelling asynchronous requests can lead to unpredictable behavior is probably never desirable. I've already experienced plugins that block for too long and what I do is, I just remove the plugin.
And most of these can be interrupted with CTRL-C. I am not certain whether the async functions will be since you added their processing to the functions which get keypresses (I think...I'm not actually familiar with most of Vim's code). I'm not saying they're not...only that the need to be. We've already had a few "Vim hangs on this regex" bugs in 7.4, and I could easily see a plugin that uses a search() call inside one of these timer functions. Or maybe it calls an external tool that is deadlocked for some reason. Or any number of other things could go wrong that the user doesn't want to wait for.
And I'm more worried about function that takes seconds, or minutes, to complete, than I am about milliseconds. My point was that users of these functions need to be aware that the action is not actually asynchronous in the sense of running in the background and allowing user input to continue in the foreground; you just mean it can be synced to a timer rather than a specific user input, correct?
> Cancelling asynchronous requests can lead to unpredictable behavior is probably never desirable. I've already experienced plugins that block for too long and what I do is, I just remove the plugin.
>
If you're typing something, suddenly Vim stops responding to any keyboard input, you've waited most of a minute for Vim to start responding, and you have unsaved work you wish to keep...it can be more desirable to interrupt the action and then continue using Vim long enough to save your work and restart.
Maybe if the user interrupts the action with CTRL-C you need to set a flag to remove the timer from the list, to prevent it firing off again in a few milliseconds?
Ben,
You are correct in that we added a timer to the main loop. Looking over the code once again, I think we should have altered the calls to select/poll instead, but lets discuss the practical effect of this patch since we can work out the details some time later.
Async does not imply parallelism nor safety- it only provides the illusion of concurrency. Idle cpu time spent waiting is instead used to perform some other task. Async frameworks make no such guarantee that every call not block the event loop; it is incumbent upon the programmer to ensure that the event loop is not blocked indefinitely. This is the definition and standard interface of async programming for every popular framework I can think of that lives in a scripting language- JavaScript, nodejs, twisted, event machine, and luvit.
The current alternative in Vim is to block forever, or worse yet, not write useful software because it is impossible to do so as a Vim plugin. With a proper settimeout, plugins have the option of returning control to the main loop and checking in latter. I'd suggest a settimeout on the settimeouts is not needed; plugins which cause too much trouble will either be fixed or uninstalled.
I like the practical implications of being able to respond to a timer event in a plugin. Polling is useful for many, many purposes, and doing it by shelling out and calling back with --remote calls is awkward at best.
> Async does not imply parallelism nor safety- it only provides the illusion of concurrency. Idle cpu time spent waiting is instead used to perform some other task. Async frameworks make no such guarantee that every call not block the event loop; it is incumbent upon the programmer to ensure that the event loop is not blocked indefinitely. This is the definition and standard interface of async programming for every popular framework I can think of that lives in a scripting language- JavaScript, nodejs, twisted, event machine, and luvit.
>
I'm more familiar with C/C++/Java, and even shell scripting, where "async" means it doesn't block the main program flow, as wikipedia says on http://en.wikipedia.org/wiki/Asynchrony: "Asynchronous actions are actions executed in a non-blocking scheme, allowing the main program flow to continue processing."
So, in my mind..."async" normally implies parallelism.
You've created an interface that is probably close enough, if developers are careful not to allow their timer callbacks to take too long, or use them to spawn and monitor a truly async process via the shell or python or whatever.
The nice thing is that you thus avoid the problem of concurrent access/thread safety, because you've not used multiple threads or processes at all. The not-so-nice thing is the inability to call long-running worker functions with your interface.
> The current alternative in Vim is to block forever, or worse yet, not write useful software because it is impossible to do so as a Vim plugin. With a proper settimeout, plugins have the option of returning control to the main loop and checking in latter.
I agree, we need something. That concurrent editing thing looked really neat! And doubtless it would be much more robust/easier to maintain with timer events in Vim.
> I'd suggest a settimeout on the settimeouts is not needed; plugins which cause too much trouble will either be fixed or uninstalled.
Possibly; but I'd rather not find the bug in the plugin by losing a lot of work when my Vim goes into an infinite loop that can't be interrupted.
Ben,
Thanks for the explanation. End users will probably never directly call set_timeout. Hopefully, plugin authors know better than to make a call that can block forever. Good documentation will help.
Maybe control-c should cancel intervals. As you mentioned, this would let users save stuff and restart vim. At worse, plugins would be dumped into an inconsistent state that could cause more harm. Also, canceling internal callbacks is quite opaque to users. I don't see any great options for dealing with runaway plugins.
-kans
Thanks for explanation.
Maybe control-c should cancel intervals. As you mentioned, this would let users save stuff and restart vim. At worse, plugins would be dumped into an inconsistent state that could cause more harm. Also, canceling internal callbacks is quite opaque to users. I don't see any great options for dealing with runaway plugins.
Hey check this out:
au CursorHold * call BlockForever()
" Blocks forever
function! BlockForever()
let c = 1
while c >= 0
let c += 1
endwhile
endfunction
You can already do this blocking, I've already experienced it with plugins I've removed. I ask again why setTimeout should be treated differently than CursorHold? You can cntrl-c out of CursorHold, but just then make that a requirement for settimeout.
> This is not asynchronous but these are timed commands. It should be
> clear that the commands are executed in the "idle" loop, when Vim waits
> for a character to be typed.
As was already pointed out, this is what asynchronous execution looks like in JavaScript. JavaScript in the browser and elsewhere has only one thread of execution, and the behavior of setTimeout in these contexts is similar to this patch. Whatever you want to call this patch, it is very powerful, and vim is missing it. Try implementing a clock for the status line in current vim. It updates only after a keystroke. With this patch, you can have a real clock, that shows you the actual time! Not the time you last pressed a key.
Try this in a Chrome console:
function blockForever() {
var c = 1;
while (c > 0) {
c++;
}
}
window.setTimeout(blockForever, 1000);
You get the same behavior that you would from this patch, the same behavior that CursorHold gives you.
I agree there needs to be documentation on how to use this. I've built vim with this patch and was able to get one plugin that uses setinterval to work, but was not able to get my own attempt to work after a quick attempt.
On Thursday, September 5, 2013 4:26:41 AM UTC-7, Bram Moolenaar wrote:
> A few thoughts about this patch.
>
>
>
> This is not asynchronous but these are timed commands. It should be
>
> clear that the commands are executed in the "idle" loop, when Vim waits
>
> for a character to be typed.
>
>
>
> There might be two types: One that has high priority, and gets handled
>
> even when the user has typed something. Another that has lower
>
> priority, only gets handled when waiting for the user to type.
>
>
>
> As someone already mentioned: If the command takes too much time, Vim
>
> will get stuck. E.g. when it reads from a socket. It might be possible
>
> to have this kind of work done in another thread, e.g. using Python.
>
>
>
> The documentation is still missing. A lot of details need to be
>
> explained there.
>
>
>
> It must be possible to get a list of these commands, so that they can be
>
> cancelled without having the id. This will then possibly break the
>
> plugin that installed the command, so this gets messy quickly.
>
>
>
> The code uses "long" to store msec, but you need 64 bits for that and
>
> long can be 32 bits.
>
>
>
> I don't see the need for 'ticktime'. The remaining time until the next
>
> timeout can be computed. If it's smaller than 'updatetime' then wait
>
> for that long.
>
>
>
>
>
> --
>
> hundred-and-one symptoms of being an internet addict:
>
> 170. You introduce your wife as "my_la.wife" and refer to your
>
> children as "forked processes."
>
>
>
> /// Bram Moolenaar -- Bra....net -- http://www.Moolenaar.net \\\
Exactly. I think as long as you can interrupt the timer with a keystroke it should be fine. But since you call your timer in between waiting for a keystroke, won't that prevent CTRL-C from interrupting it? Maybe it won't...I haven't tried it. I don't think we have any unique constraints, I just think that (a) it should be mentioned in the help that the timer blocks user input while processing, it's not a true asynchronous process and (b) it is required that CTRL-C or similar can interrupt the timer like it can already for autocmds, etc.
ctrl-c works (in this patch) because the OS converts it into a SIGINT which vim knows how to handle.
>There might be two types: One that has high priority, and gets handled
even when the user has typed something. Another that has lower
priority, only gets handled when waiting for the user to type.
As mattn pointed out, this won't work unless the internals of vim are threadsafe.
> It must be possible to get a list of these commands, so that they can be
>
> cancelled without having the id.
This wouldn't be much work, but to my knowledge, no other implementation has this ability.
> I don't see the need for 'ticktime'. The remaining time until the next
>
> timeout can be computed. If it's smaller than 'updatetime' then wait
>
> for that long.
This is true. ticktime is only useful for end users if they want to trade off efficiency for a better timer function. 50ms is probably a sane default that is good enough (and once again, no other implementation lets this bubble up to user land).
Does anyone have any feedback on how this implemented; ie, putting this logic in mch_inchar vs. RealWaitForChar? Another issue is that vim makes other blocking calls that will block the timers.
I've attached our latest patch. We moved some of the logic down to the level of poll/select (RealWaitForChar) using MAY_LOOP which is similar to what the scheme bindings do; it is cleaner and works better. We also changed the eval call to use do_cmdline_cmd which is far more convenient. Now, the timeouts don't have to be user defined functions. Also, all timeouts can now be disabled by setting 'ticktime' to -1. We discussed disabling intervals that fail, but we ultimately decided that the onus should be on programmers to make sane plugins and canceling intervals is unexpected behavior. Finally, the new features are now documented. Please let us know what you think.
Why do you keep calling it async?
Also other issues:
1. async.c does not follow coding style:
1.1. each 8 indentation spaces should be replaced with a tab
1.2. each opening figure brace should be present on its own line
1.3. no (void) in function declaration
2. async.h should be split into globals.h and proto/async.pro. async.h itself is to be removed.
3. Never use gettimeofday for intervals. There are special monotonic clocks in all operating system, for linux it is clock_gettime(CLOCK_MONOTONIC_RAW). What you use is subject for ntp daemon/ntpclient updates.
4. There is a proper way to disable a setting. Putting the whole setting into an #ifdef is not the one.
5. Documentation of 'ticktime' lacks references to +smth. I repeat, do not call smth "async". Not until you will make it execute asynchronously.
6. I do not see patch for the file with a list of features.
Why do you keep calling it async?
Now please show how you can run one sequence of commands interleaved with the other with this patch. Function that runs with given interval blocks everything until it completes, same for autocommands. You cannot say you implement preemptive multitasking if you have to wait for one task to complete before starting the other. It is not async feature.
You can, of course, do some hacks with saving a state, exiting and resuming, but you will then be forced to *emulate* preemption without any support from vim. I used to have some simple emulation even without such patch.
Note though that I would really like it merged. But not with the current name.
You can take the code from python: since 3.3 it has monotonic() function defined in C code in time module: pymonotonic function in Modules/timemodule.c. It has implementation for windows as well. Linux implementation used CLOCK_MONOTONIC (or, with higher priority, CLOCK_HIRES which is not mentioned in my clock_gettime man page; I guess there is a reason for it) without _RAW, though I would write
#ifdef CLOCK_MONOTONIC_RAW
CLOCK_MONOTONIC_RAW
#else
CLOCK_MONOTONIC
#endif
in function arguments. But it is not as severe as issue with gettimeofday: you cannot see 2 centuries hop with CLOCK_MONOTONIC, maximum slightly increased/decreased second duration when NTP daemon does his adjustments.
By the way, two pairs of figure braces may be removed from insert_timeout. Not that it is required by coding style, but this makes code look better.
> Bram,
>
> We now use long longs for keeping track of the timers. We have also documented the new functions and some of their limitations.
>
>
> Please let us know if you have further feedback.
>
> -Matt
>
I can suggest looking at VimTry* functions in src/if_py_both.h: they are used to transform all vim errors into python exceptions. Purge the python stuff and you will get C functions for wrapping arbitrary code into try/catch.
So, if I install a plugin using this feature, and some weird sequence of events puts it into an infinite loop, I just need to kill Vim and lose my work?
I really, really want a way to force all timers to stop firing. Maybe not CTRL-C but SOMETHING should be possible for the user to kill misbehaving periodic tasks.
Ben,
Ctrl-C will kill a given task, but it won't cancel future timeouts (if it was an interval). Ctrl-C can't do that or else it would happen accidentally since the end user should never notice timers firing to begin with. I'm open to suggestions, but you are asking for a solution to a problem that doesn't exist in any other text editor or IDE.
-Matt