let timeout_id = settimeout(700, 'echo("hello")')
To cancel the timeout before it's fired:
canceltimeout(timeout_id)
setinterval() also returns an id that can be used with canceltimeout.
The reason for this patch is simple: asynchronous functionality is needed to implement real-time collaborative editing in Vim. This is one of the most voted-for features (see http://www.vim.org/sponsor/vote_results.php).
Along with Matt Kaniaris, I founded Floobits to build real-time collaboration into every editor. We wrote a plugin for Vim, but we had to use hacks to get async behavior (abusing feedkeys or client-server). These methods had side-effects such as breaking leaderkeys or other shortcuts. After a lot of experimenting, we decided to try patching Vim.
Since Vim is character-driven, we had to munge some low-level input functions to get the desired behavior. We changed gui_wait_for_chars() and mch_inchar() so that call_timeouts() is run every ticktime milliseconds. The default ticktime is 100ms.
This patch isn't finished yet, but it works on unix-based OSes. If the reaction is positive, our intention is to change mch_inchar() (or something similar) in other OS-specific files. That will get async functions working for everyone.
Even if our patch isn't the best approach, we'd love to help get async functions in Vim. Doing so will open the door to a lot of cool plugins.
Oh, and this is the first time either myself or Matt have submitted a patch to Vim, so please be gentle.
Sincerely,
Geoff Greer
Thanks for the feedback. I've attached a new patch with some basic comments and non-ANSI C function definitions.
I'd really like to get feedback on the changes to gui_wait_for_chars() and mch_inchar(). That's where the meat of this patch is.
Sincerely,
Geoff
I live this patch. Bram, we discused about this similar topic at several times. And you didn't answer whether vim will include this feature clearly.
However, I think this feature is extendable way which vim get modern software designs. Let's include this feature!
Thanks.
- Yasuhiro Matsumoto
Forgive me if I'm wrong, I didn't spend more than a few minutes looking over the patch.
It looks like this is not really "asynchronous" behavior, but rather adding a timer-like hook into the main loop. Pro: no worries about thread safety, race conditions, etc. Con: if the function called by the timer event takes a long time, Vim hangs until it's done.
If I'm correct, and this gets included, the help will need to be very explicit about making sure to limit the time taken by the function called! And there should definitely be a way to interrupt the action.
Ben, why should asynchronous receive any type of restriction not already placed on code that runs some milliseconds after a key press (an existing hook). Cancelling asynchronous requests can lead to unpredictable behavior is probably never desirable. I've already experienced plugins that block for too long and what I do is, I just remove the plugin.
And most of these can be interrupted with CTRL-C. I am not certain whether the async functions will be since you added their processing to the functions which get keypresses (I think...I'm not actually familiar with most of Vim's code). I'm not saying they're not...only that the need to be. We've already had a few "Vim hangs on this regex" bugs in 7.4, and I could easily see a plugin that uses a search() call inside one of these timer functions. Or maybe it calls an external tool that is deadlocked for some reason. Or any number of other things could go wrong that the user doesn't want to wait for.
And I'm more worried about function that takes seconds, or minutes, to complete, than I am about milliseconds. My point was that users of these functions need to be aware that the action is not actually asynchronous in the sense of running in the background and allowing user input to continue in the foreground; you just mean it can be synced to a timer rather than a specific user input, correct?
> Cancelling asynchronous requests can lead to unpredictable behavior is probably never desirable. I've already experienced plugins that block for too long and what I do is, I just remove the plugin.
>
If you're typing something, suddenly Vim stops responding to any keyboard input, you've waited most of a minute for Vim to start responding, and you have unsaved work you wish to keep...it can be more desirable to interrupt the action and then continue using Vim long enough to save your work and restart.
Maybe if the user interrupts the action with CTRL-C you need to set a flag to remove the timer from the list, to prevent it firing off again in a few milliseconds?
Ben,
You are correct in that we added a timer to the main loop. Looking over the code once again, I think we should have altered the calls to select/poll instead, but lets discuss the practical effect of this patch since we can work out the details some time later.
Async does not imply parallelism nor safety- it only provides the illusion of concurrency. Idle cpu time spent waiting is instead used to perform some other task. Async frameworks make no such guarantee that every call not block the event loop; it is incumbent upon the programmer to ensure that the event loop is not blocked indefinitely. This is the definition and standard interface of async programming for every popular framework I can think of that lives in a scripting language- JavaScript, nodejs, twisted, event machine, and luvit.
The current alternative in Vim is to block forever, or worse yet, not write useful software because it is impossible to do so as a Vim plugin. With a proper settimeout, plugins have the option of returning control to the main loop and checking in latter. I'd suggest a settimeout on the settimeouts is not needed; plugins which cause too much trouble will either be fixed or uninstalled.
I like the practical implications of being able to respond to a timer event in a plugin. Polling is useful for many, many purposes, and doing it by shelling out and calling back with --remote calls is awkward at best.
> Async does not imply parallelism nor safety- it only provides the illusion of concurrency. Idle cpu time spent waiting is instead used to perform some other task. Async frameworks make no such guarantee that every call not block the event loop; it is incumbent upon the programmer to ensure that the event loop is not blocked indefinitely. This is the definition and standard interface of async programming for every popular framework I can think of that lives in a scripting language- JavaScript, nodejs, twisted, event machine, and luvit.
>
I'm more familiar with C/C++/Java, and even shell scripting, where "async" means it doesn't block the main program flow, as wikipedia says on http://en.wikipedia.org/wiki/Asynchrony: "Asynchronous actions are actions executed in a non-blocking scheme, allowing the main program flow to continue processing."
So, in my mind..."async" normally implies parallelism.
You've created an interface that is probably close enough, if developers are careful not to allow their timer callbacks to take too long, or use them to spawn and monitor a truly async process via the shell or python or whatever.
The nice thing is that you thus avoid the problem of concurrent access/thread safety, because you've not used multiple threads or processes at all. The not-so-nice thing is the inability to call long-running worker functions with your interface.
> The current alternative in Vim is to block forever, or worse yet, not write useful software because it is impossible to do so as a Vim plugin. With a proper settimeout, plugins have the option of returning control to the main loop and checking in latter.
I agree, we need something. That concurrent editing thing looked really neat! And doubtless it would be much more robust/easier to maintain with timer events in Vim.
> I'd suggest a settimeout on the settimeouts is not needed; plugins which cause too much trouble will either be fixed or uninstalled.
Possibly; but I'd rather not find the bug in the plugin by losing a lot of work when my Vim goes into an infinite loop that can't be interrupted.
Ben,
Thanks for the explanation. End users will probably never directly call set_timeout. Hopefully, plugin authors know better than to make a call that can block forever. Good documentation will help.
Maybe control-c should cancel intervals. As you mentioned, this would let users save stuff and restart vim. At worse, plugins would be dumped into an inconsistent state that could cause more harm. Also, canceling internal callbacks is quite opaque to users. I don't see any great options for dealing with runaway plugins.
-kans
Thanks for explanation.
Maybe control-c should cancel intervals. As you mentioned, this would let users save stuff and restart vim. At worse, plugins would be dumped into an inconsistent state that could cause more harm. Also, canceling internal callbacks is quite opaque to users. I don't see any great options for dealing with runaway plugins.
Hey check this out:
au CursorHold * call BlockForever()
" Blocks forever
function! BlockForever()
let c = 1
while c >= 0
let c += 1
endwhile
endfunction
You can already do this blocking, I've already experienced it with plugins I've removed. I ask again why setTimeout should be treated differently than CursorHold? You can cntrl-c out of CursorHold, but just then make that a requirement for settimeout.
> This is not asynchronous but these are timed commands. It should be
> clear that the commands are executed in the "idle" loop, when Vim waits
> for a character to be typed.
As was already pointed out, this is what asynchronous execution looks like in JavaScript. JavaScript in the browser and elsewhere has only one thread of execution, and the behavior of setTimeout in these contexts is similar to this patch. Whatever you want to call this patch, it is very powerful, and vim is missing it. Try implementing a clock for the status line in current vim. It updates only after a keystroke. With this patch, you can have a real clock, that shows you the actual time! Not the time you last pressed a key.
Try this in a Chrome console:
function blockForever() {
var c = 1;
while (c > 0) {
c++;
}
}
window.setTimeout(blockForever, 1000);
You get the same behavior that you would from this patch, the same behavior that CursorHold gives you.
I agree there needs to be documentation on how to use this. I've built vim with this patch and was able to get one plugin that uses setinterval to work, but was not able to get my own attempt to work after a quick attempt.
On Thursday, September 5, 2013 4:26:41 AM UTC-7, Bram Moolenaar wrote:
> A few thoughts about this patch.
>
>
>
> This is not asynchronous but these are timed commands. It should be
>
> clear that the commands are executed in the "idle" loop, when Vim waits
>
> for a character to be typed.
>
>
>
> There might be two types: One that has high priority, and gets handled
>
> even when the user has typed something. Another that has lower
>
> priority, only gets handled when waiting for the user to type.
>
>
>
> As someone already mentioned: If the command takes too much time, Vim
>
> will get stuck. E.g. when it reads from a socket. It might be possible
>
> to have this kind of work done in another thread, e.g. using Python.
>
>
>
> The documentation is still missing. A lot of details need to be
>
> explained there.
>
>
>
> It must be possible to get a list of these commands, so that they can be
>
> cancelled without having the id. This will then possibly break the
>
> plugin that installed the command, so this gets messy quickly.
>
>
>
> The code uses "long" to store msec, but you need 64 bits for that and
>
> long can be 32 bits.
>
>
>
> I don't see the need for 'ticktime'. The remaining time until the next
>
> timeout can be computed. If it's smaller than 'updatetime' then wait
>
> for that long.
>
>
>
>
>
> --
>
> hundred-and-one symptoms of being an internet addict:
>
> 170. You introduce your wife as "my_la.wife" and refer to your
>
> children as "forked processes."
>
>
>
> /// Bram Moolenaar -- Bra....net -- http://www.Moolenaar.net \\\
Exactly. I think as long as you can interrupt the timer with a keystroke it should be fine. But since you call your timer in between waiting for a keystroke, won't that prevent CTRL-C from interrupting it? Maybe it won't...I haven't tried it. I don't think we have any unique constraints, I just think that (a) it should be mentioned in the help that the timer blocks user input while processing, it's not a true asynchronous process and (b) it is required that CTRL-C or similar can interrupt the timer like it can already for autocmds, etc.
ctrl-c works (in this patch) because the OS converts it into a SIGINT which vim knows how to handle.
>There might be two types: One that has high priority, and gets handled
even when the user has typed something. Another that has lower
priority, only gets handled when waiting for the user to type.
As mattn pointed out, this won't work unless the internals of vim are threadsafe.
> It must be possible to get a list of these commands, so that they can be
>
> cancelled without having the id.
This wouldn't be much work, but to my knowledge, no other implementation has this ability.
> I don't see the need for 'ticktime'. The remaining time until the next
>
> timeout can be computed. If it's smaller than 'updatetime' then wait
>
> for that long.
This is true. ticktime is only useful for end users if they want to trade off efficiency for a better timer function. 50ms is probably a sane default that is good enough (and once again, no other implementation lets this bubble up to user land).
Does anyone have any feedback on how this implemented; ie, putting this logic in mch_inchar vs. RealWaitForChar? Another issue is that vim makes other blocking calls that will block the timers.
I've attached our latest patch. We moved some of the logic down to the level of poll/select (RealWaitForChar) using MAY_LOOP which is similar to what the scheme bindings do; it is cleaner and works better. We also changed the eval call to use do_cmdline_cmd which is far more convenient. Now, the timeouts don't have to be user defined functions. Also, all timeouts can now be disabled by setting 'ticktime' to -1. We discussed disabling intervals that fail, but we ultimately decided that the onus should be on programmers to make sane plugins and canceling intervals is unexpected behavior. Finally, the new features are now documented. Please let us know what you think.
Why do you keep calling it async?
Also other issues:
1. async.c does not follow coding style:
1.1. each 8 indentation spaces should be replaced with a tab
1.2. each opening figure brace should be present on its own line
1.3. no (void) in function declaration
2. async.h should be split into globals.h and proto/async.pro. async.h itself is to be removed.
3. Never use gettimeofday for intervals. There are special monotonic clocks in all operating system, for linux it is clock_gettime(CLOCK_MONOTONIC_RAW). What you use is subject for ntp daemon/ntpclient updates.
4. There is a proper way to disable a setting. Putting the whole setting into an #ifdef is not the one.
5. Documentation of 'ticktime' lacks references to +smth. I repeat, do not call smth "async". Not until you will make it execute asynchronously.
6. I do not see patch for the file with a list of features.
Why do you keep calling it async?
Now please show how you can run one sequence of commands interleaved with the other with this patch. Function that runs with given interval blocks everything until it completes, same for autocommands. You cannot say you implement preemptive multitasking if you have to wait for one task to complete before starting the other. It is not async feature.
You can, of course, do some hacks with saving a state, exiting and resuming, but you will then be forced to *emulate* preemption without any support from vim. I used to have some simple emulation even without such patch.
Note though that I would really like it merged. But not with the current name.
You can take the code from python: since 3.3 it has monotonic() function defined in C code in time module: pymonotonic function in Modules/timemodule.c. It has implementation for windows as well. Linux implementation used CLOCK_MONOTONIC (or, with higher priority, CLOCK_HIRES which is not mentioned in my clock_gettime man page; I guess there is a reason for it) without _RAW, though I would write
#ifdef CLOCK_MONOTONIC_RAW
CLOCK_MONOTONIC_RAW
#else
CLOCK_MONOTONIC
#endif
in function arguments. But it is not as severe as issue with gettimeofday: you cannot see 2 centuries hop with CLOCK_MONOTONIC, maximum slightly increased/decreased second duration when NTP daemon does his adjustments.
By the way, two pairs of figure braces may be removed from insert_timeout. Not that it is required by coding style, but this makes code look better.
> Bram,
>
> We now use long longs for keeping track of the timers. We have also documented the new functions and some of their limitations.
>
>
> Please let us know if you have further feedback.
>
> -Matt
>
I can suggest looking at VimTry* functions in src/if_py_both.h: they are used to transform all vim errors into python exceptions. Purge the python stuff and you will get C functions for wrapping arbitrary code into try/catch.
So, if I install a plugin using this feature, and some weird sequence of events puts it into an infinite loop, I just need to kill Vim and lose my work?
I really, really want a way to force all timers to stop firing. Maybe not CTRL-C but SOMETHING should be possible for the user to kill misbehaving periodic tasks.
Ben,
Ctrl-C will kill a given task, but it won't cancel future timeouts (if it was an interval). Ctrl-C can't do that or else it would happen accidentally since the end user should never notice timers firing to begin with. I'm open to suggestions, but you are asking for a solution to a problem that doesn't exist in any other text editor or IDE.
-Matt
That's because other editors and IDEs that have periodic timers don't freeze all user input while waiting for the periodic task to finish.
You're right about CTRL-C canceling all future timeouts would be a bad thing, since that would happen accidentally more than on purpose.
So probably a new keyboard command is needed. It can't be just a function or ex command, because if user input is frozen due to a bad timer, the user will never see the command-line.
Or, if a timer is interrupted with CTRL-C, you could prompt to continue running timers. Kind of like some browsers prompt you "an error occurred running javascript, continue running scripts on this page?" when they load a bad website.
We can change the vocabulary if you like, but we wanted to follow a pattern that developers were already familiar with. We used the JavaScript names of setTimeout/setInterval/cancelTimeout. Sublime Text uses the same names. Emacs has a similar API.
>
>
> There is no need for 'ticktime', one can compute the time until the next
>
> event and use that. It's also more accurate and has less overhead.
>
Matt is working on a fix for this.
>
> Like others I'm worried that some plugin creates a timer with a problem,
>
> and it keeps triggering over and over. The user must have a way to stop
>
> it.
We'll change ctrl+c so that it stops future intervals as well as canceling the current timer. That should prevent Vim from hanging and let users save their work.
We thought about something more complicated, such as showing a list of pending timers and letting the user cancel them one-by-one. Unfortunately, users can only guess at the problem timer(s). They'd need the equivalent of `top` to find the culprit(s). Worse, canceling the wrong one could easily break a well-behaved plugin. The only fix for the well-behaved plugin would be to restart Vim.
Most other editors and frameworks have taken the stance that with great power comes great responsibility. If a timer causes problems, then it causes problems. Fault lies with the developer of the code that set the timer. Sublime Text, Emacs, JavaScript, Twisted Python (and more) work this way.
> Also, there will be plugins that have interference with what some
>
> timer is doing, there needs to be a way to temporary disable them.
>
> It might be required to give the timers a name instead of an ID, so that
>
> specific timers can be disabled by name, instead of having to lookup
>
> their ID (how would one do that anyway?).
>
I think names for timers would cause more problems than they solve. Plugins could conflict with each other if they chose the same names for their timers. IDs assigned at runtime make conflicts impossible. Also, the pattern of returning an ID is not foreign to those who have used other setTimeout implementations.
>
>
> What happens with timers when at a prompt, e.g. for ":s/pat/repl/gc"?
>
>
Timers are run while the prompt is prompting. It's the responsibility of the plugin developer to not cause problems. Great power, great responsibility and all that.
>
>
>
> This is the kind of functionality where we may need to fix many bugs
>
> that are uncovered over time, like with the conceal feature. I am not
>
> going to include it before I'm convinced it works properly.
>
>
I don't think there will be many issues, but I do see where you're coming from. You're responsible for maintaining Vim indefinitely, while people like Matt and myself can submit a patch and disappear. A conservative approach to new features is entirely understandable. If I were in your position, I'd probably behave similarly.
But please be aware of the costs of *not* adding timers to Vim. Without them, whole categories of plugins are simply impossible: Indexing files in the background. Collaborative editing. Deep integration with debuggers and browsers. That's just what I can imagine in a few minutes. The entire Vim community could imagine (and build) so much more.
There's a new generation of plugins waiting to be made. They just need timers. I hope you'll soften your stance on this patch.
Sincerely,
The named has been changed from async to timers everywhere.
>
>
> There is no need for 'ticktime', one can compute the time until the next
>
> event and use that. It's also more accurate and has less overhead.
Agreed. 'ticktime' is now the minimum time between calls to call_timeouts() and is set to 20ms. Vim waits the appropriate amount of time (which could be ticktime, -1, 0, or the time until the next timeout). Say we have intervals every .5ms; the overhead of calling select that often is way too high given we don't care about that level of precision.
>
>
> Like others I'm worried that some plugin creates a timer with a problem,
>
> and it keeps triggering over and over. The user must have a way to stop
>
> it.
The new behavior cancels intervals if they error in any way. Additionally, upon futher thought, there is no way around ctrl-C. That is, we have to use signals and ctrl-c (SIGINT) is the only convenient one. In the newest version, ctrl-c will cancel a given timeout/interval and all future calls.
> Also, there will be plugins that have interference with what some
>
> timer is doing, there needs to be a way to temporary disable them.
The new solution is good enough from the user's perspective. If a rogue timer exists, the only possible action is to cancel it, save buffers, and restart vim after removing the offending plugin. No plugin is going to continue to work after its timers are canceled.
Disabling a specific timer would violate internal assumptions plugins make. We could disable all timers for X seconds, but this functionality isn't necessary and would result in plugins battling each other over when, if ever, timeouts should be called.
>
> It might be required to give the timers a name instead of an ID, so that
>
> specific timers can be disabled by name, instead of having to lookup
>
> their ID (how would one do that anyway?).
I don't follow. Letting plugins cancel non-local timeouts will result in plugin wars, unintentional or otherwise. I don't ever see a user canceling a given timeout based on some list; at the very least, its a poor user experience. Why should users ever need to know timers exist in the first place? Allowing mutable global state with side effects *across plugins* causes more problems than it solves.
> Like others I'm worried that some plugin creates a timer with a problem,>
> and it keeps triggering over and over. The user must have a way to stop
>
> it.
The new behavior cancels intervals if they error in any way. Additionally, upon further thought, there is no way around ctrl-C. That is, we have to use signals and ctrl-c (SIGINT) is the only convenient one. In the newest version, ctrl-c will cancel a given timeout/interval and all future calls.
> Also, there will be plugins that have interference with what someThe new solution is good enough from the user's perspective. If a rogue timer exists, the only possible action is to cancel it, save buffers, and restart vim after removing the offending plugin. No plugin is going to continue to work after its timers are canceled.
>
> timer is doing, there needs to be a way to temporary disable them.
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to a topic in the Google Groups "vim_dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vim_dev/-4pqDJfHCsM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vim_dev+u...@googlegroups.com.