[PATCH] Asynchronous functions (settimeout, setinterval, and cancelinterval)
Geoff Greer
let timeout_id = settimeout(700, 'echo("hello")')
To cancel the timeout before it's fired:
canceltimeout(timeout_id)
setinterval() also returns an id that can be used with canceltimeout.
The reason for this patch is simple: asynchronous functionality is needed to implement real-time collaborative editing in Vim. This is one of the most voted-for features (see http://www.vim.org/sponsor/vote_results.php).
Along with Matt Kaniaris, I founded Floobits to build real-time collaboration into every editor. We wrote a plugin for Vim, but we had to use hacks to get async behavior (abusing feedkeys or client-server). These methods had side-effects such as breaking leaderkeys or other shortcuts. After a lot of experimenting, we decided to try patching Vim.
Since Vim is character-driven, we had to munge some low-level input functions to get the desired behavior. We changed gui_wait_for_chars() and mch_inchar() so that call_timeouts() is run every ticktime milliseconds. The default ticktime is 100ms.
This patch isn't finished yet, but it works on unix-based OSes. If the reaction is positive, our intention is to change mch_inchar() (or something similar) in other OS-specific files. That will get async functions working for everyone.
Even if our patch isn't the best approach, we'd love to help get async functions in Vim. Doing so will open the door to a lot of cool plugins.
Oh, and this is the first time either myself or Matt have submitted a patch to Vim, so please be gentle.
Sincerely,
Geoff Greer
Tony Mechelynck
Wrong:
void
insert_timeouts(timeout_T *to) {
timeout_T *cur = timeouts
timeout_T *prev = NULL
OK:
/*
* Explanation of what the function is used for
* and of how to call it
*/
void
insert_timeouts(to)
timeout_T *to; /* short comment about to */
{
timeout_T *cur = timeouts; /* short comment about cur */
timeout_T *prev = NULL; /* short comment about prev */
NOTE: Don't use ANSI style function declarations. A few people still
have to
use a compiler that doesn't support it.
Best regards,
Tony.
--
Consensus Terrorism:
The process that decides in-office attitudes and behavior.
-- Douglas Coupland, "Generation X: Tales for an Accelerated
Culture"
Geoff Greer
Thanks for the feedback. I've attached a new patch with some basic comments and non-ANSI C function definitions.
I'd really like to get feedback on the changes to gui_wait_for_chars() and mch_inchar(). That's where the meat of this patch is.
Sincerely,
Geoff
mattn
I live this patch. Bram, we discused about this similar topic at several times. And you didn't answer whether vim will include this feature clearly.
However, I think this feature is extendable way which vim get modern software designs. Let's include this feature!
Thanks.
- Yasuhiro Matsumoto
Ben Fritz
Forgive me if I'm wrong, I didn't spend more than a few minutes looking over the patch.
It looks like this is not really "asynchronous" behavior, but rather adding a timer-like hook into the main loop. Pro: no worries about thread safety, race conditions, etc. Con: if the function called by the timer event takes a long time, Vim hangs until it's done.
If I'm correct, and this gets included, the help will need to be very explicit about making sure to limit the time taken by the function called! And there should definitely be a way to interrupt the action.
Bjorn Tipling
Ben, why should asynchronous receive any type of restriction not already placed on code that runs some milliseconds after a key press (an existing hook). Cancelling asynchronous requests can lead to unpredictable behavior is probably never desirable. I've already experienced plugins that block for too long and what I do is, I just remove the plugin.
Ben Fritz
> I am pretty excited about this patch and hope it makes it in.
>
>
> Ben, why should asynchronous receive any type of restriction not already placed on code that runs some milliseconds after a key press (an existing hook).
And most of these can be interrupted with CTRL-C. I am not certain whether the async functions will be since you added their processing to the functions which get keypresses (I think...I'm not actually familiar with most of Vim's code). I'm not saying they're not...only that the need to be. We've already had a few "Vim hangs on this regex" bugs in 7.4, and I could easily see a plugin that uses a search() call inside one of these timer functions. Or maybe it calls an external tool that is deadlocked for some reason. Or any number of other things could go wrong that the user doesn't want to wait for.
And I'm more worried about function that takes seconds, or minutes, to complete, than I am about milliseconds. My point was that users of these functions need to be aware that the action is not actually asynchronous in the sense of running in the background and allowing user input to continue in the foreground; you just mean it can be synced to a timer rather than a specific user input, correct?
> Cancelling asynchronous requests can lead to unpredictable behavior is probably never desirable. I've already experienced plugins that block for too long and what I do is, I just remove the plugin.
>
If you're typing something, suddenly Vim stops responding to any keyboard input, you've waited most of a minute for Vim to start responding, and you have unsaved work you wish to keep...it can be more desirable to interrupt the action and then continue using Vim long enough to save your work and restart.
Maybe if the user interrupts the action with CTRL-C you need to set a flag to remove the timer from the list, to prevent it firing off again in a few milliseconds?
kans
Ben,
You are correct in that we added a timer to the main loop. Looking over the code once again, I think we should have altered the calls to select/poll instead, but lets discuss the practical effect of this patch since we can work out the details some time later.
Async does not imply parallelism nor safety- it only provides the illusion of concurrency. Idle cpu time spent waiting is instead used to perform some other task. Async frameworks make no such guarantee that every call not block the event loop; it is incumbent upon the programmer to ensure that the event loop is not blocked indefinitely. This is the definition and standard interface of async programming for every popular framework I can think of that lives in a scripting language- JavaScript, nodejs, twisted, event machine, and luvit.
The current alternative in Vim is to block forever, or worse yet, not write useful software because it is impossible to do so as a Vim plugin. With a proper settimeout, plugins have the option of returning control to the main loop and checking in latter. I'd suggest a settimeout on the settimeouts is not needed; plugins which cause too much trouble will either be fixed or uninstalled.
Ben Fritz
> On Tuesday, September 3, 2013 9:04:40 AM UTC-7, Ben Fritz wrote:
> > On Sunday, September 1, 2013 8:42:25 PM UTC-5, Geoff Greer wrote:
> > > This patch adds asynchronous functions to vimscript. If you want to perform an action in 700ms, simply:
> > >
> > > let timeout_id = settimeout(700, 'echo("hello")')
> > >
> > > To cancel the timeout before it's fired:
> > >
> > > canceltimeout(timeout_id)
> > >
> > > setinterval() also returns an id that can be used with canceltimeout.
> > >
> > > The reason for this patch is simple: asynchronous functionality is needed to implement real-time collaborative editing in Vim. This is one of the most voted-for features (see http://www.vim.org/sponsor/vote_results.php).
> > >
> > > Along with Matt Kaniaris, I founded Floobits to build real-time collaboration into every editor. We wrote a plugin for Vim, but we had to use hacks to get async behavior (abusing feedkeys or client-server). These methods had side-effects such as breaking leaderkeys or other shortcuts. After a lot of experimenting, we decided to try patching Vim.
> > >
> > > Since Vim is character-driven, we had to munge some low-level input functions to get the desired behavior. We changed gui_wait_for_chars() and mch_inchar() so that call_timeouts() is run every ticktime milliseconds. The default ticktime is 100ms.
> > >
> > > This patch isn't finished yet, but it works on unix-based OSes. If the reaction is positive, our intention is to change mch_inchar() (or something similar) in other OS-specific files. That will get async functions working for everyone.
> > >
> > > Even if our patch isn't the best approach, we'd love to help get async functions in Vim. Doing so will open the door to a lot of cool plugins.
> > >
> > > Oh, and this is the first time either myself or Matt have submitted a patch to Vim, so please be gentle.
> > >
> > > Sincerely,
> > >
> > > Geoff Greer
> >
> > Forgive me if I'm wrong, I didn't spend more than a few minutes looking over the patch.
> >
> > It looks like this is not really "asynchronous" behavior, but rather adding a timer-like hook into the main loop. Pro: no worries about thread safety, race conditions, etc. Con: if the function called by the timer event takes a long time, Vim hangs until it's done.
> >
> > If I'm correct, and this gets included, the help will need to be very explicit about making sure to limit the time taken by the function called! And there should definitely be a way to interrupt the action.
>
> Ben,
>
> You are correct in that we added a timer to the main loop. Looking over the code once again, I think we should have altered the calls to select/poll instead, but lets discuss the practical effect of this patch since we can work out the details some time later.
>
I like the practical implications of being able to respond to a timer event in a plugin. Polling is useful for many, many purposes, and doing it by shelling out and calling back with --remote calls is awkward at best.
> Async does not imply parallelism nor safety- it only provides the illusion of concurrency. Idle cpu time spent waiting is instead used to perform some other task. Async frameworks make no such guarantee that every call not block the event loop; it is incumbent upon the programmer to ensure that the event loop is not blocked indefinitely. This is the definition and standard interface of async programming for every popular framework I can think of that lives in a scripting language- JavaScript, nodejs, twisted, event machine, and luvit.
>
I'm more familiar with C/C++/Java, and even shell scripting, where "async" means it doesn't block the main program flow, as wikipedia says on http://en.wikipedia.org/wiki/Asynchrony: "Asynchronous actions are actions executed in a non-blocking scheme, allowing the main program flow to continue processing."
So, in my mind..."async" normally implies parallelism.
You've created an interface that is probably close enough, if developers are careful not to allow their timer callbacks to take too long, or use them to spawn and monitor a truly async process via the shell or python or whatever.
The nice thing is that you thus avoid the problem of concurrent access/thread safety, because you've not used multiple threads or processes at all. The not-so-nice thing is the inability to call long-running worker functions with your interface.
> The current alternative in Vim is to block forever, or worse yet, not write useful software because it is impossible to do so as a Vim plugin. With a proper settimeout, plugins have the option of returning control to the main loop and checking in latter.
I agree, we need something. That concurrent editing thing looked really neat! And doubtless it would be much more robust/easier to maintain with timer events in Vim.
> I'd suggest a settimeout on the settimeouts is not needed; plugins which cause too much trouble will either be fixed or uninstalled.
Possibly; but I'd rather not find the bug in the plugin by losing a lot of work when my Vim goes into an infinite loop that can't be interrupted.
kans
Ben,
Thanks for the explanation. End users will probably never directly call set_timeout. Hopefully, plugin authors know better than to make a call that can block forever. Good documentation will help.
Maybe control-c should cancel intervals. As you mentioned, this would let users save stuff and restart vim. At worse, plugins would be dumped into an inconsistent state that could cause more harm. Also, canceling internal callbacks is quite opaque to users. I don't see any great options for dealing with runaway plugins.
-kans
Thanks for explanation.
David Fishburn
Maybe control-c should cancel intervals. As you mentioned, this would let users save stuff and restart vim. At worse, plugins would be dumped into an inconsistent state that could cause more harm. Also, canceling internal callbacks is quite opaque to users. I don't see any great options for dealing with runaway plugins.
Bram Moolenaar
A few thoughts about this patch.
This is not asynchronous but these are timed commands. It should be
clear that the commands are executed in the "idle" loop, when Vim waits
for a character to be typed.
There might be two types: One that has high priority, and gets handled
even when the user has typed something. Another that has lower
priority, only gets handled when waiting for the user to type.
As someone already mentioned: If the command takes too much time, Vim
will get stuck. E.g. when it reads from a socket. It might be possible
to have this kind of work done in another thread, e.g. using Python.
The documentation is still missing. A lot of details need to be
explained there.
It must be possible to get a list of these commands, so that they can be
cancelled without having the id. This will then possibly break the
plugin that installed the command, so this gets messy quickly.
The code uses "long" to store msec, but you need 64 bits for that and
long can be 32 bits.
I don't see the need for 'ticktime'. The remaining time until the next
timeout can be computed. If it's smaller than 'updatetime' then wait
for that long.
--
hundred-and-one symptoms of being an internet addict:
170. You introduce your wife as "my_...@home.wife" and refer to your
children as "forked processes."
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
Bjorn Tipling
> will get stuck. E.g. when it reads from a socket. It might be possible
> to have this kind of work done in another thread, e.g. using Python.
Hey check this out:
au CursorHold * call BlockForever()
" Blocks forever
function! BlockForever()
let c = 1
while c >= 0
let c += 1
endwhile
endfunction
You can already do this blocking, I've already experienced it with plugins I've removed. I ask again why setTimeout should be treated differently than CursorHold? You can cntrl-c out of CursorHold, but just then make that a requirement for settimeout.
> This is not asynchronous but these are timed commands. It should be
> clear that the commands are executed in the "idle" loop, when Vim waits
> for a character to be typed.
As was already pointed out, this is what asynchronous execution looks like in JavaScript. JavaScript in the browser and elsewhere has only one thread of execution, and the behavior of setTimeout in these contexts is similar to this patch. Whatever you want to call this patch, it is very powerful, and vim is missing it. Try implementing a clock for the status line in current vim. It updates only after a keystroke. With this patch, you can have a real clock, that shows you the actual time! Not the time you last pressed a key.
Try this in a Chrome console:
function blockForever() {
var c = 1;
while (c > 0) {
c++;
}
}
window.setTimeout(blockForever, 1000);
You get the same behavior that you would from this patch, the same behavior that CursorHold gives you.
I agree there needs to be documentation on how to use this. I've built vim with this patch and was able to get one plugin that uses setinterval to work, but was not able to get my own attempt to work after a quick attempt.
On Thursday, September 5, 2013 4:26:41 AM UTC-7, Bram Moolenaar wrote:
> A few thoughts about this patch.
>
>
>
> This is not asynchronous but these are timed commands. It should be
>
> clear that the commands are executed in the "idle" loop, when Vim waits
>
> for a character to be typed.
>
>
>
> There might be two types: One that has high priority, and gets handled
>
> even when the user has typed something. Another that has lower
>
> priority, only gets handled when waiting for the user to type.
>
>
>
> As someone already mentioned: If the command takes too much time, Vim
>
> will get stuck. E.g. when it reads from a socket. It might be possible
>
> to have this kind of work done in another thread, e.g. using Python.
>
>
>
> The documentation is still missing. A lot of details need to be
>
> explained there.
>
>
>
> It must be possible to get a list of these commands, so that they can be
>
> cancelled without having the id. This will then possibly break the
>
> plugin that installed the command, so this gets messy quickly.
>
>
>
> The code uses "long" to store msec, but you need 64 bits for that and
>
> long can be 32 bits.
>
>
>
> I don't see the need for 'ticktime'. The remaining time until the next
>
> timeout can be computed. If it's smaller than 'updatetime' then wait
>
> for that long.
>
>
>
>
>
> --
>
> hundred-and-one symptoms of being an internet addict:
>
> 170. You introduce your wife as "my_la.wife" and refer to your
>
> children as "forked processes."
>
>
>
> /// Bram Moolenaar -- Bra....net -- http://www.Moolenaar.net \\\
Yasuhiro MATSUMOTO
can't get events safety from the thread.
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups
> "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vim_dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
- Yasuhiro Matsumoto
Ben Fritz
> > As someone already mentioned: If the command takes too much time, Vim
> > will get stuck. E.g. when it reads from a socket. It might be possible
> > to have this kind of work done in another thread, e.g. using Python.
>
>
> Hey check this out:
>
> au CursorHold * call BlockForever()
>
>
> " Blocks forever
> function! BlockForever()
> let c = 1
> while c >= 0
> let c += 1
> endwhile
> endfunction
>
> You can already do this blocking, I've already experienced it with plugins I've removed. I ask again why setTimeout should be treated differently than CursorHold? You can cntrl-c out of CursorHold, but just then make that a requirement for settimeout.
>
Exactly. I think as long as you can interrupt the timer with a keystroke it should be fine. But since you call your timer in between waiting for a keystroke, won't that prevent CTRL-C from interrupting it? Maybe it won't...I haven't tried it. I don't think we have any unique constraints, I just think that (a) it should be mentioned in the help that the timer blocks user input while processing, it's not a true asynchronous process and (b) it is required that CTRL-C or similar can interrupt the timer like it can already for autocmds, etc.
kans
> On Thursday, September 5, 2013 8:44:15 AM UTC-5, Bjorn Tipling wrote:
> > > As someone already mentioned: If the command takes too much time, Vim
> > > will get stuck. E.g. when it reads from a socket. It might be possible
> > > to have this kind of work done in another thread, e.g. using Python.
> >
> >
> > Hey check this out:
> >
> > au CursorHold * call BlockForever()
> >
> >
> > " Blocks forever
> > function! BlockForever()
> > let c = 1
> > while c >= 0
> > let c += 1
> > endwhile
> > endfunction
> >
> > You can already do this blocking, I've already experienced it with plugins I've removed. I ask again why setTimeout should be treated differently than CursorHold? You can cntrl-c out of CursorHold, but just then make that a requirement for settimeout.
> >
>
> Exactly. I think as long as you can interrupt the timer with a keystroke it should be fine. But since you call your timer in between waiting for a keystroke, won't that prevent CTRL-C from interrupting it? Maybe it won't...I haven't tried it.
ctrl-c works (in this patch) because the OS converts it into a SIGINT which vim knows how to handle.
>There might be two types: One that has high priority, and gets handled
even when the user has typed something. Another that has lower
priority, only gets handled when waiting for the user to type.
As mattn pointed out, this won't work unless the internals of vim are threadsafe.
> It must be possible to get a list of these commands, so that they can be
>
> cancelled without having the id.
This wouldn't be much work, but to my knowledge, no other implementation has this ability.
> I don't see the need for 'ticktime'. The remaining time until the next
>
> timeout can be computed. If it's smaller than 'updatetime' then wait
>
> for that long.
This is true. ticktime is only useful for end users if they want to trade off efficiency for a better timer function. 50ms is probably a sane default that is good enough (and once again, no other implementation lets this bubble up to user land).
Does anyone have any feedback on how this implemented; ie, putting this logic in mch_inchar vs. RealWaitForChar? Another issue is that vim makes other blocking calls that will block the timers.
kans
I've attached our latest patch. We moved some of the logic down to the level of poll/select (RealWaitForChar) using MAY_LOOP which is similar to what the scheme bindings do; it is cleaner and works better. We also changed the eval call to use do_cmdline_cmd which is far more convenient. Now, the timeouts don't have to be user defined functions. Also, all timeouts can now be disabled by setting 'ticktime' to -1. We discussed disabling intervals that fail, but we ultimately decided that the onus should be on programmers to make sane plugins and canceling intervals is unexpected behavior. Finally, the new features are now documented. Please let us know what you think.
Nikolay Pavlov
Why do you keep calling it async?
Also other issues:
1. async.c does not follow coding style:
1.1. each 8 indentation spaces should be replaced with a tab
1.2. each opening figure brace should be present on its own line
1.3. no (void) in function declaration
2. async.h should be split into globals.h and proto/async.pro. async.h itself is to be removed.
3. Never use gettimeofday for intervals. There are special monotonic clocks in all operating system, for linux it is clock_gettime(CLOCK_MONOTONIC_RAW). What you use is subject for ntp daemon/ntpclient updates.
4. There is a proper way to disable a setting. Putting the whole setting into an #ifdef is not the one.
5. Documentation of 'ticktime' lacks references to +smth. I repeat, do not call smth "async". Not until you will make it execute asynchronously.
6. I do not see patch for the file with a list of features.
Thomas
Why do you keep calling it async?
http://cs.brown.edu/courses/cs168/f12/handouts/async.pdf
Bjorn Tipling
Nikolay Pavlov
Now please show how you can run one sequence of commands interleaved with the other with this patch. Function that runs with given interval blocks everything until it completes, same for autocommands. You cannot say you implement preemptive multitasking if you have to wait for one task to complete before starting the other. It is not async feature.
You can, of course, do some hacks with saving a state, exiting and resuming, but you will then be forced to *emulate* preemption without any support from vim. I used to have some simple emulation even without such patch.
Note though that I would really like it merged. But not with the current name.
kans
The new patch addresses your criticism apart from changing the name. I believe we obey Vim code conventions everywhere. We have also implemented a monotonically increasing timer for unix and osx which are the only two OSes we have on hand. Otherwise, we fall back to gettimeofday. Async is only #defined if gettimeofday exists.
Bram,
We now use long longs for keeping track of the timers. We have also documented the new functions and some of their limitations.
Please let us know if you have further feedback.
-Matt
Nikolay Pavlov
You can take the code from python: since 3.3 it has monotonic() function defined in C code in time module: pymonotonic function in Modules/timemodule.c. It has implementation for windows as well. Linux implementation used CLOCK_MONOTONIC (or, with higher priority, CLOCK_HIRES which is not mentioned in my clock_gettime man page; I guess there is a reason for it) without _RAW, though I would write
#ifdef CLOCK_MONOTONIC_RAW
CLOCK_MONOTONIC_RAW
#else
CLOCK_MONOTONIC
#endif
in function arguments. But it is not as severe as issue with gettimeofday: you cannot see 2 centuries hop with CLOCK_MONOTONIC, maximum slightly increased/decreased second duration when NTP daemon does his adjustments.
By the way, two pairs of figure braces may be removed from insert_timeout. Not that it is required by coding style, but this makes code look better.
> Bram,
>
> We now use long longs for keeping track of the timers. We have also documented the new functions and some of their limitations.
>
>
> Please let us know if you have further feedback.
>
> -Matt
>
kans
I added a monotonic timer for windows; I basically took the one from python. I also changed all the make files to include the new stuff as well. The patch is 1200 lines long now (37 files changed, 476 insertions, 24 deletions). The current state includes monotonically increasing timers for OsX, Win32, and *nix. It really doesn't make sense to add any more features, but I'm happy to do cleanup as needed.
-Matt
kans
-Matt
Nikolay Pavlov
I can suggest looking at VimTry* functions in src/if_py_both.h: they are used to transform all vim errors into python exceptions. Purge the python stuff and you will get C functions for wrapping arbitrary code into try/catch.
kans
Thanks for the help. Intervals are no longer run if they error of any sort (apart from a user interrupt which will likely happen accidentally). I was also forced to change configure.in as the monotonic timer on Linux needs to sometimes link against rt. I ran autoconf and also dumped those changes into auto/configure file. They run in both Linux and OsX.
Originally, I had hopped that this patch will be 100 lines tops, but it is closer to 600. A good chunk of that is documentation and touching the various make files. I think we have addressed all the concerns (apart from changing the name). What would it take to get this merged?
-Matt
Ben Fritz
>
> Thanks for the help. Intervals are no longer run if they error of any sort (apart from a user interrupt which will likely happen accidentally).
So, if I install a plugin using this feature, and some weird sequence of events puts it into an infinite loop, I just need to kill Vim and lose my work?
I really, really want a way to force all timers to stop firing. Maybe not CTRL-C but SOMETHING should be possible for the user to kill misbehaving periodic tasks.
kans
Ben,
Ctrl-C will kill a given task, but it won't cancel future timeouts (if it was an interval). Ctrl-C can't do that or else it would happen accidentally since the end user should never notice timers firing to begin with. I'm open to suggestions, but you are asking for a solution to a problem that doesn't exist in any other text editor or IDE.
-Matt
Bram Moolenaar
kans wrote:
> Zyx,
>
> Thanks for the help. Intervals are no longer run if they error of any
> sort (apart from a user interrupt which will likely happen
> accidentally). I was also forced to change configure.in as the
> monotonic timer on Linux needs to sometimes link against rt. I ran
> autoconf and also dumped those changes into auto/configure file. They
> run in both Linux and OsX.
>
>
> Originally, I had hopped that this patch will be 100 lines tops, but
> it is closer to 600. A good chunk of that is documentation and
> touching the various make files. I think we have addressed all the
> concerns (apart from changing the name). What would it take to get
> this merged?
I would prefer this to be called timed events instead of async, since
it's not really asynchronous. A timeout is for when you do something
and it takes too long. What this is doing is setting a timer or a
repeated timer.
There is no need for 'ticktime', one can compute the time until the next
event and use that. It's also more accurate and has less overhead.
Like others I'm worried that some plugin creates a timer with a problem,
and it keeps triggering over and over. The user must have a way to stop
it. Also, there will be plugins that have interference with what some
timer is doing, there needs to be a way to temporary disable them.
It might be required to give the timers a name instead of an ID, so that
specific timers can be disabled by name, instead of having to lookup
their ID (how would one do that anyway?).
What happens with timers when at a prompt, e.g. for ":s/pat/repl/gc"?
This is the kind of functionality where we may need to fix many bugs
that are uncovered over time, like with the conceal feature. I am not
going to include it before I'm convinced it works properly.
--
hundred-and-one symptoms of being an internet addict:
sit in front of the PC until you're just too tired to care.
Ben Fritz
That's because other editors and IDEs that have periodic timers don't freeze all user input while waiting for the periodic task to finish.
You're right about CTRL-C canceling all future timeouts would be a bad thing, since that would happen accidentally more than on purpose.
So probably a new keyboard command is needed. It can't be just a function or ex command, because if user input is frozen due to a bad timer, the user will never see the command-line.
Or, if a timer is interrupted with CTRL-C, you could prompt to continue running timers. Kind of like some browsers prompt you "an error occurred running javascript, continue running scripts on this page?" when they load a bad website.
Geoff Greer
> kans wrote:
>
>
>
> > Zyx,
>
> >
>
> > Thanks for the help. Intervals are no longer run if they error of any
>
> > sort (apart from a user interrupt which will likely happen
>
> > accidentally). I was also forced to change configure.in as the
>
> > monotonic timer on Linux needs to sometimes link against rt. I ran
>
> > autoconf and also dumped those changes into auto/configure file. They
>
> > run in both Linux and OsX.
>
> >
>
> >
>
> > Originally, I had hopped that this patch will be 100 lines tops, but
>
> > it is closer to 600. A good chunk of that is documentation and
>
> > touching the various make files. I think we have addressed all the
>
> > concerns (apart from changing the name). What would it take to get
>
> > this merged?
>
>
>
> I still have several problems with including this functionality.
>
>
>
> I would prefer this to be called timed events instead of async, since
>
> it's not really asynchronous. A timeout is for when you do something
>
> and it takes too long. What this is doing is setting a timer or a
>
> repeated timer.
>
We can change the vocabulary if you like, but we wanted to follow a pattern that developers were already familiar with. We used the JavaScript names of setTimeout/setInterval/cancelTimeout. Sublime Text uses the same names. Emacs has a similar API.
>
>
> There is no need for 'ticktime', one can compute the time until the next
>
> event and use that. It's also more accurate and has less overhead.
>
Matt is working on a fix for this.
>
> Like others I'm worried that some plugin creates a timer with a problem,
>
> and it keeps triggering over and over. The user must have a way to stop
>
> it.
We'll change ctrl+c so that it stops future intervals as well as canceling the current timer. That should prevent Vim from hanging and let users save their work.
We thought about something more complicated, such as showing a list of pending timers and letting the user cancel them one-by-one. Unfortunately, users can only guess at the problem timer(s). They'd need the equivalent of `top` to find the culprit(s). Worse, canceling the wrong one could easily break a well-behaved plugin. The only fix for the well-behaved plugin would be to restart Vim.
Most other editors and frameworks have taken the stance that with great power comes great responsibility. If a timer causes problems, then it causes problems. Fault lies with the developer of the code that set the timer. Sublime Text, Emacs, JavaScript, Twisted Python (and more) work this way.
> Also, there will be plugins that have interference with what some
>
> timer is doing, there needs to be a way to temporary disable them.
>
> It might be required to give the timers a name instead of an ID, so that
>
> specific timers can be disabled by name, instead of having to lookup
>
> their ID (how would one do that anyway?).
>
I think names for timers would cause more problems than they solve. Plugins could conflict with each other if they chose the same names for their timers. IDs assigned at runtime make conflicts impossible. Also, the pattern of returning an ID is not foreign to those who have used other setTimeout implementations.
>
>
> What happens with timers when at a prompt, e.g. for ":s/pat/repl/gc"?
>
>
Timers are run while the prompt is prompting. It's the responsibility of the plugin developer to not cause problems. Great power, great responsibility and all that.
>
>
>
> This is the kind of functionality where we may need to fix many bugs
>
> that are uncovered over time, like with the conceal feature. I am not
>
> going to include it before I'm convinced it works properly.
>
>
I don't think there will be many issues, but I do see where you're coming from. You're responsible for maintaining Vim indefinitely, while people like Matt and myself can submit a patch and disappear. A conservative approach to new features is entirely understandable. If I were in your position, I'd probably behave similarly.
But please be aware of the costs of *not* adding timers to Vim. Without them, whole categories of plugins are simply impossible: Indexing files in the background. Collaborative editing. Deep integration with debuggers and browsers. That's just what I can imagine in a few minutes. The entire Vim community could imagine (and build) so much more.
There's a new generation of plugins waiting to be made. They just need timers. I hope you'll soften your stance on this patch.
Sincerely,
kans
>
>
>
> > Zyx,
>
> >
>
> > Thanks for the help. Intervals are no longer run if they error of any
>
> > sort (apart from a user interrupt which will likely happen
>
> > accidentally). I was also forced to change configure.in as the
>
> > monotonic timer on Linux needs to sometimes link against rt. I ran
>
> > autoconf and also dumped those changes into auto/configure file. They
>
> > run in both Linux and OsX.
>
> >
>
> >
>
> > Originally, I had hopped that this patch will be 100 lines tops, but
>
> > it is closer to 600. A good chunk of that is documentation and
>
> > touching the various make files. I think we have addressed all the
>
> > concerns (apart from changing the name). What would it take to get
>
> > this merged?
>
>
>
> I still have several problems with including this functionality.
>
>
>
> I would prefer this to be called timed events instead of async, since
>
> it's not really asynchronous. A timeout is for when you do something
>
> and it takes too long. What this is doing is setting a timer or a
>
> repeated timer.
The named has been changed from async to timers everywhere.
>
>
> There is no need for 'ticktime', one can compute the time until the next
>
> event and use that. It's also more accurate and has less overhead.
Agreed. 'ticktime' is now the minimum time between calls to call_timeouts() and is set to 20ms. Vim waits the appropriate amount of time (which could be ticktime, -1, 0, or the time until the next timeout). Say we have intervals every .5ms; the overhead of calling select that often is way too high given we don't care about that level of precision.
>
>
> Like others I'm worried that some plugin creates a timer with a problem,
>
> and it keeps triggering over and over. The user must have a way to stop
>
> it.
The new behavior cancels intervals if they error in any way. Additionally, upon futher thought, there is no way around ctrl-C. That is, we have to use signals and ctrl-c (SIGINT) is the only convenient one. In the newest version, ctrl-c will cancel a given timeout/interval and all future calls.
> Also, there will be plugins that have interference with what some
>
> timer is doing, there needs to be a way to temporary disable them.
The new solution is good enough from the user's perspective. If a rogue timer exists, the only possible action is to cancel it, save buffers, and restart vim after removing the offending plugin. No plugin is going to continue to work after its timers are canceled.
Disabling a specific timer would violate internal assumptions plugins make. We could disable all timers for X seconds, but this functionality isn't necessary and would result in plugins battling each other over when, if ever, timeouts should be called.
>
> It might be required to give the timers a name instead of an ID, so that
>
> specific timers can be disabled by name, instead of having to lookup
>
> their ID (how would one do that anyway?).
I don't follow. Letting plugins cancel non-local timeouts will result in plugin wars, unintentional or otherwise. I don't ever see a user canceling a given timeout based on some list; at the very least, its a poor user experience. Why should users ever need to know timers exist in the first place? Allowing mutable global state with side effects *across plugins* causes more problems than it solves.
David Fishburn
> Like others I'm worried that some plugin creates a timer with a problem,>
> and it keeps triggering over and over. The user must have a way to stop
>
> it.
The new behavior cancels intervals if they error in any way. Additionally, upon further thought, there is no way around ctrl-C. That is, we have to use signals and ctrl-c (SIGINT) is the only convenient one. In the newest version, ctrl-c will cancel a given timeout/interval and all future calls.
Cancelled 21_DB_timedEvent
> Also, there will be plugins that have interference with what someThe new solution is good enough from the user's perspective. If a rogue timer exists, the only possible action is to cancel it, save buffers, and restart vim after removing the offending plugin. No plugin is going to continue to work after its timers are canceled.
>
> timer is doing, there needs to be a way to temporary disable them.
Matthew Kaniaris
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to a topic in the Google Groups "vim_dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vim_dev/-4pqDJfHCsM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vim_dev+u...@googlegroups.com.
Bram Moolenaar
Geoff Greer wrote:
> On Sunday, September 22, 2013 6:38:32 AM UTC-7, Bram Moolenaar wrote:
> > I still have several problems with including this functionality.
> >
> > I would prefer this to be called timed events instead of async, since
> > it's not really asynchronous. A timeout is for when you do something
> > and it takes too long. What this is doing is setting a timer or a
> > repeated timer.
>
> We can change the vocabulary if you like, but we wanted to follow a
> pattern that developers were already familiar with. We used the
> JavaScript names of setTimeout/setInterval/cancelTimeout. Sublime Text
> uses the same names. Emacs has a similar API.
JavaScript wasn't designed properly, it has lots of confusing stuff, I
would never use it as an example for proper design. Only for ideas.
What is Emacs? :-)
unfortunately things break once in a while. At some point in this
design you have to think about what might go wrong and how to handle
that properly. This is not something to be done as an afterthought.
Plugins WILL interfere in ways you cannot imagine. It's naive to assume
that they will all work properly. And the plugin writer cannot possibly
overview all corner cases, no matter how clever he is. We must prepare
for some plugin, or combination of plugins, to malfunction and provide
the user with a way out.
At least it must be possible for the user to find out what is causing
problems, find the source of it and fix it or disable it. That is why
we have the :verbose command, for example to find out who last set an
option. For timers there must be at least a command that lists all
active timers, when they trigger and where they were defined.
Interrupting a timer with CTRL-C is hit-and-miss, by the time the key
press gets through another timer may be active and you kill the wrong
one. Having a prompt to ask the user what he wants to kill might help.
But also interrupt the normal working sequence. And on Windows CTRL-C
means copy...
> > This is the kind of functionality where we may need to fix many bugs
> > that are uncovered over time, like with the conceal feature. I am not
> > going to include it before I'm convinced it works properly.
>
> I don't think there will be many issues, but I do see where you're
> coming from. You're responsible for maintaining Vim indefinitely,
> while people like Matt and myself can submit a patch and disappear. A
> conservative approach to new features is entirely understandable. If I
> were in your position, I'd probably behave similarly.
>
> But please be aware of the costs of *not* adding timers to Vim.
> Without them, whole categories of plugins are simply impossible:
> Indexing files in the background. Collaborative editing. Deep
> integration with debuggers and browsers. That's just what I can
> imagine in a few minutes. The entire Vim community could imagine (and
> build) so much more.
- Indexing in the background may hang, run at 100% CPU, change files
that are being used, etc.
- Collaborative editing will use network functions, which may hang.
- Debugger integration is very complex, anything can go wrong.
> There's a new generation of plugins waiting to be made. They just need
> timers. I hope you'll soften your stance on this patch.
--
Women are probably the main cause of free software starvation.
Nikolay Pavlov
I would also add information about times to such command: it or another command should list average, maximum and time consumed for last call for debugging purposes. Possibly with the number of times they were triggered and maybe when they were defined (in format "N time ago" (e.g. "1 minute 5 seconds ago" and not "25.09.2013 16:16")) (assuming timers are constantly added and deleted it is good to know that timer A was defined nearly when you started to experience input lags).
> Interrupting a timer with CTRL-C is hit-and-miss, by the time the key
> press gets through another timer may be active and you kill the wrong
> one. Having a prompt to ask the user what he wants to kill might help.
> But also interrupt the normal working sequence. And on Windows CTRL-C
> means copy...
Is not interrupt happening because the job done by timer did hang? This way interrupting wrong timer is much unlikely.
>
> > > This is the kind of functionality where we may need to fix many bugs
> > > that are uncovered over time, like with the conceal feature. I am not
> > > going to include it before I'm convinced it works properly.
> >
> > I don't think there will be many issues, but I do see where you're
> > coming from. You're responsible for maintaining Vim indefinitely,
> > while people like Matt and myself can submit a patch and disappear. A
> > conservative approach to new features is entirely understandable. If I
> > were in your position, I'd probably behave similarly.
> >
> > But please be aware of the costs of *not* adding timers to Vim.
> > Without them, whole categories of plugins are simply impossible:
> > Indexing files in the background. Collaborative editing. Deep
> > integration with debuggers and browsers. That's just what I can
> > imagine in a few minutes. The entire Vim community could imagine (and
> > build) so much more.
>
> You are exactly mentioning things that can go haywire:
> - Indexing in the background may hang, run at 100% CPU, change files
> that are being used, etc.
Since the patch does not add preemptive multitasking I doubt anybody sane will write indexing in the background using these timers. But they are handy at updating in-app copy of index.
In fact if I was writing plugin doing any job mentioned here I would use timers solely for updating state after all necessary computations were done in some other thread/process. Though you cannot and should not expect other plugin writers share the same thoughts.
> - Collaborative editing will use network functions, which may hang.
> - Debugger integration is very complex, anything can go wrong.
>
> > There's a new generation of plugins waiting to be made. They just need
> > timers. I hope you'll soften your stance on this patch.
>
> I won't include it before it has been done properly.
>
> --
> Women are probably the main cause of free software starvation.
>
> /// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
> /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> \\\ an exciting new programming language -- http://www.Zimbu.org ///
> \\\ help me help AIDS victims -- http://ICCF-Holland.org ///
>
> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.
Tux.
>Java is the most used language these days and uses the Timer class.
--
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.
Tux.
Bram Moolenaar <Br...@moolenaar.net> schrieb:
>Java is the most used language these days and uses the Timer class.
Bjorn Tipling
Given that your concern is to not present a situation to the user for which they have no remedy, it seems like providing information might be a possible remedy The more information the user has, the easier it might be to figure out how to get out of a possible sticky situation. You mentioned being able to list a list of names of timers might be a workable solution for you to get this patch in? Maybe being able to remove a timer by name? Would these additions suffice? Just curious. I'm not involved in the development of this plugin, but I would like to see it make it into vim. Maybe the timer can print its name as it starts to :messages so the user might have an idea as to what is currently happening if things hang.
kans
>
> JavaScript wasn't designed properly, it has lots of confusing stuff, I
>
> would never use it as an example for proper design. Only for ideas.
Its true that JS was designed in a weekend, but it is easily the most popular event driven language (where asynchronous programming is the norm). We chose setTimeout and setInterval as our model because they are the closest thing to what we wanted, are familiar to most devs, and can be reasonably implemented inside Vim. Also, they aren't one of the bad parts (http://oreilly.com/javascript/excerpts/javascript-good-parts/bad-parts.html) :P
>
> > Timers are run while the prompt is prompting. It's the responsibility
>
> > of the plugin developer to not cause problems. Great power, great
>
> > responsibility and all that.
>
>
>
> Your attitude appears to be that you are making things work. Well,
>
> unfortunately things break once in a while. At some point in this
>
> design you have to think about what might go wrong and how to handle
>
> that properly. This is not something to be done as an afterthought.
I recognize that things do go wrong, but I don't expect end users to do anything about it other than exit and change their ~/.vimrc. Fixing broken plugins and writing sane ones is squarely the responsibility of the plugin authors. Installing plugins (working or otherwise) is up to end users, and this has always been the case. I don't see why this patch should be treated any differently.
If there is no way to diagnose or get out of infinite loops in general in Vim, what necessitates such a system exist here? Currently, any plugin can trivially create a busy loop that blocks forever. It is the job of the plugin authors to write code that doesn't do that.
> At least it must be possible for the user to find out what is causing
>
> problems, find the source of it and fix it or disable it. That is why
>
> we have the :verbose command, for example to find out who last set an
>
> option. For timers there must be at least a command that lists all
>
> active timers, when they trigger and where they were defined.
I disagree that a list of active timers is particularly helpful to end users for the following reasons:
1. Timers aren't like options or autocommands which are both static. They are mostly set at startup and never changed. autocommands exist on the human time scale. Timers come and go on the computer time scale. A well behaving plugin could realistically run a timer ever 30ms on the low end without ever bothering the user.
2. How will the user see the list of timers if Vim is blocked by a timer? Diagnosing troublesome timers is the only time a user would want this feature.
3. How will the user identify which timer is causing the slowdown? A timeout is only fired once; previous (troublesome) timeouts won't be in the list. Moreover, the list can be huge.
4. How does the end user even know that a timer is the problem in the first place? From his perspective, all he sees is that vim is unresponsive.
5. Canceling the wrong timer will lead to even more trouble (unlike autocommands which have no expectation of ever firing) and will make diagnosing the problem even harder.
6. The ability to cancel third party timers is dangerous. Imagine if different pages in a web browser could cancel timers they didn't set.
Perhaps this feature would be OK at helping the end user diagnose slightly problematic intervals- ie, ones that block for a second or two, but it wouldn't do anything for infinite loops. I tend to think its easy enough to diagnose that problem without this feature.
If a timers list is a hard requirement, I'm willing to write the code, but I don't think it adds much.
> Interrupting a timer with CTRL-C is hit-and-miss, by the time the key
>
> press gets through another timer may be active and you kill the wrong
>
> one. Having a prompt to ask the user what he wants to kill might help.
>
> But also interrupt the normal working sequence. And on Windows CTRL-C
>
> means copy...
Yeah, this is trouble and I'm open to suggestions. What does Vim currently offer for letting users get out of infinite loops? I would like to implement a solution, but I can't think of a satisfactory one that would take less than a month to write. I would guess the difficulty in implementing something like this is why it doesn't already exist (for unresponsive vim scripts).
-Matt
Nikolay Pavlov
On Sep 26, 2013 3:30 AM, "kans" <mkan...@gmail.com> wrote:
>
> > Java is the most used language these days and uses the Timer class.
> >
> > JavaScript wasn't designed properly, it has lots of confusing stuff, I
> >
> > would never use it as an example for proper design. Only for ideas.
>
> Its true that JS was designed in a weekend, but it is easily the most popular event driven language (where asynchronous programming is the norm). We chose setTimeout and setInterval as our model because they are the closest thing to what we wanted, are familiar to most devs, and can be reasonably implemented inside Vim. Also, they aren't one of the bad parts (http://oreilly.com/javascript/excerpts/javascript-good-parts/bad-parts.html) :P
>
> >
> > > Timers are run while the prompt is prompting. It's the responsibility
> >
> > > of the plugin developer to not cause problems. Great power, great
> >
> > > responsibility and all that.
> >
> >
> >
> > Your attitude appears to be that you are making things work. Well,
> >
> > unfortunately things break once in a while. At some point in this
> >
> > design you have to think about what might go wrong and how to handle
> >
> > that properly. This is not something to be done as an afterthought.
>
> I recognize that things do go wrong, but I don't expect end users to do anything about it other than exit and change their ~/.vimrc. Fixing broken plugins and writing sane ones is squarely the responsibility of the plugin authors. Installing plugins (working or otherwise) is up to end users, and this has always been the case. I don't see why this patch should be treated any differently.
>
> If there is no way to diagnose or get out of infinite loops in general in Vim, what necessitates such a system exist here? Currently, any plugin can trivially create a busy loop that blocks forever. It is the job of the plugin authors to write code that doesn't do that.
This is only true for python. Infinite loop can be canceled by CTRL-C if it is written in VimL and author is not intercepting interrupts just to make the loop unkillable.
> > At least it must be possible for the user to find out what is causing
> >
> > problems, find the source of it and fix it or disable it. That is why
> >
> > we have the :verbose command, for example to find out who last set an
> >
> > option. For timers there must be at least a command that lists all
> >
> > active timers, when they trigger and where they were defined.
>
> I disagree that a list of active timers is particularly helpful to end users for the following reasons:
>
> 1. Timers aren't like options or autocommands which are both static. They are mostly set at startup and never changed. autocommands exist on the human time scale. Timers come and go on the computer time scale. A well behaving plugin could realistically run a timer ever 30ms on the low end without ever bothering the user.
Plugin should not be expected to be well behaving.
> 2. How will the user see the list of timers if Vim is blocked by a timer? Diagnosing troublesome timers is the only time a user would want this feature.
He will use CTRL-C. Then see the list.
> 3. How will the user identify which timer is causing the slowdown? A timeout is only fired once; previous (troublesome) timeouts won't be in the list. Moreover, the list can be huge.
Use string identifiers as suggested. This is how you can collect stats I said to be fine to see in previous message.
> 4. How does the end user even know that a timer is the problem in the first place? From his perspective, all he sees is that vim is unresponsive.
If he is not already experienced he will be asked to launch command that shows stats when he comes with his problem to stackoverflow.
> 5. Canceling the wrong timer will lead to even more trouble (unlike autocommands which have no expectation of ever firing) and will make diagnosing the problem even harder.
You can undefine plugin functions. If one wants to make problems for the plugin he already has enough capabilities to do this.
> 6. The ability to cancel third party timers is dangerous. Imagine if different pages in a web browser could cancel timers they didn't set.
Commands, functions, autocommands, mappings - they all can be undefined/altered by a third party timers. This is less dangerous then expecting deferred function to take 30 ms when launched.
>
> Perhaps this feature would be OK at helping the end user diagnose slightly problematic intervals- ie, ones that block for a second or two, but it wouldn't do anything for infinite loops. I tend to think its easy enough to diagnose that problem without this feature.
>
> If a timers list is a hard requirement, I'm willing to write the code, but I don't think it adds much.
>
> > Interrupting a timer with CTRL-C is hit-and-miss, by the time the key
> >
> > press gets through another timer may be active and you kill the wrong
> >
> > one. Having a prompt to ask the user what he wants to kill might help.
> >
> > But also interrupt the normal working sequence. And on Windows CTRL-C
> >
> > means copy...
>
> Yeah, this is trouble and I'm open to suggestions. What does Vim currently offer for letting users get out of infinite loops? I would like to implement a solution, but I can't think of a satisfactory one that would take less than a month to write. I would guess the difficulty in implementing something like this is why it doesn't already exist (for unresponsive vim scripts).
I am wondering why you are acting here like if CTRL-C does not work. From my experience this always helps against problematic VimL scripts (but not python plugins; likely ruby/lua/perl/... have the same property).
It makes sense though to collect such stats only on request. Currently there is only one thing that can add problems without easy method (I mean profiling) to determine that it is it: statusline that does not use VimL functions. AFAIR you execute Ex commands in timer. If I am right how one is going to debug if problematic timer is running only :ruby $command_t.update_index (an example, I do not claim Command-T has such method) (except for the usual disabling of the plugins)? Also are timers run shown when you :set verbose to some high enough value?
It would be fine if such stats collection functions are written in a way that can be later attached to autocommands: if e.g. InsertCharPre event fires only some python/... script this will also come unprofilable and only seen in verbose mode.
Note that stats provide only information for the user. Developer in such cases will use in-lang profiling capabilities.
> -Matt
kans
As Bram pointed out ctrl-c doesn't work on windows. I'm not going to try to fix that.
Also, as Bram pointed out, ctrl-c breaking the normal flow (to ask the user if he really wanted to cancel a timeout) is annoying. Instead, we print a warning when a user cancels a timer via ctrl-c.
The newest patch includes an ex command that lists all pending timeouts and a bunch of meta info around them. We could add extra info about running times, but the current patch is already 1400 lines is good enough.
-Matt
Ben Fritz
>
> As Bram pointed out ctrl-c doesn't work on windows. I'm not going to try to fix that.
>
CTRL-C works on Windows as long as a user doesn't source the infamous mswin.vim somewhere in their config. Or if they do, they need to edit out the mapping for CTRL-C.
kans
I believe we have addressed all major concerns- the last one being the ex_timers command which shows all pending timers. If we have missed something, or you have other concerns, please let us know. As I wrote before, we'd really like to see this feature in Vim.
-Matt
kans
We recently stumbled into one strange bug with timers. Everything we've thrown at them works as expected apart from :Explore. When run in a timer, netrw stuff slows down by about two orders of magnitude. The commands for timers are executed like so with no other setup:
do_cmdline_cmd(timeouts->cmd)
None of the extra time is spent in self and the cpu usage doesn't spike. There isn't anything too crazy in Instruments.app (strace). Does anyone have any idea about the cause? I've included some basis profiling info below:
:call settimeout(0, "Explore ~")
count total (s) self (s) function
1 2.620059 0.027059 netrw#Explore()
1 2.593000 0.000131 netrw#LocalBrowseCheck()
1 2.446097 0.000347 <SNR>26_NetrwBrowse()
1 1.443353 0.062179 <SNR>26_PerformListing()
1 0.978938 0.000448 <SNR>26_NetrwGetBuffer()
1 0.978304 0.000426 <SNR>26_NetrwEnew()
3 0.957588 0.001165 <SNR>26_NetrwOptionRestore()
1 0.748296 <SNR>26_NetrwSetSort()
1 0.271453 <SNR>26_NetrwListHide()
2 0.168370 0.042646 <SNR>13_SynSet()
1 0.128560 0.128315 <SNR>26_LocalListing()
1 0.126958 0.000222 <SNR>26_NetrwSafeOptions()
2 0.125646 netrw#NetrwRestorePosn()
1 0.083888 0.021126 netrw#NetrwSavePosn()
2 0.041997 <SNR>10_LoadFTPlugin()
1 0.021434 0.021221 <SNR>26_NetrwBookHistRead()
68 0.001621 <SNR>26_NetrwInit()
1 0.001439 0.001433 <SNR>26_NetrwMaps()
2 0.000417 <SNR>26_NetrwOptionSave()
1 0.000228 <SNR>26_LocalFastBrowser()
:Explore ~
count total (s) self (s) function
1 0.021241 0.006975 netrw#Explore()
1 0.014266 0.000074 netrw#LocalBrowseCheck()
1 0.013968 0.000296 <SNR>26_NetrwBrowse()
1 0.008547 0.000448 <SNR>26_PerformListing()
1 0.003741 0.003255 <SNR>26_NetrwBookHistSave()
1 0.003012 0.000424 <SNR>26_NetrwGetBuffer()
1 0.002976 0.002806 <SNR>26_LocalListing()
2 0.002871 0.000588 <SNR>26_NetrwEnew()
4 0.002356 0.001092 <SNR>26_NetrwOptionRestore()
1 0.001364 0.001358 <SNR>26_NetrwMaps()
1 0.001356 0.000218 <SNR>26_NetrwSafeOptions()
2 0.001330 0.000472 <SNR>13_SynSet()
1 0.001310 <SNR>26_NetrwSetSort()
68 0.001304 <SNR>26_NetrwInit()
2 0.000763 <SNR>10_LoadFTPlugin()
1 0.000630 0.000156 <SNR>4_mergelists()
3 0.000572 <SNR>26_NetrwOptionSave()
1 0.000393 0.000133 ctrlp#mrufiles#cachefile()
1 0.000334 0.000008 <SNR>4_savetofile()
1 0.000326 0.000313 ctrlp#utils#writecache()
mattn
I can't assent.
Bjorn T
B
Bram Moolenaar
Yasuhiro Matsumoto wrote:
> Bram, why you remove this patch from todo list?
>
> https://code.google.com/p/vim/source/diff?spec=svn173c9c860e42e2af31d598aa6924b99d7e73bd1b&r=173c9c860e42e2af31d598aa6924b99d7e73bd1b&format=side&path=/runtime/doc/todo.txt
>
> I can't assent.
--
A computer programmer is a device for turning requirements into
undocumented features. It runs on cola, pizza and Dilbert cartoons.
Bram Moolenaar
Christian Brabandt
On Mi, 09 Okt 2013, Bram Moolenaar wrote:
>
> Yasuhiro Matsumoto wrote:
>
> > Bram, why you remove this patch from todo list?
> >
> > https://code.google.com/p/vim/source/diff?spec=svn173c9c860e42e2af31d598aa6924b99d7e73bd1b&r=173c9c860e42e2af31d598aa6924b99d7e73bd1b&format=side&path=/runtime/doc/todo.txt
> >
> > I can't assent.
>
> There are several changes. Can you quote the one you refer to?
regards,
Christian
--
Weiber gew�hnen sich Gleichg�ltigkeit und Unaufmerksamkeit gegen
Wissenschaft und Taubheit an, weil die M�nner zu oft vor ihnen von
wissenschaftlichen Dingen reden, die ihnen unbekannt.
-- Jean Paul
Bram Moolenaar
Christian Brabandt wrote:
> On Mi, 09 Okt 2013, Bram Moolenaar wrote:
> >
> > Yasuhiro Matsumoto wrote:
> >
> > > Bram, why you remove this patch from todo list?
> > >
> > > https://code.google.com/p/vim/source/diff?spec=svn173c9c860e42e2af31d598aa6924b99d7e73bd1b&r=173c9c860e42e2af31d598aa6924b99d7e73bd1b&format=side&path=/runtime/doc/todo.txt
> > >
> > > I can't assent.
> >
> > There are several changes. Can you quote the one you refer to?
>
> I guess, he means why you removed the async patch from the todo list.
because it isn't. I'll add something. It's actually similar to the
request for the :timer command.
--
WOMAN: Dennis, there's some lovely filth down here. Oh -- how d'you do?
ARTHUR: How do you do, good lady. I am Arthur, King of the Britons.
Who's castle is that?
WOMAN: King of the who?
The Quest for the Holy Grail (Monty Python)
kans
>
> request for the :timer command.
We renamed the feature to +timers. Are you referring to a different patch or to this one?
At any rate, I tracked down the final bug. We were actually stealing cpu time by calling select far too often (20ms) when select is call reentrantly. Apparently, some timeouts cause select to be called again. We were correctly dealing with not calling timeouts in this case, but not in scheduling the next call to call_timeouts. This was really hard to track because the cpu time wasn't spent inside Vim and the only thing that exhibited this problem was :Explore which is very hard to grok. To the best of my knowledge, this patch is now bug free!
-Matt
Ken Takata
Your patch still have some coding style errors and I fixed them.
Please check the attached patch.
Additionally, there are still some issues I think:
1. Copyright information is missing in the timers.c.
2. MCH_MONOTONIC_TIME is defined in os_macosx.m and os_unix.c, but it should be
defined in header files because the definition is used in timers.c.
3. Do we still need to support MSVC6? (Note that 7.4.044 is a fix for VC6.)
If it is needed, we can't use "long long". We should define a 64-bit type.
Similar (but not the same) thing is written in the todo.txt:
> On 64 bit MS-Windows "long" is only 32 bits, but we sometimes need to store a
> 64 bits value. Change all number options to use nropt_T and define it to the
> right type.
Regards,
Ken Takata
cptstubing
Ken Takata
2013/10/11 Fri 10:13:34 UTC+9 Ken Takata wrote:
> Your patch still have some coding style errors and I fixed them.
> Please check the attached patch.
>
> Additionally, there are still some issues I think:
>
> 1. Copyright information is missing in the timers.c.
> 2. MCH_MONOTONIC_TIME is defined in os_macosx.m and os_unix.c, but it should be
> defined in header files because the definition is used in timers.c.
> 3. Do we still need to support MSVC6? (Note that 7.4.044 is a fix for VC6.)
> If it is needed, we can't use "long long". We should define a 64-bit type.
> Similar (but not the same) thing is written in the todo.txt:
>
> > On 64 bit MS-Windows "long" is only 32 bits, but we sometimes need to store a
> > 64 bits value. Change all number options to use nropt_T and define it to the
> > right type.
I have updated the patch:
* Couldn't compile with MSVC.
* Add has("timers").
* Update documents.
* Fix a warning.
* etc.
Regards,
Ken Takata
glts
> I believe we have addressed all major concerns- the last one being the ex_timers command which shows all pending timers. If we have missed something, or you have other concerns, please let us know. As I wrote before, we'd really like to see this feature in Vim.
Quick question. You chose to make the string argument to settimeout()/
setinterval() a colon command:
:call settimeout(1000, "echo 2")
Thus, what settimeout() does is "execute a command", but the help states
it "evaluate[s] the expression". This confusion needs to be cleared up;
for consistency, I think that settimeout() and setinterval() should
indeed accept a Vim expression, not a colon command, e.g.,
:call settimeout(1000, "indexer#Update()")
where Update() is a user function. Compare remote_expr() and friends, as
well as the '*expr' options. But I may be missing something.
Second, do you know of existing plugins that rely on this patch, so that
we can grab it and try it out in our own settings without having to
write our own dummy plugins?
Thanks,
Dominique Pellé
I've just tried the "timers.patch". It compiles fine with clang
but with a few easy to fix warnings
ex_docmd.c:11522:11: warning: unused parameter 'eap' [-Wunused-parameter]
exarg_T *eap;
^
os_unix.c:5076:24: warning: unused variable 'now' [-Wunused-variable]
unsigned long long now;
^
timers.c:139:13: warning: comparison of integers of different signs:
'unsigned long long' and 'long' [-Wsign-compare]
if (towait < p_tt)
~~~~~~ ^ ~~~~
timers.c:143:32: warning: comparison of integers of different signs:
'unsigned long long' and 'long' [-Wsign-compare]
if (max_to_wait > 0 && towait > max_to_wait)
~~~~~~ ^ ~~~~~~~~~~~
Also, in the documentation (eval.txt) of settimeout(),
the folllowing example is incorrect as it gives error E117:
:call settimeout(1000, "call echo(2)")
It should be:
:call settimeout(1000, "echo(2)")
Same error in example in help of setinterval(...).
I also see a typo in eval.txt:
Compiled with support for timer funtions.
... should be:
Compiled with support for timer functions.
These are all minor remarks. The timer feature
would be a good addition to Vim!
Dominique
Andre Sihera
>
> I believe we have addressed all major concerns- the last one being the ex_timers command which shows all pending timers. If we have missed something, or you have other concerns, please let us know. As I wrote before, we'd really like to see this feature in Vim.
>
> -Matt
>
Please confirm the method by which I can unconditionally disable all
timers, and all
attempts to thereafter create a timer, using a single command. For
example, a
command like "timers off" in my .vimrc could override any attempt by any
plug-in
to enable a timer and force ViM into a guaranteed timer-free state.
It doesn't matter how well the feature itself is written, or how many
additional
commands have been created to "monitor" the timers, nothing can protect
me, a
person who uses ViM for primarily editing files, from:
1) A shoddy programmer who doesn't use the timers in the most
"VIM-friendly" way.
2) A bug in a plug-in that causes that plug-in, other plug-ins, or even
VIM itself, to
behave unpredictably and obstruct the work I am doing.
Incidentally, unacceptable solutions to this concern include, but are
not limited to:
a) VIM -u NONE
b) "Why don't you just build your own version with FEAT_TIMERS not defined?"
etc, etc.
Thanks in advance.
Nikolay Pavlov
On Oct 13, 2013 2:57 PM, "Andre Sihera" <andre....@hotmail.co.jp> wrote:
>
> On 03/10/13 07:40, kans wrote:
>>
>>
>> I believe we have addressed all major concerns- the last one being the ex_timers command which shows all pending timers. If we have missed something, or you have other concerns, please let us know. As I wrote before, we'd really like to see this feature in Vim.
>>
>> -Matt
>>
>
> Actually, I have a concern regarding these timers.
>
> Please confirm the method by which I can unconditionally disable all timers, and all
> attempts to thereafter create a timer, using a single command. For example, a
> command like "timers off" in my .vimrc could override any attempt by any plug-in
> to enable a timer and force ViM into a guaranteed timer-free state.
You have no way to unconditionally disable mappings, statusline, CursorHold events and so on; all these features can be harmful if used not in proper way. If plugin developer wants to make intrusive plugin he will make it. Thus such command makes exactly no sense. CursorHold is what is currently used to emulate timers.
> It doesn't matter how well the feature itself is written, or how many additional
> commands have been created to "monitor" the timers, nothing can protect me, a
> person who uses ViM for primarily editing files, from:
>
> 1) A shoddy programmer who doesn't use the timers in the most "VIM-friendly" way.
>
> 2) A bug in a plug-in that causes that plug-in, other plug-ins, or even VIM itself, to
> behave unpredictably and obstruct the work I am doing.
>
> Incidentally, unacceptable solutions to this concern include, but are not limited to:
>
> a) VIM -u NONE
>
> b) "Why don't you just build your own version with FEAT_TIMERS not defined?"
>
> etc, etc.
>
> Thanks in advance.
>
>
Nikolay Pavlov
On Oct 13, 2013 3:05 PM, "Nikolay Pavlov" <zyx...@gmail.com> wrote:
>
>
> On Oct 13, 2013 2:57 PM, "Andre Sihera" <andre....@hotmail.co.jp> wrote:
> >
> > On 03/10/13 07:40, kans wrote:
> >>
> >>
> >> I believe we have addressed all major concerns- the last one being the ex_timers command which shows all pending timers. If we have missed something, or you have other concerns, please let us know. As I wrote before, we'd really like to see this feature in Vim.
> >>
> >> -Matt
> >>
> >
> > Actually, I have a concern regarding these timers.
> >
> > Please confirm the method by which I can unconditionally disable all timers, and all
> > attempts to thereafter create a timer, using a single command. For example, a
> > command like "timers off" in my .vimrc could override any attempt by any plug-in
> > to enable a timer and force ViM into a guaranteed timer-free state.
>
> You have no way to unconditionally disable mappings, statusline, CursorHold events and so on; all these features can be harmful if used not in proper way. If plugin developer wants to make intrusive plugin he will make it. Thus such command makes exactly no sense. CursorHold is what is currently used to emulate timers.
Forgot a thing: CursorHold may be disabled with eventignore. Though nothing may prevent a plugin from resetting this option.
glts
> On Thu, Oct 3, 2013 at 12:40 AM, kans <mkan...@gmail.com> wrote:
>> I believe we have addressed all major concerns- the last one being the ex_timers command which shows all pending timers. If we have missed something, or you have other concerns, please let us know. As I wrote before, we'd really like to see this feature in Vim.
>
> we can grab it and try it out in our own settings without having to
> write our own dummy plugins?
one of the ideas upthread: show the date and time in the status line.
:call setinterval(200, "set stl=%{strftime('%c')}")
doesn't work because the statusline is only updated when the cursor
is moved, or on redraw.
:call setinterval(200, "set stl=%{strftime('%c')} | redraw")
does work but constantly redraws the screen which causes flickering.
Also, in this situation I would not be able to use :timers to diagnose
the situation, because the redraw clears :timers output immediately. I'd
be glad if you could show me what's wrong with my naive approach.
Anyway, I think a working real-world example would really help to show
what a plugin with timers can and can't do.
Nikolay Pavlov
There is :redrawstatus.
> Anyway, I think a working real-world example would really help to show
> what a plugin with timers can and can't do.
>
glts
>
> On Oct 13, 2013 3:33 PM, "glts" <676c...@gmail.com> wrote:
>>
>> On Sun, Oct 13, 2013 at 11:15 AM, glts <676c...@gmail.com> wrote:
>> > On Thu, Oct 3, 2013 at 12:40 AM, kans <mkan...@gmail.com> wrote:
>> >> I believe we have addressed all major concerns- the last one being the
>> >> ex_timers command which shows all pending timers. If we have missed
>> >> something, or you have other concerns, please let us know. As I wrote
>> >> before, we'd really like to see this feature in Vim.
>> >
>> > Second, do you know of existing plugins that rely on this patch, so that
>> > we can grab it and try it out in our own settings without having to
>> > write our own dummy plugins?
>>
>> Played around with this for a few minutes to see if I could implement
>> one of the ideas upthread: show the date and time in the status line.
>>
>> :call setinterval(200, "set stl=%{strftime('%c')}")
>>
>> doesn't work because the statusline is only updated when the cursor
>> is moved, or on redraw.
>>
>> :call setinterval(200, "set stl=%{strftime('%c')} | redraw")
>>
>> does work but constantly redraws the screen which causes flickering.
>> Also, in this situation I would not be able to use :timers to diagnose
>> the situation, because the redraw clears :timers output immediately. I'd
>> be glad if you could show me what's wrong with my naive approach.
>
> There is :redrawstatus.
:timers in this situation. I suppose one should just never use :redraw
in combination with timers ... got it, thanks.
Andre Sihera
On Oct 13, 2013 3:05 PM, "Nikolay Pavlov" <zyx...@gmail.com> wrote:
>
>
> On Oct 13, 2013 2:57 PM, "Andre Sihera" <andre....@hotmail.co.jp> wrote:
> >
> > On 03/10/13 07:40, kans wrote:
> >>
> >>
> >> I believe we have addressed all major concerns- the last one being the ex_timers command which shows all pending timers. If we have missed something, or you have other concerns, please let us know. As I wrote before, we'd really like to see this feature in Vim.
> >>
> >> -Matt
> >>
> >
> > Actually, I have a concern regarding these timers.
> >
> > Please confirm the method by which I can unconditionally disable all timers, and all
> > attempts to thereafter create a timer, using a single command. For example, a
> > command like "timers off" in my .vimrc could override any attempt by any plug-in
> > to enable a timer and force ViM into a guaranteed timer-free state.
>
> You have no way to unconditionally disable mappings, statusline, CursorHold events and so on; all these features can be harmful if used not in proper way. If plugin developer wants to make intrusive plugin he will make it. Thus such command makes exactly no sense. CursorHold is what is currently used to emulate timers.Forgot a thing: CursorHold may be disabled with eventignore. Though nothing may prevent a plugin from resetting this option.
A plug-in developer may want to create an intrusive plug-in, in which case there had
better be a way that normal users have to protect themselves against such plug-ins,
irrespective of whether those plug-ins were created with malicious intent or not.
Timers turn a finite-state system into a real-time system, and guaranteeing the reliability
of any real-time system is impossible. Which is why, in every real-time system, however
simple, there is always a get-out clause; and ViM should have one as well for when things
go wrong. There is a lack of control on some commands now but that is no excuse for not
building in appropriate controls into future commands to create a more robust, reliable,
and predictable User experience.
When things inevitably do go wrong and become a hassle to diagnose and/or fix, Users
may dump ViM in favour of other editors; editors that don't pretend to be all-singing and
all-dancing for the benefit of a few at the expense of being a robust, reliable, and
predictable tool for the many.
glts
Vim: Caught deadly signal SEGV
Segmentation fault (core dumped)
#0 call_timeouts (max_to_wait=max_to_wait@entry=71) at timers.c:100
#1 0x0000000000524aea in RealWaitForChar (fd=0, msec=71,
msec@entry=4000, check_for_gpm=0x0) at os_unix.c:5285
#2 0x0000000000524f9c in WaitForChar (msec=4000) at os_unix.c:5022
#3 WaitForChar (msec=<optimized out>) at os_unix.c:4973
#4 0x0000000000525f19 in mch_inchar (buf=buf@entry=0x876434
<typebuf_init+84> "", maxlen=maxlen@entry=60, wtime=wtime@entry=-1,
tb_change_cnt=tb_change_cnt@entry=91) at os_unix.c:413
#5 0x000000000059e660 in ui_inchar (buf=buf@entry=0x876434
<typebuf_init+84> "", maxlen=60, wtime=wtime@entry=-1,
tb_change_cnt=tb_change_cnt@entry=91) at ui.c:199
#6 0x00000000004ba18f in inchar (buf=0x876434 <typebuf_init+84> "",
maxlen=180, wait_time=-1, tb_change_cnt=91) at getchar.c:3039
#7 0x00000000004bc114 in vgetorpeek (advance=1) at getchar.c:2814
#8 vgetorpeek (advance=1) at getchar.c:1918
#9 0x00000000004bce76 in vgetc () at getchar.c:1590
#10 0x00000000004bd2d9 in safe_vgetc () at getchar.c:1795
#11 0x0000000000504770 in normal_cmd (oap=oap@entry=0x7fffffffe030,
toplevel=toplevel@entry=1) at normal.c:666
#12 0x00000000005e005d in main_loop (cmdwin=cmdwin@entry=0,
noexmode=noexmode@entry=0) at main.c:1329
#13 0x000000000043add7 in main (argc=<optimized out>, argv=<optimized
out>) at main.c:1020
I used the following toy code, the idea being: write the buffer to a
temp file and let an external tool do work on it. Communication happens
via a lock file created by the external component; the timed command
keeps polling for the lock file until it disappears.
function! DispatchWorker()
silent write !cat > testfile
call system('./work.sh &')
let g:worker_id = setinterval(300, 'call PollWorker()')
endfunction
function! PollWorker()
if !filereadable('.lock')
call canceltimeout(g:worker_id)
echo "Job done!"
endif
endfunction
command! Go call DispatchWorker()
The segfault happens on the call to canceltimeout().
I won't have time to look into this more today, so I'm just leaving the
report here.
Christian Brabandt
On So, 13 Okt 2013, Nikolay Pavlov wrote:
> You have no way to unconditionally disable mappings,
> statusline,
set ls=0 stl=
> CursorHold
set ei=CursorHold
> Thus such command makes exactly no sense.
makes perfectly sense to be able to turn timers off globally using e.g.
:timers off
regards,
Christian
--
Christliche Mystiker sollte es gar nicht geben, da die Religion
selbst Mysterien darbietet. Auch gehen sie immer gleich ins Abstruse,
in den Abgrund des Subjekts.
-- Goethe, Maximen und Reflektionen, Nr. 475
Ben Fritz
>
>
>
> I'll take your "Thus such command makes exactly no sense" response
> as a "No", shall I?
>
>
>
> A plug-in developer may want to create an intrusive plug-in, in
> which case there had
>
> better be a way that normal users have to protect themselves against
> such plug-ins,
>
> irrespective of whether those plug-ins were created with malicious
> intent or not.
>
Protection: don't install a plugin that uses timers!
>
>
> Timers turn a finite-state system into a real-time system, and
> guaranteeing the reliability
>
> of any real-time system is impossible. Which is why, in every
> real-time system, however
>
> simple, there is always a get-out clause; and ViM should have one as
> well for when things
>
> go wrong. There is a lack of control on some commands now but that
> is no excuse for not
>
> building in appropriate controls into future commands to create a
> more robust, reliable,
>
> and predictable User experience.
>
A "get out" command was something we discussed extensively. If I understand correctly, if Vim starts misbehaving due to timers, press CTRL-C and the currently executing timer, and all future timer events, will be cancelled.
This is just like what you would do if syntax highlighting rules or a / search or an infinite loop in plugin code were to make Vim unresponsive due to a user command.
>
>
> When things inevitably do go wrong and become a hassle to diagnose
> and/or fix, Users
>
> may dump ViM in favour of other editors; editors that don't pretend
> to be all-singing and
>
> all-dancing for the benefit of a few at the expense of being a
> robust, reliable, and
>
> predictable tool for the many.
Vim out-of-the-box has NO plugins which include a timer. If a user installs a bad plugin, then that is their fault. Nobody is going to dump Vim because they installed a bad plugin. They will dump the bad plugin.
Syntax highlighting can make Vim hang (until you press CTRL-C). Do you disable all syntax highlighting?
Any arbitrary plugin can set a CursorHold or CursorMoved autocmd that does:
while 1
endwhile
This will make Vim hang (until you press CTRL-C). Do you disable all autocmds?
Any plugin can set a user mapping like :nnoremap j :wqa!<CR>
Do you remove all mappings after loading your plugins?
Out-of-the-box Vim should never contain timers that will make Vim hang without the user doing something stupid. Installing a plugin is always a risk the user takes on. If the user installs a plugin and it doesn't work, they need to uninstall the plugin and notify the plugin author that their plugin is broken.
ZyX
>
> set paste
Does not work for normal mode. Can be overridden from the plugins.
> > statusline,
>
> set ls=0 stl=
`statusline' here means that plugin is setting statusline for its purposes. Thus `set stl=` is a no-op. `set ls=0` does not prevent statusline from displaying if there is more then one window arranged vertically. Thus disabling plugin is the only choice.
> > CursorHold
>
> set ei=CursorHold
I mentioned it in follow-up message. And this can also be overridden.
ZyX
> makes perfectly sense to be able to turn timers off globally using e.g.
> :timers off
I still think that timers should have a string ID generated on plugin side. Since plugins will 99% of time reuse one ID for one task it allows collecting stats I described earlier. And with ID :timers off may then be reduced to
set ignoretimers=*
(where 'ignoretimers' setting has 'wildignore'-like syntax and obvious purpose) if you think it is good to have some way to ignore timers: this way you can target specific plugin only. Like I will use after/ directory and :au!/:unmap if I see some too intrusive CursorHold event/mapping (though more likely I will remove the plugin) and definitely not 'eventignore' that will hurt all plugins.
Note that all of :au!/:unmap/set eventignore/:filetype off/:syntax off/etc rely on good will of plugin authors: only laziness may prevent them from coding self-recovering in case user has disabled parts of the functionality.
Joseph Pea
> This patch adds asynchronous functions to vimscript. If you want to perform an action in 700ms, simply:
>
> let timeout_id = settimeout(700, 'echo("hello")')
>
> To cancel the timeout before it's fired:
>
> canceltimeout(timeout_id)
>
> setinterval() also returns an id that can be used with canceltimeout.
>
> The reason for this patch is simple: asynchronous functionality is needed to implement real-time collaborative editing in Vim. This is one of the most voted-for features (see http://www.vim.org/sponsor/vote_results.php).
>
> Along with Matt Kaniaris, I founded Floobits to build real-time collaboration into every editor. We wrote a plugin for Vim, but we had to use hacks to get async behavior (abusing feedkeys or client-server). These methods had side-effects such as breaking leaderkeys or other shortcuts. After a lot of experimenting, we decided to try patching Vim.
>
> Since Vim is character-driven, we had to munge some low-level input functions to get the desired behavior. We changed gui_wait_for_chars() and mch_inchar() so that call_timeouts() is run every ticktime milliseconds. The default ticktime is 100ms.
>
> This patch isn't finished yet, but it works on unix-based OSes. If the reaction is positive, our intention is to change mch_inchar() (or something similar) in other OS-specific files. That will get async functions working for everyone.
>
> Even if our patch isn't the best approach, we'd love to help get async functions in Vim. Doing so will open the door to a lot of cool plugins.
>
> Oh, and this is the first time either myself or Matt have submitted a patch to Vim, so please be gentle.
>
> Sincerely,
>
> Geoff Greer
Dear God, I really want to see this in action. I could imagine JavaScript somehow playing a role in this...
Nikolay Pavlov
On Oct 14, 2013 1:00 AM, "ZyX" <zyx...@gmail.com> wrote:
>
> > We also have :syntax on, filetype plugin on and so on. So I think it
> > makes perfectly sense to be able to turn timers off globally using e.g.
> > :timers off
>
> I still think that timers should have a string ID generated on plugin side. Since plugins will 99% of time reuse one ID for one task it allows collecting stats I described earlier. And with ID :timers off may then be reduced to
s/allows collecting/allows collecting settimeout() stats and makes all (settimeout()+setinterval()) collected stats more readable/
> set ignoretimers=*
>
> (where 'ignoretimers' setting has 'wildignore'-like syntax and obvious purpose) if you think it is good to have some way to ignore timers: this way you can target specific plugin only. Like I will use after/ directory and :au!/:unmap if I see some too intrusive CursorHold event/mapping (though more likely I will remove the plugin) and definitely not 'eventignore' that will hurt all plugins.
>
> Note that all of :au!/:unmap/set eventignore/:filetype off/:syntax off/etc rely on good will of plugin authors: only laziness may prevent them from coding self-recovering in case user has disabled parts of the functionality.
>
kans
Bram,
I happy to see there is still some hope for this patch getting merged.
Ken,
Thanks for doing the work to clean up the patch.
Dominique,
Thanks for the warnings. They should be gone now.
glts,
1. I believe your timer does the equivalent of this:
:call setinterval(0, "call canceltimeout(0)")
This caused the problem of double freeing the timer and has been fixed. Timers can now be created and canceled safely from within a timer. Canceling a timeout that is running does nothing. Canceling a running interval causes it to not be rescheduled.
2. Concerning flickering, this is a bit of a problem for some use cases. We may need to write a couple of minor features to work around this (probably not as part of this patch).
3. Check out https://github.com/Floobits/floobits-vim for a demo. It allows real time collaborative editing in Vim (and between emacs/sublime). You can ignore all the caveats since as of this +timers, neither +clientserver nor cursor hold is needed by the plugin. See https://news.floobits.com/2013/09/16/adding-realtime-collaboration-to-vim/ for an explanation of how we got here and how painful the current solution is.
Andre,
I agree with Ben and Zyx. If you don't want to deal with timers, don't install plugins that use them. There can be no expectation that any plugin will work if you disable its timers. If a plugin is more trouble than its worth, uninstall the plugin. Moreover, any interval can be canceled with ctrl-c and it won't be rescheduled (so this should never be an issue).
At any rate, about 20 replies ago, I gave up trying to defend this position and implemented a way to disable all timers. They will eat up memory, but will never be called. Per the docs, "To globally disable all timers for debugging, set |ticktime| to -1." I still maintain using this for anything other than debugging is insane.
Zyx,
I don't think giving timers names is necessarily a bad idea, but this patch is too big as is.
-Matt
Andre Sihera
There has been too much action in this thread to reply to everyone inline. Bram, I happy to see there is still some hope for this patch getting merged. Ken, Thanks for doing the work to clean up the patch. Dominique, Thanks for the warnings. They should be gone now. glts, 1. I believe your timer does the equivalent of this: :call setinterval(0, "call canceltimeout(0)") This caused the problem of double freeing the timer and has been fixed. Timers can now be created and canceled safely from within a timer. Canceling a timeout that is running does nothing. Canceling a running interval causes it to not be rescheduled. 2. Concerning flickering, this is a bit of a problem for some use cases. We may need to write a couple of minor features to work around this (probably not as part of this patch). 3. Check out https://github.com/Floobits/floobits-vim for a demo. It allows real time collaborative editing in Vim (and between emacs/sublime). You can ignore all the caveats since as of this +timers, neither +clientserver nor cursor hold is needed by the plugin. See https://news.floobits.com/2013/09/16/adding-realtime-collaboration-to-vim/ for an explanation of how we got here and how painful the current solution is. Andre, I agree with Ben and Zyx. If you don't want to deal with timers, don't install plugins that use them. There can be no expectation that any plugin will work if you disable its timers. If a plugin is more trouble than its worth, uninstall the plugin. Moreover, any interval can be canceled with ctrl-c and it won't be rescheduled (so this should never be an issue).
timer plug-in will only happen in a plug-in that the user chooses to install? Just
because the pre-shipped plug-ins have been checked against themselves it
does not mean they're bug-free. Welcome to the world of real-time systems.
The pre-built plug-ins may work when tested against themselves but a plug-in I
install that is proven correct may cause the original pre-built plug-ins to fail
because, being a real-time system, it was impossible to test the pre-built plug-
ins against every future plug-in that was ever going to be written (note: future
tense intended).
At any rate, about 20 replies ago, I gave up trying to defend this position and implemented a way to disable all timers. They will eat up memory, but will never be called. Per the docs, "To globally disable all timers for debugging, set|ticktime|to -1." I still maintain using this for anything other than debugging is insane.
command and not debug only.
I originally supposed that you could put such a command in a .vimrc to which
the standard response has been "but a plug-in can override it". Well, analysing
my own behaviour, when things go wrong the first thing I want to do is type ":"
to get a prompt and disable the problem. This means I:
1) Don't have to exit ViM, restart ViM, or re-read .vimrc or any plug-ins/scripts.
2) Don't potentially lose any work I'm doing.
And for the person who quoted the syntax highlight case, the answer is yes,
when a syntax highlighting hangs my current session, I DO disable highlighting
by typing ":" to get a command prompt and typing "syntax off". That immediately
solves all problems, session is released and I can continue until the next "quit".
In the same way, if half way through a session these timers cause me a problem
I want instant out by ":timers off" at the prompt, not necessarily from my .vimrc.
Nikolay Pavlov
If I am not mistaking, this patch includes "timers" string in has() list two times.
In f_canceltimeout in a first if in a first if in a while cycle indentation is wrong: 4 spaces go before tab. Later indentation is wrong for
+ free(tmp->sourcing_name);
line: spaces are used in place of tabs. Same for line
+ return (long)towait;
in call_timeouts().
Andre Sihera
> I agree with Ben and Zyx. If you don't want to deal with timers, don't install plugins that use them. There can be no expectation that any plugin will work if you disable its timers. If a plugin is more trouble than its worth, uninstall the plugin. Moreover, any interval can be canceled with ctrl-c and it won't be rescheduled (so this should never be an issue).
or additional command.
Simply, to make sure that no plug-in or filetype extension that
uses a timer ever makes it into the standard ViM install packages.
That way, the system will be at its most robust and predictable
after installation as it is currently. If the user installs then a plug-in
themselves and things start to go wrong they know exactly which
plug-in needs to be uninstalled.
Nikolay Pavlov
On Oct 14, 2013 7:02 AM, "kans" <mkan...@gmail.com> wrote:
>
What about profiling? It is already not fine we need to be plugin developers to profile autocommands* and statusline, but timers are far more demanding in this case. If you do not add profiling/statistics right now it is fine, but if you do not add string IDs you will be unable to have nice output forever. IDs are needed to connect timers that seem different (e.g. in place of passing parameters in globals they pass parameters embedded in settimeout()/setinterval() string: the way I would prefer because I do not like using globals as parameters): those that you cannot connect based on cmd+SID pair.
* Write just one :python/:ruby/... command in :au rhs and user no longer sees how much time did it take to run it (alternative is wrapping rhs into a function which will call :python/...). Plugin developers though may edit their own files to use python own profiling. Statusline is the same: e.g. powerline sets &stl to a "%!pyeval(smth)". If something is slow I need to guess which autocmd is it based on my knowledge of when they are launched.
> -Matt
Michael Henry
I've been wondering about the use of CTRL-C regarding timers.
There has been a lot of discussion on ways to regain control in
the face of a runaway timer, which is of course an important
consideration. I've been wondering, however, about any negative
effects of accidental cancellation. If I understand correctly, when
a user presses CTRL-C while a timer happens to be running, that
timer will be canceled and not rescheduled. Can this happen
accidentally when the user is pressing CTRL-C for other reasons
(e.g., to abandon an ex-mode command he was typing)? Since
timers might be running at arbitrary times, how can a user be
sure it's safe to press CTRL-C without running the risk of canceling
a timer by mistake? Should it require multiple CTRL-C presses in
a row before aborting current and future timers, or is there some
other way to make sure the user won't accidentally cancel timers
that are operating properly?
Michael Henry
Ben Fritz
I think this is a valid concern.
I'm not exactly sure how this works now, I remember discussion about a list of canceled timers. Perhaps a single invocation could be canceled allowing this list to be displayed, then a second CTRL-C would be required (with the list displayed) to actually cancel all future invocations as well.
Ben Fritz
> On 14/10/13 12:02, kans wrote:
>
> > Andre,
>
> > I agree with Ben and Zyx. If you don't want to deal with timers, don't install plugins that use them. There can be no expectation that any plugin will work if you disable its timers. If a plugin is more trouble than its worth, uninstall the plugin. Moreover, any interval can be canceled with ctrl-c and it won't be rescheduled (so this should never be an issue).
>
>
>
> There is another solution which is far simpler than any code fix
>
> or additional command.
>
>
>
> Simply, to make sure that no plug-in or filetype extension that
>
> uses a timer ever makes it into the standard ViM install packages.
>
I agree that no timer functionality of any standard Vim plugins should be enabled by default in a new Vim install.
But I'd accept some timer-based features in the standard plugins if they were opt-in. Kind of like matchit is distributed with Vim but you need to take an extra step to actually use it.
>
>
> That way, the system will be at its most robust and predictable
>
> after installation as it is currently. If the user installs then a plug-in
>
> themselves and things start to go wrong they know exactly which
>
> plug-in needs to be uninstalled.
Agree.
Ben Fritz
>
> I originally supposed that you could put such a command in a .vimrc
> to which
>
> the standard response has been "but a plug-in can override it".
> Well, analysing
>
> my own behaviour, when things go wrong the first thing I want to do
> is type ":"
>
> to get a prompt and disable the problem. This means I:
>
>
>
> 1) Don't have to exit ViM, restart ViM, or re-read .vimrc or any
> plug-ins/scripts.
>
>
>
> 2) Don't potentially lose any work I'm doing.
>
>
>
> And for the person who quoted the syntax highlight case, the answer
> is yes,
>
> when a syntax highlighting hangs my current session, I DO disable
> highlighting
>
> by typing ":" to get a command prompt and typing "syntax off". That
> immediately
>
> solves all problems, session is released and I can continue until
> the next "quit".
>
>
>
> In the same way, if half way through a session these timers cause me
> a problem
>
> I want instant out by ":timers off" at the prompt, not necessarily
> from my .vimrc.
>
>
Yes, but with syntax, if it's bad enough, you'll first need to CTRL-C to get a prompt at all. No different here with timers, and after the CTRL-C your problem goes away until you :q
I suppose, that with syntax, you have the ability to now temporarily disable it in your .vimrc if you don't want to track down the issue right this minute. So there is that difference.
Marcin Szamotulski
to normal mode?
Best regards,
Marcin
Ernie Rael
> On Monday, October 14, 2013 5:59:48 AM UTC-5, Michael Henry wrote:
>> All,
>>
>>
>>
>> I've been wondering about the use of CTRL-C regarding timers.
>>
>>
> I think this is a valid concern.
>
> I'm not exactly sure how this works now, I remember discussion about a list of canceled timers. Perhaps a single invocation could be canceled allowing this list to be displayed, then a second CTRL-C would be required (with the list displayed) to actually cancel all future invocations as well.
>
or only partially run. That could make plugins using timers pretty
tricky to write.
Ben Fritz
Well I didn't mention it, but I was actually envisioning this message would block timers or other commands until dismissed somehow. Otherwise a timer or mapping could easily do :redraw! and the message would be gone anyway.
I press CTRL-C very infrequently. I'd be willing to give it a try first without such a feature and see how often I accidentally interrupt timers.
We get a message if timers are cancelled, yes? Is it possible to restart them somehow or are you just out of luck if this happens?
Matthew Kaniaris
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to a topic in the Google Groups "vim_dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vim_dev/-4pqDJfHCsM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vim_dev+u...@googlegroups.com.
Anton Bobrov
Relying on good chance is much worse than timer plugins in stock vim. Unreproducible glitches, fuck yeah!
Ben Fritz
> > I press CTRL-C very infrequently. I'd be willing to give it a try first without such a feature and see how often I accidentally interrupt timers.
>
> Relying on good chance is much worse than timer plugins in stock vim. Unreproducible glitches, fuck yeah!
It wouldn't be unreproducible, nor would it be a glitch. I'd get a prominent error message if I pressed CTRL-C that I had canceled timers.
In response, I'd say "whoops!", save all my work, and restart Vim.
And then, I'd remember to use <Esc> instead of <C-C> to cancel stuff next time.
I don't think anybody is talking about timers for default functionality in stock Vim.
Anton Bobrov
Not all users have excellent error message detection skills like yours.
> In response, I'd say "whoops!", save all my work, and restart Vim.
I'd say 'bloody vim, why you kill my tiny sweet timers? They are not guilty!'
> And then, I'd remember to use <Esc> instead of <C-C> to cancel stuff next time.
You are man! My <esc> is <nop>.
Seriously, there is a simple solution: to kill only long running timers. One second timeout will be sane default.
Matthew Kaniaris
What do you mean by long running timers? Are you suggesting we look at the running time of the current timer?
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to a topic in the Google Groups "vim_dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vim_dev/-4pqDJfHCsM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vim_dev+u...@googlegroups.com.
Anton Bobrov
> Anton,
>
> What do you mean by long running timers? Are you suggesting we look at the
> running time of the current timer?
>
> -Matt
timer's start time and can calculate running time in case of <c-c>.
Matthew Kaniaris
Anton Bobrov
> suggest for the heuristic on not rescheduling? A timer that takes up 500ms
> out of every 1000 is also something we'd want to kill.
>
> -Matt
complex documentation and users will shoot their legs in any case.
Timer running threshold solves 'do not kill good plugins' problem that
completely different from the above.
Matthew Kaniaris
Anton Bobrov
> would have to use threads and locking and vim isn't thread safe. This is
> way more work than I am willing to put in even if it means this patch
> doesn't get merged.
>
> -Matt
active timer (if any). And this decision is based on start time solely.
Ben Fritz
OK, so it sounds like maybe I like your idea:
1. If CTRL-C is pressed while a timer is running, check how long it has been running.
2. If it has been running longer than a threshold, cancel that timer for now and forevermore.
This way it wouldn't cancel anything if hit accidentally on a well-behaved timer.
3. Also cancel all other timers? Maybe this isn't needed. But if a plugin has multiple timers, you could hose that plugin worse by stopping only one of its timers I bet.
Perhaps a new feature could be added? Autocmds have groups, how about timer groups? Then CTRL-C could cancel all timers in the same group. If a timer doesn't have a group, it cancels all ungrouped timers. Then CTRL-C would end only one plugin most of the time.
Anton Bobrov
ideal condition after emergencies. This is plugin author responsibility to take
control of execution times and timer dependences.
Bram Moolenaar
CTRL-C would "pause" a timer if it's clear that was the intention of
hitting CTRL-C. As suggested, that the timer has been running for more
than a certain time would work. Anything that's stuck must quit when
CTRL-C is pressed. That's what it's for.
But if the timer runs for a short time, but very often, it may still
cause the user to hit CTRL-C to stop it. Perhaps we can detect that the
user types CTRL-C twice in a row, then we know we need to cancel more
than just the current operation. If there are multiple things that
could be interrupted, a "what do you want to interrupt?" dialog could be
used (although that's not really user friendly).
I would prefer to have CTRL-C pause the timer instead of completely
killing it. That gives a way to recover. E.g. if the timer uses a
socket that is temporarily unavailable. Would require a "resume timer"
command.
--
Did you ever stop to think... and forget to start again?
-- Steven Wright
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
Geoff Greer
Bram,
Thanks for taking time to look at our patch and give feedback. Besides pausing/resuming timers, are there any other blockers for merging this patch? Matt and I really want to get it merged, but there's been a recurring pattern where we address one thing only to have another brought up.
If this is the last thing, we'll gladly add it. If it's not the last thing, please give us a complete list of blockers. Then we can determine if we want to continue addressing your issues or just maintain our own fork of Vim.
Sincerely,
Geoff Greer
Andre Sihera
> If this is the last thing, we'll gladly add it. If it's not the last thing, please give us a complete list of blockers. Then we can determine if we want to continue addressing your issues or just maintain our own fork of Vim.
stability
might be the best idea.
Rushing to get it merged when there isn't (to my knowledge) a set of
test cases to
prove its correctness even to a limited degree seems like folly given
the amount
of hard graft it has taken to get VIM to the stability it has today. The
core-dumping
that has been reported to this mailing list alone during its development
has done
nothing to instil confidence in its long-term image with regards to
robustness.
To verify every execution path within any real-time system is
impossible. However,
as VIM as a set of test cases for most of its individual commands, and
any command
can technically be executed from within a timer, I would at least expect
that a set of
test cases be drawn up where the standard test cases, when each is
executed within
a timer, produces the same result as when the test is executed as a
standalone test.
Furthermore, a timer being effectively an "interrupt", there should be
an extra test
case that executes underneath the timer when it fires that can be shown
to produce
a known result when the timer interrupt returns, hence showing that the
execution
of commands within a timer has no unknown side-effects on the
interrupted code.
If this set of test cases could be drawn up and could be run like the
other test cases
I think it's robustness could be proven enough to merge it sooner rather
than later.
Ken Takata
2013/10/14 Mon 16:39:04 UTC+9 ZyX wrote:
> If I am not mistaking, this patch includes "timers" string in has() list two times.
> In f_canceltimeout in a first if in a first if in a while cycle indentation is wrong: 4 spaces go before tab. Later indentation is wrong for
>
> + free(tmp->sourcing_name);
>
> line: spaces are used in place of tabs. Same for line
>
> + return (long)towait;
>
> in call_timeouts().
* Fix duplicated has('timers').
* Fix duplicated setinterval() in the eval.txt.
(Hmm, there's a line which exceeds 80 columns.)
* Fix examples of setinterval() in the eval.txt.
* Fix an indent.
I think remaining items are:
* How to stop a timer which has a problem.
* Support for old compilers like MSVC6. (Is it needed?)
Regards,
Ken Takata
Ben Fritz
>
>
> Furthermore, a timer being effectively an "interrupt", there should be
>
> an extra test
>
> case that executes underneath the timer when it fires that can be shown
>
> to produce
>
> a known result when the timer interrupt returns, hence showing that the
>
> execution
>
> of commands within a timer has no unknown side-effects on the
>
> interrupted code.
>
If I understand correctly, the timer runs in the "idle" loop of Vim, so it should not ever interrupt any code running outside the timer.