Vim9 script performance vs. legacy

[Salman Halim]

Hello,

I've been writing my scripts using Vim 9 recently (though without classes) and was wondering if anybody had any performance metrics/benchmarks they would be willing to share that compare Vim 9's speed compared to the same thing written in legacy code.

I find that a significant amount of my script-writing time is used in calling functions such as matchlist, popup_create and the such, which seem to me to be identical (unless you tell me that Vim 9 has access to more optimised version of these).

Maybe the 'for' loop where I iterate over a list of strings is faster in Vim 9 because it's both compiled and statically typed?

One of the things that caught me recently was that popup_create takes a parameter called 'line' which can be either a number or the word 'cursor'. It cannot be the STRING '2', it has to be the NUMBER 2 while 'cursor' is a string. In legacy script, no problem. Here, I had to make it a string variable and then see do something like 'lineNumber != "cursor" ? str2nr(lineNumber) : lineNumber' to convert it to a number if needed. i'm trying to figure out whether there is a measurable speed advantage to going through these motions or if I should just write my scripts in legacy Vim script.

What do people use for their own stuff these days?

Thanks so much, everybody.

--

 
Salman

I, too, shall something make and glory in the making.

[Christian Brabandt]

I haven't done any measurements but I noticed converting to Vim9 script
helps, if you are calling your scripts many times.

So I have rewritten parts of vim-airline for those parts, that are
executed many many times and that are known to cause performance
degrations (see https://github.com/vim-airline/vim-airline/wiki/FAQ)

Best,
Christian
--
<knghtbrd> add a GF2/3, a sizable hard drive, and a 15" flat panel and
you've got a pretty damned portable machine.
<Coderjoe> a GeForce Two-Thirds?
<knghtbrd> Coderjoe: yes, a GeForce two-thirds, ie, any card from ATI.

[Salman Halim]

On Tue, Aug 29, 2023 at 10:33 AM Christian Brabandt <cbl...@256bit.org> wrote:

On Di, 29 Aug 2023, Salman Halim wrote:

> Hello,
>
> I've been writing my scripts using Vim 9 recently (though without classes) and
> was wondering if anybody had any performance metrics/benchmarks they would be
> willing to share that compare Vim 9's speed compared to the same thing written
> in legacy code.
>

...

> Maybe the 'for' loop where I iterate over a list of strings is faster in Vim 9
> because it's both compiled and statically typed?
>

...

> What do people use for their own stuff these days?

I haven't done any measurements but I noticed converting to Vim9 script
helps, if you are calling your scripts many times.

So I have rewritten parts of vim-airline for those parts, that are
executed many many times and that are known to cause performance
degrations (see https://github.com/vim-airline/vim-airline/wiki/FAQ)

Best,
Christian

I took matters into my own hands and did a HIGHLY scientific test where I wrote these two functions that just add up the lengths of all the lines in the current file:

function! CountLengths()
  let total = 0

  for i in range( 1, line( '$' ) )
    let total += getline( i ) ->strchars()
  endfor

  return total

endfunction 

export def g:V9CountLengths(): number
  var total: number = 0

  for i in range( 1, line( '$' ) )
    total += getline( i ) ->strchars()
  endfor

  return total
enddef

I tried to keep the implementations as close to identical as possible. Then, I opened up a fairly large file (13 megs) where wordcount() returns:

{'chars': 13705032, 'cursor_chars': 1, 'words': 1515632, 'cursor_words': 1, 'bytes': 14136260, 'cursor_bytes': 1} <-- Guess where in the file I had my cursor!

I ran the legacy function ONCE (several times to verify) and got a time of 1.5754 seconds, which is definitely noticeable from when I pressed enter and when I got control back. I ran the Vim 9 version and it appeared to finish almost immediately, so I ran it TEN times in a loop and got a total time of 1.272675.

Thus, here, the Vim 9 function is more than twelve times faster. This result makes me happy and tells me that I should keep writing in Vim 9.

The only reason I can think of to stick with legacy is if people are now concerned about portability with Neovim if they're thinking that now, because of recent sad events, is a time to think about making a move.

For the curious, my timing function (legacy Vim, I know):

function! HowLong( command, numberOfTimes )
  " We don't want to be prompted by a message if the command being tried is an echo as that would slow things down while
  " waiting for user input.
  let more = &more
  set nomore

  let startTime = reltime()

  for i in range( a:numberOfTimes )
    execute a:command
  endfor

  let result = reltime( startTime )

  let &more = more

  return result
endfunction

And I tested it like this:

let g:result = HowLong('call CountLengths()', 1) ->reltimestr()
let g:result9 = HowLong('call V9CountLengths()', 10) ->reltimestr()

All the best,

Salman

[Owajigbanam Ogbuluijah]

Amazing! Thank you for doing this benchmarking. 

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/CANuxnEfC28j7D24rWj2h1EeS-SVaaTuJOQxLHPz2ZNfmLH_WPw%40mail.gmail.com.

[Lifepillar]

On 2023-08-29, Salman Halim <salma...@gmail.com> wrote:
> Hello,
>
> I've been writing my scripts using Vim 9 recently (though without classes)
> and was wondering if anybody had any performance metrics/benchmarks they
> would be willing to share that compare Vim 9's speed compared to the same
> thing written in legacy code.

I have rewritten Colortemplate v3¹, which is a somewhat complex project,
in Vim 9 script. This is how long it takes to build some sets of color
schemes on my laptop:

Template | Legacy Vim | Vim 9 script
--------------------------------------------
Gruvbox 8 | 17.2s | 3.7s
Solarized 8 | 17.4s | 2.2s
Vim colorschemes²| 20.5s | 3.2s

¹ https://github.com/lifepillar/vim-colortemplate
² https://github.com/vim/colorschemes

And this comparison is a bit apples vs oranges because it was not
a porting, but a complete rewrite. For instance, instead of an hoc
top-down recursive-descent parser, I am now using a parser combinators
library in functional style³—something that would be cumbersome to write
in legacy Vim script, and it would most likely perform horribly.
Functions calls are much more efficient in Vim 9 script.

For a more apples-to-apples comparison, below is the execution time of
`libcolor.Neighbours()` from my libcolor library,³ which is a literal
translation of a legacy colortemplate#colorspace#k_neighbours() from
Colortemplate v2:

k | Legacy Vim script | Vim 9 script
---|-------------------|-------------
1 | 73.8 ms | 5.1 ms
10 | 648.4 ms | 42.9 ms
20 | 1164.1 ms | 97.7 ms

The advantage of Vim 9 script is very significant. The benchmarking code
is at the end of this message for reference.

³ https://github.com/lifepillar/vim-devel

> I find that a significant amount of my script-writing time is used in
> calling functions such as matchlist, popup_create and the such, which seem
> to me to be identical (unless you tell me that Vim 9 has access to more
> optimised version of these).

I believe that built-in functions and commands should mostly show
similar performance.

> Maybe the 'for' loop where I iterate over a list of strings is faster in
> Vim 9 because it's both compiled and statically typed?

Likely so. Not an answer to your question, but you may take a look at
some Vim 9 script benchmarks here:

https://github.com/lacygoill/wiki/blob/main/vim/vim9.md

in the "What's the fastest between" section, and maybe compare with
similar loops in legacy Vim script.

> One of the things that caught me recently was that popup_create takes a
> parameter called 'line' which can be either a number or the word 'cursor'.
> It cannot be the STRING '2', it has to be the NUMBER 2 while 'cursor' is a
> string. In legacy script, no problem. Here, I had to make it a string
> variable and then see do something like 'lineNumber != "cursor" ?
> str2nr(lineNumber) : lineNumber' to convert it to a number if needed. i'm
> trying to figure out whether there is a measurable speed advantage to going
> through these motions or if I should just write my scripts in legacy Vim
> script.

From my experience on porting scripts to the new syntax, the stricter
typing rules help finding bugs and write cleaner code. It is a win,
regardless of speed.

But Vim 9 script also has to coexist with the ecosystem of Vim
functions, and in some cases there is some unavoidable friction, as in
your example. For your specific example, I'd probably wrap the annoying
conversion into a function (did I say that function calls are cheap?):

def PopupLine(lineNumber: string): any
if lineNumber == 'cursor'
return lineNumber
endif
return str2nr(lineNumber)
enddef

Btw, differently from legacy script, an `if` command is not slower than
using `?`: both constructs gets compiled into the same bytecode.

> What do people use for their own stuff these days?

Only Vim 9 script. Bram gave us a wonderful gift, and I am very glad to
see that other developers are actively maintaining it and refining it.

Happy Vim scripting!
Life.

---
vim9script

# Benchmarking a naive k-neighbors algorithm
# in legacy Vim script vs Vim 9 script

import 'libcolor.vim' as libcolor

def Benchmark(Fn: func, args: list<any> = [], repeat = 1): float
var i = 0
const start = reltime()

while i < repeat
call(Fn, args)
++i
endwhile

return 1000 * reltime(start)->reltimefloat() / repeat
enddef

const n_repeat = 10

for k in [1, 10, 20]
# Vim 9 script
echomsg Benchmark(libcolor.Neighbours, ['#f54f29', k], n_repeat) "ms"

# Legacy Vim script
echomsg Benchmark(
colortemplate#colorspace#k_neighbours, ['#f54f29', k], n_repeat
) "ms"
endfor

[Salman Halim]

On Wed, Aug 30, 2023 at 5:25 PM Lifepillar <lifep...@lifepillar.me> wrote:

I have rewritten Colortemplate v3¹, which is a somewhat complex project,
in Vim 9 script. This is how long it takes to build some sets of color
schemes on my laptop:

Template         | Legacy Vim | Vim 9 script
--------------------------------------------
Gruvbox 8        |     17.2s  |       3.7s
Solarized 8      |     17.4s  |       2.2s
Vim colorschemes²|     20.5s  |       3.2s

¹ https://github.com/lifepillar/vim-colortemplate
² https://github.com/vim/colorschemes

And this comparison is a bit apples vs oranges because it was not
a porting, but a complete rewrite. For instance, instead of an hoc
top-down recursive-descent parser, I am now using a parser combinators
library in functional style³—something that would be cumbersome to write
in legacy Vim script, and it would most likely perform horribly.
Functions calls are much more efficient in Vim 9 script.

This is really interesting; the fact that you have a real-life example instead of my contrived test makes your findings more valuable. About five to six times faster, notwithstanding your different implementation approach.

 

For a more apples-to-apples comparison, below is the execution time of
`libcolor.Neighbours()` from my libcolor library,³ which is a literal
translation of a legacy colortemplate#colorspace#k_neighbours() from
Colortemplate v2:

 k | Legacy Vim script | Vim 9 script
---|-------------------|-------------
 1 |          73.8 ms  |    5.1 ms
10 |         648.4 ms  |   42.9 ms
20 |        1164.1 ms  |   97.7 ms

The advantage of Vim 9 script is very significant. The benchmarking code
is at the end of this message for reference.

This is between 11 to 15+ times faster. Interestingly, running it 20 times reduced the advantage slightly. Just an anomaly where the CPU started doing something else, d'you suppose?

 

³ https://github.com/lifepillar/vim-devel

> I find that a significant amount of my script-writing time is used in
> calling functions such as matchlist, popup_create and the such, which seem
> to me to be identical (unless you tell me that Vim 9 has access to more
> optimised version of these).

I believe that built-in functions and commands should mostly show
similar performance.

I expect that, also; unless Vim 9 versions of these are written that are statically typed (which only matters in some cases where an argument could be one of several types), these aren't going to be the factors that determine the response time.

 

> Maybe the 'for' loop where I iterate over a list of strings is faster in
> Vim 9 because it's both compiled and statically typed?

Likely so. Not an answer to your question, but you may take a look at
some Vim 9 script benchmarks here:

    https://github.com/lacygoill/wiki/blob/main/vim/vim9.md

in the "What's the fastest between" section, and maybe compare with
similar loops in legacy Vim script.

Super useful link and pretty much answers a lot of the questions I had, including some I didn't manage to articulate!

 

From my experience on porting scripts to the new syntax, the stricter
typing rules help finding bugs and write cleaner code. It is a win,
regardless of speed.

I have actually ported a few of my existing scripts over to Vim 9, but I've tried to restrict it to more intensive tasks as legacy scripts are often fast enough for everyday work.

 

But Vim 9 script also has to coexist with the ecosystem of Vim
functions, and in some cases there is some unavoidable friction, as in
your example. For your specific example, I'd probably wrap the annoying
conversion into a function (did I say that function calls are cheap?):

def PopupLine(lineNumber: string): any
  if lineNumber == 'cursor'
    return lineNumber
  endif
  return str2nr(lineNumber)
enddef

Btw, differently from legacy script, an `if` command is not slower than
using `?`: both constructs gets compiled into the same bytecode.

I actually rewrote a routine with a ternary into an if/else earlier today for the sake of readability, but felt like I was damning it to a slower execution cycle while I did it. It's good to know that I might not have sacrificed performance for maintainability.

I have been trying to avoid 'any' as a type because I feel that it loses some of the speed advantages of Vim 9, but it can't always be helped. I had actually come up with version of your PopupLine that uses the fact that str2nr just returns 0 if the conversion fails as this allows more than just a value of 'cursor' (to make it a more generic function):

def GetStrOrNr(in: string): any
  if (in == '0')
    return 0
  endif

  var line: number = str2nr( in )

  return line == 0 ? in : line
enddef

 

> What do people use for their own stuff these days?

Only Vim 9 script. Bram gave us a wonderful gift, and I am very glad to
see that other developers are actively maintaining it and refining it.

I'm definitely using it for all my new stuff, also. Glad to be in good company.

 

Happy Vim scripting!
Life.

Thank you for the detailed response. Much appreciated.

All the best,

Salman

[Lifepillar]

On 2023-08-31, Salman Halim <salma...@gmail.com> wrote:
> On Wed, Aug 30, 2023 at 5:25 PM Lifepillar <lifep...@lifepillar.me> wrote:
>> For a more apples-to-apples comparison, below is the execution time of
>> `libcolor.Neighbours()` from my libcolor library,³ which is a literal
>> translation of a legacy colortemplate#colorspace#k_neighbours() from
>> Colortemplate v2:
>>
>> k | Legacy Vim script | Vim 9 script
>> ---|-------------------|-------------
>> 1 | 73.8 ms | 5.1 ms
>> 10 | 648.4 ms | 42.9 ms
>> 20 | 1164.1 ms | 97.7 ms
>>
>> The advantage of Vim 9 script is very significant. The benchmarking code
>> is at the end of this message for reference.
>>
>
> This is between 11 to 15+ times faster. Interestingly, running it 20 times
> reduced the advantage slightly. Just an anomaly where the CPU started doing
> something else, d'you suppose?

That k is an argument of the function (the function finds the k colors
that are most similar to a given color). Each reported figure is an
average of ten executions. I have not reported (and not computed)
confidence intervals, so 11–15x may just be within noise limits.

Interestingly, I ran those benchmarks on a version of Vim compiled by
myself. If I use my system's Vim I get better results:

k | Legacy Vim script | Vim 9 script
---|-------------------|-------------

1 | 33.1 ms | 4.0 ms
10 | 283.9 ms | 20.7 ms
20 | 512.8 ms | 51.0 ms

The gap narrows a bit: Vim 9 script is 8–10x faster. Still quite a lot.

Now, I have to find out how to optimize my custom build...

Life.

[Salman Halim]

My (less rigorous) findings were on a Windows 10 machine with a GVim 9.0.1677 that I built myself, but didn't tweak to optimise beforehand. (Stock 64-bit GUI build with Python enabled, built using Ming)

Salman

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vim_use/ucqo6c%248i9%241%40ciao.gmane.io.