Re: substitue( optimize

[Luc Hermitte]

Hello Nicholas,

> After reading a file and getting 1 million lines' List, I need to
> make similar substitutions over matched lines ~500 times with this
> func:
>
>
> fun! foo#set_new_val(ressource) abort "{{{
>
> " position
> let pos = foo#getpos_ress(a:ressource)
> let pres = foo#geth_presence(a:ressource)
> let s:xml[pos] = substitute(s:xml[pos],'false\|true',pres,"") <<<
> HERE
>
> " Slave/Voie01 to Voie08
> for idx in [1,2,3,4,5,6,7,8] <<< HERE
> let pos = foo#getpos_ress(a:ressource.'/bar0'.string(idx))
> let s:xml[pos] = substitute(s:xml[pos],'false\|true',pres,"") <<<
> HERE
> endfor
>
> endfu"}}}
>
>
> 1/ Can I optimize this function ?

So far my experiments on the subject taught me that we are best avoiding :for and :while constructs when performances matter.
If you can use map() instead, do!

HTH,

--
Luc Hermitte

[Ni Va]

Yess Luc !

I finally reached to the same conclusion.

1/ But mapping Dict or List, inside the substitute func is needed no ?
2/ Other point, passing by register with emmbedded pattern is it faster ?

let @a = '\w\+\(<\/PRES\)'
let @b = pres.'\1'

call map(voies_no, 'substitute(s:xml[v:val],"\=@a","\=@b","")')


Thanks you

[Nikolay Aleksandrovich Pavlov]

Using registers here makes no sense, you can just use variables. `\=`
is not going to work like you typed here. For loop with preceding
substitute from original function look like they can be rewritten into

call map(
\[a:ressource] + map(range(1, 8), 'a:ressource."/bar0".v:val'),
\'extend(
\ s:xml, {
\ extend(l:, {"pos": foo#getpos_ress(v:val)}).pos
\ : substitute(s:xml[pos],''\c\vfalse|true'',pres,"")})')

Note that my main VimL optimization principle is “less commands is
better”. “Avoid loops” is one of the special cases only: by “less
commands” I mean not “less commands in code”, but “less commands to
execute”. I used to write a `:while` loop which contained nothing, but
conditional expression and all the work was done there (`extend(l:,
{})`, list literals as a replacement for comma operator and no
optimizer messing with sequence points rock (until you need to debug
this mess)). Note that for this reason lambdas (with the current
implementation) are a no-go: they create functions with `return`
command. If you need optimization and you have a choice between lambda
and some kind of `eval()` (e.g. map string literal) then use `eval()`.

>
>
> Thanks you
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Ni Va]

Entirely agree with your VimL principle : less command is fast/better, inner system apis... Principle that works for any Language.

Out the fact that it is not here a xml treenode parser but inlines binary file modifications, do you think modifying large xml file (1Million line  40Mo) can be as fast as libxml (ox ruby, python) or xpath parser ?

> To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---

You received this message because you are subscribed to a topic in the Google Groups "vim_use" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vim_use/dY1x4ok__7w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vim_use+unsubscribe@googlegroups.com.

[Nikolay Aleksandrovich Pavlov]

2018-02-14 11:31 GMT+03:00 Ni Va <niva...@gmail.com>:
> Entirely agree with your VimL principle : less command is fast/better, inner
> system apis... Principle that works for any Language.

Not actually. What I mean here is that you should use VimL expressions
where normally you would use commands: `let [a, b] = [1, 2]` should be
a bit faster then `let a = 1|let b = 2`, though both are semantically
equivalent and first even allocates a list to be disposed of when
command finishes. In many cases this reasoning applies to `call
map(copy(list), …)` vs `for val in list`, sometimes even if `for`
variant contains `:break` command while `map()` obviously can’t.

The principle may be reworded as “parsing VimL expressions is far
faster then parsing VimL commands”. Reasoning is simple: in modern
scripting languages code follows the path

source code ---(parser)--> AST ---(translator)--> bytecode[
---(compiler)--> machine code]

Only after undergoing all these translations code is actually
executed. In VimL there is no parser, no translator and no compiler.
Vim just stores source code strings (possibly with some minor
conversions like reencoding (:h :scriptencoding) or joining lines (:h
line-continuation)) and executes them as they are parsed, never
storing parsing results in any format for future executions. Thus
while in other programming languages equivalent commands and
expressions will produce nearly the same or even identical
byte/machine code, no matter how parsing commands is more/less
complicated then parsing expressions (if there is separate between
commands and expressions at all), in VimL you pay a price for parsing
command every time you execute it and commands parser is just slower
(`map()` vs `:for`), in addition to the fact that “one big expression
with side-effects” approach in many cases means that you need to call
expressions parser single time vs multiple times for multiple commands
with the same effect.

>
> Out the fact that it is not here a xml treenode parser but inlines binary
> file modifications, do you think modifying large xml file (1Million line
> 40Mo) can be as fast as libxml (ox ruby, python) or xpath parser ?

For the above reasons (no parser and no translator) VimL can never
hope to achieve performance comparable to a C library. This may change
in Neovim as I am working on adding parser and translator after some
time, but work is going rather slowly; currently there only is parser
for expressions and it is only used for command-line highlighting (I
mean, highlighting expressions in `<C-r>=` and the like prompts): no
translator is available and neither there is code that executes AST
without translating.

>> > an email to vim_use+u...@googlegroups.com.

>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> --
>> You received this message from the "vim_use" maillist.
>> Do not top-post! Type your reply below the text you are replying to.
>> For more information, visit http://www.vim.org/maillist.php
>>
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "vim_use" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/vim_use/dY1x4ok__7w/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> vim_use+u...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---

> You received this message because you are subscribed to the Google Groups
> "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to vim_use+u...@googlegroups.com.

[Ni Va]

Yes that's why I switch Choice One : VimL binary file parse to Choice 2 : Ruby Ox parser seems to have very good perf.

currently workin on, I leave VimL func...

Thank you for the first explanation of set List full ready to work some job. I agree 

>> > an email to vim_use+unsubscribe@googlegroups.com.

>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> --
>> You received this message from the "vim_use" maillist.
>> Do not top-post! Type your reply below the text you are replying to.
>> For more information, visit http://www.vim.org/maillist.php
>>
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "vim_use" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/vim_use/dY1x4ok__7w/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> vim_use+unsubscribe@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups
> "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to vim_use+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to a topic in the Google Groups "vim_use" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vim_use/dY1x4ok__7w/unsubscribe.

To unsubscribe from this group and all its topics, send an email to vim_use+unsubscribe@googlegroups.com.