improving Vim - Kickstarter - brainstorming - goals - who wants to join?

606 views
Skip to first unread message

Marc Weber

unread,
Nov 25, 2013, 8:26:42 AM11/25/13
to vim...@vim.org, vim...@googlegroups.com
I've been complaining about Vim related issues for a long time,
I think its time to stop complaining and just fix it.

Join by providing feedback:
http://mawercer.de/vim.php

or adding additional issues to be fixed here:
http://vim-wiki.mawercer.de/wiki/topic/in-which-way-does-vim-suck.html

The idea is to create a kickstarter project to funding all work.
IMHO Vim is worth keeping alive, and that means we must find a way
to move Vim into the future.

If this requires writing a new language, because C lacks abstracktions,
and C++ is complex, then that's the task to be done IMHO.

This project makes me think we might have success:
http://www.kickstarter.com/projects/maxcantor/beautiful-vim-cheat-sheet-poster?ref=live

Thus if you're either a developper or a user who wants to help join and
tell me what you want to work on - even if its "testing new features you
care about" only.

Goals are:
- focus on productivity
- code reusage

I consider refactoring viml a key thing, eg creating a viml library
which contains the interpreter only.

If you'll help me with this I'll be working on Vim related topics the
following month, otherwise I'll get any alternative job.

I feel I've hit a border meaning some core items must be fixed in order
to improve even further, and most people cannot afford dropping out of
job and work 8 weeks on Vim.

The work will be
- defining goals
- fixing them

This work might end
- in rewriting huge parts
- introducing threading (python, ruby ctrl-c does not work)
- have your whatever beloved interpreter as standard interpreter
- add js support (v8)
- compare with Yzis
- think about whether gobjectIntrospection interface can be used to
share work on interfacing with interpreters in the future.
- maybe introducing a new higher level language which is friendly to C
and less complex than C++ - yes, sry - I consider C to be a problem
because its hard to share code, eg reuse the syntax highlighting from
within JS and so on.
- ..

Of course if you think "vim is great the way it is" (I agree)
and if you think "nothing should be changed" I tend to disagree.

If you don't receive much feedback I'll try the kickstarter project
adding features I think are most useful to start with.

Marc Weber

Nikolay Pavlov

unread,
Nov 25, 2013, 11:51:30 AM11/25/13
to vim...@googlegroups.com, vim-dev Mailingliste

This has to be expanded. No doubt it needs to be done, but this statement is too broad.

> - introducing threading (python, ruby ctrl-c does not work)

Ctrl-C is not as much an issue as thread unsafety when it comes to threads.

I guess you will have to start with removing globals scattered over all files. This part can be iterative (a few globals per patch assuming we are not forking and have permission from Bram) and is relatively simple at start.

I was thinking about starting a new series of patches with sole purpose of removing globals, but without a) experience in C multithreading (i.e. knowledge of what ultimate goal requires me to do) and b) permission from Bram or some will to do a job of convincing him (with a) badly hitting my positions) I cannot start it.

> - have your whatever beloved interpreter as standard interpreter

No much sense. VimL is better with its :s/:g/:... commands for quickly writing something simple for text processing. Having command like :pyrepl to run python REPL (and same for other interpreters) would be handy to quickly writing something relatively complex though.

Making some interpreter standard not only risks slowing user down, but also makes it close to impossible to use other people's vim.

> - add js support (v8)

This has to be a low priority, especially considering v8: Gentoo maintainers have masked it for removal as it "does not have stable API resulting in compile breakages in reverse dependencies". There is no way you can refactor vim and fight with this at the same time, and I guess not only Gentoo maintainers will not be willing to deal with it.

Especially considering that there are no plugins written in javascript for an obvious reason and javascript is not the best language you can pick for embedding.

> - compare with Yzis
> - think about whether gobjectIntrospection interface can be used to
>   share work on interfacing with interpreters in the future.
> - maybe introducing a new higher level language which is friendly to C
>   and less complex than C++ - yes, sry - I consider C to be a problem
>   because its hard to share code, eg reuse the syntax highlighting from
>   within JS and so on.

I would consider D in first place. Not that I actually worked with it, but I have read lots of articles convincing me that it is way better then C++.

There are other concerns though: C is portable (with lots of macros, but they are primary isolated), there is a way to use a C function from nearly anywhere (e.g. like with python ctypes, etc), there is a big bunch of things that have C bindings and sometimes are even written in C and it does not forbid any optimization you may come up with.

I do not understand the statement about reusing syntax highlighting though: you either write in JS, something that compiles to it (C *does*) or forget about the idea AFAIK (assuming you are talking about browser js).

I would suggest limiting options to only languages that work on all kernel+architecture combinations debian works.

> - ..
>
> Of course if you think "vim is great the way it is" (I agree)
> and if you think "nothing should be changed" I tend to disagree.
>
> If you don't receive much feedback I'll try the kickstarter project
> adding features I think are most useful to start with.
>
> Marc Weber
>

> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

Marc Weber

unread,
Nov 25, 2013, 12:25:40 PM11/25/13
to vim_use
> Making some interpreter standard not only risks slowing user down, but also
> makes it close to impossible to use other people's vim.
Well - that's already the case - and I don't care. People remap :; in
normal mode only (to reduce stress) and the like.

But starting up Vim is fast, adding conditional code in ~/.vimrc
like if $VIM_CONFIG == "me" ... seems to be reasonable.

VAM/vundle/pathogen support having plugins in directories. So this
is no problem at all.

> There are other concerns though: C is portable (with lots of macros, but
> they are primary isolated), there is a way to use a C function from nearly
> anywhere (e.g. like with python ctypes, etc), there is a big bunch of
> things that have C bindings and sometimes are even written in C and it does
> not forbid any optimization you may come up with.
That's why I want something "D/Haskell/disciple/urweb" like which can
compile down to C, and which knows about "things close to C" so that you
can replace pieces only without having to rewrite all code at once.
And that language does not exist yet to my knowledge.

> I do not understand the statement about reusing syntax highlighting though:
> you either write in JS, something that compiles to it (C *does*) or forget
> about the idea AFAIK (assuming you are talking about browser js).
Ideally syntax is easy: start parsing a document, build up an ast,
compile ast down to something like char 1-20 red, char 21 to 50 blue and
so on.

Thus the interface would look like:

parse_line(parser_state, new_line, more_lines_if_needed){
return {new_parses_state: ..., highlighting_of_this_line}
}

unless you need lookahead or such ..

I can easily imagine writing the parser in a functional language, then
compiling it down to either JS or C.

But yes - getting this right would be a lot of work but eventually worth
trying.

> I would suggest limiting options to only languages that work on all
> kernel+architecture combinations debian works.
:) That's why if a new language was choosen it should be able to target
C, so that compatibility is not affected much.

But I'd like to abstract over lists, hashes, ... without having to start
type casting or write hacky preprocessor macros.

Ideally I'd like to have something which represents vim behavior, then
just tell it "compile to C using C strings" or "compile to C using
arrays as strings so that we can represent \0 bytes".

Something like ocaml/urweb modules, but when instantiating those modules
allow additional arguments such as "memory-management-system: ref-count,
boehm, other"

Yes - this is still uncertain and just an idea.

If somebody knows a project which tries to achieve this let me know.

Marc Weber

Nikolay Pavlov

unread,
Nov 25, 2013, 12:55:56 PM11/25/13
to vim...@googlegroups.com


On Nov 25, 2013 9:23 PM, "Marc Weber" <marco-...@gmx.de> wrote:
>
> > Making some interpreter standard not only risks slowing user down, but also
> > makes it close to impossible to use other people's vim.
> Well - that's already the case - and I don't care. People remap :; in
> normal mode only (to reduce stress) and the like.

I mainly omit mappings changing core commands behavior too much from the original. It does not mean there are no mappings I am too used to. What you suggest is very different: start with vim -u NORC and you get no mappings and trapped in a language you are not familiar with. More, unlike VimL there is exactly no requirement on having other languages. Thus if with vim I can use my own vimrc or, at least, input some mappings by hand, what can I do if there is and *may be no* python? Not all interpreters work on the same OS+arch combination vim does.

> But starting up Vim is fast, adding conditional code in ~/.vimrc
> like if $VIM_CONFIG == "me" ... seems to be reasonable.
>
> VAM/vundle/pathogen support having plugins in directories. So this
> is no problem at all.
>
> > There are other concerns though: C is portable (with lots of macros, but
> > they are primary isolated), there is a way to use a C function from nearly
> > anywhere (e.g. like with python ctypes, etc), there is a big bunch of
> > things that have C bindings and sometimes are even written in C and it does
> > not forbid any optimization you may come up with.
> That's why I want something "D/Haskell/disciple/urweb" like which can
> compile down to C, and which knows about "things close to C" so that you
> can replace pieces only without having to rewrite all code at once.
> And that language does not exist yet to my knowledge.

Bram is creating some language which may or may not fit, do not remember the details right now though.

Marc Weber

unread,
Nov 25, 2013, 1:22:07 PM11/25/13
to vim_use
Excerpts from Nikolay Pavlov's message of Mon Nov 25 18:55:56 +0100 2013:
> I do if there is and *may be no* python? Not all interpreters work on the
> same OS+arch combination vim does.
Then there will be no support - why bother?
Then Vim will be "an editor only" on those platforms. But honestly I
think moste people do use Vim on systems where most interpreters (like
python) are available, too.

Also pay attention that Vim already is a set of compromises.

Example 1:
if_ implementations seem to support therading (scheme), others
don't or don't do that well such as python. It works, but never call
back into Vim from a background Vim thread ..

Example 2: Mappings. In gvim some mappings are available which are not
in vim <m-*>

Yet nobody asked "what about console? Some mappings will not be
available".

If something is not implemented (such as rm -fr), you have to worry
about it in VimL- and VimL might be bad at it.

> Bram is creating some language which may or may not fit, do not remember
> the details right now though.
http://www.zimbu.org/

zimbu is about "everything is an object" and everything "has an
interface". While that's nice it prevents some kind of optimizations.
Some existing systems tend to either "allow optimizing everything" or
"nothing". The perfect system does not "write code", it describes code
which can than be transformed into whatever you need.
For the OO part I agree much with zimbu, however I'd also like to see a
functional/haskell/ocaml like part: everything is data, and at compile
time the compiler chooses the right code to operate on it using type
classes. If you don't have inheritance the difference should not be that
big.

Marc Weber

Nikolay Pavlov

unread,
Nov 25, 2013, 1:46:51 PM11/25/13
to vim...@googlegroups.com


On Nov 25, 2013 10:20 PM, "Marc Weber" <marco-...@gmx.de> wrote:
>
> Excerpts from Nikolay Pavlov's message of Mon Nov 25 18:55:56 +0100 2013:
> > I do if there is and *may be no* python? Not all interpreters work on the
> > same OS+arch combination vim does.
> Then there will be no support - why bother?
> Then Vim will be "an editor only" on those platforms. But honestly I
> think moste people do use Vim on systems where most interpreters (like
> python) are available, too.
>
> Also pay attention that Vim already is a set of compromises.
>
> Example 1:
>   if_ implementations seem to support therading (scheme), others
>   don't or don't do that well such as python. It works, but never call
>   back into Vim from a background Vim thread ..

Do not know about mzscheme (racket), but if_py does not actually support threading: it is cpython that supports it: you are forced to bother with GIL. Though releasing it temporary to work with some vim C functions is not required I can hardly call these bits support.

> Example 2: Mappings. In gvim some mappings are available which are not
> in vim <m-*>
>
> Yet nobody asked "what about console? Some mappings will not be
> available".
>
> If something is not implemented (such as rm -fr), you have to worry
> about it in VimL- and VimL might be bad at it.

There is a big difference between one mapping disabled and missing support for the whole language with a day-to-day use. If one has to know VimL because it is the only option he will survive such situation. If one has not and thus do not know VimL a tiny bit...

And you still have not addressed another part: how will you spell things like :?abc?,/def/s/a/b/i or :g/ghi/norm A$ with python considering it is the default? I do not know a good solution. VimL is good unless you want to write a script with this. Python is very good for scripts, but never try to write a one-liner with it. Perl is good for one-liners as well as the scripts, but it is good for one-liners working with *streams* (e.g. pipes or files) (though there are interesting things like if(/abc/../def/), they are not much helpful if there are multiple lines matching /abc/ above the cursor and you only need the closest one).


>
> > Bram is creating some language which may or may not fit, do not remember
> > the details right now though.
> http://www.zimbu.org/
>
> zimbu is about "everything is an object" and everything "has an
> interface". While that's nice it prevents some kind of optimizations.
> Some existing systems tend to either "allow optimizing everything" or
> "nothing". The perfect system does not "write code", it describes code
> which can than be transformed into whatever you need.
> For the OO part I agree much with zimbu, however I'd also like to see a
> functional/haskell/ocaml like part: everything is data, and at compile
> time the compiler chooses the right code to operate on it using type
> classes. If you don't have inheritance the difference should not be that
> big.
>
> Marc Weber
>

Paolo Bolzoni

unread,
Nov 25, 2013, 5:12:49 PM11/25/13
to vim...@googlegroups.com, vim-dev Mailingliste
On Mon, Nov 25, 2013 at 5:51 PM, Nikolay Pavlov <zyx...@gmail.com> wrote:
> On Nov 25, 2013 5:24 PM, "Marc Weber" <marco-...@gmx.de> wrote:
> I would consider D in first place. Not that I actually worked with it, but I
> have read lots of articles convincing me that it is way better then C++.

I used D for a project and I quickly went back to C++. When it works
fine it is probably better than C++, but in my experience it is rare.
I my opinion all the reason to dislike C++ disappeared with C++11.

Marc Weber

unread,
Nov 25, 2013, 9:00:59 PM11/25/13
to vim_use
Excerpts from Paolo Bolzoni's message of Mon Nov 25 23:12:49 +0100 2013:
> On Mon, Nov 25, 2013 at 5:51 PM, Nikolay Pavlov <zyx...@gmail.com> wrote:
> > On Nov 25, 2013 5:24 PM, "Marc Weber" <marco-...@gmx.de> wrote:
> > I would consider D in first place. Not that I actually worked with it, but I
> > have read lots of articles convincing me that it is way better then C++.
Do you still remember some cases why you disliked D?
Would you mind adding some comments about why you think D lacks?
http://vim-wiki.mawercer.de/wiki/vim-development/ideas/replacing-c-by.html
Trying to collect arguments..

C++11: About C++ I dislike that its a lot more verbose than eg haxe.
Example:
var m = new Hash<Int,Sting>();
^^^ and these types can even be omitted
m.set(2,"abc");

But I agree that C++(11) at least fixes the most urgent problems, it
provides typed lists/hashes/...

I also always wondered why I have to maintain .h and .c* files.
Why doesn't the compiler generate the .h files? (probably CPP #if reasons)

Marc Weber

Marc Weber

unread,
Nov 25, 2013, 9:16:02 PM11/25/13
to vim_use
Excerpts from Nikolay Pavlov's message of Mon Nov 25 19:46:51 +0100 2013:
> Do not know about mzscheme (racket), but if_py does not actually support
> threading: it is cpython that supports it:
The problem is that Vim does not expect called by Vim unless its VimL
calling into Vim or such. Python does support passing CPU to multiple
python threads.
But if such thread wants to tell vim "hey, i'm ready", it cannot be done
easily. You have to put the thread into "waiting/ready state", then you
have to make Vim poll for that state. Otherwise Vim might crash.
And that polling is hard to implement without delay. I know about on
idel events and such, but then you might collide with other plugins.
So whatever you do its a hack only.

> And you still have not addressed another part: how will you spell things
> like :?abc?,/def/s/a/b/i or :g/ghi/norm A$ with python considering it is

Let's say it this way:
:%s/* are just "viml commands"
Thus ":%s/" would no exist if you changed default interpreter.
Eg let's assume you'd choose python as default, then you would have to:
:vim.eval('%....')
or wirte a substitute function in python in global scope.

q: /: ?: already show how you can have multiple "mini buffers" for
different tasks. So why not have a p: or a v: to get python/viml command
lines ? (I know p is paste ..)

Marc Weber

Nikolay Pavlov

unread,
Nov 25, 2013, 10:46:05 PM11/25/13
to vim...@googlegroups.com


On Nov 26, 2013 6:14 AM, "Marc Weber" <marco-...@gmx.de> wrote:
>
> Excerpts from Nikolay Pavlov's message of Mon Nov 25 19:46:51 +0100 2013:
> > Do not know about mzscheme (racket), but if_py does not actually support
> > threading: it is cpython that supports it:
> The problem is that Vim does not expect called by Vim unless its VimL
> calling into Vim or such. Python does support passing CPU to multiple
> python threads.
> But if such thread wants to tell vim "hey, i'm ready", it cannot be done
> easily. You have to put the thread into "waiting/ready state", then you
> have to make Vim poll for that state. Otherwise Vim might crash.
> And that polling is hard to implement without delay. I know about on
> idel events and such, but then you might collide with other plugins.
> So whatever you do its a hack only.
>
> > And you still have not addressed another part: how will you spell things
> > like :?abc?,/def/s/a/b/i or :g/ghi/norm A$ with python considering it is
>
> Let's say it this way:
>   :%s/* are just "viml commands"
> Thus ":%s/" would no exist if you changed default interpreter.
> Eg let's assume you'd choose python as default, then you would have to:
>   :vim.eval('%....')
> or wirte a substitute function in python in global scope.

Function is going to be verbose. And what is the point in picking default interpreter if you have to use VimL in such a weird way?

Also note that you cannot use python re without joining buffer into one big string and then splitting. Guess the same with other interpreters.

> q: /: ?: already show how you can have multiple "mini buffers" for
> different tasks. So why not have a p: or a v: to get python/viml command
> lines ? (I know p is paste ..)

I already said that having command like :pyrepl may be handy: there even exist some plugins for this task. No need to take any keys for this though.

lith

unread,
Nov 26, 2013, 8:45:53 AM11/26/13
to vim...@googlegroups.com
There definitely always is some space for improvements. I'd nevertheless like to ask if you really think that more of the same is a significant improvement. Vim is a versatile editor the way it is. I still wish it would have emacs-like overlays and maybe js as scripting languagee, since I'm convinced js will gain more dominant in the years to come but it really doesn't matter that much.

Rather than a slightly improved version of vim, I'd like to see a free version of vim for eclipse that is able to run any vim plugin and fully integrates with eclipse (though eclim already is great) and/or a browser/web based version of vim that runs any vim plugin and that works with the local file system or files on some server.

Just my 2c :-)

Jeff Schwartz

unread,
Nov 26, 2013, 9:02:47 AM11/26/13
to vim...@googlegroups.com
To me, vim is more than an editor... Vim is a language which I as well as many have come to embrace. Yet, as much as I love the modal dialect of vim, I rarely use vim proper anymore and instead use Sublime Text 3 with vim emulation and though it isn't a full blown vim experience, the emulation on top of a modern editor is enough for me as well, apparently, as it is for many others. I'd love a new vim shell crafted to the likes of sublime text that fully supports package management and command and visual mode, something which most vim emulators do not. My most frustrating editing moments are whenever I use vim and that's just sad. Give me a reason to go back to vim and I will and I'd gladly offer to help in the effort.

Gary Johnson

unread,
Nov 26, 2013, 11:46:29 AM11/26/13
to vim...@googlegroups.com
Someone might be able to give you such a reason, but it's impossible
to do so without knowing _why_ you are frustrated using Vim and
_what_ those frustrating moments are.

Regards,
Gary

Paolo Bolzoni

unread,
Nov 26, 2013, 11:56:41 AM11/26/13
to vim...@googlegroups.com
Since we are in the topic of improvements, what about a real
rectangular selection?

Example:
sssssssss
xxxx
yyyyyyyyyyyy
zzzzz

Select everything a part of the right-most s and
the four left-most y?

Marc Weber

unread,
Nov 27, 2013, 1:04:23 AM11/27/13
to vim_use
Excerpts from Paolo Bolzoni's message of Tue Nov 26 17:56:41 +0100 2013:
> Example:
> sssssssss
> xxxx
> yyyyyyyyyyyy
> zzzzz
>
> Select everything a part of the right-most s and
> the four left-most y?
I don't understand what you're looking for.
Can you mark the characters to select by making them upcase?

Vim also has :h 'virtualedit'

Marc Weber

Konovalov, Vadim

unread,
Nov 27, 2013, 1:02:18 AM11/27/13
to vim...@googlegroups.com, vim...@vim.org, vim...@googlegroups.com
> From: vim...@googlegroups.com On Behalf Of Michael Jarvis

> On Monday, November 25, 2013 7:26:42 AM UTC-6, MarcWeber wrote:
> > I've been complaining about Vim related issues for a long time,
> > I think its time to stop complaining and just fix it.
....

> > The idea is to create a kickstarter project to funding all work.
> > IMHO Vim is worth keeping alive, and that means we must find a way
> > to move Vim into the future.
> >
> > If this requires writing a new language, because C lacks abstracktions,
> > and C++ is complex, then that's the task to be done IMHO.

Sounds naïve.
Does the idea have something brilliant in it that makes it different from
a dozen of already failed projects in this list -

http://www.freehackers.org/VimIntegration

?

>
>
> Personally I think that rewriting in a new language would mean that it's not
> Vim any more. If you want to create a Vim-like IDE, written in another
> language, with more functionality, then it would be a new product. Maybe
> call it Vim++, or SuperVim, or something similar, but it should be a fork of
> Vim and not a total rewrite of the current editor.

I add that it would be *yet another* fork, without a chance to reach code stability even, IMO.

Vadim.

Marc Weber

unread,
Nov 27, 2013, 1:07:59 AM11/27/13
to vim_use
Excerpts from Nikolay Pavlov's message of Mon Nov 25 17:51:30 +0100 2013:
> Making some interpreter standard not only risks slowing user down, but also
> makes it close to impossible to use other people's vim.
Are you talking about pairing? I mean it would be possible to either
introduce "collaborative editing", or it would be possible to have
multiple user profiles eventually so that you can switch mappings to
match user's A preferences or usver's B preferences without restarting
Vim. (Of course there a are a lot of details to be considered).
But if that's a common issue maybe we can address it.

Marc Weber

Marc Weber

unread,
Nov 27, 2013, 1:28:14 AM11/27/13
to vim_use
Dear Konovalov, Vadim

> Sounds naïve.
Maybe it is. But if you don't try you've failed from the very beginning ;)

But let me give you an example:

sturt foo {
char bar[LENGTH]
int something_else;
}

foo s;

for (i = 0, i <= LENGTH, i++){
s.bar[i] = '\x0';
}

Of coure there is a bug, and this happens.

Now the code above overrides s.something_else by accident.
This can lead to hard to find bugs.

How could such be fixed? Introducing type classes:

class MemoryManagementFor foo {

memory_layout = # compile time thing describing how to layout the
# struct fields in memory.
# if you have "debug mode", separate struct fields by
# 8 bytes
# thus this code gets evaluated at compile time and
# returns a list of "struct fields" and memory
# position/sizes.
# This would also add a check sum field, see new/free

new :: # alloc such a thing, zero byte it, in debug mode fill those
# dummy 8 bytes by random bytes, and calculate a check sum

free :: # free such a thing, if debug mode, check that the check sum
# still matches the initial random bytes

}

Now if free detects failure, you know where to start looking for bugs.
When could such an implementation possibly fail? If you want to "stream"
to a file, having random bytes could be harmfull. Again, you just create
a new type class

class WriteToFileHandle where
write handle struct = # if debug then omit random bytes

and you're done. Now tell me how I can do that with C/C++?
Maybe such solutions exist. I'm pretty confident that such tooling built
into a compiler does help gitting stable code much more than valgrind.

Now you can continue with

foo a = new(); // and get the debug layout eventually.
free(a): // and this would check that no memory corruption happened.

But if you know a tool which is as smart as I told above, let me know
and teach me, please. I know that this can be done. And in fact I want
to do that. Thus if thus tooling is not available yet, I have to write
it.

> Does the idea have something brilliant in it that makes it different from
> a dozen of already failed projects in this list -
> http://www.freehackers.org/VimIntegration
Thanks for sharing this nice list. I've added this link to my wiki.

Still collecting ideas from community. I'v already published my own list
of things I'd like to fix. Whether that's enough to become more
brilliant is something you have to judge.

Marc Weber

Christian Brabandt

unread,
Nov 27, 2013, 2:08:00 AM11/27/13
to vim...@googlegroups.com
On Tue, November 26, 2013 17:56, Paolo Bolzoni wrote:
> Since we are in the topic of improvements, what about a real
> rectangular selection?
>
> Example:
> sssssssss
> xxxx
> yyyyyyyyyyyy
> zzzzz
>
> Select everything a part of the right-most s and
> the four left-most y?

You can already do that. :set virtualedit=all and then continue
as usual.

Best,
Christian

Paolo Bolzoni

unread,
Nov 27, 2013, 7:53:22 AM11/27/13
to vim...@googlegroups.com
I like virtualedit=block more, but thanks for
pointing me in the right direction!

herm...@free.fr

unread,
Nov 27, 2013, 8:44:42 AM11/27/13
to vim use
Hello,


> But let me give you an example:
>
> sturt foo {
> char bar[LENGTH]
> int something_else;
> }
>
> foo s;
>
> for (i = 0, i <= LENGTH, i++){
> s.bar[i] = '\x0';
> }
> [...]
> How could such be fixed?
> Now tell me how I can do that with C/C++?

This is a easy one: embrace C++11 !
struct foo {
std::array<char, LENGTH> bar;
int somethine_else;
};

Then, you have the choice between the following buffer-overrun-resistant solutions:
- s.bar.fill('\x0');
- for(char & c : s.bar) c = '\x0';
- std::fill(begin(s.bar), end(s.bar), '\x0')

There are other solutions based on iterators where typographic errors may have the same overflow result
for (char *c = std::begin(s.bar), E=std::end(s.bar)
; c != E // <- here, old school developers may write c<E, or worse: c<=E
; ++c)
*c = '\x0';


The difficulty with C++, is to force ourselves to forget we also know C.
(BTW, C/C++ is a beast that shall be hunt down. Either code in C, or in C++. Never try to import C good practices in C++ because they don't apply)

You could also have used plain static arrays, the second and third solutions would have still worked.


> Maybe such solutions exist. I'm pretty confident that such tooling
> built into a compiler does help gitting stable code much more than
> valgrind.

Indeed. It's always better to find errors at compilation stage, or at least in test units.


> Now you can continue with
>
> foo a = new(); // and get the debug layout eventually.
> free(a): // and this would check that no memory corruption happened.

Again. Forget free/delete exist and embrace smart-pointers. Not necessarily std::shared_ptr<>, but at least the light and fast std::unique_ptr<>.


> But if you know a tool which is as smart as I told above, let me know
> and teach me, please. I know that this can be done. And in fact I
> want to do that. Thus if thus tooling is not available yet, I have to
> write it.

The way that C++ is taught is the problem. One of the most interesting article that indirectly demonstrates why we should embrace other ways to organize our code is the following : http://ra3s.com/wordpress/dysfunctional-programming/return-code-vs-exception-handling/

Anyway. Use whatever language you master. But don't invent/create a new one for this project -- I hope I misread the first message about choosing a new language to implement this fork of vim.

------

On another topic as a plugin writer that has always forced himself to stick to viml (for portability issues), it would be nice to have viml supported (for ascendant compatibility). I've written to many things, and I don't want to leave and loose everything.


--
Luc Hermitte
http://lh-vim.googlecode.com/

Paolo Bolzoni

unread,
Nov 27, 2013, 10:12:18 AM11/27/13
to vim...@googlegroups.com
It is nice to see someone which actually knows C++ once in a while...
In my environment there is no-one.

Saad Malik

unread,
Nov 27, 2013, 10:52:59 AM11/27/13
to vim...@googlegroups.com
Yes! I'd contribute to a Kickstarter campaign for enhancing ViM.
* true async support in the core. Remove notions of updatetime since everything will be async
* remove backwards compatibility to Vi. I'm assuming that vi backwards compatibility is hard to maintain? At the very least make nocompatible the default
* replace cryptic one letter options like coptions
* integrate popular plugins into the core: syntastic, ctrlp, nerdtree, etc
* ability to renumber buffer numbers. This enhancement alone I would pay for.
* concept of window-local buffers and tab-local buffers
* similar to window location list, a window header list that sits on top of the window. Plugins like bufexplorer could show window local buffers in this header

Marc Weber

unread,
Nov 27, 2013, 11:35:53 AM11/27/13
to vim_use
Excerpts from hermitte's message of Wed Nov 27 14:44:42 +0100 2013:
> The difficulty with C++, is to force ourselves to forget we also know C.
> (BTW, C/C++ is a beast that shall be hunt down. Either code in C, or
> in C++. Never try to import C good practices in C++ because they don't
> apply)
Historically I was told that C++ was written to "reuse the knowledge
of programmers because C programmers did know C".

So how do you rewrite an application such as Vim using C++ without
rewriting it if you should be using only one of C or C++ ?

If I choose to create a new artifical language, I possibly could write
something turning C into my new language, then do what I said to test
huge amounts of C code by adding runtime checks.

Your points are valid. In Haskell I can do:

data Struct =
name :: String,
list :: List (of) Int,
age :: Int

deriving (Show, Read)
^ this is built into the compiler

$(derive [serialize-to-binary, memory-alloc, memory-check])
^ this is "template haskell", which derives class instances for me.

So how I can I possibly make C++ derive a "new(my struct)" which
calls s.bar.fill('\x0') (and does this for each field of the struct) ?

I know that postgresql invents his own preprocessor to make it easier
using SQL in C/C++. But somehow I feel that template haskell like macro
preprocessing is the cleaner solution.

That's what I'm missing and looking for, too.

> Indeed. It's always better to find errors at compilation stage, or at
> least in test units.
Again - who can afford rewriting C? So at least I want to have the
tooling.. but I don't. You should have notice that I'm probably not a
perfect C programmer, but I cannot even imagine becoming one for those
easons.

> Again. Forget free/delete exist and embrace smart-pointers. Not
> necessarily std::shared_ptr<>, but at least the light and fast
> std::unique_ptr<>.
So you think C++ is more than ever a valid replacement for C in the Vim
case. I twould still be interesting to learn why Yzis failed, and how to
prevent such failure.

> Anyway. Use whatever language you master. But don't invent/create a
> new one for this project -- I hope I misread the first message about
> choosing a new language to implement this fork of vim.
I'm wondering how synergy can be shared between Yi and Vim for example.
One is C++, one is Haskell. There might be pieces to share.
In which language to write those in?
That's why I came up with the idea of a language which translates to
both Haskell/Elisp/C++ to at least share "semantic syntax highlighting"
which gets discussed more and more often.

Eg why not allow having some code "run in the browser" ?
I know about llvm -> JS, I don't consider that to be an option, though.

Those are the reasons why I think about "new languages", too.
> On another topic as a plugin writer that has always forced himself to
> stick to viml (for portability issues), it would be nice to have viml
> supported (for ascendant compatibility). I've written to many things,
> and I don't want to leave and loose everything.
I know. VimL is lacking features. But I agree. That's why I consider
this to be "must have". That's why I think truning viml into a
python/ruby/lua/ like standalone project would be worth it.

CcxCZ told me about http://pypy.org which somehow illustrates how
multiple backends could be served.

Marc Weber

zvav...@gmail.com

unread,
Nov 27, 2013, 12:14:04 PM11/27/13
to vim...@googlegroups.com

zvav...@gmail.com

unread,
Nov 27, 2013, 12:14:36 PM11/27/13
to vim...@googlegroups.com

zvav...@gmail.com

unread,
Nov 27, 2013, 12:16:38 PM11/27/13
to vim...@googlegroups.com

Marc Weber

unread,
Nov 27, 2013, 11:52:28 AM11/27/13
to vim_use
> * true async support in the core.
> Remove notions of updatetime since everything will be async
I don't understand this second sentence, example?

> * remove backwards compatibility to Vi. I'm assuming that vi backwards
> compatibility is hard to maintain? At the very least make nocompatible
> the default
:-) Yeah. I agree about a lot - or make non compatible setting the
default.

> * replace cryptic one letter options like coptions
by what?

> * integrate popular plugins into the core: syntastic, ctrlp, nerdtree,
> etc
I think this would be fixed by moving vimruntime to github soon .. :)

> * ability to renumber buffer numbers. This enhancement alone I would
> pay for.
Use case? Why don't you just use aliases like b:.. and write your own
function to access buffers by alias?
Would be about 10 lines of VimL code probably ?
[1]

command

> * concept of window-local buffers and tab-local buffers
Please elaborate

Also add your ideas to this page, please:
http://vim-wiki.mawercer.de/wiki/topic/in-which-way-does-vim-suck.html

Then they won't get lost ..

Marc Weber

[1]

if !exists(g:my_buf_list)
let g:my_buf_list = {}
endif

fun SetBNr(nr)
let b:nr = a:nr
let g:my_buf_list[a:nr] = buffer_number('.')
endf

command -nargs=1 SetBufNr call SetBNr(<f-args>)

" completion missing:
command -nargs=1 GotoBuf exec '.g:my_buf_list[<f-args>]

Also see github.com/MarcWeber/vim-addon-other which implements <m-1>
<m-2> access for tabs or use :b some-chars-only which also works
reasonably well often.

Don't forgett about mA mB and access files by 'A 'B

Marc Weber

herm...@free.fr

unread,
Nov 27, 2013, 12:23:28 PM11/27/13
to vim use
> Excerpts from hermitte's message of Wed Nov 27 14:44:42 +0100 2013:
> > The difficulty with C++, is to force ourselves to forget we also
> > know C.
> > (BTW, C/C++ is a beast that shall be hunt down. Either code in C,
> > or in C++. Never try to import C good practices in C++ because
> > they don't apply)
> Historically I was told that C++ was written to "reuse the knowledge
> of programmers because C programmers did know C".

This strength is also a weakness. It's easy to add C++ constructs there and there in a C program. But in the end, good practices about programming robust applications are not the same.


> So how do you rewrite an application such as Vim using C++ without
> rewriting it if you should be using only one of C or C++ ?

That's a good question. I've never delved into Vim code, and as such I can't tell you what's best: a complete rewrite or adding things there and there. In case it's of any help: GCC has been migrated to C++ ( https://lwn.net/Articles/542457/ ).


> If I choose to create a new artificial language, I possibly could
> write something turning C into my new language, then do what I
> said to test huge amounts of C code by adding runtime checks.

If you write a new language, you'll have two kind of bugs to track down : the ones from your language, and the one from the new-vim application written in that language. This represents a tremendous effort. Moreover, there is a huge risk that (almost) nobody will join until the final product has again enough stability, and community.


> Your points are valid. In Haskell I can do:
>
> data Struct =
> name :: String,
> list :: List (of) Int,
> age :: Int
>
> deriving (Show, Read)
> ^ this is built into the compiler
>
> $(derive [serialize-to-binary, memory-alloc, memory-check])
> ^ this is "template haskell", which derives class instances for me.
>
> So how I can I possibly make C++ derive a "new(my struct)" which
> calls s.bar.fill('\x0') (and does this for each field of the struct)
> ?

[I may have not understood your question as I don't know haskell]
By using the constructors ? So that nothing will be left uninitialized. One of the good practices in C++ (which is not as spread in C because of C89 inability to have variables declared anywhere but at the beginning of scopes) is to never have variables exist before you are able to initialize them to a pertinent value. If you can initialize them to a final value, it's even better.

> > Again. Forget free/delete exist and embrace smart-pointers. Not
> > necessarily std::shared_ptr<>, but at least the light and fast
> > std::unique_ptr<>.
> So you think C++ is more than ever a valid replacement for C in the
> Vim case. I twould still be interesting to learn why Yzis failed, and how
> to prevent such failure.

If you ask me, C++ is a more valid candidate than C for any _new_ development -- the reasons lie in the article I've referenced in my previous post: C++ RAII makes it simple to manage resources. I reserve C for the cases I really have absolutely no other choice (mainly because of client wishes, or because of very exotic platforms).
But I don't want to start a flame war, this is not the place.

Here, what do you want to do ? Start a new editor from scratch ? Fork vim and have new dynamics regarding patch integration & so on ?
In the later case, C++ may not be a solution, at all.

Regarding Yzis, I'm not sure the problem was technical. Gathering a community around a forked product that breaks habits (like for instance plugins, didn't it?) is certainly not that trivial.

ZyX

unread,
Nov 27, 2013, 12:42:24 PM11/27/13
to vim...@googlegroups.com
On Wednesday, November 27, 2013 7:52:59 PM UTC+4, Saad Malik wrote:
> Yes! I'd contribute to a Kickstarter campaign for enhancing ViM.
> * true async support in the core. Remove notions of updatetime since everything will be async

True asynchronous support is badly needed. But for compatibility with existing plugins updatetime must not be removed.

> * remove backwards compatibility to Vi. I'm assuming that vi backwards compatibility is hard to maintain? At the very least make nocompatible the default

I do not remember when I have last seen bugs here related to 'cpoptions'. Thus I assume it is not.

> * replace cryptic one letter options like coptions

Not replace, but rather add new interface for them.

> * integrate popular plugins into the core: syntastic, ctrlp, nerdtree, etc

Vim philosophy never contained “batteries included” like python does. If you want them out-of-the-box you have to use things like janus and it is good. Integrating them into the core will badly hit their competitors and I definitely not willing to see ctrlp in the core since I use Command-T and saw ctrlp source code.

> * ability to renumber buffer numbers. This enhancement alone I would pay for.

After I started using Command-T I nearly never use buffer numbers (except for the plugins). And doing this will introduce a huge amount of bugs because nearly *every* plugin that deals with buffers assumes that buffers cannot be renumbered.

> * concept of window-local buffers and tab-local buffers

You can emulate it to some extent. I would rather request API for digging deeper into vim core to emulate it to greater extent if you need. Never saw a need for such a thing since switching buffers with Command-T is pretty fast.

> * similar to window location list, a window header list that sits on top of the window. Plugins like bufexplorer could show window local buffers in this header

Better to add events like WinMovePre/WinMovePost so that one can emulate sticking to window anywhere he needs and not only to the top. Also note that as quickfix window with locations list is mostly a regular window it can be moved anywhere from the location it was opened (including moving to the other tab, though it would render this window useless as far as I see).

Bram Moolenaar

unread,
Nov 28, 2013, 6:10:38 AM11/28/13
to Michael Jarvis, vim...@googlegroups.com, vim...@vim.org, vim...@googlegroups.com

Michael Jarvis wrote:

> On Monday, November 25, 2013 7:26:42 AM UTC-6, MarcWeber wrote:
> > I've been complaining about Vim related issues for a long time,
> >
> > I think its time to stop complaining and just fix it.
> >
> >
> >
> > Join by providing feedback:
> >
> > http://mawercer.de/vim.php
> >
> >
> >
> > or adding additional issues to be fixed here:
> >
> > http://vim-wiki.mawercer.de/wiki/topic/in-which-way-does-vim-suck.html
> >
> >
> >
> > The idea is to create a kickstarter project to funding all work.
> >
> > IMHO Vim is worth keeping alive, and that means we must find a way
> >
> > to move Vim into the future.
> >
> >
> >
> > If this requires writing a new language, because C lacks abstracktions,
> >
> > and C++ is complex, then that's the task to be done IMHO.
>
>
> Personally I think that rewriting in a new language would mean that
> it's not Vim any more. If you want to create a Vim-like IDE, written
> in another language, with more functionality, then it would be a new
> product. Maybe call it Vim++, or SuperVim, or something similar, but
> it should be a fork of Vim and not a total rewrite of the current
> editor.
>
> I think Vim is wonderful as it is for my purposes, and it's currently
> the "default" Vi on many many Linux computers across the world. We
> don't want to change that by starting over, even if we do try to
> re-use some of the code.
>
> This article from Joel Spolsky sums up why I think rewriting would be
> a bad idea: http://www.joelonsoftware.com/articles/fog0000000069.html

Correct. Vim overtook Elvis, because the author of Elvis decided to do
a complete rewrite. After a couple of years of development something
came out that didn't quite work and was missing features.

I do think that C is an old language and writing high level code in it
is tough. Especially when it needs to be portable. A possible way out
is to write code in Zimbu, which compiles into C, so that parts can be
(re)written in Zimbu while still using existing code in C. But Zimbu
isn't ready yet...


--
Often you're less important than your furniture. If you think about it, you
can get fired but your furniture stays behind, gainfully employed at the
company that didn't need _you_ anymore.
(Scott Adams - The Dilbert principle)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

Ed Kostas

unread,
Dec 1, 2013, 10:01:42 AM12/1/13
to vim...@googlegroups.com, vim...@vim.org
On Monday, November 25, 2013 11:26:42 AM UTC-2, MarcWeber wrote:
> I've been complaining about Vim related issues for a long time,
>
> I think its time to stop complaining and just fix it.
>
>

I agree. Then let us fix it.


>
> Join by providing feedback:
>
> http://mawercer.de/vim.php
>

I want to join, but I receive the following message when I try:

Forbidden

You don't have permission to access /x/php.fcgi/index.php on this server.


>
> or adding additional issues to be fixed here:
>
> http://vim-wiki.mawercer.de/wiki/topic/in-which-way-does-vim-suck.html
>

I receive the above message when I try to access this link too.


Please, let me know what can I do for joining. Let me be clear in what ways I can help. I am a reasonable programmer. I worked as a programmer at Cornell University (this was a long time ago ... :-). Recently, I worked at Utah State University as a programmer (well, my title was Visiting Professor, but what I did was programming). Finally, I helped lawyers in creating databases, textual search, syntactic analyzers for legal texts, communication programs to access higher courts, etc. As for languages, my experience are: Pascal, Visual Prolog, Haskell, C, Clean, Lisp, Ocaml, Scheme.

A suggestion: Write a small starting point in the language of your choice, and post it in your page. This starting point should include cursor movements, erase operations, saving operations, clipboard operations, parentheses matching, and a primitive syntax highlighting scheme. You could invite people to post this kind of editor kernel in your page too. Use ncurses for the kernel. I am talking about a couple of hours of work.

Based on this kernel, people involved could have a better base for choosing the language. I even don't discard implementations in many languages. For instance, if a group wants to write an Ocaml version, let them do it.

Please, fix the links so I can visit your pages.



> The idea is to create a kickstarter project to funding all work.
>
> IMHO Vim is worth keeping alive, and that means we must find a way
>
> to move Vim into the future.
>
>
>
> If this requires writing a new language, because C lacks abstracktions,
>
> and C++ is complex, then that's the task to be done IMHO.
>
>
>

> This project makes me think we might have success:
>
> http://www.kickstarter.com/projects/maxcantor/beautiful-vim-cheat-sheet-poster?ref=live
>
>
>
> Thus if you're either a developper or a user who wants to help join and
>
> tell me what you want to work on - even if its "testing new features you
>
> care about" only.
>
>
>
> Goals are:
>
> - focus on productivity
>
> - code reusage
>
>
>
> I consider refactoring viml a key thing, eg creating a viml library
>
> which contains the interpreter only.
>
>
>
> If you'll help me with this I'll be working on Vim related topics the
>
> following month, otherwise I'll get any alternative job.
>
>
>
> I feel I've hit a border meaning some core items must be fixed in order
>
> to improve even further, and most people cannot afford dropping out of
>
> job and work 8 weeks on Vim.
>
>
>
> The work will be
>
> - defining goals
>
> - fixing them
>
>
>
> This work might end
>
> - in rewriting huge parts
>
> - introducing threading (python, ruby ctrl-c does not work)
>
> - have your whatever beloved interpreter as standard interpreter
>
> - add js support (v8)
>
> - compare with Yzis
>
> - think about whether gobjectIntrospection interface can be used to
>
> share work on interfacing with interpreters in the future.
>
> - maybe introducing a new higher level language which is friendly to C
>
> and less complex than C++ - yes, sry - I consider C to be a problem
>
> because its hard to share code, eg reuse the syntax highlighting from
>
> within JS and so on.
>
> - ..
>
>
>
> Of course if you think "vim is great the way it is" (I agree)
>
> and if you think "nothing should be changed" I tend to disagree.
>
>
>
> If you don't receive much feedback I'll try the kickstarter project
>
> adding features I think are most useful to start with.
>
>
>
> Marc Weber

Marc Weber

unread,
Dec 2, 2013, 12:25:03 AM12/2/13
to vim_use
Excerpts from Ed Kostas's message of Sun Dec 01 16:01:42 +0100 2013:
> I want to join, but I receive the following message when I try:
> You don't have permission to access /x/php.fcgi/index.php on this server.
Your message to the mailinglist has been enough, I took notes.

The pages will be back soon, I've problem with php-fpm access rights for
some wired reasons after trying an update.

> > http://vim-wiki.mawercer.de/wiki/topic/in-which-way-does-vim-suck.html
> I receive the above message when I try to access this link too.

You can read the source @ github, too:
https://github.com/MarcWeber/vim-git-wiki/blob/master/vim-online-wiki-source/topic/in-which-way-does-vim-suck

I think the first important thing is to collect things to fix, then
workout how to fix [1] them, then estimate costs and try to get funding.

Thanks for your interest.

Marc Weber

[1]: And I don't know this yet. This will take some time.

Nikolay Pavlov

unread,
Dec 2, 2013, 2:41:43 AM12/2/13
to vim...@googlegroups.com


On Dec 2, 2013 9:25 AM, "Marc Weber" <marco-...@gmx.de> wrote:
>
> Excerpts from Ed Kostas's message of Sun Dec 01 16:01:42 +0100 2013:
> > I want to join, but I receive the following message when I try:
> > You don't have permission to access /x/php.fcgi/index.php on this server.
> Your message to the mailinglist has been enough, I took notes.
>
> The pages will be back soon, I've problem with php-fpm access rights for
> some wired reasons after trying an update.
>
> > > http://vim-wiki.mawercer.de/wiki/topic/in-which-way-does-vim-suck.html
> > I receive the above message when I try to access this link too.
>
> You can read the source @ github, too:
> https://github.com/MarcWeber/vim-git-wiki/blob/master/vim-online-wiki-source/topic/in-which-way-does-vim-suck
>
> I think the first important thing is to collect things to fix, then
> workout how to fix [1] them, then estimate costs and try to get funding.

I think that the most problematic thing to fix is threading. You cannot add thread safety without a very huge rework that starts with removing global state (unless we are going to say that GIL is good enough). And it will be problematic to add other changes while refactoring is in progress.

From my point of view this list should look like this:

- Threading.
- Making vim embeddable (making it second point since it is even more refactoring).
- Input (I mean this constantly popping out discussion with vim not supporting certain keys).
- More input (iminsert and keymaps are not good at all at handling people using more then one keyboard layout: gvim needs a way to get actual keys pressed and process them as if they were user English layout (not hardcoded us keymap) regardless of actual layout when not in insert or command(?) modes).
- Stripping requirement to use VimL in Python/Ruby/Lua/etc interface. I.e. finishing what I started (I guess I can do this) for python and doing the similar thing for other languages.
- Zero byte handling. Low priority since this is uncommon to have in text files, but in the current state you have to write ugly hacks to make it work. Should probably be the third as it implies refactoring string implementation. Possible solutions I see: escaping like in zsh (note that it is done the way you never notice) or struct {size_t len; char str[1];} (where you allocate string using usual alloc(sizeof(str_T) + len) hack).
- Unicode support in regular expressions.
- Possibly sane unicode indexing (may require further string refactoring to be efficient).
- Non-string funcrefs (i.e. inclusion of my patch (without lambda part)). It contains fixes for some problems.
- Stripping requirement to use shell to run applications.

> Thanks for your interest.
>
> Marc Weber
>
> [1]: And I don't know this yet. This will take some time.
>

Marc Weber

unread,
Dec 2, 2013, 8:25:37 AM12/2/13
to vim_use
Hi Nikolay

I've turned your suggestion into a page about order of steps to fix Vim:
http://vim-wiki.mawercer.de/wiki/vim-development/fixing-vim-next-steps.html

Have a look at the (Comment ..) text, please add some samples. Code can
be added by
{{{
...
}}}
I've also added you as collaborator to https://github.com/MarcWeber/vim-git-wiki.
See http://vim-wiki.mawercer.de/wiki/this-wiki/contributing.html

Duplicating my comments here so that items can be discussed.
(If you reply here I'll just update the comments)

Excerpts from Nikolay Pavlov's message of Mon Dec 02 08:41:43 +0100 2013:
> - Making vim embeddable (making it second point since it is even more
> refactoring).
Keep in mind that Yzis team failed on this, see
http://www.freehackers.org/VimIntegration (which was posted in this
thread earlier)

> - Stripping requirement to use VimL in Python/Ruby/Lua/etc interface. I.e.
> finishing what I started (I guess I can do this) for python and doing the
> similar thing for other languages.
I don't get you, can you elaborate?

> - Unicode support in regular expressions.
Can you also provide a simple use case, here ?

> - Possibly sane unicode indexing (may require further string refactoring to
> be efficient).
Use case ?

> - Non-string funcrefs (i.e. inclusion of my patch (without lambda part)).
> It contains fixes for some problems.
Again, just add a section to http://vim-wiki.mawercer.de/wiki/topic/in-which-way-does-vim-suck.html

> - Stripping requirement to use shell to run applications.
Again, a use case would help much understanding what you mean exactly.

Marc Weber

Mikołaj Machowski

unread,
Dec 2, 2013, 9:28:17 AM12/2/13
to vim_use
Dnia Poniedzia�ek, 2 Grudnia 2013 14:25 Marc Weber <marco-...@gmx.de> napisa�(a)
>
> Excerpts from Nikolay Pavlov's message of Mon Dec 02 08:41:43 +0100 2013:
> > - Making vim embeddable (making it second point since it is even more
> > refactoring).
> Keep in mind that Yzis team failed on this, see
> http://www.freehackers.org/VimIntegration (which was posted in this
> thread earlier)

Yzis failed mainly because it was ugly hack around Qt event loop which meant (among others) it was sloooooow. Since then Qt evolved and building and maintenance of 'qvim' should be much easier.

> > - Unicode support in regular expressions.
> Can you also provide a simple use case, here ?

Just guessing: [a-z] automatically cover national diacritics without need to list them separately eg [a-z��濟�] (this case would be covered by [:alpha:] but it is simplest example of usage)

> > - Possibly sane unicode indexing (may require further string refactoring to
> > be efficient).
> Use case ?

Vim doesn't recognize how many bytes are in character and indexing of strings may broke easily which is awkward: Vim is *text* editor and should deal with characters not bytes.

> > - Stripping requirement to use shell to run applications.
> Again, a use case would help much understanding what you mean exactly.

Wouldn't proper threading take care about that problem?

m.


Nikolay Pavlov

unread,
Dec 2, 2013, 9:56:25 AM12/2/13
to vim...@googlegroups.com


On Dec 2, 2013 5:25 PM, "Marc Weber" <marco-...@gmx.de> wrote:
>
> Hi Nikolay
>
> I've turned your suggestion into a page about order of steps to fix Vim:
> http://vim-wiki.mawercer.de/wiki/vim-development/fixing-vim-next-steps.html
>
> Have a look at the (Comment ..) text, please add some samples. Code can
> be added by
> {{{
> ...
> }}}
> I've also added you as collaborator to https://github.com/MarcWeber/vim-git-wiki.
> See http://vim-wiki.mawercer.de/wiki/this-wiki/contributing.html
>
> Duplicating my comments here so that items can be discussed.
> (If you reply here I'll just update the comments)
>
> Excerpts from Nikolay Pavlov's message of Mon Dec 02 08:41:43 +0100 2013:
> > - Making vim embeddable (making it second point since it is even more
> > refactoring).
> Keep in mind that Yzis team failed on this, see
> http://www.freehackers.org/VimIntegration (which was posted in this
> thread earlier)
>
> > - Stripping requirement to use VimL in Python/Ruby/Lua/etc interface. I.e.
> > finishing what I started (I guess I can do this) for python and doing the
> > similar thing for other languages.
> I don't get you, can you elaborate?

Currently such interfaces are pretty much useless without using vim.eval or vim.command (or equivalents in your language of choice). I started creating wrappers around various C structures (like that vim.options thingie wrapping functions that manipulate options array) that strip out requirement for vim.eval/vim.command.

> > - Unicode support in regular expressions.
> Can you also provide a simple use case, here ?

How do you match a Russian word? This will be /\b\p{Cyrillic}+\b/ in Perl. There is a big bunch of Unicode properties that you can place inside figure braces there and they are there for a reason. The most severe are more general groups: e.g. what is the regex for matching variable name in python 3 (ignore reserved words for simplicity)? Hint: it is not \<\a\w*\>: Python 3 supports using non-ASCII in variable names.

>
> > - Possibly sane unicode indexing (may require further string refactoring to
> > be efficient).
> Use case ?

You are at the column N and want to get character before the cursor to decide the behavior of Tab. Assuming you are writing text in LaTeX in Russian and want to detect whether it is Russian letter just before the cursor to do something special. The best idea would be using indexing, but it is wrong, so you have to use matchstr(). matchstr(getline('.')[:col('.')-2], '.$') vs getline('.')[col('.')-2].

This can be worked around, but workarounds are ugly. And there is big bunch of plugin developers who do not care about unicode and thus do not have such workarounds (consider the relatively recent issue where one Russian man wanted to use IMAP() to map some Russian letters and was unable to for this very reason). Good language is not about "care about things or they will not work". It is about "you see how many things work without you actually caring about them".

I have yet to see a language which is not going to bother me with encoding issues.

>
> > - Non-string funcrefs (i.e. inclusion of my patch (without lambda part)).
> > It contains fixes for some problems.
> Again, just add a section to http://vim-wiki.mawercer.de/wiki/topic/in-which-way-does-vim-suck.html
>
> > - Stripping requirement to use shell to run applications.
> Again, a use case would help much understanding what you mean exactly.

system("frobnicate ".shellescape(@%)) vs system(['frobnicate', @%]). It is again very easy and common to forget about escaping or do it wrong. But it is even easier to have your plugin screwed up just because vim has no way other then passing argument to cmd.exe with clumsy escaping. Check python subprocess module to see how to do it right.

---

Also using pipes and not temporary files is faster and sometimes more reliable. It has to be a separate issue I have not mentioned.

>
> Marc Weber

Nikolay Pavlov

unread,
Dec 2, 2013, 10:11:49 AM12/2/13
to vim...@googlegroups.com


On Dec 2, 2013 6:28 PM, "Mikołaj Machowski" <mik...@wp.pl> wrote:
>
> Dnia Poniedziałek, 2 Grudnia 2013 14:25 Marc Weber <marco-...@gmx.de> napisał(a)


> >
> > Excerpts from Nikolay Pavlov's message of Mon Dec 02 08:41:43 +0100 2013:
> > > - Making vim embeddable (making it second point since it is even more
> > > refactoring).
> > Keep in mind that Yzis team failed on this, see
> > http://www.freehackers.org/VimIntegration (which was posted in this
> > thread earlier)
>
> Yzis failed mainly because it was ugly hack around Qt event loop which meant (among others) it was sloooooow. Since then Qt evolved and building and maintenance of 'qvim' should be much easier.
>
> > > - Unicode support in regular expressions.
> > Can you also provide a simple use case, here ?
>

> Just guessing: [a-z] automatically cover national diacritics without need to list them separately eg [a-ząęćżźół] (this case would be covered by [:alpha:] but it is simplest example of usage)


>
> > > - Possibly sane unicode indexing (may require further string refactoring to
> > > be efficient).
> > Use case ?
>
> Vim doesn't recognize how many bytes are in character and indexing of strings may broke easily which is awkward: Vim is *text* editor and should deal with characters not bytes.
>
> > > - Stripping requirement to use shell to run applications.
> > Again, a use case would help much understanding what you mean exactly.
>
> Wouldn't proper threading take care about that problem?

Which problem? Process is run using C functions like execve. How vim runs system('command')? It runs execve("/bin/zsh", {"/bin/zsh", "-c", "(command) >/tmp/... 2&>1"}, env) (checked out with strace). You see: while one can call command directly it is calling shell and it calls the command. This has nothing to do with threading, it is more about efficiency (which is even worse then you imagine as some users prefer to use interactive versions of shell just to be able to use their aliases) and escaping (I was thinking about python subprocess-like first argument handling in system(): it takes lists as this argument and either escapes it (no other way on windows) for the program (not for the *shell*) or constructs execve call with this list transformed into array of strings).

>
> m.

BPJ

unread,
Dec 2, 2013, 10:36:17 AM12/2/13
to vim...@googlegroups.com
2013-12-02 15:56, Nikolay Pavlov skrev:
> > > - Unicode support in regular expressions.
> > Can you also provide a simple use case, here ?
>
> How do you match a Russian word? This will be /\b\p{Cyrillic}+\b/
> in Perl. There is a big bunch of Unicode properties that you can
> place inside figure braces there and they are there for a reason.
> The most severe are more general groups: e.g. what is the regex
> for matching variable name in python 3 (ignore reserved words for
> simplicity)? Hint: it is not \<\a\w*\>: Python 3 supports using
> non-ASCII in variable names.
>

Maybe it would be possible to find some way to build Vim with PCRE
with Unicode support instead of or in addition to Vim's native
regex engine?

/bpj

Nikolay Pavlov

unread,
Dec 2, 2013, 10:51:17 AM12/2/13
to vim...@googlegroups.com

I do not think making PCRE work with vim buffer representation is easier. In fact, I think the opposite. Not to mention that you need make it work with buffers representation and additionally support vim-specific stuff like \%V.

> /bpj

Marc Weber

unread,
Dec 2, 2013, 11:44:17 AM12/2/13
to vim_use
Excerpts from Nikolay Pavlov's message of Mon Dec 02 15:56:25 +0100 2013:
> Currently such interfaces are pretty much useless without using vim.eval or
> vim.command (or equivalents in your language of choice). I started creating
> wrappers around various C structures (like that vim.options thingie
> wrapping functions that manipulate options array) that strip out
> requirement for vim.eval/vim.command.
=> TODO: would gobjectIntrospection system help? AFAIK it can generate
beindings to gobject like interfaces automatically for most scripting
languages

> > > - Unicode support in regular expressions.
> > Can you also provide a simple use case, here ?
> How do you match a Russian word? This will be /\b\p{Cyrillic}+\b/ in Perl.
So what you're aksing for is [a-zA-Z] but include language specific
characters such as ß in German or whatever is used in Russia?
Something smarter than [^ \t.;:!"'] (trying to match non word chars ..
but that's not going to work well)

> PCRE:
In fact that was what I proposed to Bram, I even wrote to the PCRE
mailinglist to understand how to use it (feed data in incrementally).
Bram disliked the idea ..
I'm in favour of "reusing code" if possible. I'm not in favour of
maintaining everything myself.

Also they were refactoring the interface.

Well "just improve" is not enough. We have one use case {Cyrillic}, but
which additional ones to support?

> > > - Non-string funcrefs (i.e. inclusion of my patch (without lambda
> part)).
Oh by the way: I introduced kind of, see vim-addon-mw-utils
funcref#Function or such. It allows storing a function to be called and
some args to be passed first. It also allows to evaluate strings.

Its not perfect, but quite useful.

> system("frobnicate ".shellescape(@%)) vs system(['frobnicate', @%])
Have a look at VAM's shell dsl. it comes very close.

> again very easy and common to forget about escaping or do it wrong
I agree. Also system vs :! requires different quoting etc.

> Also using pipes and not temporary files is faster and sometimes more
> reliable.
r! ...
:cbuffer
:-)

Added this to the sucks page

Marc Weber

Nikolay Pavlov

unread,
Dec 2, 2013, 12:12:00 PM12/2/13
to vim...@googlegroups.com


On Dec 2, 2013 8:44 PM, "Marc Weber" <marco-...@gmx.de> wrote:
>
> Excerpts from Nikolay Pavlov's message of Mon Dec 02 15:56:25 +0100 2013:
> > Currently such interfaces are pretty much useless without using vim.eval or
> > vim.command (or equivalents in your language of choice). I started creating
> > wrappers around various C structures (like that vim.options thingie
> > wrapping functions that manipulate options array) that strip out
> > requirement for vim.eval/vim.command.
> => TODO: would gobjectIntrospection system help? AFAIK it can generate
> beindings to gobject like interfaces automatically for most scripting
> languages

Wondering whether gobject may make python-vim API look like rather regular python module and not like something ctypes-based with many implementation details leaking.

>
> > > > - Unicode support in regular expressions.
> > > Can you also provide a simple use case, here ?
> > How do you match a Russian word? This will be /\b\p{Cyrillic}+\b/ in Perl.
> So what you're aksing for is [a-zA-Z] but include language specific
> characters such as ß in German or whatever is used in Russia?
> Something smarter than [^ \t.;:!"'] (trying to match non word chars ..
> but that's not going to work well)

It is locale and not unicode support, though they intersect. I understand that locale support will break plugins, thus I asked only about unicode. This is not about ranges, but about character properties.

>
> > PCRE:
> In fact that was what I proposed to Bram, I even wrote to the PCRE
> mailinglist to understand how to use it (feed data in incrementally).
> Bram disliked the idea ..
> I'm in favour of "reusing code" if possible. I'm not in favour of
> maintaining everything myself.

I do not remember clearly, but was not it suggested to take PCRE syntax? I do not remember any discussions suggesting taking PCRE and replacing parser code, but I may have missed something.

>
> Also they were refactoring the interface.
>
> Well "just improve" is not enough. We have one use case {Cyrillic}, but
> which additional ones to support?

All unicode properties. I guess you may use icu to fetch characters with a given property into a collection. Maybe existing API allows faster variants.

It is useful for things like stripping diacritics or finding variable names. There are more uses I cannot imagine thus there is no need for restricting support to just some unicode properties unless there is some technical reason for it.

>
> > > > - Non-string funcrefs (i.e. inclusion of my patch (without lambda
> > part)).
> Oh by the way: I introduced kind of, see vim-addon-mw-utils
> funcref#Function or such. It allows storing a function to be called and
> some args to be passed first. It also allows to evaluate strings.
>
> Its not perfect, but quite useful.

It is not this. Problems solved include:

- Possible deletion of function that is referenced somewhere.
- Weird anonymous function names.
- Restriction for deleting functions being profiled.
- Using python callables as vim functions.

This list does *not* include partial application, not without lambda subbranch.

>
> > system("frobnicate ".shellescape(@%)) vs system(['frobnicate', @%])
> Have a look at VAM's shell dsl. it comes very close.

I know it. It is not close, not a tiny bit. Main problem is not escaping, it is shell itself. There is no way you can use system()/:! and not use shell.

>
> > again very easy and common to forget about escaping or do it wrong
> I agree. Also system vs :! requires different quoting etc.
>
> > Also using pipes and not temporary files is faster and sometimes more
> > reliable.
> r! ...
> :cbuffer
> :-)

It is too smart about line endings.

> Added this to the sucks page
>
> Marc Weber
>

Saad Malik

unread,
Dec 2, 2013, 12:29:16 PM12/2/13
to vim...@googlegroups.com, vim...@vim.org
Marc,
Is it possible to address the 'Esc key timeout' issue in terminal ViM? ViM has a short delay when exiting Insert mode because it needs to determine if the user pressed escape key or if this is the start of some new sequence.

I'm sure fixing this would have destabilizing ripple effects on all plugins, but we are trying to "enhance" ViM =)

Thiago Padilha

unread,
Dec 2, 2013, 1:04:05 PM12/2/13
to vim...@googlegroups.com, vim...@vim.org
Over the past years of working with many tools(from visual studio to vim + tmux
+ zsh) I've built a mental model of what would be the perfect tool for my
work as a programmer/system administrator, so I will share its concepts here
as a means of contributing to the idea. Hopefully this will be helpful and
inspiring to someone.

The perfect editor, which can adapt itself to any workflow and will evolve
naturally with new technologies, should meet the following basic requirements:

- Client-server model from the core, following tmux ideas regarding
sessions/windows.

- Small core written in C. Like Marc Weber, I'm not exactly thrilled by writing
large programs in C, but C is a very simple language that anyone can learn
quickly, and a common denominator on all platforms. Besides the core should be
small(so C is fine) only handling basic things:
- Editor server state(which should be sessions, windows, and buffers)
- API that plugins can use to control the editor state.
- Loading plugins
- Emitting events from the client to the plugins and back to the client.
- API for system tasks like dealing with files(?)

- Handle concurrency/multitasking using event loops which is more
efficient and simple than threading. Personally, I would choose
libuv not only for the event loop, but because it provides a platform
abstraction layer for doing most system tasks. Its the simplest way to write a
concurrent C program that runs anywhere.

- Extreme extensibility based on editor events. Everything that can be a plugin,
should be a plugin. The plugin system should be simple enough:
- Based on dlfcn which works on any POSIX and on windows through mingw
- Plugin discovery done by searching a set of directories for
shared libraries.
- Each library can implement any of the functions defined by the plugin
interface to listen for a particular event.
- Scripting support in other languages should be implemented as external
plugins. Each language plugin is responsible for discovering/loading its
own scripts and delegating events from the core to the
scripts(For an example
of this in practice, see 'weechat')

- Protocol for client/server communication, which should express user
input which
comes from the client and editor output which comes from the server processing
events with plugins.

- UI independent. This is a direct consequence of the client/server model. The
motivation is simple: Longevity. Who can be sure if in the future we will be
using keyboard/mouse to write programs? Its best to use the protocol mentioned
above for expressing things like syntax highlighting, popup windows, user
input... and let the client, which can be written in any language or toolkit
decide how it should be displayed(Maybe in the future someone will write an
UI that uses voice-recognition to avoid typing :)).


As some may have noticed, the above doesn't necessarily describe a text
editor, but a generic framework for building any kind of editor or IDE.

Any feedback is appreciated.

Thiago

Marc Weber

unread,
Dec 2, 2013, 1:41:01 PM12/2/13
to vim_use
Excerpts from Nikolay Pavlov's message of Mon Dec 02 18:12:00 +0100 2013:
> Wondering whether gobject may make python-vim API look like rather regular
> python module and not like something ctypes-based with many implementation
> details leaking.
Writing a small python layer on top should be easy.

> I do not remember clearly, but was not it suggested to take PCRE syntax? I
> do not remember any discussions suggesting taking PCRE and replacing parser
> code, but I may have missed something.
I don't remember wether it was public. Facts: PCRE is fast, it already
implements what Bram had in mind, it even has JIT support.
But yes, it would be PCRE syntax, too. (Is this good or bad?).

> All unicode properties.
Add references to documentation what you have in mind (wiki).

> It is not this. Problems solved include:
> - Possible deletion of function that is referenced somewhere.
What for? Don't understand what you mean.

> - Weird anonymous function names.
Talking about dictionaries? :function{23} will provide more information

> - Restriction for deleting functions being profiled.
> - Using python callables as vim functions.
You can do this all - kind of.:

You can define helper function

fun! CallPyAndReturn(py-expr)
py a = a:py-expr
py return a
endf

Then

let my_funcref = funcref#Function('return CallPyAndReturn('python-thing', ARGS[0], ARGS[1])', [2])
# my_funcref contains first arg. Second arg still has to be passed:
let x= funcref#Call(my_funcref, [arg_2])

> This list does *not* include partial application, not without lambda
> subbranch.
Please document your work on the wiki. We need to have a primary place
to share ideas/ work/ ..

Marc Weber

Marc Weber

unread,
Dec 2, 2013, 1:55:42 PM12/2/13
to vim_use
> irc: Have you seen my mail
Please don't expect me to reply within 1h. 24 to 48h should must be
enough :)

> - Client-server model from the core, following tmux ideas regarding
> sessions/windows.
You think its enough to say "client-server" model? Please try this:

client-server model for these use cases:
- ..
- ..
- ..

> - Small core written in C.
No way. C is a risk. In industry even the "C" kernel gets rewritten
using newer technologies. If you want to go that route without me.

I agree that "C" should be one way to compile the editor for portability
reasons which is why I think C++11 or something compliing down to C is
the perfect tool. But that's research (my TODO item)

> but C is a very simple language that anyone can learn
I'm not interested in languages "anybody can learn" to reinvent each
wheel yet another time. There are facts, and those are that you're 5-10
times slower writing an app in C compared to high level languages for
various reasons. And those arguments are stronger IMHO.

> - Editor server state(which should be sessions, windows, and buffers)
> - API that plugins can use to control the editor state.
> - Loading plugins
> - Emitting events from the client to the plugins and back to the client.
> - API for system tasks like dealing with files(?)
Try to find particular use cases. In the end there must be one decision:
Keep improving Vim or start from scratch. Starting from scratch will
take a couple of years (See yzis history, jedit, ..)

> - Handle concurrency/multitasking using event loops which is more
> efficient and simple than threading.
I don't think that "comping up with event loops" is a design yet.
If you want such join Yi, so that we can find out whether even
improving Yi (eg making it understand VimL) could be a possible
upstream. Haskell already has most of what you're asking for:
Simple event loops, typing, cheap threads etc.

Eg what about concurrent editing?

> libuv
I've added this to be evaluated. I don't know it yet.

> - Extreme extensibility based on editor events. Everything that can be a plugin,
> should be a plugin.
So this means "break" and rewrite?

> - UI independent. This is a direct consequence of the client/server model.
Yes, but why stick to C then? Why not be independent of target "JS in
browser vs console vs gtk/qt/.." ?

> motivation is simple: Longevity.
That's why I even think about "invent a new language" so that code can
be shared between Emacs/Vim/Yi. Eg write code which can be compiled down
to C (which could be used by Vim then), but share dev work with other
editors. That would allow some upstream for Vim, and still allow to
write a new core, too.

I agree on this. Being modular is key.

> Who can be sure if in the future we will be
> using keyboard/mouse to write programs?
Right, .. I agree. Its me who would like to replace keyboard/ mouse.
Eg see "plover stenoknight" as exampl, but that design dates back very
much, too. I know there are many alteranatives. (Yet I don't use 2
keyboard /mice input system for my pc) - why? Because when I tried and
typed something into a firefox address bar a "completion popup happens",
and focus was assigne randomly to the other keyboard.
That all can be fixed - but who is going to do all the work ? ..
Just to give you an idea what kind of "new issues" can happen.
That's why C is no option, because you there is not enough abstraction.
No way. It can only be a target IMHO.

> As some may have noticed, the above doesn't necessarily describe a text
> editor, but a generic framework for building any kind of editor or IDE.
Well - something which could be turned into IDEA/Eclipse/... etc ?
Not doable in a couple of years.

Marc Weber

Nikolay Pavlov

unread,
Dec 2, 2013, 2:35:18 PM12/2/13
to vim...@googlegroups.com


On Dec 2, 2013 10:41 PM, "Marc Weber" <marco-...@gmx.de> wrote:
>
> Excerpts from Nikolay Pavlov's message of Mon Dec 02 18:12:00 +0100 2013:
> > Wondering whether gobject may make python-vim API look like rather regular
> > python module and not like something ctypes-based with many implementation
> > details leaking.
> Writing a small python layer on top should be easy.
>
> > I do not remember clearly, but was not it suggested to take PCRE syntax? I
> > do not remember any discussions suggesting taking PCRE and replacing parser
> > code, but I may have missed something.
> I don't remember wether it was public. Facts: PCRE is fast, it already
> implements what Bram had in mind, it even has JIT support.
> But yes, it would be PCRE syntax, too. (Is this good or bad?).

It is bad. Everybody is using vim syntax and thus PCRE one is not acceptable for it throwing away backwards compatibility with exactly no nice ways to work around.

>
> > All unicode properties.
> Add references to documentation what you have in mind (wiki).

Unicode property is an already defined entity. Maybe need a link (though do not know where to as perldoc perlre and some icu examples assume you know what exactly "unicode property" is).

>
> > It is not this. Problems solved include:
> > - Possible deletion of function that is referenced somewhere.
> What for? Don't understand what you mean.

You can define a function, reference it and do :delfunction. Now reference is unusable. My branch holds a reference to function definition in funcref variable so that as far as you hold the reference you are guaranteed to run exactly the same function you referenced.

Note: this means that if there is reference somewhere when you redefine function that reference will still use old definition. I consider such stability an advantage.

>
> > - Weird anonymous function names.
> Talking about dictionaries? :function{23} will provide more information

Which is very inconvenient. And it does not provide a name. I am losing the ability to use :fu without holding a reference in that branch, but it is possible to have :fu working with the error output and have normal function names at one time with a more or less minor patch, but I like current state much more since it a) is less damaging to debugging then numeric names are (too easy to workaround and trying to use :fu in the only case I actually *may* have preferred it over browsing source code (long generated functions) is very inconvenient without syntax highlighting (do not recommend :redir to a file and then opening it: it is far less convenient then what I have)) and b) makes anonymous functions not global.

>
> > - Restriction for deleting functions being profiled.
> > - Using python callables as vim functions.
> You can do this all - kind of.:
>
> You can define helper function
>
> fun! CallPyAndReturn(py-expr)
>   py a = a:py-expr
>   py return a
> endf
>
> Then
>
> let my_funcref = funcref#Function('return CallPyAndReturn('python-thing', ARGS[0], ARGS[1])', [2])
> # my_funcref contains first arg. Second arg still has to be passed:
> let x= funcref#Call(my_funcref, [arg_2])

Make this a) automatic b) automatic and fast and c) make it friends with GC so that you cannot dispose python object when you hold a reference and you will dispose it when you don't.

And your code is not going to work. There are way too many mistakes for such small piece of code. I do not even get an idea, so I am pretending the idea behind this code is the same as I would have written: store python callable into a dictionary, generate an anonymous function which calls it with "*vim.bindeval('a:000')" as arguments (am not sure whether star accepts any iterable and not just tuple and list, so this may as well be "*tuple(vim.bindeval('a:000'))") and return the result of the call. Like this:

py << EOF
callables = dict()
def store_callable(c):
  key = id(c)
  callables[key] = c
  return c
EOF
fun GenPyCallable(expr)
  let id=pyeval('store_callable('.expr.')')
  let d={}
  exe "fu d.f(...)\nreturn pyeval('callables[".id."](*vim.bindeval(\"a:000\"))\nendfu"
  return d.f
endfun

. I am not sure whether I have everything here right (especially escaping), but this is exactly why I first express the idea in English.

Note that there is also no way I know that allows to do this and keep context (i.e. l: and a: locals or absence of them for use with vim.*eval). This is poind d).

>
> > This list does *not* include partial application, not without lambda
> > subbranch.
> Please document your work on the wiki. We need to have a primary place
> to share ideas/ work/ ..
>
> Marc Weber
>

Nikolay Pavlov

unread,
Dec 2, 2013, 2:50:52 PM12/2/13
to vim...@googlegroups.com

There actually is a way to keep context: make it a *python* function and not vimL. This makes it harder to use though and if you cope with a) (which is not very hard) you still will not be able to construct a drop-in pyeval replacement for this reason. I.e. GenPyCallable('foo') transforms into pyeval('gen_py_callable(foo)') (note: foo is not a string argument here: using eval in different context omitted).

Thiago Padilha

unread,
Dec 2, 2013, 8:13:00 PM12/2/13
to vim...@googlegroups.com
On Mon, Dec 2, 2013 at 3:55 PM, Marc Weber <marco-...@gmx.de> wrote:
> You think its enough to say "client-server" model? Please try this:
>
> client-server model for these use cases:
> - ..
> - ..
> - ..
- Multiple clients connected to the same vim instance would provide an easy way
to have collaborative editing(like pair programming).
- Vim running in a remote server with local GUI, without the security
implications of X forwarding
- Use vim as a terminal multiplexer for detaching from a remote server without
closing the running programs.
- This is actually a very particular use case of mine, but it should illustrate
another scenario were it would be useful: I do all my development work inside
a headless linux VM running in virtual box in a windows laptop. Having a
client/server architecture would let me detach a windows-native client, save
the VM state and restore everything later even across reboots. This is already
how I work, except that I use a combination of TMUX, vcxsrv(windows
port of xorg), urxvt and terminal vim. This would let me replace all those
programs by vim.

>> - Small core written in C.
> No way. C is a risk. In industry even the "C" kernel gets rewritten
> using newer technologies. If you want to go that route without me.
>
> I agree that "C" should be one way to compile the editor for portability
> reasons which is why I think C++11 or something compliing down to C is
> the perfect tool. But that's research (my TODO item)
>
>> but C is a very simple language that anyone can learn
> I'm not interested in languages "anybody can learn" to reinvent each
> wheel yet another time. There are facts, and those are that you're 5-10
> times slower writing an app in C compared to high level languages for
> various reasons. And those arguments are stronger IMHO.

Like I said, I'm not particularly fond of C either, but consider the following
points:

- You wont be reinventing the wheel very much because the C core will not be
modified very often. Most of the development will be done in scripting
languages.
- C will force you to keep the core program very small and clean. If you use
C++ you will end up writing a lot of functionality in it,
functionality which
should be written as a plugin in a higher level language like python.
- Most of the functionality will be implemented as plugins in third party
languages, choosing C will give the editor easy interoperability with any
language you can think, improving the chances of having bigger communities.

Of course, thats just my personal view on the matter, I would have no problem in
using another language for writing the core editor.

>> - Handle concurrency/multitasking using event loops which is more
>> efficient and simple than threading.
> I don't think that "comping up with event loops" is a design yet.
> If you want such join Yi, so that we can find out whether even
> improving Yi (eg making it understand VimL) could be a possible
> upstream. Haskell already has most of what you're asking for:
> Simple event loops, typing, cheap threads etc.
>
> Eg what about concurrent editing?

Event loops simpler because theres no need for locking or synchronization.
Take TMUX as an example:
its server can talk to hundreds of programs simultaneously in a single thread
and is very efficient in doing so. I'm not sure what you mean by concurrent
editing but that would be no problem for an event loop since any kind of
concurrency problem can be modeled on top of it.

>> - UI independent. This is a direct consequence of the client/server model.
> Yes, but why stick to C then? Why not be independent of target "JS in
> browser vs console vs gtk/qt/.." ?

I suggested C only for the core/plugin loading which would be very small.
Nothing would stop one from writing a websocket plugin and have a
javascript/browser client talk to it.

>> motivation is simple: Longevity.
> That's why I even think about "invent a new language" so that code can
> be shared between Emacs/Vim/Yi. Eg write code which can be compiled down
> to C (which could be used by Vim then), but share dev work with other
> editors. That would allow some upstream for Vim, and still allow to
> write a new core, too.

I think it would be extremely challening writing an editor extension language
independent of the editor.

>
> I agree on this. Being modular is key.
>
>> Who can be sure if in the future we will be
>> using keyboard/mouse to write programs?
> Right, .. I agree. Its me who would like to replace keyboard/ mouse.
> Eg see "plover stenoknight" as exampl, but that design dates back very
> much, too. I know there are many alteranatives. (Yet I don't use 2
> keyboard /mice input system for my pc) - why? Because when I tried and
> typed something into a firefox address bar a "completion popup happens",
> and focus was assigne randomly to the other keyboard.
> That all can be fixed - but who is going to do all the work ? ..
> Just to give you an idea what kind of "new issues" can happen.
> That's why C is no option, because you there is not enough abstraction.
> No way. It can only be a target IMHO.
>
>> As some may have noticed, the above doesn't necessarily describe a text
>> editor, but a generic framework for building any kind of editor or IDE.
> Well - something which could be turned into IDEA/Eclipse/... etc ?
> Not doable in a couple of years.
>
> Marc Weber
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

Anyway, thanks for the feedback.

My goal with the proposed design is simple: Combine VIM and TMUX into one
program, removing overlapping features and moving every possible feature to
plugins. And by "every possible feature" I mean even things such as terminal
emulation, so vim core would not be cluttered with terminal emulator stuff.

But lets abandon crazy rewrite ideas that would take too long to complete and
focus on a feature that would greatly improve VIM IDE capabilities.

Consider that VIM could run multiple background jobs simultaneously and perform
asynchronous UI updates(without using CursorHold hacks). Here are some things
this would enable:

- Run terminals in buffers with full vimL(and other languages) scripting
support.
- Powerful asynchronous code completion/analisys tools could run as
background processes without slowing down the UI(YouCompleteMe is an example
of such an engine).
- A background job could constantly regenerate tags when a file is changed,
notifying vim for updating plugins such as tagbar.
- Plugins such as CtrlP would not block when calculating big datasets.
- Vim could be extended in any language imaginable(even non-embedable
platforms just as node.js) by providing something like a RESTFUL extension
API. Extensions could do extensive background work without affecting vim's
lightning fast speed.

Since one could script terminals and background processes from withing vim, the
possibilities would be endless.

So concurrency support is a feature that would greatly benefit vim. I havent
ever looked into vim source code, but I believe the best approach to adding
concurrency would be through an event loop library like libuv, as it wouldn't
require adding locks or syncronization to current code. Could a vim developer
give some feedback on this idea? Depending on the answer I could feel motivated
to dig into VIM's source code and provide patches for these changes.

Thiago

Tim Chase

unread,
Dec 2, 2013, 10:40:17 PM12/2/13
to vim...@googlegroups.com
It seems odd to build a lot of these things into vim when excellent
solutions exist with more generic applications. You (Thiago) mention
already using tmux+vim, and I find that solves most of the issues you
list, and thus I'd find adding those to vim to be superfluous.

On 2013-12-02 23:13, Thiago Padilha wrote:
> - Multiple clients connected to the same vim instance would provide
> an easy way to have collaborative editing(like pair programming).

GNU screen and tmux both provide this functionality

http://www.howtoforge.com/sharing-terminal-sessions-with-tmux-and-screen

That way, I can not only screen-share my Vim session, but my terminal
commands for building/testing, as well as other shell stuff like
network, process & configuration management.

> - Vim running in a remote server with local GUI, without the
> security implications of X forwarding

Though I don't usually need a GUI (the terminal version does just
about everything I need), I find that using ssh's X-forwarding
securely does everything I need without having to open things up
broadly/dangerously via xhost.

> - Use vim as a terminal multiplexer for detaching from a remote
> server without closing the running programs.

As above, I recommend just using screen/tmux. Why duplicate the
behavior in Vim, without the ability to multiplex all apps like
screen/tmux already does?

> - This is actually a very particular use case of mine, but it
> should illustrate another scenario were it would be useful: I do
> all my development work inside a headless linux VM running in
> virtual box in a windows laptop. Having a client/server
> architecture would let me detach a windows-native client, save the
> VM state and restore everything later even across reboots. This is
> already how I work, except that I use a combination of TMUX,
> vcxsrv(windows port of xorg), urxvt and terminal vim. This would
> let me replace all those programs by vim.

Most of that seem to involve getting a GUI. With terminal vim,
that's just tmux and vim.

Maybe it's my grumpy-old-man stance, but I don't see them as issues
since they already have robust solutions that do far more than Vim
would encompass.

-tim


Thiago Padilha

unread,
Dec 2, 2013, 11:10:29 PM12/2/13
to vim...@googlegroups.com
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

Tim,

While tmux+vim combo serves me very well for now, sometimes I see myself using
very hacky solutions for things that would be easy if the two were integrated.
For example, [this gist](https://gist.github.com/tarruda/5158535) shows how to
integrate tmux and vim, and both with the x11 clipboard. If I could
properly run terminals
in vim buffers none of that would be needed.

I'm not saying that vim should implement a terminal emulator or advanced
features like semantic code completion, but if it provided some basic way of
updating UI or executing a command after a background job completes then it
would integrate much better with other plugins such as youcompleteme and
conqueshell which extend its functionality in extreme ways


Thiago

Ping

unread,
Dec 2, 2013, 11:35:06 PM12/2/13
to vim...@googlegroups.com, vim...@googlegroups.com
agree
I think it'll move vim to a new milestone if more cool stuff can be integrated into vim. Throw away other sick hackings and stay just with vim . Like emacs . How about lightweight version (current) and a new(heavy ) version , toggle-able via a new 'compatible ' knob?

from iPhone

Marc Weber

unread,
Dec 3, 2013, 1:07:08 AM12/3/13
to vim_use
Excerpts from Nikolay Pavlov's message of Mon Dec 02 20:35:18 +0100 2013:
> Note: this means that if there is reference somewhere when you redefine
> function that reference will still use old definition. I consider such
> stability an advantage.
I always assumed that
- restarting Vim is that fast that I do not bother about such changes at
runtime. Instead I do care about "setting up my dev env" faster, eg by
using local vimrcs, spending more time on workflows to define the
compiler (vim-addon-actions) and the like.

> Note that there is also no way I know that allows to do this and keep
> context (i.e. l: and a: locals or absence of them for use with vim.*eval).
> This is poind d).
No, there is not. You have to pass context as arguments manually.
I didn't try my code. Fact is I'm using it in many ways.
If you're interested you can find the implementation easily.

But anyway: Your work makes me think that having a separate viml library
is the way to go, so that improvements can be created and debugged more
easily.

Marc Weber

Erik Christiansen

unread,
Dec 3, 2013, 1:08:09 AM12/3/13
to vim_use
On 02.12.13 19:41, Marc Weber wrote:
> > I do not remember clearly, but was not it suggested to take PCRE syntax? I
> > do not remember any discussions suggesting taking PCRE and replacing parser
> > code, but I may have missed something.
> I don't remember wether it was public. Facts: PCRE is fast, it already
> implements what Bram had in mind, it even has JIT support.
> But yes, it would be PCRE syntax, too. (Is this good or bad?).

Bad, I suggest.
Posix EREs are powerful, and standard across many utilities.
What is important here is that a new regex dialect need not be learnt
for every utility we use. Productivity results from leveraging learning
across tools.

Erik

--
The Kyoto Protocol, drafted in 1997, aimed to reduce the world's greenhouse gas
emissions by around five per cent from 1990 levels by the end of 2012.
Instead, global greenhouse gases have risen around 58 per cent in that time.
- http://www.abc.net.au/news/2012-12-08/troubled-un-climate-talks-spill-over/4416696

Nikolay Pavlov

unread,
Dec 3, 2013, 8:01:32 AM12/3/13
to vim...@googlegroups.com


On Dec 3, 2013 10:08 AM, "Erik Christiansen" <dva...@internode.on.net> wrote:
>
> On 02.12.13 19:41, Marc Weber wrote:
> > > I do not remember clearly, but was not it suggested to take PCRE syntax? I
> > > do not remember any discussions suggesting taking PCRE and replacing parser
> > > code, but I may have missed something.
> > I don't remember wether it was public. Facts: PCRE is fast, it already
> > implements what Bram had in mind, it even has JIT support.
> > But yes, it would be PCRE syntax, too. (Is this good or bad?).
>
> Bad, I suggest.
> Posix EREs are powerful, and standard across many utilities.
> What is important here is that a new regex dialect need not be learnt
> for every utility we use. Productivity results from leveraging learning
> across tools.

Vim syntax looks like BRE (*not* ERE) on steroids unless you use very magic modifier at the start of regex. It looks like nothing else if you do. It is vim who has unique syntax.

PCRE has a bunch of similarities with ERE (nearly any ERE regex can be used without modifications with PCRE) and is pretty common: Perl 5, Python, Ruby, go, PHP, grep (if build with PCRE support and provided -P switch), zsh (if build with PCRE support and some options enabled or specific comparison switch is used).

Note that neither Python nor Perl are actually using PCRE: Python just borrowed the syntax and Perl is where PCRE borrowed syntax from. Do not know which of the other languages I mentioned with PCRE-like regex syntax are actually using PCRE.

Thus by bringing this argument you contradict yourself: if you want to leverage learning across utilities you must use PCRE.

> Erik
>
> --
> The Kyoto Protocol, drafted in 1997, aimed to reduce the world's greenhouse gas
> emissions by around five per cent from 1990 levels by the end of 2012.
> Instead, global greenhouse gases have risen around 58 per cent in that time.
>  - http://www.abc.net.au/news/2012-12-08/troubled-un-climate-talks-spill-over/4416696
>

Nikolay Pavlov

unread,
Dec 3, 2013, 8:14:06 AM12/3/13
to vim...@googlegroups.com


On Dec 3, 2013 10:07 AM, "Marc Weber" <marco-...@gmx.de> wrote:
>
> Excerpts from Nikolay Pavlov's message of Mon Dec 02 20:35:18 +0100 2013:
> > Note: this means that if there is reference somewhere when you redefine
> > function that reference will still use old definition. I consider such
> > stability an advantage.
> I always assumed that
> - restarting Vim is that fast that I do not bother about such changes at
>   runtime. Instead I do care about "setting up my dev env" faster, eg by
>   using local vimrcs, spending more time on workflows to define the
>   compiler (vim-addon-actions) and the like.

I normally trade startup time for other conveniences. As I am using anonymous funcrefs in my plugin this problem does not bother me, but it is what I expect function reference to do in other languages.

>
> > Note that there is also no way I know that allows to do this and keep
> > context (i.e. l: and a: locals or absence of them for use with vim.*eval).
> > This is poind d).
> No, there is not. You have to pass context as arguments manually.
> I didn't try my code. Fact is I'm using it in many ways.
> If you're interested you can find the implementation easily.

It is impossible to pass context and use vim.*eval() like if it was regular pyeval(). I.e. you cannot construct drop-in replacement for pyeval() from my branch which makes the code harder to use.

>
> But anyway: Your work makes me think that having a separate viml library
> is the way to go, so that improvements can be created and debugged more
> easily.

Do not know. Next set of changes to python interface is going to mostly interface things outside of eval.c, the same ones that eval.c interfaces.

I have to solve the problem with crashing compatibility library for older vims first though: I have written most of it, but cannot even start testing.

Reply all
Reply to author
Forward
0 new messages