[vim/vim] XDiff and Rust (and licensing) (Discussion #18628)
Yee Cheng Chin
Recently, Git made some news as there is some movement (email chain) to migrate part of the codebase to Rust (currently it is primarily written in C). My understanding is that this was primarily spawned from an effort to rewrite/improve parts of xdiff (the diff engine in Git) in Rust (email chain).
Since Vim uses xdiff as its diff engine too, this will obviously impact us. The changes I mentioned in Git are still work-in-progress, but if they go through with Rustification (it sounds like they are likely to do so), this will make xdiff itself have a dependency on Rust, which means Vim will too if we take upstream changes. Currently Vim takes upstream xdiff updates only rarely, with the last one being #12181 two years ago when we updated from Git 2.33 to 2.40. Part of the reason is that xdiff in general hasn't received a lot of changes in recent years, but presumably we would want keep the option to get upstream changes as needed.
Since this change is not in Git yet, this is still a bit hypothetical, but I just wanted to start a discussion on this topic to see what people think (Rust tends to invite controversy 😅). Currently Vim is built with only C, and not even require a full C99 compiler (one reason being OpenVMS doesn't support it). I would imagine introducing Rust would complicate the tool chain quite a bit and cut out some legacy platforms if they want to build Vim with +diff (there are similar discussions on the Git email chains as well but I think Vim works on more platforms than Git). An alternative is to simply stay on the current version and maintain a C-only forked version of xdiff. It's a relatively mature code base and does what we want for the most part.
There is also the licensing issue. Skimming through the xdiff patches, it does not look like the new Rust files have LGPL license headers associated with them. I don't know if this is intentional or not as it's still work-in-progress, but if they don't license the new Rust code as LGPL that may prevent us from taking it (since Git codebase is GPL). In fact, I'm currently working on a Vim PR to pull in new xdiff changes from upstream (which are all still in C), and running into a snag because someone introduced in xdiff a dependency to regular Git code, which are not LGPL. I'm still investigating options for it.
Neovim might be interested in this discussion as they are downstream of Vim. They are using xdiff too, but they didn't even merge in #12181 (the update to Git 2.40) when that went in to Vim so I think they are still on Git 2.33. Doesn't seem like anyone is bothered.
I'm not sure if it is premature to post about this but may as well start getting some discussion started.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Maxim Kim
An alternative is to simply stay on the current version and maintain a C-only forked version of xdiff
this
I don't have rust and don't plan to have it. If rust is a hard toolchain dependency at some point, I would just rely on a package manager of my distro and old vim it has. It has the benefit of me not doing all sorts of vim testing, freeing the time for the other things :).
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Rhialto The M.
Several big problems with Rust are that it is a fast-moving target and it is also bad at supporting anything they seem "obscure". Which includes my favourite operating system. A program that builds with today's rust compiler may not built with next week's, so you're on a constant upgrade treadmill which on some platforms is impossible.
An alternative is to simply stay on the current version and maintain a C-only forked version of xdiff.
This is the only possible portable solution.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Christian Brabandt
I am aware of this planned change, I even reached out to the git mailing list when this was initially discussed. But as xdiff seems to be otherwise a slowly changing target, I am fine with keeping the xdiff code here (which was effectively being forked by us anyhow) and use this one. I think the libgit project is also affected by this change and even some third party platforms are not happy by this proposed change, but it seems this is what we will get with git 3.0. Also the newly proposed rust dependency includes some xdiff speedups...
As for the wider rust adoption, I have no opinion on this. But Vim is too big of a project that I don't have plans to introduce a rust dependency, even though I am sympathetic with the additional memory safety this might bring. (I know there are some government policies to push open source projects to use some safe coding features).
Overall however I think the use of the current xdiff tree should not cause an issue for us and we have quite a few additional diff features available now.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Yee Cheng Chin
That makes sense. I will prepare a PR to do an upstream merge anyway. The Rust investigation regarding the xdiff hashing function improvement did lead to a parallel change to the C version which also led to a measurable performance improvement (mid single digit % from local testing) in xdiff, so I think they will be useful to get.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.