[vim/vim] Arabic characters are misaligned (Issue #14115)

avidseeker

unread,

Feb 29, 2024, 4:16:16 AM2/29/24

to vim/vim, Subscribed

Steps to reproduce

Paste this text

السَّلامُ عليْكُم

:set arabic

Expected behaviour

Two problems:

Diacritics are mispositioned.
Positioning cursor on a letter, then inserting or appending puts the cursor ahead of the wrong letter.

Version of Vim

9.1

Environment

Terminal: Alacritty.

But I think it would work for any terminal with no Arabic shaping support. As I explained here (#9761 (comment)), Vim :set arabic seems to expect that the terminal won't do any shaping, and so by far, it's the only way (I know of) to correctly display Arabic in Vim.

Logs and stack traces

No response

—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.

Christian Brabandt

unread,

Feb 29, 2024, 6:04:25 AM2/29/24

to vim/vim, Subscribed

I think this will need someone to actively work on arabic support. I believe it is an area, that is basically un-maintained and will need help from someone who knows proper rules for this. Don't expect a solution to this soon unless you want to work on it yourself.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

samanera

unread,

Mar 21, 2024, 10:01:08 PM3/21/24

to vim/vim, Subscribed

Screenshot.2024-03-21.at.7.51.33.p.m.png (view on web)

@avidseeker , I'm not sure about Arabic, but I think the diacritics are rendered correctly on my Vim setup.

To provide more information (if any is needed for debugging):

Vim version is (9.1, patches: 1-193) compiled with +arabic.
Font is SF Mono, also tested with Noto Sans Mono.
Terminal application is Terminal.app with xterm-256color terminfo.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

avidseeker

unread,

Mar 22, 2024, 4:14:30 PM3/22/24

to vim/vim, Subscribed

As I said here, the letter shaping applied by Vim conflicts with the shaping of the terminal. So try setting :set arabicshape! to properly display the text.

At any case, my problem is not with the rendering of diacritics but with their mispositioning. Here's an example of another font showing diacritics offset by one character backwards:

image.png (view on web)

Compare with the correct rendering provided e.g by Libreoffice:

image.png (view on web)

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Tony Mechelynck

unread,

Mar 24, 2024, 4:09:39 AM3/24/24

to vim/vim, Subscribed

Maybe you know some or all of what I'm gonna write here but let's recap what I know about using Vim to write Arabic:

There is a setting to tell Vim whether the terminal supports bidirectional Unicode text, namely 'termbidi'.

This option must not be set in gvim;
It must also not be set in Vim-in-terminal if the terminal does not support Unicode bidirectional text;
It must be set in Vim-in-terminal if the terminal (such as mlterm) can handle Unicode bidirectional text itself;
By defult, this option is off unless $TERM is mlterm in which case it is on.

If 'termbidi' is on, 'arabicshape' will be ignored and 'arabic' will only set the standard Arabic keymap, so you can just as well keep 'arabic' off and set your preferred Arabic keymap yourself (including if you have one you wrote yourself).

If bidi is not handled by the terminal, and in particular in gvim, setting 'arabic' will make Vim shape spacing letters (including the laam-alif digraph) correctly, but the positioning of diacritics (vowels, sukuun, shadda, nonspacing hamza) is more problematic. In order to be correctly placed in the text (and to appear correctly when displayed in other software e.g. not only LibreOffice but also browers) diacritics must be placed in the text-on-disk immediately after the concerned consonant, even if Vim displays them elsewher than it should.

Best regards,
Tony.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

avidseeker

unread,

Mar 24, 2024, 3:42:43 PM3/24/24

to vim/vim, Subscribed

Yes. This is what I also know about Arabic in Vim. As for your last point, the logical sequence of characters saved on disk is correct. The problem is that Vim renders it incorrectly.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Mahdi Bahramian

unread,

Oct 20, 2024, 1:29:21 AM10/20/24

to vim/vim, Subscribed

how would someone try to implement bidi in Vim?

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Mahdi Bahramian

unread,

Oct 20, 2024, 1:30:27 AM10/20/24

to vim/vim, Subscribed

how would someone try to implement bidi in Vim?

—

Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Tony Mechelynck

unread,

Oct 20, 2024, 11:17:57 AM10/20/24

to vim/vim, Subscribed

I think it would be extremely difficult, because not only some characters are RTL and others LTR, but there are even characters (including IIUC the space, comma and full stop; the Arabic comma is different but let's not forget Hebrew which is also RTL) which can be LTR or RTL depending on what is around them. In HTML, we have tags <span dir="RTL">...</span> and <span dir="LTR">...</span> to care for that, but I'm for instance not sure how to tell Vim where to display the above-mentioned "ambiguous" characters when they are located between LTR and RTL plaintext.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Tony Mechelynck

unread,

Oct 21, 2024, 5:11:26 AM10/21/24

to vim/vim, Subscribed

@avidseeker @MahdiGMK et al: maybe you know all the following, but I'm recapitulating the present state of affairs for whoever would know it in less detail:

At the moment, what Vim (compiled with +rightleft and +arabic, of course) can do correctly is display text which is either LTR or RTL, but not both in the same view-window, by means of the window-local boolean option 'rightleft' (or 'rl'). In addition, in 'rightleft' Arabic text, the laam-alif character pair and the junctions, or not, between successive consonants in a word, are correctly displayed. Vocalised Arabic is less clear-cut, and in particular, even when Nadim Shaikli was actively developing the Arabic subsystem, it has never been possible to display correctly the sequence laam-fatha-alif as a laam-alif digram with a fatha on the laam (the fatha would break the digram). Nowadays, IIUC (but I still hope I understand wrongly), Nadim is AWOL and the Arabic subsystem is up for whoever wants to maintain it, but to maintain it seriously, after applying, not to our beloved Bram anymore (may he rest in peace) but to Christian.

It is possible to have two (or more) view-windows on a single text editfile, one with 'rightleft' and another with 'norightleft' and to compare various parts of a single bidi text this way by switching windows. This works quite well when mixing Latin (e.g. English) text with unvocalised Hebrew or Arabic. As I said above, vocalised Arabic (and I don't know Hebrew) still has a few problems.

As I said (IIRC) in an earlier post, another possibility is to run vim (not gvim) in a bidi-capable terminal such as mlterm; in such a case the terminal will handle the full-bidi all by itself, and if 'term' is correctly set, Vim will set 'termbidi' and not meddle with the terminal's handling of the bidi text. In this case, even 'rightleft' need not be set.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.

Enno

unread,

Oct 21, 2024, 4:48:38 PM10/21/24

to vim/vim, Subscribed

I am having a hard time finding bidi terminals other than Mlterm. There's this Urxvt bidi extension which however mangles composed characters.