bug(?): Unicode Symbol as half-width in CJK

[Korean user1 (in CJK)]

In CJK, this bug is very old, but it has not been fixed yet. I believed someone would be interested in it and solve it, but until now I'm inquiring here because no one seems to be interested.

As far as I know, in text editors like vim Only fixed-width fonts are available.

In the CJK area, CJK characters and Unicode symbols It is called 'full-width character', and in comparison, English characters are called 'half-width characters'.

In other words, the width of the CJK character is exactly twice the width of the English character. In addition, the 'Unicode symbol' should also be marked twice as wide. In vim, however, most of the 'unicode symbols' are treated as half-width characters and are not displayed correctly.

I have attached a GIF animation file for easy understanding.

[Tony Mechelynck]

AFAIK, Vim is aware of the double width of most CJK characters, but you may need to use gvim to display them correctly, because vim-in-console depends on the underlying console for display.

Which Vim version are you using ? From time to time, new versions of Vim include definitions of newly defined Unicode characters concerning narrow vs. wide characters, lowercase vs. uppercase correlations, etc. Version 9.0.2173 has been published on 2023-12-17 or shortly before that.

Best regards,

Tony.

[Christian Brabandt]

On So, 17 Dez 2023, Korean user1 (in CJK) wrote:

> In CJK, this bug is very old, but it has not been fixed yet. I believed someone would be interested in it and solve it, but until now I'm inquiring here because no one seems to be interested.
>
> As far as I know, in text editors like vim Only fixed-width fonts are available.
>
> In the CJK area, CJK characters and Unicode symbols It is called 'full-width character', and in comparison, English characters are called 'half-width characters'.
>

> In other words, the width of the CJvim-input-unicode-symbol-halfwidth-problems-korean.gifK character is exactly twice the width of the English character. In addition, the 'Unicode symbol' should also be marked twice as wide. In vim,

> however, most of the 'unicode symbols' are treated as half-width characters and are not displayed correctly.

What symbol (name or codepoint) is this? Perhaps it's wrong in the
Unicode database? we are regularly syncing with the last released
unicode database, last change was v9.0.2013 for Unicode release v15.1
(released 23.09.2023)

Thanks,
Christian
--
Haste makes waste.
-- John Heywood

[Korean user1 (in CJK)]

Tested in vim version 9.0.2173.

2023년 12월 18일 월요일 오전 10시 15분 54초 UTC+9에 Tony Mechelynck님이 작성:

[Korean user1 (in CJK)]

The range of Unicode symbols is extensive. I haven't tested all Unicode symbols for half-width display.
I tested it in the corrected vim yesterday by a Japanese named Kaoriya, and there was no half-width marking problem I mentioned.
I received it in the following url.

https://www.kaoriya.net/software/vim/
http://vim-jp.org/redirects/koron/vim-kaoriya/latest/win64/
(vim82-kaoriya-win64-8.2.1287-20200724.zip)

There are several diff files in the patch folder, but I think the part I pointed out as a bug is related to the 2003-charspace.diff file.
I tested it on the vim of Kaoriya and attached 2 gif files.

2023년 12월 18일 월요일 오후 5시 26분 7초 UTC+9에 Christian Brabandt님이 작성:

[Korean user1 (in CJK)]

-----------------------------------------------------------------------------
vim82-kaoriya-win64-8.2.1287-20200724.zip

/patch/2003-charspace.diff

-----------------------------------------------------------------------------

# HG changeset patch
# Parent 4788b87d17786243e3d3b6fca78c4ff73026644b
implement 'charspace' for Windows GUI

diff --git a/runtime/doc/options.txt b/runtime/doc/options.txt
index 8598713bc..1d83efb2a 100644
--- a/runtime/doc/options.txt
+++ b/runtime/doc/options.txt
@@ -1498,6 +1498,20 @@ A jump table for the options with a short description can be found at |Q_op|.
  This option cannot be set from a |modeline| or in the |sandbox|, for
  security reasons.
 
+ *'charspace'* *'csp'*
+'charspace' 'csp' number (default: 0)
+ global
+ {not in Vi}
+ {only in Windows GUI version and when compiled with
+ the |+kaoriya| feature}
+ Number of horizontal pixels inserted between characters.  Useful if
+ the font uses the full character cell width, making characters touch
+ each other.
+ When non-zero there is room for side of characters.
+ With some fonts there can  be too much room between characters (to
+ have space for left and right).  Then it make sense to set 'charspace'
+ to a negative value.  This may cause display problems though!
+
     *'cindent'* *'cin'* *'nocindent'* *'nocin'*
 'cindent' 'cin' boolean (default off)
  local to buffer
diff --git a/src/gui_w32.c b/src/gui_w32.c
index d9d835bd7..10c495c45 100644
--- a/src/gui_w32.c
+++ b/src/gui_w32.c
@@ -1472,6 +1472,8 @@ GetFontSize(GuiFont font)
      "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz",
      52, &size);
     gui.char_width = (size.cx / 26 + 1) / 2 + tm.tmOverhang;
+    // add extra pixels for 'charspace' option.
+    gui.char_width += p_charspace;
 
     gui.char_height = tm.tmHeight + p_linespace;
 
diff --git a/src/option.c b/src/option.c
index c581e47a1..93de1e307 100644
--- a/src/option.c
+++ b/src/option.c
@@ -3256,7 +3256,7 @@ set_num_option(
     }
 
 #ifdef FEAT_GUI
-    else if (pp == &p_linespace)
+    else if (pp == &p_linespace || pp == &p_charspace)
     {
  // Recompute gui.char_height and resize the Vim window to keep the
  // same number of lines.
diff --git a/src/option.h b/src/option.h
index 27aae4099..0844b743c 100644
--- a/src/option.h
+++ b/src/option.h
@@ -456,6 +456,9 @@ EXTERN int p_deco; // 'delcombine'
 #ifdef FEAT_EVAL
 EXTERN char_u *p_ccv; // 'charconvert'
 #endif
+#ifdef FEAT_GUI
+EXTERN long p_charspace; // 'charspace'
+#endif
 EXTERN char_u *p_cino; // 'cinoptions'
 #ifdef FEAT_CMDWIN
 EXTERN char_u *p_cedit; // 'cedit'
diff --git a/src/optiondefs.h b/src/optiondefs.h
index 6e9701ab3..e0c893f0d 100644
--- a/src/optiondefs.h
+++ b/src/optiondefs.h
@@ -566,6 +566,14 @@ static struct vimoption options[] =
      {(char_u *)0L, (char_u *)0L}
 #endif
      SCTX_INIT},
+    {"charspace",   "csp",  P_NUM|P_NODEFAULT|P_VIM|P_RCLR,
+#ifdef FEAT_GUI
+     (char_u *)&p_charspace, PV_NONE,
+#else
+     (char_u *)NULL, PV_NONE,
+#endif
+     {(char_u *)0L, (char_u *)0L}
+     SCTX_INIT},
     {"cindent",     "cin",  P_BOOL|P_VI_DEF|P_VIM,
 #ifdef FEAT_CINDENT
      (char_u *)&p_cin, PV_CIN,

2023년 12월 18일 월요일 오후 8시 10분 0초 UTC+9에 Korean user1 (in CJK)님이 작성:

[Christian Brabandt]

On Mo, 18 Dez 2023, Korean user1 (in CJK) wrote:

> The range of Unicode symbols is extensive. I haven't tested all Unicode symbols for half-width display.

Please give an example of such a char then.

> I tested it in the corrected vim yesterday by a Japanese named Kaoriya, and there was no half-width marking problem I mentioned.
> I received it in the following url.
>
> https://www.kaoriya.net/software/vim/
> http://vim-jp.org/redirects/koron/vim-kaoriya/latest/win64/
> (vim82-kaoriya-win64-8.2.1287-20200724.zip)
>
> There are several diff files in the patch folder, but I think the part I pointed out as a bug is related to the 2003-charspace.diff file.
> I tested it on the vim of Kaoriya and attached 2 gif files.
>

> vim-kaoriya-fork-display-unicode-symbol-1.gif
> vim-kaoriya-fork-display-unicode-symbol-2.gif

I am not using a gui based mail client, that does not help me.

Thanks,
Christian
--
The moon is made of green cheese.
-- John Heywood

[Christian Brabandt]

That patch seem to do something different, e.g. simply adding horizontal
whitespace between gui cells on Windows (only). I don't see how this is
related to displaying half-width chars.

Thanks,
Chris


Thanks,
Christian
--
Information Center, n.:
A room staffed by professional computer people whose job it is to
tell you why you cannot have the information you require.

[Korean user1 (in CJK)]

2023년 12월 18일 월요일 오후 8시 28분 31초 UTC+9에 Christian Brabandt님이 작성:

[Christian Brabandt]

On Mo, 18 Dez 2023, Korean user1 (in CJK) wrote:

> vim-cmd-cjk-symbol-halfwidth-bug-1.gif

????


Thanks,
Christian
--
"I don't think so," said Ren' e Descartes. Just then, he vanished.

[Eric Pruitt]

On Mon, Dec 18, 2023 at 12:28:24PM +0100, Christian Brabandt wrote:

I am not using a gui based mail client, that does not help me.

https://groups.google.com/g/vim_dev/c/xyIQwAIp8-w/m/j46eg7wcAQAJ

Eric

[Ken Takata]

Hi,

This kind of displaying error can happen by the ambiguous width characters.

This can happen when the character width defined by the selected font and the width that Vim is recognizing don't match.

(For console version of Vim, the width that the terminal emulator is recognizing must also match.)

Normally, this can be controlled by the 'ambiwidth' option.

If you use CJK fonts, setting this to "double" might be better.

KaoriYa Vim provides an additional setting to the 'ambiwidth' option.

It provides ambiwidth=auto setting which automatically recognize the width from the actual font.

(ambiwidth=auto is the default setting for KaoriYa Vim.)

If you use DirectWrite, the issue can be more complicated.

If a glyph isn't found in the current font, Windows automatically find the glyph from other fonts.

E.g. If you select a double-width font, but if the glyph from a single-width font is used,

the width can be changed.

(KaoriYa Vim's ambiwidth=auto patch doesn't support DirectWrite, but I have an additional patch to support it.)

If you need a fine tuning, you can also use setcellwidths().

Regards,

Ken Takata

2023年12月18日月曜日 10:01:58 UTC+9 Korean user1 (in CJK):

[Korean user1 (in CJK)]

Thank you very much! 

I did what you told me to do, 

and my long-standing worries have finally been resolved!

I checked the description of the ambiwidth option in the options.txt document file of the doc folder. There was no "auto" as the value of the ambiwidth option, but "single" and "double". The default was "single". I finally solved the troublesome problem by typing like this as you told me.

:set ambiwidth=double

I attached a gif file that captured the screen where this problem was solved.

HA HA HA ! :)

I feel very good. Thank you again.

2023년 12월 19일 화요일 오후 3시 30분 43초 UTC+9에 Ken Takata님이 작성: