Help with suppressing display of certain characters.

35 views
Skip to first unread message

itabtabai

unread,
Jun 12, 2008, 7:27:00 AM6/12/08
to vim_multibyte
Hello,

I do edits using regexp on large files. However, I am facing problems
with Arabic diacritics display on MacVim, and I am assuming it is
inherited from vim so there is no work around it. Therefore, I wonder
if I can suppress certain characters from being displayed so the text
would show neatly and editing files would be easy on the eyes.

any help would be appreciated ...

thanks

björn

unread,
Jun 12, 2008, 7:37:55 AM6/12/08
to vim_mu...@googlegroups.com
2008/6/12 itabtabai <itab...@gmail.com>:

I do not know for sure but this is possibly a problem with MacVim. It
is know that MacVim has problems with certain nonspacing marks to
which set Arabic diacritics may belong. If you could post a sample of
some problematic text we could confirm whether or not this is a MacVim
specific problem.

Björn

itabtabai

unread,
Jun 12, 2008, 7:48:37 AM6/12/08
to vim_multibyte
Thanks for the prompt reply.

What do you mean by "sample of problematic code"? A sample of the
arabic text that is not displaying well? Screenshot?

On Jun 12, 2:37 pm, "björn" <bjorn.winck...@gmail.com> wrote:
> 2008/6/12 itabtabai <itabta...@gmail.com>:

Nico Weber

unread,
Jun 12, 2008, 7:50:58 AM6/12/08
to vim_mu...@googlegroups.com
> What do you mean by "sample of problematic code"? A sample of the
> arabic text that is not displaying well? Screenshot?

A screenshot would do, but a short text file containing the
problematic characters would be even better.

Nico

itabtabai

unread,
Jun 12, 2008, 8:21:11 AM6/12/08
to vim_multibyte
Sorry guys, but it is my first time here, I've got two comparative
screenshots and a sample text file. How do I post them?

Nico Weber

unread,
Jun 12, 2008, 8:38:12 AM6/12/08
to vim_mu...@googlegroups.com
Hi,

> Sorry guys, but it is my first time here, I've got two comparative
> screenshots and a sample text file. How do I post them?

you could use the "upload file" button at http://groups.google.com/group/vim_mac
, post them to rapidshare, or append them to a mail if the files are
not too big.

Nico

itabtabai

unread,
Jun 12, 2008, 9:09:50 AM6/12/08
to vim_multibyte
Can't upload, e-mail is rejected!
My first time using rapidshare, I hope the link is the right one:

http://rapidshare.com/files/121921145/macvim.gif.html

On Jun 12, 3:38 pm, Nico Weber <nicolaswe...@gmx.de> wrote:
> Hi,
>
> > Sorry guys, but it is my first time here, I've got two comparative
> > screenshots and a sample text file. How do I post them?
>
> you could use the "upload file" button athttp://groups.google.com/group/vim_mac

Tony Mechelynck

unread,
Jun 12, 2008, 9:46:48 AM6/12/08
to vim_mu...@googlegroups.com
On 12/06/08 15:09, itabtabai wrote:
> Can't upload, e-mail is rejected!
> My first time using rapidshare, I hope the link is the right one:
>
> http://rapidshare.com/files/121921145/macvim.gif.html
>

Hmmm....
- Is your Vim version compiled with +arabic ?
- It could be a problem with your console if you weren't using gvim
- I'm attaching a _small_ text file in UTF-8, which contains one line in
vocalized Arabic. My gvim (in 'rightleft' mode of course) displays it
correctly, with the vowels above or below the proper consonants and the
latters properly joined where appropriate. Do you see it correctly in
GUI mode? The text is a commonly "polite" form of salutation,
"as-salaamu `alaykum wa-rahmatu 'llaahi wa-barakaatuhu", or in English,
"Peace upon you and God's benevolence and His blessings"; if your mailer
displays the text attachment inline, that ought to give you a point of
comparison.


Best regards,
Tony.
--
The bureaucracy is expanding to meet the needs of an expanding
bureaucracy.

salaam-rahma.txt

björn

unread,
Jun 12, 2008, 9:56:15 AM6/12/08
to vim_mu...@googlegroups.com
2008/6/12 Tony Mechelynck <antoine.m...@gmail.com>:

Judging from how Tony's example renders in TextEdit and in MacVim.app
I'd say this is a problem with MacVim. Unfortunately, I think this
kind of problem is going to be tremendously hard to fix with the
current renderer but it might be possible with the ATSUI based
renderer. In any case, I don't think I can do much about it in MacVim
right now. Sorry. :-(

Jiang (if you are reading this), do you think you can get Tony's file
to render with ATSUI?

Björn

itabtabai

unread,
Jun 12, 2008, 10:15:17 AM6/12/08
to vim_multibyte
It might be a MacVim problem after all since it renders well on your
"gvim".
I am not sure if there is a "+arabic" literal configuration option. Do
you mean "multibyte" "huge" ./configure options? The "set arabic"
command works, so I am assuming that I've built with arabic support.

Note that the images I've sent are using the most decent fonts that
MacVim would display. I've been through more bad fonts at display,
e.g. some fonts would eat up the first character on the line. Not to
mention that when "arabic" is set and display becomes RTL the line
numbering would get messed up.

Should I compile MacVim with gvim support, maybe that would solve the
problem?

On Jun 12, 4:46 pm, Tony Mechelynck <antoine.mechely...@gmail.com>
wrote:
> [salaam-rahma.txt]أَلسَّلام٠ Ø¹ÙŽÙ„َيْك٠مْ وَرَحْمَة٠ Ù±Ù„له٠ ÙˆÙŽÙŽØ¨ÙŽØ±ÙŽÙƒØ§ØªÙ Ù‡Ù

itabtabai

unread,
Jun 12, 2008, 10:42:27 AM6/12/08
to vim_multibyte
Just a follow-up:
:version
would display +arabic.
Sorry I didn't understand your "+arabic" question.

Tony Mechelynck

unread,
Jun 12, 2008, 10:47:19 AM6/12/08
to vim_mu...@googlegroups.com
On 12/06/08 16:15, itabtabai wrote:
> It might be a MacVim problem after all since it renders well on your
> "gvim".
> I am not sure if there is a "+arabic" literal configuration option. Do
> you mean "multibyte" "huge" ./configure options? The "set arabic"
> command works, so I am assuming that I've built with arabic support.

To see if it is compiled with +arabic enabled, look at the output of
":version". On the first line after "Features enabled [+] or not [-]:",
the first item is usually either "+arabic" (feature enabled) or
"-arabic" (feature disabled). If it is disabled, your version of Vim
probably knows nothing about how to shape Arabic letters within words or
about how to place harakat (i.e., short vowels with or without tanueen,
as well as shadda and sukuun) on top of consonants (or under them as the
case may be).

>
> Note that the images I've sent are using the most decent fonts that
> MacVim would display. I've been through more bad fonts at display,
> e.g. some fonts would eat up the first character on the line. Not to
> mention that when "arabic" is set and display becomes RTL the line
> numbering would get messed up.

If setting 'arabic' makes the display RTL, your version is probably
compiled with Arabic script support.

>
> Should I compile MacVim with gvim support, maybe that would solve the
> problem?

Compiling Vim with GUI support (be it the X11 GUI or the native-Mac GUI)
would at least eliminate any problems due to running Vim inside
Terminal.app. Or are you already running in GUI mode but with some
dark-background colorscheme?


Best regards,
Tony.
--
BEDEVERE: And that, my lord, is how we know the Earth to be banana-shaped.
"Monty Python and the Holy Grail" PYTHON (MONTY)
PICTURES LTD

itabtabai

unread,
Jun 12, 2008, 11:01:21 AM6/12/08
to vim_multibyte
Running MacVim.app with dark background color scheme.

Running vim inside Terminal.app for arabic support is worthless. Apple
doesn't even properly support arabic in it's applications to say the
least.

On Jun 12, 5:47 pm, Tony Mechelynck <antoine.mechely...@gmail.com>
wrote:

itabtabai

unread,
Jun 12, 2008, 5:13:10 PM6/12/08
to vim_multibyte
Just wanted to thank everyone.

I got gvim, using fink, and bitstream sans mono ... the display is so
crisp it makes me cry!

Kenneth Reid Beesley

unread,
Jun 12, 2008, 5:24:45 PM6/12/08
to vim_mu...@googlegroups.com

You might just post a short sequence of Unicode code-point values, e.g.

(beginning)

0x0643 (kaaf)
0x064F (Damma)
0x062A (taa')
0x0650 (kasra)
0x0628 (baa')
0x064E (fatHa)
0x062A (taa')
0x0652 (sukuun)

(end)

and explain what is not being rendered correctly. (Here the kaaf-taa'-
baa'-taa' should
be displayed properly connected, right-to-left; the Damma should
appear above the (initial-form) kaaf, the kasra should appear below
the (medial-form) taa', the fatHa should appear above the (medial-
form) baa', and the sukuun should appear above the (final-form) taa'.

Please specify which font you are using.

Best,

Ken


******************************
Kenneth R. Beesley, D.Phil.
P.O. Box 540475
North Salt Lake, UT
84054 USA

Tony Mechelynck

unread,
Jun 12, 2008, 5:36:42 PM6/12/08
to vim_mu...@googlegroups.com
On 12/06/08 23:13, itabtabai wrote:
> Just wanted to thank everyone.
>
> I got gvim, using fink, and bitstream sans mono ... the display is so
> crisp it makes me cry!

Well, so much the better for you.

FWIW, in Bitstream Vera Sans Mono I see the harakat (Arabic short vowels
etc.) one cell left of where they should be, but in Courier New I see
them just where they belong -- using the same text file and the same
gvim binary.

Best regards,
Tony.
--
Philogyny recapitulates erogeny; erogeny recapitulates philogyny.

Jjgod Jiang

unread,
Jun 12, 2008, 7:16:11 PM6/12/08
to vim_mu...@googlegroups.com
Hi,

On Thu, Jun 12, 2008 at 6:56 AM, björn <bjorn.w...@gmail.com> wrote:
> Judging from how Tony's example renders in TextEdit and in MacVim.app
> I'd say this is a problem with MacVim. Unfortunately, I think this
> kind of problem is going to be tremendously hard to fix with the
> current renderer but it might be possible with the ATSUI based
> renderer. In any case, I don't think I can do much about it in MacVim
> right now. Sorry. :-(
>
> Jiang (if you are reading this), do you think you can get Tony's file
> to render with ATSUI?

Sorry for the delayed reply. I don't think it will work properly like
TextEdit did.
At least not for now, but I'll try to investigate this (Just as I told
Ben in my last
mail on vim_mac list).

- Jiang

itabtabai

unread,
Jun 13, 2008, 7:48:01 AM6/13/08
to vim_multibyte
Hi Ken,

I've uploaded screenshots of the display with different fonts on both
MacVim and gvim.

http://rapidshare.com/files/122137390/Archive.tbz.html

All fonts are displayed at size 20. My comments on the screenshots:
gvim-bitsreamB.png: Crisp, nice, spread, and easy on the eyes.
macvim-dejavuB.png: The more diacritics (harakaat) I put the more the
line number is pushed off screen and the letters start having gaps.
macvim-dejavuB2.png: Only just starting to type.
macvim-dejavuB3.png: Only one letter and it's ""haraka" and the line
number is already pushed.

Tony,
The rest for you:
gvim-courierNR.png: Not bad. But you note the gaps due to "harakaat".
gvim-courierNB.png: Nice, crisp, but not as easy on the eyes as
bitstream/dejavu. Bitstream/dejavu spreading/cursivness is very neat.
gvim-courier10B.png: Gaps are very clear.

So, my conclusion from all the above is that the defect of display is
not only MacVim specific. It also relates to the fonts used in a less
intrusive manner. While some fonts do produce gaps as worst case
scenario on gvim, almost all fonts don't work properly with diacritics/
harakaat on MacVim. Also the fact that none monospaced fonts don't
display properly in MacVim nor gvim.
Reply all
Reply to author
Forward
0 new messages