gvim and ASCII glyphs

[François Gannaz]

Hello

In a few words, here is a patch that makes gvim work better with ligatures
in fonts, which can be useful even for programmers. Details follow.

I tried to use a Hasklig[^1], a font with ligatures intended for the
Haskell language. It serves the same objective as the Haskell Conceal
script[^2], but with the added benefit that even a mouse copy-paste works
as intended.
[^1]: https://github.com/i-tu/hasklig
[^2]: http://www.vim.org/scripts/script.php?script_id=3200

Unfortunately, gvim doesn't support ligatures on ASCII characters. The
following assertion fails at run-time:

ascii_glyph_table_init: assertion 'gui.ascii_glyphs->num_glyphs ==
sizeof(ascii_chars)' failed

and many characters are displayed with the wrong glyphs.
The attached patch limits the function ascii_glyph_table_init() to
spaces and alphanumeric chars. It solves the problem here.

Yet I wonder if the current hack with ASCII characters is really useful.
Is there any performance test to check if a simpler behaviour wouldn't be
suitable, at least for modern desktop installations?
As the code comment mentions spaces, maybe it should be restricted to
blank lines?

Regards
--
François

[Bram Moolenaar]

This stuff is too complicated to guess what works best.
Any changes need to be tested with various GTK versions and with
different display drivers to check the speed. Even modern desktops
sometimes have slow graphics (using generic drivers instead of
proprietary ones).

Note the remark about drawing the cursor, check that the test doesn't
change when moving the cursor over it.

Perhaps we should add an option to disable the optimization?


--
hundred-and-one symptoms of being an internet addict:
80. At parties, you introduce your spouse as your "service provider."

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[François Gannaz]

Le 2014-04-21, Bram Moolenaar <Br...@moolenaar.net> a écrit :
> >
> > Yet I wonder if the current hack with ASCII characters is really useful.
> > Is there any performance test to check if a simpler behaviour wouldn't be
> > suitable, at least for modern desktop installations?
> > As the code comment mentions spaces, maybe it should be restricted to
> > blank lines?
>
> This stuff is too complicated to guess what works best.
> Any changes need to be tested with various GTK versions and with
> different display drivers to check the speed. Even modern desktops
> sometimes have slow graphics (using generic drivers instead of
> proprietary ones).
>
> Note the remark about drawing the cursor, check that the test doesn't
> change when moving the cursor over it.

It was clear this stuff was complicated, that's why I opted for a minimal
change. It didn't have any visible effect on my desktop, and now I can
use some fonts that were mangled in gvim before (mangled on lines with no
non-ASCII chars!).

> Perhaps we should add an option to disable the optimization?

In my humble opinion, removing the current optimization hack should be a
long-term goal. The optimization adds complexity to the display code,
with various side-effects. And as Pango and such have evolved, I don't
know if there is still a need for a distinct display routine on ASCII
lines.

Thanks
--
François

[Hörmetjan Yiltiz]

This would be so great to see implemented.

[manuelschi...@googlemail.com]

It certainly would (and apologies for reopening such an old discussion).

On my system, the patch François sent doesn't work for the ligatures in PragmataPro (but flawlessly for other fonts like Hasklig), apparently because the ASCII-to-glyph mapping won't quite work - I suspect that's due to PragmataPro having ligatures in that range, too. I solved this by inserting a space between characters that we hand to pango for shaping. Sane fonts will not have ligatures between a space and a printable character, this way we still get one glyph per ASCII character. I've attached the modified patch. This will work with normal fonts and ones with programming ligatures like Hasklig and PragmataPro; I've tested it with the gtk2 and gtk3 front ends.

Please let me know if it would be possible to include this, and if not, at least the patch is public now where people can find it if they want it.

Best regards,

Manuel

[Bram Moolenaar]

Let's hear from a few people whether this works in various contexts.

--
Are leaders born or made? And if they're made, can we return them under
warranty?
(Scott Adams - The Dilbert principle)

[manuelschi...@googlemail.com]

On Sunday, 7 August 2016 15:19:48 UTC+2, Bram Moolenaar wrote:

> Manu Schiller wrote:
> > Please let me know if it would be possible to include this, and if
> > not, at least the patch is public now where people can find it if they
> > want it.
>
> Let's hear from a few people whether this works in various contexts.

That's all I can ask for. :) (And it's anyway the drill I'm used to from my contributions to ROOT.) I'm willing to try and help fix any breakage, should we get any.

Manuel

[Kazunobu Kuriyama]

According to http://www.fsd.it/shop/fonts/pragmatapro/ ,  PragmataPro has "One typeface, 3 families for the right need":

- PragmataPro is a spaced modularly font family with working ligatures.

- PragmataPro Mono is a very monospaced without ligatures.

- Essential PragmataPro is the cheap version to coding for English only. 

As you know, Vim is implemented by design only to support monospace font families.

So it looks to me that, among those three font families,  the only possible and suitable choice for Vim to use with is PragmataPro Mono alone.

If that is the case, however, that font family is described as "without ligatures," which makes your patch no sense.

Accordingly, it appears my assumption was wrong; it might be better for me to conclude that what you meant by PragmataPro was literally PragmataPro, not its monospace version, or PragmataPro Mono.

Then, another question arises:  What  does "a spaced modularly font" mean?  Does it imply that the font family in question is technically considered something equivalent to a monospaced font and hence deserve Vim's support?

I'd appreciate it if you would give me some clarification or correct my possible misunderstanding.

Best regards,

Kazunobu Kuriyama

apparently because the ASCII-to-glyph mapping won't quite work - I suspect that's due to PragmataPro having ligatures in that range, too. I solved this by inserting a space between characters that we hand to pango for shaping. Sane fonts will not have ligatures between a space and a printable character, this way we still get one glyph per ASCII character. I've attached the modified patch. This will work with normal fonts and ones with programming ligatures like Hasklig and PragmataPro; I've tested it with the gtk2 and gtk3 front ends.

Please let me know if it would be possible to include this, and if not, at least the patch is public now where people can find it if they want it.

Best regards,

Manuel

--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[manuelschi...@googlemail.com]

In PragmataPro, all ASCII glyphs etc. are monospaced. Just as they are in PragmatePro Mono or Courier, or whatever your favourite monospaced font is. The difference between the two is what happens during shaping into glyphs inside pango (or another library): For fonts like Hasklig or PragmataPro, some combinations of ASCII characters get assigned a new glyph (the ligature) which looks better (e.g. "<=" might be replaced with a new glyph that more closely mimics the mathematical notation), but fits into the normal monospaced character grid. These ligature characters are two and sometimes three times the width of a single ASCII glyph, depending on how many characters they replace (if it's a monospace font - in a proportional font like Times, the ligature for something like "fl" can be any width the font designer likes, and I'm not proposing to support this latter case).

That kind of character/glyph juggling is okay because it fits with what vim is expecting to draw anyway (so vim's drawing code can largely remain unaware of the font shaping going on) - it's expecting to draw something that's e.g. 2 characters wide, and it draws (a single) glyph that wide.

(In PragmataPro Mono, these ligature tables are absent, so shaping just returns the normal single-width glyphs.)

The point of this patch is that it ensures that vim gets the right single width glyphs for all monospaced fonts, even in the code that bypasses the glyph shaping process and does its own thing.

I've attached screenshots how unpatched vim (as shipped with the latest stable debian) and patched vim look with these fonts. In all cases, I do a ":version" before I take the screenshot. As you can see, it's totally garbled in case of the unpatched versions of the plots, despite the fact that the two fonts are monospaced (and are counted as such by all major terminal programs).

This is a corner case where the optimisation to speed up rendering of ASCII glyphs in vim (by bypassing the shaping process) gets it wrong (namely the corner case where the font designer specified ligatures for a font). What happens in these cases is that pango returns a list with fewer glyphs than characters (because some character combinations return only a single glyph), and vim happily reads past the end of the glyph list and caches the result to render ascii glyphs. The effect is unreadable gibberish, as you can see. I think it should just be fixed, because it's ugly and needlessly astonishes unsuspecting users (neglecting for a moment the fact that there may also be security implications of such accesses past the end of a data structure)...

> If that is the case, however, that font family is described as "without ligatures," which makes your patch no sense.

(See above.)

> Accordingly, it appears my assumption was wrong; it might be better for me to conclude that what you meant by PragmataPro was literally PragmataPro, not its monospace version, or PragmataPro Mono.

(See above again.)

> Then, another question arises:  What  does "a spaced modularly font" mean?  Does it imply that the font family in question is technically considered something equivalent to a monospaced font and hence deserve Vim's support?

I'll try and render it yet another way: If you ask pango/... to draw a single character in a monospaced font, it's always the same width. But, in some monospaced fonts, for some two (and sometimes three) letter combinations, the font rendering library substitutes a glyph that looks better, but occupies exactly the width of two (three) individual characters. While you still fit totally into the monospaced character grid, you draw fewer glyphs than input characters now...

I think this implies that such modern programmer's fonts like Hasklig or PragmataPro are effectively a monospaced fonts, and people will increasingly want to use them. (In fact, you're talking to a former die-hard text mode console adept, who after 15+ years started to see the point of GUIs...) These are questions that people will keep asking (probably increasingly often). In my opinion, vim would do well to include support for this kind of effectively monospaced fonts, so the answer to those questions can become "update to the latest version", and you guys can get back to some serious hacking again... ;)

> I'd appreciate it if you would give me some clarification or correct my possible misunderstanding.

I hope this somehow clarifies the issue. I'm not proposing to implement proportional font support or anything like that. But the assumption in the current code that one ascii character renders to a single glyph is no longer true in today's world, and I think that needs to be fixed somehow. The fact that it also happens to make Hasklig or PragmataPro render correctly is, for me, a welcome side effect (and also happens to be the reason why I got started with this). But the longer I think about this patch, the more I tend to think of the current (unpatched) drawing code as a liability for vim because it reads past the end of a data structure without any checks. We're lucky that only the graphical output is garbled.

I hope this clarifies things a bit.

Best regards,

Manuel

> Best regards,
> Kazunobu Kuriyama
>
> apparently because the ASCII-to-glyph mapping won't quite work - I suspect that's due to PragmataPro having ligatures in that range, too. I solved this by inserting a space between characters that we hand to pango for shaping. Sane fonts will not have ligatures between a space and a printable character, this way we still get one glyph per ASCII character. I've attached the modified patch. This will work with normal fonts and ones with programming ligatures like Hasklig and PragmataPro; I've tested it with the gtk2 and gtk3 front ends.
>
>
>
> Please let me know if it would be possible to include this, and if not, at least the patch is public now where people can find it if they want it.
>
>
>
> Best regards,
>
>
>
> Manuel
>
>
>
> --
>
> --
>
> You received this message from the "vim_dev" maillist.
>
> Do not top-post! Type your reply below the text you are replying to.
>
> For more information, visit http://www.vim.org/maillist.php
>
>
>
> ---
>
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
>

> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.

[Tony Mechelynck]

Hm, the "unpatched hasklig" case seems to be just off-by-one
everywhere, while the other "unpatched" one looks pathologic.

In current versions of Vim, there is no necessary equivalence between
the number of bytes and the number of screen cells used by a given
character. CJK "wide" characters take up two screen cells each
regardless of the number of bytes, and everything else takes up one
cell per character regardless of the number of bytes. (For instance,
in UTF-8 which is the recommended 'encoding' these days, codepoints
U+0080 to U+07FF use two bytes each, and anything from U+0800 up uses
at least three. FWIW, the "usual" Kangxi characters start at U+4E00
with the single-line 一 character [Kangxi radical 1 plus no strokes]
meaning "one", but there are "extension CJK" characters as low as
U+3400, and the "Compatibility Ideographs Supplement" block starts at
U+2F800 and then goes up I don't know how high.) The horizontal
tabulation U+0009 is of course a special case, in a category by
itself.

Best regards,
Tony.

[Kazunobu Kuriyama]

Sorry, but I still don't understand how you justify your patch that adds 0x20 to each of alphanumeric characters and send them to Pango.

Actually, what is your point to make every text data length double?  That is 100% inefficient and totally ruins the existing optimization hack.  Just as what you wrote in your patch yourself, that's a ugly hack, indeed.

If I wrote a patch for ligature support, I would remove the optimization hack and add some code to cursor movement over a ligature, selection mechanisms for visual mode and text objects when either of the region boundaries has a ligatures on it, line wrapping handling at a ligature (to name but a few) in order to decently accommodate ligatures in vim.

What consideration is paid to those in your patch?   I don't see any.

My question in the previous mail was whether or not PragmataPro was a monospace font family as binary data.  Not that what you felt about it nor what the world saw about it.  That was a purely technical question.  Was it too hard for you to check your PragmataPro file(s) with an appropriate utility?

I'm afraid your reply to that, which sounds somehow insulting old-school people is completely out of place.

I'm sorry that you don't care about technical aspects of the issue.  I really feel sick when I hear someone talks about old vs new to justify his view.

Why don't you send us a patch on that?

 

To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@googlegroups.com.

[manuelschi...@googlemail.com]

On Monday, 8 August 2016 12:49:15 UTC+2, Kazunobu Kuriyama wrote:
> Sorry, but I still don't understand how you justify your patch that adds 0x20 to each of alphanumeric characters and send them to Pango.

In my opinion, the optimisation in and of itself is a bit of a hack. The trouble is that it does the wrong thing in case of ligatures, since you get fewer glyphs than ASCII characters, which breaks the 1-to-1 mapping. The spaces are added to prevent ligatures (since most fonts with ligatures do not define a ligature between a space and another character).

In some sense, this is still a hack, and what you would really want to do (if you want to keep the optimisation hack) is to shape one character one at a time, so are guaranteed to get one glyph per ascii character...

> Actually, what is your point to make every text data length double?  That is 100% inefficient and totally ruins the existing optimization hack.  Just as what you wrote in your patch yourself, that's a ugly hack, indeed.

It is, but bypassing the shaping to gain a bit of speed is ugly to start with, and adds quite a bit of code, data structures and complexity to vim's rendering routines. If I just disable that optimisation, vim doesn't have any issue with monospaced fonts, ligatures or otherwise.

It's an optimisation, and one that isn't quite right. My point is that my patch contains more of the correct solution than does the code that's currently there. I don't claim it's nice, or beautiful, or anything. The double-length array is only a cost in memory, some small multiple of 128 bytes (likely 4 times that), which I doubt will be a consideration if you're running gvim, and an integer multiply (by 2, so likely a left shift) in the code that renders glyphs in the code path that is used by the optimisation...

> If I wrote a patch for ligature support, I would remove the optimization hack and add some code to cursor movement over a ligature, selection mechanisms for visual mode and text objects when either of the region boundaries has a ligatures on it, line wrapping handling at a ligature (to name but a few) in order to decently accommodate ligatures in vim.
>
> What consideration is paid to those in your patch?   I don't see any.

The trick is to activate the optimisation not for all ASCII characters (chars < 128), but only for [A-Z], [a-z], [0-9] and space. If a string to be displayed contains other characters, it'll pass through the full blown shaping and has its glyphs drawn, including ligatures.

> My question in the previous mail was whether or not PragmataPro was a monospace font family as binary data.  Not that what you felt about it nor what the world saw about it.  That was a purely technical question.  Was it too hard for you to check your PragmataPro file(s) with an appropriate utility?

I tried to answer your question as best as I could, and I'm sorry if I didn't quite hit the mark. In fact, with your new question, I don't even know what "monospace font family as binary data" means, maybe you could clarify. Unless you can read TrueType/OpenType fonts in hexdumps, and glean meaning from it, this will tell you very little. It's not a bitmap font, if that's what you mean.

> I'm afraid your reply to that, which sounds somehow insulting old-school people is completely out of place.

I certainly never intended to insult anyone, and apologise if I have given offence. In fact, I've been programming for more than 25 years now, and am considered old-school by many of my colleagues. Changing one's preference in text editor user interface is certainly nothing that is carved in stone for all eternity, and people should have a freedom of choice.

> I'm sorry that you don't care about technical aspects of the issue.  I really feel sick when I hear someone talks about old vs new to justify his view.

I tried to present the technical issues at stake to the best of my (limited) ability and knowledge. I'm sorry if I didn't quite hit the mark. But to assert that I don't care about the technical aspects of the issue is a gross oversimplification. In fact, I'd welcome any constructive input I can get on this matter. Speaking of which: have you tried the patch with your favourite font? Does it work for you? (The patch may very well need improving, and I'm happy to do just that. But if you haven't given the patch a try, you might want to do that before judging and its author in such a harsh tone.)

Concerning your request to send a patch that just fixes the assumption about "one ascii character == one glyph": One could either get rid of that speed optimisation code altogether (in which case all ligatures will work), or rewrite most of that optimisation, doing one shaping call into pango for each of (or some of) the characters with codes < 128. The latter versoin would amount to some overhead for all those calls, and reassembling the returned data structures in a suitable way...

Instead, I preferred to go with a more minimal patch, but I'm more than willing to listen to whatever consensus emerges.

Best regards,

Manuel

[manuelschi...@googlemail.com]

Hi Tony,

your musings on UTF-8 sound reasonable, although I'm far from an expert. In fact for unicode characters with character code >= 128, even the current vim code base will always do the full-blown character shaping, and apply whatever magic it applies to display the text. And it'll hopefully get things correct even without my patch.

To render strings with codes < 128, however, the current code will apply an optimisation for skipping the shaping step inside pango. The character code -> glyph mapping is no longer 1:1 for a font with ligatures in that range, and that is why the cache of that mapping appears to display gibberish.

So it's less a question of the mapping of characters (whatever their encoding) to screen cells, but that a speed optimisation inside the gtk gui code can be fooled by some fonts...

Best regards,

Manuel

[Tony Mechelynck]

Well, if you let Pango do glyph reshaping for U+0020 to U+007F you
might end up with what you said you didn't want, i.e. fi fl st ffi ffl
digraphs and trigraphs, which in my experience are actually uglier (in
monospaced fonts) than just letting the individual letters stand side
by side. Try the following in a gvim (even unpatched) with 'encoding'
set to utf-8 to see what I mean:
1. Enter Insert mode (i or a or o or whatever)
2. Type the following sequences (with no spaces). Each sequence should
output one glyph.
Ctrl-V u fb00
Ctrl-V u fb01
Ctrl-V u fb02
Ctrl-V u fb03
Ctrl-V u fb04
Ctrl-V u fb05
Ctrl-V u fb06
You should see, in that order, the glyphs for ff fi fl ft st ffi ffl.
I absolutely don't like them (in the monospaced fonts that have them;
in serif or sans-serif fonts it's different, especially in serif
italic). OTOH, I think that expanding them to two or three character
cells would be ugly the opposite way.

Best regards,
Tony.

On Mon, Aug 8, 2016 at 1:50 PM, manuelschiller.pimail via vim_dev

[manuelschi...@googlemail.com]

On Monday, 8 August 2016 14:51:01 UTC+2, Tony Mechelynck wrote:
> Well, if you let Pango do glyph reshaping for U+0020 to U+007F you
> might end up with what you said you didn't want, i.e. fi fl st ffi ffl
> digraphs and trigraphs, which in my experience are actually uglier (in
> monospaced fonts) than just letting the individual letters stand side
> by side. Try the following in a gvim (even unpatched) with 'encoding'
> set to utf-8 to see what I mean:
> 1. Enter Insert mode (i or a or o or whatever)
> 2. Type the following sequences (with no spaces). Each sequence should
> output one glyph.
> Ctrl-V u fb00
> Ctrl-V u fb01
> Ctrl-V u fb02
> Ctrl-V u fb03
> Ctrl-V u fb04
> Ctrl-V u fb05
> Ctrl-V u fb06
> You should see, in that order, the glyphs for ff fi fl ft st ffi ffl.
> I absolutely don't like them (in the monospaced fonts that have them;
> in serif or sans-serif fonts it's different, especially in serif
> italic). OTOH, I think that expanding them to two or three character
> cells would be ugly the opposite way.

Point taken, but this is not what the patch does. The sets [A-Z], [a-z], [0-9], and ' ' are still drawn without glyph shaping, when there's nothing else in the string being drawn that requires it - just as the old code did. (And the old code also fell back to full glyph shaping once you move above character code 128, i.e. into the range where you start to get UTF-8 encoded characters...) So in that sense, there's no substantial change in strategy.

Also, I'm not changing the width of any glyph drawn, so there is no expansion of glyphs going on. It's just a question of how exactly the cache works that bypasses the glyph shaping (and which strings are allowed to bypass the glyph shaping process by going through the cache...).

Are you proposing changes to the patch, or are you more concerned about how it'll look? In the former case, maybe you could outline which changes you'd like to see? In the latter case, why not just give the patch a try (e.g by typing one of the character combinations, and see if it does what you'd like it to do), and tell me which bits you don't like (which is much more productive than discussing in the abstract)?

Kind regards,

Manuel

[Kazunobu Kuriyama]

2016-08-08 20:37 GMT+09:00 manuelschiller.pimail via vim_dev <vim...@googlegroups.com>:

On Monday, 8 August 2016 12:49:15 UTC+2, Kazunobu Kuriyama  wrote:
> Sorry, but I still don't understand how you justify your patch that adds 0x20 to each of alphanumeric characters and send them to Pango.

In my opinion, the optimisation in and of itself is a bit of a hack. The trouble is that it does the wrong thing in case of ligatures, since you get fewer glyphs than ASCII characters, which breaks the 1-to-1 mapping. The spaces are added to prevent ligatures (since most fonts with ligatures do not define a ligature between a space and another character).

In some sense, this is still a hack, and what you would really want to do (if you want to keep the optimisation hack) is to shape one character one at a time, so are guaranteed to get one glyph per ascii character...

> Actually, what is your point to make every text data length double?  That is 100% inefficient and totally ruins the existing optimization hack.  Just as what you wrote in your patch yourself, that's a ugly hack, indeed.

It is, but bypassing the shaping to gain a bit of speed is ugly to start with, and adds quite a bit of code, data structures and complexity to vim's rendering routines. If I just disable that optimisation, vim doesn't have any issue with monospaced fonts, ligatures or otherwise.

It's an optimisation, and one that isn't quite right. My point is that my patch contains more of the correct solution than does the code that's currently there. I don't claim it's nice, or beautiful, or anything. The double-length array is only a cost in memory, some small multiple of 128 bytes (likely 4 times that), which I doubt will be a consideration if you're running gvim, and an integer multiply (by 2, so likely a left shift) in the code that renders glyphs in the code path that is used by the optimisation...

> If I wrote a patch for ligature support, I would remove the optimization hack and add some code to cursor movement over a ligature, selection mechanisms for visual mode and text objects when either of the region boundaries has a ligatures on it, line wrapping handling at a ligature (to name but a few) in order to decently accommodate ligatures in vim.
>
> What consideration is paid to those in your patch?   I don't see any.

The trick is to activate the optimisation not for all ASCII characters (chars < 128), but only for [A-Z], [a-z], [0-9] and space. If a string to be displayed contains other characters, it'll pass through the full blown shaping and has its glyphs drawn, including ligatures.

> My question in the previous mail was whether or not PragmataPro was a monospace font family as binary data.  Not that what you felt about it nor what the world saw about it.  That was a purely technical question.  Was it too hard for you to check your PragmataPro file(s) with an appropriate utility?

I tried to answer your question as best as I could, and I'm sorry if I didn't quite hit the mark. In fact, with your new question, I don't even know what "monospace font family as binary data" means, maybe you could clarify. Unless you can read TrueType/OpenType fonts in hexdumps, and glean meaning from it, this will tell you very little. It's not a bitmap font, if that's what you mean.

If your windowing system is a recent X11 with fontconfig, do

    $ fc-scan <your favorite ligature font file>

then you'll have something human readable.

Take a look at an item called 'spacing.'

If you have none or the number zero, then the font is proportional.

If you have the number 90, then the font is dual.

If you have the number 100, then the font is monospace.

If you have the number 110, then the font is charcell

To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@googlegroups.com.

[manuelschi...@googlemail.com]

On Monday, 8 August 2016 15:39:02 UTC+2, Kazunobu Kuriyama wrote:
> If your windowing system is a recent X11 with fontconfig, do
>
>     $ fc-scan <your favorite ligature font file>
>
>
> then you'll have something human readable.
>
>
> Take a look at an item called 'spacing.'
>
>
> If you have none or the number zero, then the font is proportional.
> If you have the number 90, then the font is dual.
> If you have the number 100, then the font is monospace.
> If you have the number 110, then the font is charcell

Thanks, this is getting us somewhere. :)

I've run a couple of checks with the commands you suggested, and I get the following results:
- DejaVu and PragmataPro Mono do indeed have the spacing attribute set to 100.
- Hasklig and PragmataPro do not, in fact they do not show a spacing attribute. (However, as I tried to explain in that previous post in my own frightfully limited way, they're effectively monospaced, and after glyph shaping where appropriate, things will line up with the character cell layout that the rendering code assumes, I think...)

So, apparently, there's monospaced fonts, and fonts that just happen to have individual glyphs an intger multiple of the same width. I freely admit that I did not know about this, and apologise if I have given offence in the process.

To my mind, the question the list now needs to decide is this: Do we want to support these "effectively monospaced" fonts like Hasklig, or not? (I think we should, but I'll abide by your decision, of course. :)

If the answer is yes, we can discuss which bits of the patch you'd like cleaned up, and I'll do my best to provide something that's acceptable. Sorry again for the "exchange of fire" earlier, I certainly didn't mean to hurt anyone's feelings.

Best regards,

Manuel

[Kazunobu Kuriyama]

2016-08-08 23:06 GMT+09:00 manuelschiller.pimail via vim_dev <vim...@googlegroups.com>:

On Monday, 8 August 2016 15:39:02 UTC+2, Kazunobu Kuriyama  wrote:
> If your windowing system is a recent X11 with fontconfig, do
>
>     $ fc-scan <your favorite ligature font file>
>
>
> then you'll have something human readable.
>
>
> Take a look at an item called 'spacing.'
>
>
> If you have none or the number zero, then the font is proportional.
> If you have the number 90, then the font is dual.
> If you have the number 100, then the font is monospace.
> If you have the number 110, then the font is charcell

Thanks, this is getting us somewhere. :)

I've run a couple of checks with the commands you suggested, and I get the following results:
- DejaVu and PragmataPro Mono do indeed have the spacing attribute set to 100.
- Hasklig and PragmataPro do not, in fact they do not show a spacing attribute. (However, as I tried to explain in that previous post in my own frightfully limited way, they're effectively monospaced, and after glyph shaping where appropriate, things will line up with the character cell layout that the rendering code assumes, I think...)

By "if you have none..." I meant "(fc-scan) do(es) not show a spacing attribute."

Therefore, the conclusion is that both Hasklig and PragmataPro are proportional.  No matter how they appear "effectively" monospaced to our eyes, no rendering engine handles them as such.

Accordingly, Vim can't support them in an appropriate way and needn't.

Do you know any other fonts which have ligatures and are truly monospaced?   For such fonts, if any, I think there's room for consideration.

So, apparently, there's monospaced fonts, and fonts that just happen to have individual glyphs an intger multiple of the same width. I freely admit that I did not know about this, and apologise if I have given offence in the process.

To my mind, the question the list now needs to decide is this: Do we want to support these "effectively monospaced" fonts like Hasklig, or not? (I think we should, but I'll abide by your decision, of course. :)

I haven't seen this community works like that.  Needless to say, it's only you who can make a decision as to your way to go.

Cheers,

Kazunobu

If the answer is yes, we can discuss which bits of the patch you'd like cleaned up, and I'll do my best to provide something that's acceptable. Sorry again for the "exchange of fire" earlier, I certainly didn't mean to hurt anyone's feelings.

Best regards,

Manuel

--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_dev" group.

[manuelschi...@googlemail.com]

Hi Kazunobu,

On Monday, 8 August 2016 18:31:40 UTC+2, Kazunobu Kuriyama wrote:
> 2016-08-08 23:06 GMT+09:00 manuelschiller.pimail via vim_dev <vim...@googlegroups.com>:
>
>
> On Monday, 8 August 2016 15:39:02 UTC+2, Kazunobu Kuriyama  wrote:
>
> > If your windowing system is a recent X11 with fontconfig, do
>
> >
>
> >     $ fc-scan <your favorite ligature font file>
>
> >
>
> >
>
> > then you'll have something human readable.
>
> >
>
> >
>
> > Take a look at an item called 'spacing.'
>
> >
>
> >
>
> > If you have none or the number zero, then the font is proportional.
>
> > If you have the number 90, then the font is dual.
>
> > If you have the number 100, then the font is monospace.
>
> > If you have the number 110, then the font is charcell
>
>
>
> Thanks, this is getting us somewhere. :)
>
>
>
> I've run a couple of checks with the commands you suggested, and I get the following results:
>
> - DejaVu and PragmataPro Mono do indeed have the spacing attribute set to 100.
>
> - Hasklig and PragmataPro do not, in fact they do not show a spacing attribute. (However, as I tried to explain in that previous post in my own frightfully limited way, they're effectively monospaced, and after glyph shaping where appropriate, things will line up with the character cell layout that the rendering code assumes, I think...)
>
>
>
> By "if you have none..." I meant "(fc-scan) do(es) not show a spacing attribute."
>
>
> Therefore, the conclusion is that both Hasklig and PragmataPro are proportional.  No matter how they appear "effectively" monospaced to our eyes, no rendering engine handles them as such.

I understand that now.

>
> Accordingly, Vim can't support them in an appropriate way and needn't.
>

No, it need not do so, and for the general case of proportional fonts, I'm not trying to convince you, or anyone else. But the change required isn't a big one, and all it costs at runtime is a slightly more complicated initialisation of the cache bypassing the shaping, and a bit more memory for it (both of which are likely unnoticable among all the other things that gvim needs to do before it can start drawing characters in the edit section of the window). It also costs one left shift for the index into the cache. That's not what I'd call resource-hungry, or bloated. While I understand that vim wouldn't claim support, wouldn't it be nice if these effectively monospaced fonts would work to that extent at least?

>
> Do you know any other fonts which have ligatures and are truly monospaced?   For such fonts, if any, I think there's room for consideration.
>
>
>
>
> So, apparently, there's monospaced fonts, and fonts that just happen to have individual glyphs an intger multiple of the same width. I freely admit that I did not know about this, and apologise if I have given offence in the process.
>
>
>
> To my mind, the question the list now needs to decide is this: Do we want to support these "effectively monospaced" fonts like Hasklig, or not? (I think we should, but I'll abide by your decision, of course. :)
>
>
>
> I haven't seen this community works like that.  Needless to say, it's only you who can make a decision as to your way to go.
>

Well, I have my patch which does what I want. :) If you think there's something salvagable from it (e.g. reworking the bit of the cache that has the assumption that pango shaping will give one glyph per input character), let me know, and I'll see what I can do in my free time.

But from what I gather from your reaction (if I read between the lines correctly), you'd rather not include any of it because the fonts in question are not truely monospaced in the sense you indicated. Fair enough.

All the best,

Manuel

>
> Cheers,
> Kazunobu
>
>
>
>
> If the answer is yes, we can discuss which bits of the patch you'd like cleaned up, and I'll do my best to provide something that's acceptable. Sorry again for the "exchange of fire" earlier, I certainly didn't mean to hurt anyone's feelings.
>
>
>
> Best regards,
>
>
>
> Manuel
>
>
>
> --
>
> --
>
> You received this message from the "vim_dev" maillist.
>
> Do not top-post! Type your reply below the text you are replying to.
>
> For more information, visit http://www.vim.org/maillist.php
>
>
>
> ---
>
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
>
>
>

> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.

[Tony Mechelynck]

I'm concerned about how it will look, and I wasn't aware that you were
intentionally short-circuiting Pango for all letters and digits.

Yes, these 6 codepoints (U+FB01 to U+FB06) are above the ASCII range
but they show examples of what one could get by replacing some
particular pairs of letters. Similarly, the Arabic subsystem of Vim
fetches its glyphs by (IIUC) altering their codepoint numbers
depending on whether they're joined to a preceding character, or to a
following character, or whether a lam and an alif are found together
in that order, thus implementing the rules for Arabic character
shaping by means of "presentation form" codepoints.

[manuelschi...@googlemail.com]

Okay, now I understand - thanks for the clarification. Well, current vim code (and my patch) bypasses pango's glyph shaping completely in some cases. Current vim code does this if the string only contains code points <= 127. My patch restricts the range somewhat, so that glyph shaping occurs in more cases as with vanilla vim code. As far as I can tell, strings containing code points of 128 and above are left alone with or without my patch, and pango deals with them the usual way by shaping, and then drawing the resulting glyphs...

Thanks,

Manuel

[Bram Moolenaar]

Manuel Schiller wrote:

[...]

> Concerning your request to send a patch that just fixes the assumption
> about "one ascii character == one glyph": One could either get rid of
> that speed optimisation code altogether (in which case all ligatures
> will work), or rewrite most of that optimisation, doing one shaping
> call into pango for each of (or some of) the characters with codes <
> 128. The latter versoin would amount to some overhead for all those
> calls, and reassembling the returned data structures in a suitable
> way...

I don't see how it can work with a font where it is not true that each
ASCII character is one display cell. E.g., when highlighting or drawing
the cursor. If you draw some combination of two ASCII characters with
one double-wide glyph it's not possible to highlight half of it.

It appears pango has no way to disable ligatures.

So the table we create should work, putting in extra spaces to avoid the
shaping to take place. But I don't see why we would be doing this only
for alphanumeric characters, this should work for all ASCII characters.


--
The only way the average employee can speak to an executive is by taking a
second job as a golf caddie.

[manuelschi...@googlemail.com]

On Monday, 8 August 2016 22:27:11 UTC+2, Bram Moolenaar wrote:
> Manuel Schiller wrote:
>
> [...]
>
> > Concerning your request to send a patch that just fixes the assumption
> > about "one ascii character == one glyph": One could either get rid of
> > that speed optimisation code altogether (in which case all ligatures
> > will work), or rewrite most of that optimisation, doing one shaping
> > call into pango for each of (or some of) the characters with codes <
> > 128. The latter versoin would amount to some overhead for all those
> > calls, and reassembling the returned data structures in a suitable
> > way...
>
> I don't see how it can work with a font where it is not true that each
> ASCII character is one display cell. E.g., when highlighting or drawing
> the cursor. If you draw some combination of two ASCII characters with
> one double-wide glyph it's not possible to highlight half of it.
>
> It appears pango has no way to disable ligatures.
>
> So the table we create should work, putting in extra spaces to avoid the
> shaping to take place. But I don't see why we would be doing this only
> for alphanumeric characters, this should work for all ASCII characters.
>
>

Hi Bram,

well, it seems I upset quite a bee's nest with this (much of which may be my own fault ;)...

Neither patch nor the current vim code will work when one ASCII character translates to more than one display cell.

But when you have a ligature between two characters, drawing a two-cell glyph might be considered an improvement by some. As to what happens when you change highlighting, or move the cursor across such a ligature: Currently, the redrawn character cell gets repainted with the "non-ligaturised" version of that single glyph, which results in the ligature partially reverting back to its component glyphs. Moving the cursor off, and a (partial) redraw like from pressing ^L causes the ligature glyph to reappear. It's not perfect, but it's consistent with the behaviour I've seen from e.g. putty and Konsole/qterminal (the latter two do their own metrics calculation, and get the spacing wrong if there's ligatures on the line - patched gvim doesn't do that, so I'm very happy with it ;). My assertion is that people using these programming fonts will be happy to see these ligatures assemble/disassemble as they edit across them.

Regarding your question about the range of ascii characters to which the tabulated cache is applied: Restricting this to subranges of [0,127] that I mentioned should get you most of the speed improvement that can be had from bypassing pango's shaping, and from what I understood, the space ' ' is especially important, since it's used to clear the screen.

But the restriction to those subranges also means that strings containing the popular "programming ligatures" like "<=", ">=", "!=", "->" etc. will be subject to pango's shaping, thus ligatures are displayed... In a way, we're selectively disabling the "shaping bypass" for less obvious character ranges where ligatures are more likely (and more likely to be missed by programmers).

I hope this clarifies things a bit.

Thanks for the interest,

Manuel

[Kazunobu Kuriyama]

2016-08-09 6:35 GMT+09:00 manuelschiller.pimail via vim_dev <vim...@googlegroups.com>:

On Monday, 8 August 2016 22:27:11 UTC+2, Bram Moolenaar  wrote:
> Manuel Schiller wrote:
>
> [...]
>
> > Concerning your request to send a patch that just fixes the assumption
> > about "one ascii character == one glyph": One could either get rid of
> > that speed optimisation code altogether (in which case all ligatures
> > will work), or rewrite most of that optimisation, doing one shaping
> > call into pango for each of (or some of) the characters with codes <
> > 128. The latter versoin would amount to some overhead for all those
> > calls, and reassembling the returned data structures in a suitable
> > way...
>
> I don't see how it can work with a font where it is not true that each
> ASCII character is one display cell.  E.g., when highlighting or drawing
> the cursor.  If you draw some combination of two ASCII characters with
> one double-wide glyph it's not possible to highlight half of it.
>
> It appears pango has no way to disable ligatures.
>
> So the table we create should work, putting in extra spaces to avoid the
> shaping to take place.  But I don't see why we would be doing this only
> for alphanumeric characters, this should work for all ASCII characters.
>
>

Hi Bram,

well, it seems I upset quite a bee's nest with this (much of which may be my own fault ;)...

Neither patch nor the current vim code will work when one ASCII character translates to more than one display cell.

But when you have a ligature between two characters, drawing a two-cell glyph might be considered an improvement by some. As to what happens when you change highlighting, or move the cursor across such a ligature: Currently, the redrawn character cell gets repainted with the "non-ligaturised" version of that single glyph, which results in the ligature partially reverting back to its component glyphs. Moving the cursor off, and a (partial) redraw like from pressing ^L causes the ligature glyph to reappear. It's not perfect, but it's consistent with the behaviour I've seen from e.g. putty and Konsole/qterminal (the latter two do their own metrics calculation, and get the spacing wrong if there's ligatures on the line - patched gvim doesn't do that, so I'm very happy with it ;).

putty, konsole, qterminal, and gvim...There's a obvious difference between the first three and the last; the former are terminals and the latter is an editor.

If you think of the difference of their usages, you'll notice that even small hiccups on terminals could much matter on editors, in particular for users who must have the cursor go to and fro to view and edit files for hours.

That's something we should not underestimate, isn't that?  Probably, some of the users wouldn't be patient enough to put up with it and would soon file bug reports once your patch was merged into the upstream.

You mentioned that gvim has less issues than the terminals.  But that's is a side effect of redrawing by gvim for its own necessity, not for the sake of ligatures.

Your patch doesn't contain any code for that.  That has no redraw mechanism to ensure ligatures to appear on the screen as/when they should.

I'm glad to hear, "patched gvim doesn't do that, so I'm very happy with it:)."  But you should be aware that your happiness comes to you only by accident or fortune, not by design or code.

The reason I'm sticking to monospace fonts comes here.  If we were making a text viewer, your patch would be sufficient.  But actually what we are making is an editor.  We have to address other issues such as the one you mentioned above to make vim truly and decently support ligatures.  For that purpose, the precondition that all fonts to be used are monospace, could be helpful and make our work much simpler and easier.  After that, we could do something about "effectively" monospace fonts if necessary, couldn't we?

By the way, why didn't you honestly talk to the people here about the issue you've already noticed till now?  As you guessed, I didn't install the fonts in question on my PC nor tried your patch, because I easily predicted from the code that your patch had such an issue you mentioned above.  That's why I'm sticking to monospace; it's for getting us ready for addressing possible other issues your patch has.  Not a pretext for rejecting your patch on behalf of old-school people.

That said, it's OK for me to stop talking right now if you think I'm not qualified for a reviewer because of those.  That helps me enjoy the coming vacation without anything bothering me.

 

My assertion is that people using these programming fonts will be happy to see these ligatures assemble/disassemble as they edit across them.

Once people view ligatures on Vim, they will ask us more for the purpose of editing them in a way each individual thinks better...

Best,

Kazunobu

Regarding your question about the range of ascii characters to which the tabulated cache is applied: Restricting this to subranges of [0,127] that I mentioned should get you most of the speed improvement that can be had from bypassing pango's shaping, and from what I understood, the space ' ' is especially important, since it's used to clear the screen.

But the restriction to those subranges also means that strings containing the popular "programming ligatures" like "<=", ">=", "!=", "->" etc. will be subject to pango's shaping, thus ligatures are displayed... In a way, we're selectively disabling the "shaping bypass" for less obvious character ranges where ligatures are more likely (and more likely to be missed by programmers).

I hope this clarifies things a bit.

Thanks for the interest,

Manuel

--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@googlegroups.com.

[manuelschi...@googlemail.com]

Dear Kazunobu,

On Tuesday, 9 August 2016 07:28:43 UTC+2, Kazunobu Kuriyama wrote:
> 2016-08-09 6:35 GMT+09:00 manuelschiller.pimail via vim_dev <vim...@googlegroups.com>:
>
> On Monday, 8 August 2016 22:27:11 UTC+2, Bram Moolenaar  wrote:
>
> > Manuel Schiller wrote:
> >
> > [...]
> >
>

> Hi Bram,
>
> well, it seems I upset quite a bee's nest with this (much of which may be my own fault ;)...
>
> Neither patch nor the current vim code will work when one ASCII character translates to more than one display cell.
>
> But when you have a ligature between two characters, drawing a two-cell glyph might be considered an improvement by some. As to what happens when you change highlighting, or move the cursor across such a ligature: Currently, the redrawn character cell gets repainted with the "non-ligaturised" version of that single glyph, which results in the ligature partially reverting back to its component glyphs. Moving the cursor off, and a (partial) redraw like from pressing ^L causes the ligature glyph to reappear. It's not perfect, but it's consistent with the behaviour I've seen from e.g. putty and Konsole/qterminal (the latter two do their own metrics calculation, and get the spacing wrong if there's ligatures on the line - patched gvim doesn't do that, so I'm very happy with it ;).
>
>
> putty, konsole, qterminal, and gvim...There's a obvious difference between the first three and the last; the former are terminals and the latter is an editor.
>
> If you think of the difference of their usages, you'll notice that even small hiccups on terminals could much matter on editors, in particular for users who must have the cursor go to and fro to view and edit files for hours.

Let's not confuse terminals and gvim here.

On terminals (even those that "paint" ligatures, even if it is with some glitches due to the way in which those terminals treat them), (text mode) vim is very usable. I should know because I stare at vim in a terminal for a big part of my day, especially when things tunnel through ssh. My patch (whether it is applied or not) does not change vim's behaviour when running in a terminal.

My point is that gvim with the patch is no worse than those terminals with the "glitches" when the cursor moves across a ligature. gvim without the patch is quite a bit worse because you cannot read what's in the window with some fonts.

For gvim, I could envisage a couple of options that people may decide on:
* we could leave things as they are:
- true monospaced fonts will work fine
- my "effectively monospaced fonts" will display gibberish
- people may start filing bug reports which you may or may not choose to close
right away again with the hint that gvim only supports monospaced fonts
* if we had a patch that did ligatures in "effectively monospaced fonts" perfectly,
you might choose to apply it (but who knows when we'll have it?)
* we may use an imperfect patch along the lines that I suggested - it's not the
last word on the matter, but it'll improve the display with these fonts from
completely unreadable to something readable that is approaching what was
intended, even if I'm not quite there in solving all the issues yet with this patch...

> That's something we should not underestimate, isn't that?  Probably, some of the users wouldn't be patient enough to put up with it and would soon file bug reports once your patch was merged into the upstream.

Well, I could file such a bug report right now, and it wouldn't complain about little imperfections like what happens when the cursor moves across a ligature. The gvim window is unreadable without a patch along the lines suggested if you have such a font.

> You mentioned that gvim has less issues than the terminals.  But that's is a side effect of redrawing by gvim for its own necessity, not for the sake of ligatures.

I think that's because gvim with the patch is actually doing something a little more correct than these terminals do - it respects the character cell size for monospaced and effectively monospaced fonts, no matter what. So that's not due to vim redrawing when it needs it, but because gvim actually manages to calculate the right position in the window at which to draw the rendered glyphs.

> Your patch doesn't contain any code for that.  That has no redraw mechanism to ensure ligatures to appear on the screen as/when they should.

No, it does not. That kind of thing happens inside pango at the moment. If someone more knowledgable comes along and has a good idea which fixes this, I'd be more than happy.

> I'm glad to hear, "patched gvim doesn't do that, so I'm very happy with it:)."  But you should be aware that your happiness comes to you only by accident or fortune, not by design or code.

Point taken. OTOH, saying the current glyph caching code for ascii characters also fails horribly in some cases where as a non-technical user who just happens to like (g)vim, you'd not expect it to fail - monospace fonts or not. The question becomes a bit "which is the unhappier accident or bad fortune?"...

> The reason I'm sticking to monospace fonts comes here.  If we were making a text viewer, your patch would be sufficient.  But actually what we are making is an editor.  We have to address other issues such as the one you mentioned above to make vim truly and decently support ligatures.  For that purpose, the precondition that all fonts to be used are monospace, could be helpful and make our work much simpler and easier.  After that, we could do something about "effectively" monospace fonts if necessary, couldn't we?

Feel free to do as you see fit. My point is that this little patch doesn't change
anything in the big picture of things, so you're still free to plan whatever
architectural changes you may have in the pipeline, and I'm sure we'll all be happy
when they materialise, whoever comes up with them. At the same time, that ugly little
patch fixes a issue with some fonts not rendering to anything intelligible. In fact,
if I want I can do a "set guifont=Helvetica\ 12", and I get something I can read. I
know you don't officially support proportional fonts, and it sure doesn't look
pretty. That's what I get for trying. But, and this is a big but: It's readable.
Every single last character of it. For these fonts with ligatures in the ascii
range, the result is very different, and quite unreadable. You may like or not like my patch, but that is something it improves - maybe it was written and advertised for the wrong reasons, but it does improve on that.

>
> By the way, why didn't you honestly talk to the people here about the issue you've already noticed till now?  As you guessed, I didn't install the fonts in question on my PC nor tried your patch, because I easily predicted from the code that your patch had such an issue you mentioned above.  That's why I'm sticking to monospace; it's for getting us ready for addressing possible other issues your patch has.  Not a pretext for rejecting your patch on behalf of old-school people.
>

Well, I'm occasionally doing a some code review myself (in high energy physics). When somebody sends something, I generally give them the benefit of a doubt, and try their patch first. (Sometimes I also say "I don't have time right now, can we come back to this in a month or two?". That's equally acceptable.)

You also seem to expect a newcomer who's trying to figure out things in vim as he goes along to have the same level of proficiency as you do. I'm not sure that's entirely fair. But be that as it may. Since the patch was that transparent for you, you might have asked for clarification on these specific points right away, saving us both a lot of typing...

> That said, it's OK for me to stop talking right now if you think I'm not qualified for a reviewer because of those.  That helps me enjoy the coming vacation without anything bothering me.

Well, the trouble is that I think you likely are qualified to review that patch, despite me being a newcomer who cannot really tell because I don't know who does what, and who is experienced in which field. But somehow, we seem to have gotten off on the wrong foot (which may be my fault, and if it is, I'm sorry), or you seem to feel really strongly that the code should only deal with true monospaced fonts. That's a bit of a pity because I think we're both trying to make vim a little bit better (even if it's in a frightfully limited way in my case). By the way, your vacation remark I find a bit insulting in this context.

>  My assertion is that people using these programming fonts will be happy to see these ligatures assemble/disassemble as they edit across them.
>
> Once people view ligatures on Vim, they will ask us more for the purpose of editing them in a way each individual thinks better...
>

Yep, that's the sign of people doing a good job - people ask for more. :) In a sense, that's what I'm doing as well (I hope you see that as a compliment!).

Seriously, I don't mean to aggravate you or anybody else, or cloud your vacation in any way. Maybe we postpone this until you're back from vacation, and we've both had some time to think it over?

Kind regards, and enjoy your vacation,

Manuel

> Best,
> Kazunobu
>
>
>
>
> Regarding your question about the range of ascii characters to which the tabulated cache is applied: Restricting this to subranges of [0,127] that I mentioned should get you most of the speed improvement that can be had from bypassing pango's shaping, and from what I understood, the space ' ' is especially important, since it's used to clear the screen.
>
>
>
> But the restriction to those subranges also means that strings containing the popular "programming ligatures" like "<=", ">=", "!=", "->" etc. will be subject to pango's shaping, thus ligatures are displayed... In a way, we're selectively disabling the "shaping bypass" for less obvious character ranges where ligatures are more likely (and more likely to be missed by programmers).
>
>
>
> I hope this clarifies things a bit.
>
>
>
> Thanks for the interest,
>
>
>
> Manuel
>
>
>
>
>
> --
>
> --
>
> You received this message from the "vim_dev" maillist.
>
> Do not top-post! Type your reply below the text you are replying to.
>
> For more information, visit http://www.vim.org/maillist.php
>
>
>
> ---
>
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
>

> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.

[Kazunobu Kuriyama]

Hi Manual,

Apologies to you for that. 

>  My assertion is that people using these programming fonts will be happy to see these ligatures assemble/disassemble as they edit across them.
>
> Once people view ligatures on Vim, they will ask us more for the purpose of editing them in a way each individual thinks better...
>
Yep, that's the sign of people doing a good job - people ask for more. :) In a sense, that's what I'm doing as well (I hope you see that as a compliment!).

Seriously, I don't mean to aggravate you or anybody else, or cloud your vacation in any way. Maybe we postpone this until you're back from vacation, and we've both had some time to think it over? 

That sounds good...but considering the fact that the ligature patch now looks gaining momentum for the first time in two years (as Bram shows an interest in it), I think your keeping working on it while I'm off could lead to a good, even better and faster, result.  So I'd like to say to you, "Keep it up!  Over my dead body!", though it's all up to you :)

Kind regards, and enjoy your vacation,

Thanks,

Kazunobu 

Manuel

> Best,
> Kazunobu
>
>
>
>
> Regarding your question about the range of ascii characters to which the tabulated cache is applied: Restricting this to subranges of [0,127] that I mentioned should get you most of the speed improvement that can be had from bypassing pango's shaping, and from what I understood, the space ' ' is especially important, since it's used to clear the screen.
>
>
>
> But the restriction to those subranges also means that strings containing the popular "programming ligatures" like "<=", ">=", "!=", "->" etc. will be subject to pango's shaping, thus ligatures are displayed... In a way, we're selectively disabling the "shaping bypass" for less obvious character ranges where ligatures are more likely (and more likely to be missed by programmers).
>
>
>
> I hope this clarifies things a bit.
>
>
>
> Thanks for the interest,
>
>
>
> Manuel
>
>
>
>
>
> --
>
> --
>
> You received this message from the "vim_dev" maillist.
>
> Do not top-post! Type your reply below the text you are replying to.
>
> For more information, visit http://www.vim.org/maillist.php
>
>
>
> ---
>
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.

>
> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@googlegroups.com.

[manuelschi...@googlemail.com]

Dear Kazunobu,

On Tuesday, 9 August 2016 12:03:47 UTC+2, Kazunobu Kuriyama wrote:
> Hi Manual,
>
> [...]

> Well, the trouble is that I think you likely are qualified to review that patch, despite me being a newcomer who cannot really tell because I don't know who does what, and who is experienced in which field. But somehow, we seem to have gotten off on the wrong foot (which may be my fault, and if it is, I'm sorry), or you seem to feel really strongly that the code should only deal with true monospaced fonts. That's a bit of a pity because I think we're both trying to make vim a little bit better (even if it's in a frightfully limited way in my case). By the way, your vacation remark I find a bit insulting in this context.
>
>
> Apologies to you for that. 
>

No need, I rather speak up when something is getting beyond my confort zone, but before I actually feel seriously insulted. And I guess I'm at the very least sharing the blame for the escalation... :) Sorry for that again, and for the noise!

>
> >  My assertion is that people using these programming fonts will be happy to see these ligatures assemble/disassemble as they edit across them.
>
> >
>
> > Once people view ligatures on Vim, they will ask us more for the purpose of editing them in a way each individual thinks better...
>
> >
>
> Yep, that's the sign of people doing a good job - people ask for more. :) In a sense, that's what I'm doing as well (I hope you see that as a compliment!).
>
>
>
> Seriously, I don't mean to aggravate you or anybody else, or cloud your vacation in any way. Maybe we postpone this until you're back from vacation, and we've both had some time to think it over? 
>
>
> That sounds good...but considering the fact that the ligature patch now looks gaining momentum for the first time in two years (as Bram shows an interest in it), I think your keeping working on it while I'm off could lead to a good, even better and faster, result.  So I'd like to say to you, "Keep it up!  Over my dead body!", though it's all up to you :)
>

Okay, fair enough. :) It might take some time though, since I'll be moving to a different country to start a new job in the coming month.

What direction would you like to see things heading? I'll most likely be no good at finding the ultimate patch that gets everything right because I'm not enough of an expert in the font rendering library field - it only dawned on me what was going wrong in that code recently.

If you'd want the ultimate ligature-clean solution, I anticipate one would need to revamp much of the gtk display code to update the right bits of the screen and keep the relation between glyph and characters, especially for the cursor drawing routines, so the right bit gets redrawn. That's a tall order, and I feel my C and gtk/pango skills are a bit too much in the range from non-existant to rusty for that. (My C++ is good, but that doesn't help here. ;) Moreovwe, we may want to use a patch along the lines I suggested to gain a bit of experience to learn about what's important in this business before we try and design the patch that solves all ligature issues... It may involve bug reports, and users asking for features, but that's very good feedback once we understand where the problem is, guiding us toward a better solution.

In the meantime, I may be able to clean up the glyph cache for ascii codes < 128, for example, by having a shaping call per character code, so we only need to store a single glyph in the cache, if you feel that would help. One could also think about making the ascii code range which passes through the glyph cache configurable, if you guys would prefer that over the static list like the one I took over from the original patch proposition. Is there anything else I can do that you or others would feel is valuable to have? Feel free to keep the suggestions coming...

All the best,

Manuel

[Kazunobu Kuriyama]

Hi Manuel

2016-08-09 20:06 GMT+09:00 manuelschiller.pimail via vim_dev <vim...@googlegroups.com>:

Dear Kazunobu,

If you don't mind, please consider using friendlier 'Hi' instead of 'Dear.'   It's OK to me even if you omit greeting itself.

On Tuesday, 9 August 2016 12:03:47 UTC+2, Kazunobu Kuriyama  wrote:
> Hi Manual,

I'm sorry about my writing your name wrongly. 

>
> [...]
> Well, the trouble is that I think you likely are qualified to review that patch, despite me being a newcomer who cannot really tell because I don't know who does what, and who is experienced in which field. But somehow, we seem to have gotten off on the wrong foot (which may be my fault, and if it is, I'm sorry), or you seem to feel really strongly that the code should only deal with true monospaced fonts. That's a bit of a pity because I think we're both trying to make vim a little bit better (even if it's in a frightfully limited way in my case). By the way, your vacation remark I find a bit insulting in this context.
>
>
> Apologies to you for that. 
>
No need, I rather speak up when something is getting beyond my confort zone, but before I actually feel seriously insulted. And I guess I'm at the very least sharing the blame for the escalation... :) Sorry for that again, and for the noise!
>
> >  My assertion is that people using these programming fonts will be happy to see these ligatures assemble/disassemble as they edit across them.
>
> >
>
> > Once people view ligatures on Vim, they will ask us more for the purpose of editing them in a way each individual thinks better...
>
> >
>
> Yep, that's the sign of people doing a good job - people ask for more. :) In a sense, that's what I'm doing as well (I hope you see that as a compliment!).
>
>
>
> Seriously, I don't mean to aggravate you or anybody else, or cloud your vacation in any way. Maybe we postpone this until you're back from vacation, and we've both had some time to think it over? 
>
>
> That sounds good...but considering the fact that the ligature patch now looks gaining momentum for the first time in two years (as Bram shows an interest in it), I think your keeping working on it while I'm off could lead to a good, even better and faster, result.  So I'd like to say to you, "Keep it up!  Over my dead body!", though it's all up to you :)
>

Okay, fair enough. :) It might take some time though, since I'll be moving to a different country to start a new job in the coming month.

I see.  You are busy for a while.

Actually, that gives me its own convenience.  It gives me some time to make myself get accustomed to ligature fonts and their usage on Vim.  Nothing wrong.

What direction would you like to see things heading? I'll most likely be no good at finding the ultimate patch that gets everything right because I'm not enough of an expert in the font rendering library field - it only dawned on me what was going wrong in that code recently.

If you'd want the ultimate ligature-clean solution, I anticipate one would need to revamp much of the gtk display code to update the right bits of the screen and keep the relation between glyph and characters, especially for the cursor drawing routines, so the right bit gets redrawn. That's a tall order, and I feel my C and gtk/pango skills are a bit too much in the range from non-existant to rusty for that. (My C++ is good, but that doesn't help here. ;) Moreovwe, we may want to use a patch along the lines I suggested to gain a bit of experience to learn about what's important in this business before we try and design the patch that solves all ligature issues... It may involve bug reports, and users asking for features, but that's very good feedback once we understand where the problem is, guiding us toward a better solution.

If you don't have any clear direction on that, how about rethinking all the issues that have been raised in this thread so far.  You might find some of them needed to be fixed or might get a better idea to address them.

Doing that would tell us (hints of) what we need to know and what we have to do.

In the meantime, I may be able to clean up the glyph cache for ascii codes < 128, for example, by having a shaping call per character code, so we only need to store a single glyph in the cache, if you feel that would help. One could also think about making the ascii code range which passes through the glyph cache configurable, if you guys would prefer that over the static list like the one I took over from the original patch proposition.

I think it's worth a try.  Even if an effort fails, that will help us understand why the current way is not bad.

Is there anything else I can do that you or others would feel is valuable to have? Feel free to keep the suggestions coming...

I'd also like to hear the voices of people who want to see ligatures on Vim.

Best regards,

Kazunobu

All the best,

Manuel

To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@googlegroups.com.

[manuelschi...@googlemail.com]

Hi Kazunobu,

On Tuesday, 9 August 2016 15:46:38 UTC+2, Kazunobu Kuriyama wrote:
> Hi Manuel
>
> 2016-08-09 20:06 GMT+09:00 manuelschiller.pimail via vim_dev <vim...@googlegroups.com>:
>
>
> Dear Kazunobu,
>
> If you don't mind, please consider using friendlier 'Hi' instead of 'Dear.'   It's OK to me even if you omit greeting itself.
>

Done. :) Now I just have to remember...

>
> On Tuesday, 9 August 2016 12:03:47 UTC+2, Kazunobu Kuriyama  wrote:
>
> > Hi Manual,
>
> I'm sorry about my writing your name wrongly. 
>

I get that about once a week to every two weeks in my inbox, and actually consider it a bit of a compliment when people compare me to the fine manual which each and every one of the of course has read most dilligently, intentionally or otherwise... ;) So: No worries, as long as I get that you're trying to send me a message, it's all good...

>
> > [...]

>
> > Okay, fair enough. :) It might take some time though, since I'll be moving to a different country to start a new job in the coming month.
>
> I see.  You are busy for a while.
>
> > Actually, that gives me its own convenience.  It gives me some time to make myself get accustomed to ligature fonts and their usage on Vim.  Nothing wrong.
>

Have fun. Just be careful, you might not want to go back to something truely monospaced... ;)

>
> > What direction would you like to see things heading? I'll most likely be no good at finding the ultimate patch that gets everything right because I'm not enough of an expert in the font rendering library field - it only dawned on me what was going wrong in that code recently.
> >
> > If you'd want the ultimate ligature-clean solution, I anticipate one would need to revamp much of the gtk display code to update the right bits of the screen and keep the relation between glyph and characters, especially for the cursor drawing routines, so the right bit gets redrawn. That's a tall order, and I feel my C and gtk/pango skills are a bit too much in the range from non-existant to rusty for that. (My C++ is good, but that doesn't help here. ;) Moreovwe, we may want to use a patch along the lines I suggested to gain a bit of experience to learn about what's important in this business before we try and design the patch that solves all ligature issues... It may involve bug reports, and users asking for features, but that's very good feedback once we understand where the problem is, guiding us toward a better solution.
>
> If you don't have any clear direction on that, how about rethinking all the issues that have been raised in this thread so far.  You might find some of them needed to be fixed or might get a better idea to address them.
>
> Doing that would tell us (hints of) what we need to know and what we have to do.
>

Well, I'll see what I can do. For non-latin and non-greek alphabets, there's not much I can do because I know too little about those writing systems to know how things should look, though... I will need help (and potentially an expert) on that front.

>
> > In the meantime, I may be able to clean up the glyph cache for ascii codes < 128, for example, by having a shaping call per character code, so we only need to store a single glyph in the cache, if you feel that would help. One could also think about making the ascii code range which passes through the glyph cache configurable, if you guys would prefer that over the static list like the one I took over from the original patch proposition.
>
> I think it's worth a try.  Even if an effort fails, that will help us understand why the current way is not bad.
>

Okay.

> > Is there anything else I can do that you or others would feel is valuable to have? Feel free to keep the suggestions coming...
>
> I'd also like to hear the voices of people who want to see ligatures on Vim.
>

I agree. I'm just not sure how many we'll attract from off-list (or how many people on-list are interested) without having a demonstrator patch out there for which people can hand in suggestions. But we can cross that bridge when we have to.

Cool, that sounds like a plan.

Cheers,

Manuel

[Bram Moolenaar]

I think you may see this as an improvement to having the font not work
at all, but other people who think the font would just work might very
well report a bug about this behavior. For example, long time ago we
simulated bold by drawing the glyph with a one pixel offset. That
sometimes left some pixel dust behind. The bug reports didn't stop
until all of that was fixed.

> Regarding your question about the range of ascii characters to which
> the tabulated cache is applied: Restricting this to subranges of
> [0,127] that I mentioned should get you most of the speed improvement
> that can be had from bypassing pango's shaping, and from what I
> understood, the space ' ' is especially important, since it's used to
> clear the screen.
>
> But the restriction to those subranges also means that strings
> containing the popular "programming ligatures" like "<=", ">=", "!=",
> "->" etc. will be subject to pango's shaping, thus ligatures are
> displayed... In a way, we're selectively disabling the "shaping
> bypass" for less obvious character ranges where ligatures are more
> likely (and more likely to be missed by programmers).

What makes you think these are popular? Maybe it's because I have read
">=" as two characters all my programming life, but having it show up as
anything but the sequence of two characters would look weird to me.
Perhaps it would look nice in a book or other place where you only read
code.

If we don't do this, then is there still any point in trying to support
this font that isn't really mono spaced?

I'm afraid any half solution will just lead to more bugs about details
later. It might be better to just say that this font is not supported.

--
Engineers understand that their appearance only bothers other people and
therefore it is not worth optimizing.

[manuelschi...@googlemail.com]

Hi Bram,

Point taken.

> > Regarding your question about the range of ascii characters to which
> > the tabulated cache is applied: Restricting this to subranges of
> > [0,127] that I mentioned should get you most of the speed improvement
> > that can be had from bypassing pango's shaping, and from what I
> > understood, the space ' ' is especially important, since it's used to
> > clear the screen.
> >
> > But the restriction to those subranges also means that strings
> > containing the popular "programming ligatures" like "<=", ">=", "!=",
> > "->" etc. will be subject to pango's shaping, thus ligatures are
> > displayed... In a way, we're selectively disabling the "shaping
> > bypass" for less obvious character ranges where ligatures are more
> > likely (and more likely to be missed by programmers).
>
> What makes you think these are popular? Maybe it's because I have read
> ">=" as two characters all my programming life, but having it show up as
> anything but the sequence of two characters would look weird to me.
> Perhaps it would look nice in a book or other place where you only read
> code.

Well, preferences matter a lot to people, and I respect that (read on,
because I think there's a way how both of us can have our cake, and eat
it, too...).

In fact, I had a useful discussion by mail with Kazunobu earlier today, and
also did some thinking on the tram home tonight, and here's what I figured:

- I think I should have another go at the ascii (code < 128) character to
glyph cache in such a way that it forces one glyph per character, regardless
of what the font offers in terms of ligatures. That way, the corruption for
normal text for these effectively monospaced fonts with ligatures in that
range will stop (because we'd effectively disable ligatures in that range),
and we'll introduce none of the strange drawing issues with the cursor/redraw.

- Instead of having a hardcoded range of characters for which we bypass glyph
shaping (be it the entire range from 0 to 127, or just the subrange that
the patch proposed), we could have a bit map from characters 0-127 in 4
32 bit integers, where each bit specifies if the character is eligible
to bypass glyph shaping, or not. (We can discuss other data structures,
of course.)

If we make this user settable in some form, and set the default such that the
old behaviour (i.e. no ligatures, no redrawing problems) remains unless people
do something in their .vimrc, we could have correct, albeit non-fancy behaviour
by default, and have effectively monospace fonts work (without ligatures).

If users change the character range for which they bypass glyph shaping
because they want ligatures, they'll get some, or all of the ligatures, with
all the interesting behaviour that happens on redraw and cursor movement.
Thus, users would get a choice.

That way, we'd kind of have the best of both worlds with one or two relatively
simple patches that I would hope to be viable for most people, and that do the
right thing with no surprises or drawing glitches, unless people change the
default settings.

Once we have a bit of experience with that, maybe we can come up with a better
patch that supports ligatures in an even better way.

Would that be acceptable for you as an interim hack/solution, Bram?

> If we don't do this, then is there still any point in trying to support
> this font that isn't really mono spaced?

Hmm, given that a proportional font like Helvetica works (even if the result
isn't pretty), producing gibberish for these effectively monospaced fonts
with ligatures seems to violate the POLA (Principle Of Least Astonishment).
My prediction would be that not fixing it will just generate you a steady
stream of discussions through the coming years, and you'll have to explain
point to a statement each time that goes "no, we really mean monospaced
and monospaced only when it comes to supported fonts, with the proper flag
set and everything".

> I'm afraid any half solution will just lead to more bugs about details
> later. It might be better to just say that this font is not supported.

Well, that could still remain the official slogan, or we could dub it
"experimental" or something if you think it helps. But the question will
come back every now and then, I think...

Kind regards,

Manuel

[Tony Mechelynck]

Manuel:

In the past there have been "unofficial" features published as patches
which remained outside of the "official" Vim repositories but publicly
available, sometimes for years, before Bram finally decided to take
them in. The +conceal and +float features, now part of mainstream Vim,
are two examples of such which I've seen remain "unofficial
third-party patches" while several successive minor versions of Vim
came and went.

Maybe you could publish your patch (as a patch that could be applied
by running "patch -p1 ligatures.diff", or something like that, at the
top level of a Vim repository clone), upload it somewhere on github or
vim.org or wherever, and let anyone use it who wants. Then after
letting it bake there for some time, we'll know better how popular it
is. Assuming that the new version currently being made ready will be
called Vim 8.0 and be released before the end of 2016, we might then
have a poll about your patch when getting ready for 8.1 or 8.2, and
let's hope that that will arrive long before 8.0 is at patchlevel
8.0.2200 — Vim 7.4, whose original release was almost exactly three
years ago, seems to have been quite successful in its own way.

If you choose to go this way, please set it up so that it could be
disabled at compile-time (I mean, place the changes behind #ifdef
FEAT_LIGATURES or something equally distinctive), it will help it
being accepted into the main code, since anyone not wanting it would
be able to disable it at compile-time — and similarly, an option (to
enable or disable it at runtime if present at compile-time, let's say
in the vimrc or gvimrc before starting the GUI) would IMHO be equally
welcome.


Best regards,
Tony.

[Stefan Schwarzer]

I recently purchased a license for PragmataPro and ran into
the same problem as Manuel. During my research I came across
his patch (many thanks, Manuel!) and I subscribed to the list
to follow the discussion on the patch and possibly
contribute to the exchange.

Here are my thoughts on the current discussion. (I had
considered to add these "inline," but changed my mind
because I think this would make the text _less_ accessible.)

If you want to use the ligatures in PragmataPro in GVim,
your only choice is a GVim with the patch from Manuel.
Without the patch, there's no readable text.

As I understand it, both the original code and the patch
make an assumption about the possible characters in a
ligature, and these assumptions can still be wrong.

The original code with its optimization was checked in about
twelve years ago. I wonder if it nowadays would be feasible
to remove the optimization altogether, i. e. replace it with
straightforward code that doesn't make a (still potentially
wrong) assumption about "valid" or "reasonable" character
combinations for ligatures.

I'm aware that this is a trade-off. Possibly some users will
be noticeably affected by slower rendering, but on the other
hand you get simpler and more robust code. I haven't seen a
suggestion of removing the optimization yet, so I'm not sure
if it's an implicit consensus that the optimization is still
needed or if this idea didn't come to mind at all. Don't get
me wrong: I'm not even suggesting removing the optimization,
but I'm suggesting _considering_ it, even if the result of
this consideration is to keep the optimization. :-)

Like Manuel, I noticed artifacts when moving the cursor over
ligatures. Note that the cursor movement can also happen if
the cursor isn't moved by the user, that is, if `showmatch`
is set. To reproduce this:

- compile GVim with the patch from Manuel
- start this GVim without a file
- :set guifont=PragmataPro\ 10
- :set matchpairs+=<:>
- :set showmatch
- insert <= >=
- press <esc> to leave insert mode
- press ctrl-L to make the ligatures render if that didn't
happen yet
- move the cursor onto the > which was just inserted
- observation: the cursor jumps to the left onto the < in
<= and the ligature is garbled (a < followed by the
right half of the <= ligature)
- expectation: the ligature isn't changed or is only changed
as long as the cursor is on it

Regardless of the rendering issues, so far they've always
been fixed easily by pressing ctrl-L. Even with the current
imperfections, I'd rather have the patch soon in a regular
GVim version than not being able to use the ligatures at all
for a potentially long time.

In the long run, I'm absolutely fine if the ligatures are
temporarily "decomposed" if the cursor moves over them, but
it would be nice if each ligature was rendered correctly if
the cursor doesn't cover it anymore.

Regarding the thought that merging the patch will make users
ask for more: Yes, that's probably true. On the other hand,
not including the patch will also have users ask for more,
namely support of ligatures in programmer fonts. :-)

Thank you all for working on Vim. I've been using it since
about 20 years and I'm very glad it exists!

Best regards,
Stefan

[manuelschi...@googlemail.com]

Hi Tony,

you (and others) are making very good points here, and I appreciate the
feedback.

Following your suggestion, I have created a vim fork with a branch for
this kind of development:

https://github.com/manuelschiller/vim/tree/glyphs

Currently, it contains two patches:

- gui_gtk_x11: force shaping one character at a time for ASCII glyph cache

This one does what it says. It'll get fonts like PragmataPro or Hasklig
working in gvim without ligatures, and without the drawing caveats we
discussed earlier. I imagine that this patch might make inclusion in vim
quite a bit earlier (I'd hope soonish, but that may be wishful thinking)
than the next item, because I do not think it does anything controversial.
If you'd like to see style improvements etc., please let me know, I'm
happy to accomodate you. :)

- gui_gtk_x11: enable poor man's ligatures

This one is the bit that enables ligatures, and will require a couple of
iterations on my side before it's ready to be considered for inclusion.
(For example, I'd like to make the set of characters that disable the
ASCII glyph cache user-configurable, and I have to find out how C code
gets access to variables inside vimscript...) For the curious, this is
something they might want to try out, and give feedback...

I would again like to thank you all for the friendly and constructive
atmosphere. And let me know if you have suggestions, please!

Manuel

[manuelschi...@googlemail.com]

Hi Stefan,

thanks for giving it a try, I appreciate it! Keep that feedback
coming (I'll keep reading mail), and we can hopefully make
ligatures in vim happen at some point. I'll keep updating patches
at the github repo I mentioned in my last mail.

Cheers,

Manuel

[Tony Mechelynck]

Hm. gui_gtk_x11.c is of course only for gvim with GTK GUI running on
Linux-X11. GTK2, and now even GTK3 ("new in 8.0"), are of course the
preferred GUIs for Vim on Linux these days, and for X11 in general
(though a few older ones are still supported IIUC); however, what
about other flavors of gvim? Such as gvim for Windows, and, maybe
worse, MacVim (gvim for Mac-Cocoa IIUC), of which I think, but am not
sure, that all its sources are included in the current "official" Vim
sources. Do Windows and/or Mac have similar fonts? Won't they feel
left out? The mechanisms to implement the corresponding Poor Man's
Ligatures will of necessity be different because we're at too low a
level for cross-platform programming to be possible all the way. Maybe
you don't have the necessary OSes to build and test the corresponding
gvim versions (neither do I) so it might perhaps be useful if some
Windows Vim developer(s) and some Mac Vim developer(s) joined you on
this project.

I'm cross-posting on the vim_mac group because Mac people might (or
might not) be interested; but this thread was started on vim_dev. Mac
developers: please refer to vim_dev for the discussion's history.


Best regards,
Tony.

[Kazunobu Kuriyama]

Hi Tony,

As for Mac-Cocoa, I haven't succeeded in building it with recent OSXs last 6 years. That build is automatically disabled at configure.

As for MacVim, they've already merged a patch which enables ligatures into their repo. It was almost a year ago ( Aug 24, 2015).

So...it's us who was left out.

In my understanding, they made ligatures possible by tweaking an attribute of a class of character strings called NSAttributedString [2] (Actually, they are doing that through the Core API, not Cocoa).

FYI, the latest nightly build of iTerm2, which is one of the popular terminals on Mac and seen a lot of Vimmer on console using the editor on it, starts supporting ligatures by a similar way.

Rendering ligatures is done by using some information from font files.  Therefore, as can been seen in two instances above, people usually rely on an rendering engine in use for that chore.

In other words, since creating a new shaping module for the purpose of cross-platform is very tough, there's no feasible solution other than relying on GUI's rendering engines.

As for Windows stuff...anyone?

Best regards,

Kazunobu

[1] https://github.com/macvim-dev/macvim/pull/56

[2] https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Foundation/Classes/NSAttributedString_Class/index.html#//apple_ref/doc/c_ref/NSLigatureAttributeName

Best regards,
Tony.

--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@googlegroups.com.

[Tony Mechelynck]

My mistake. The various Mac GUI systems leave me stumped, I even had
to do research before i knew whether Cocoa came before or after
Carbon. That tells you.

>
> As for MacVim, they've already merged a patch which enables ligatures into
> their repo. It was almost a year ago ( Aug 24, 2015).

Ah, OK.

>
> So...it's us who was left out.
>
> In my understanding, they made ligatures possible by tweaking an attribute
> of a class of character strings called NSAttributedString [2] (Actually,
> they are doing that through the Core API, not Cocoa).
>
> FYI, the latest nightly build of iTerm2, which is one of the popular
> terminals on Mac and seen a lot of Vimmer on console using the editor on it,
> starts supporting ligatures by a similar way.
>
> Rendering ligatures is done by using some information from font files.
> Therefore, as can been seen in two instances above, people usually rely on
> an rendering engine in use for that chore.
>
> In other words, since creating a new shaping module for the purpose of
> cross-platform is very tough, there's no feasible solution other than
> relying on GUI's rendering engines.
>
> As for Windows stuff...anyone?

Not me… but I sure wouldn't want users of Windows 15 (or something) to
someday come hollering after Bram, saying "Why were we left out?" ;-)

[Ken Takata]

Hi,

I just tried Hasklig on Windows. It seems that ligatures partly works on
Windows.

After Vim 7.4.393, gvim.exe can use two types of rendering engine, the
traditional GDI engine and newer DirectWrite engine. GDI doesn't seem to
support ligatures at all, but DirectWrite supports ligature. However, when
you put the cursor on a ligature, it is shown as separate characters. If you
want to show the ligature again, you need to type <Ctrl-L>.

Screenshot attached. (I haven't try PragmataPro.)

Regards,
Ken Takata

[Kazunobu Kuriyama]

Hi Ken,

Thank you for the headsup!  Great.  Bram must feel relief now :)

I'm also playing around Hasklig on MacVim today.  It appears the ligature support is not completed yet.

But this is my Day 1 of ligature experience, my setting may be wrong.  I'd like to hear the voices from MacVim guys about that.

After Vim 7.4.393, gvim.exe can use two types of rendering engine, the
traditional GDI engine and newer DirectWrite engine.  GDI doesn't seem to
support ligatures at all, but DirectWrite supports ligature.  However, when
you put the cursor on a ligature, it is shown as separate characters.  If you
want to show the ligature again, you need to type <Ctrl-L>.

Hmm, it looks leaving everything to rendering engines is not a good solution to rendering programming ligatures...  Perhaps, Vim itself needs to know byte sequences which constitute those ligatures and instruct the rendering engine about them explicitly...

Screenshot attached.  (I haven't try PragmataPro.)

I don't either for an obvious reason..Isn't there a free version? :)

Regards,
Ken Takata

Best regards,

Kazunobu 

[Stefan Schwarzer]

On 2016-08-11 17:40, Kazunobu Kuriyama wrote:
> 2016-08-11 23:44 GMT+09:00 Ken Takata <ktakat...@gmail.com>:
>> Screenshot attached. (I haven't try PragmataPro.)
>
> I don't either for an obvious reason..Isn't there a free version? :)

As far as I know there's no free version. The cheapest is the
"PragmataPro Essential" version for 19 Euro plus tax [1]. :-/

Maybe it would make sense to ask [2] the author for a reduced
subset to try things out. If I remember correctly, someone in
the forum section (at [1]) suggested having such a "test version"
and the font author seemed open to the idea. If you want, I can
contact the author and ask.

[1] http://www.fsd.it/shop/fonts/pragmatapro/
(There's no dedicated page for the Essential version, but it's
selectable from the "Weights" list.)

[2] http://www.fsd.it/contact-me/

Best regards,
Stefan

[manuelschi...@googlemail.com]

Hi,

okay, good to know that Mac and Windows already have at least partial support for ligatures.

In the meantime, I've amended the patch that enables "poor man's ligatures" on GTK2/3. Now, the user can select for which ASCII characters < 128 the glyph cache should be bypassed (thus enabling ligatures). This is done by checking a global vim variable, g:gtk_nocache. If unset, not a list, empty, the wrong length, or set to [0, 0, 0, 0], the default behaviour is retained. For those who want to see ligatures in most cases, I recommend to set it to [0x00000000, 0xfc00ffff, 0xf8000001, 0x78000001], which will bypass the glyph cache for everything but control characters, space, [0-9A-Za-z] and ASCII code 127 (it's a simple bitmap with bits set if the cache is to be bypassed). You can find the patch on the glyphs branch in my github repository (https://github.com/manuelschiller/vim/commit/d45b7745bd8118748b234e10d8fb9bb80053fa52). It does not solve any of the redrawing/cursor movement related issues that were discussed earlier, but it will at least give users a choice if they want their favourite font glitch-free without ligatures, or with ligatures (and still the occasional ^L).

Feel free to give it a try, or let me know what's bad and good about it. In any case, I'm looking forward to feedback.

Cheers,

Manuel

[manuelschi...@googlemail.com]

Hi,

here's another observation which I hadn't made before consciously: With the gtk3 version of the GUI, the artifacts with ligatures are virtually gone: While typing, ligatures like "<=" compose themselves while typing, and the artifact of "decomposing" the ligature when moving the cursor that I described earlier (and with the gtk2 backend) is virtually gone. In the latter case, you can see a very weak "echo" of the "unligatured" character under the cursor that was used to compose the ligature, but that goes away without a trace when you move the cursor away, and over normal text. It's also absent when the cursor has "blinked off". For example, when you move the cursor over the "<=" combination with PragmataPro, you'll see the "<=" ligature very clearly. But when the cursor blinks to on, it'll also show a single "<" or "=" overlaid with the original screen contents which disappears without the trace when the cursor blinks off again, or is moved away.

Bottom line: it seems that with the gtk3 backend for the GUI, you'll have an almost glitch-free experience with programming fonts with ligatures like Hasklig or PragmataPro. You'll still need the patches I pointed to in my previous e-mail, though.

Cheers,

Manuel

[manuelschi...@googlemail.com]

Hi,

since Bram kindly included a fix that will make the ASCII glyph cache usable with fonts with ligatures, I revised my patch that enables said ligatures to the extent possible with current code to work with Bram's patches. For those that do not want to wait until vim officially supports ligatures (if it ever will - but I remain optimist ;), the patch will be maintained here:

https://github.com/manuelschiller/vim/commits/glyphs

The commit message (https://github.com/manuelschiller/vim/commit/7a25336eed7b2082c016db36f80b2bb9345a4e3f) contains hints on how to fine-tune for which ASCII characters the glyph cache will be bypassed, so you have at least some vimrc-level control over which ligatures you want to see.
I'd recommend the GTK3 GUI backend over the GTK2 one for this patch, since the number of times you want to press ^L is vastly reduced.

I'll try to keep pace with vim development in the sense that I'll update the patch as needed, and as time permits (feel free to send a reminder if things do no longer apply cleanly!).

Comments, suggestions, and patches to improve things continue to be most welcome.

Cheers,

Manuel

[Matěj Cepl]

On 2016-08-07, 11:27 GMT, manuelschiller.pimail via vim_dev wrote:
> Please let me know if it would be possible to include this,
> and if not, at least the patch is public now where people can
> find it if they want it.

https://github.com/vim/vim/issues/418#issuecomment-246748135
and the (completely unofficial) build for RHEL/CentOS-7 is on
http://koji.fedoraproject.org/koji/taskinfo?taskID=15617207

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

Quod fuimus, estis; quod sumus, vos eritis.

[Matěj Cepl]

On 2016-08-11, 15:40 GMT, Kazunobu Kuriyama wrote:
>> Screenshot attached. (I haven't try PragmataPro.)
>
> I don't either for an obvious reason..Isn't there a free version? :)

Would https://github.com/tonsky/FiraCode/ work?

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

..every Man has a Property in his own Person. This no Body has
any Right to but himself. The Labour of his Body, and the Work of
his Hands, we may say, are properly his. .... The great and chief
end therefore, of Mens uniting into Commonwealths, and putting
themselves under Government, is the Preservation of their
Property.
-- John Locke, "A Treatise Concerning Civil Government"

[Matěj Cepl]

On 2016-08-08, 12:50 GMT, Tony Mechelynck wrote:
> You should see, in that order, the glyphs for ff fi fl ft st
> ffi ffl.
> I absolutely don't like them (in the monospaced fonts that
> have them; in serif or sans-serif fonts it's different,
> especially in serif italic). OTOH, I think that expanding them
> to two or three character cells would be ugly the opposite
> way.

It could be the problem with your font. I use
https://github.com/tonsky/FiraCode/ and it seems to work for me
just fine.

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

May you come to the attention of those in authority.
May you find what you are looking for.
May you live in interesting times.
-- claimed to be a Chinese proverb, but its origin is disputed
first documented use
Frederic R. Coudert, Proceedings of the Academy of Political Science, 1939

[manuelschi...@googlemail.com]

Hi,

glad to see that people give it a try. You may want to use the updated version of the patch here:

https://github.com/manuelschiller/vim/commit/0c8d6a735f1f6452b11c0da039f8c9c9f125475a

It allows users to hand-tune for which characters they want to bypass the glyph cache by putting a line in .vimrc, e.g.

let g:gtk_nocache=[0x00000000, 0xfc00ffff, 0xf8000001, 0x78000001]

This contains a bitmap for each character < 128, which has the corresponding bit set if the glyph cache is to be bypassed (which in turn enables ligatures). The default is no ligatures, unless people set something like above in their .vimrc. With this slightly more sophisticated patch, people have the option to enable/disable certain ligatures by routing the corresponding characters through the cache (or not) without recompiling, if they so prefer.

Comments remain welcome!

Cheers,

Manuel

[Kazunobu Kuriyama]

2016-09-14 6:09 GMT+09:00 Matěj Cepl <mc...@cepl.eu>:

On 2016-08-11, 15:40 GMT, Kazunobu Kuriyama wrote:
>> Screenshot attached.  (I haven't try PragmataPro.)
>
> I don't either for an obvious reason..Isn't there a free version? :)

Would https://github.com/tonsky/FiraCode/ work?

Matěj

Thank you for the note.  I've already installed it on my Mac.  I personally think it useful for rendering tests because of its wider coverage of ligature glyphs in comparison other free ligature fonts. 

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5  BC1D 7920 5802 880B C9D8

..every Man has a Property in his own Person. This no Body has
any Right to but himself. The Labour of his Body, and the Work of
his Hands, we may say, are properly his. .... The great and chief
end therefore, of Mens uniting into Commonwealths, and putting
themselves under Government, is the Preservation of their
Property.
    -- John Locke, "A Treatise Concerning Civil Government"

[Matěj Cepl]

On 2016-09-14, 13:28 GMT, manuelschiller.pimail via vim_dev wrote:
> It allows users to hand-tune for which characters they want to
> bypass the glyph cache by putting a line in .vimrc, e.g.
>
> let g:gtk_nocache=[0x00000000, 0xfc00ffff, 0xf8000001, 0x78000001]
>
> This contains a bitmap for each character < 128, which has the
> corresponding bit set if the glyph cache is to be bypassed
> (which in turn enables ligatures). The default is no
> ligatures, unless people set something like above in their
> .vimrc. With this slightly more sophisticated patch, people
> have the option to enable/disable certain ligatures by routing
> the corresponding characters through the cache (or not)
> without recompiling, if they so prefer.

So, for example how to make >= and <= be included so that they
translate into (glyphs from :digraphs):

=< ≤ 8804
>= ≥ 8805

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

The first mistake in publick business, is the going into it.
-- Benjamin Franklin

[Matěj Cepl]

On 2016-09-14, 13:28 GMT, manuelschiller.pimail via vim_dev wrote:

> It allows users to hand-tune for which characters they want to
> bypass the glyph cache by putting a line in .vimrc, e.g.
>
> let g:gtk_nocache=[0x00000000, 0xfc00ffff, 0xf8000001, 0x78000001]

Tell me, please, you have never ever considered this variable to
be anything more than a cludge during the development? Please,
tell me you never planned to inflict this on users (yes, I know,
vim users are not normal humans, but still)! Please.

Could you also add value of the variable which would be
equivalent of the previous patch, please? I really don’t know
how to make it work as well as the previous one?

Otherwise, vim with your new patch builds on RHEL-7 without
problem
http://koji.fedoraproject.org/koji/taskinfo?taskID=15633602

Thank you,

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

It is a rare mind indeed that can render the hitherto non-existent
blindingly obvious. The cry “I could have thought of that” is a very
popular and misleading one, for the fact is that they didn’t, and a very
significant and revealing fact it is too.
-- Douglas Adams, Dirk Gently's Holistic Detective Agency

[Christian Brabandt]

Hi,

On Mi, 14 Sep 2016, manuelschiller.pimail via vim_dev wrote:

>
> let g:gtk_nocache=[0x00000000, 0xfc00ffff, 0xf8000001, 0x78000001]
>
> This contains a bitmap for each character < 128, which has the
> corresponding bit set if the glyph cache is to be bypassed (which in
> turn enables ligatures). The default is no ligatures, unless people
> set something like above in their .vimrc. With this slightly more
> sophisticated patch, people have the option to enable/disable certain
> ligatures by routing the corresponding characters through the cache
> (or not) without recompiling, if they so prefer.
>
> Comments remain welcome!

I appreciate your effort you put into this. But for the hell of it, I
can't understand how the values inside g:gtk_nocache are supposed to
work. Can you please describe more in detail, what each value stands
for?

I am afraid this needs some documentation and possibly a different
format. We don't have something similar like this. Perhaps a new type
for the 'renderoptions'?

Best,
Christian
--
Ich habe etliche mal bemerkt, daß ich Kopf-Weh bekam, wenn ich mich
lange in einem Hohl-Spiegel betrachtete.
-- Georg Christoph Lichtenberg

[manuelschi...@googlemail.com]

Hi Matěj, Christian,

I've added the value which is equivalent to the previous patch, i.e.

" this should keep character 0-31 (control characters), and [0-9A-Za-z] flowing
" through the glyph cache, and the rest < 128 will bypass it

let g:gtk_nocache=[0x00000000, 0xfc00ffff, 0xf8000001, 0x78000001]

So, if you're looking to see "<=" as a combined glyph, and your font has a ligature for that combination, you should see it with the setting above.

About how these values work: This is a bitmap, specifying with a set bit which of the first 128 characters bypass the glyph cache. To check character i, you'd check (a[i / 32] & (1u << (i % 32))), if a refers to the array g:gtk_nocache, and you assume standard C notation.

About the criticism that this is not the most user-friendly option to set: That's a very valid point. My reasoning was thus:

- Internally, we need something that's quick to handle inside a tight loop in the glyph rendering code, so something complex such as a list of characters or a regexp is out. This bitmap thing is quite nice, since it doesn't take up much memory, and it is easy and fast to do lookups. As an internal representation, that appears to do the job nicely.

- The "user-facing part" depends very much on what people would like to have. I haven't written this part for two reasons yet: The first reason is that I am unsure what people would prefer. The second reason, I'll keep to myself for another couple of sentences, and continue with my thought process. I was thinking that it should be possible to provide a little vim script/function that takes the list of characters for which to bypass the glyph cache (or those for which not to bypass it), and set g:gtk_nocache from that. What form this takes depends very much on what users want. This brings me to the second reason why the user-facing part of vim script doesn't exist yet: I'm new to the vim code base and vim script, and I'm not sure I will manage something usable on the first go. This will require some iteration... ;)

That said, I'm very happy for suggestions (or patches), and will try to have a draft ready soonish when suggestions do trickle in. :)

Does this help?

Cheers,

Manuel

[Kazunobu Kuriyama]

Hi Manuel,

I'm now giving a try to a nightly build iTerm2 with FiraCode to improve my general knowledge on the ligature issue.  The terminal has started supporting ligatures since early August.

With Vim on that particular terminal, the result is nearly perfect:

-  No drawing glitches are found when the cursor goes over ligature glyphs. 

- Other than specifying a font name at the Preferences Panel, you can enjoy ligatures with Vim just out of the box.

Only the glitch I've found so far is that, when the cursor is over a double-width glyph, the right half of the glyph is gone.

It might be good to give a glance at other implementations.

Best regards,

Kazunobu

That said, I'm very happy for suggestions (or patches), and will try to have a draft ready soonish when suggestions do trickle in. :)

Does this help?

Cheers,

Manuel

[manuelschi...@googlemail.com]

Hi Kazunobu,

I am confused. My patch shouldn't affect the terminal version of vim at all. iTerm2 is thus a good way to see how other applications (here a terminal emulator) deal with ligatures and screen updates (and if they do a good job, you'll see ligatures because the terminal handles them). But that has nothing to do with the patches I sent.

If you want to see how my patch does, you will need to have a look at the GTK version of vim to see anything happening (or not).

> With Vim on that particular terminal, the result is nearly perfect:
>
>
> -  No drawing glitches are found when the cursor goes over ligature glyphs. 
>
>
> - Other than specifying a font name at the Preferences Panel, you can enjoy ligatures with Vim just out of the box.
>
>
> Only the glitch I've found so far is that, when the cursor is over a double-width glyph, the right half of the glyph is gone.
>
>
> It might be good to give a glance at other implementations.

Agreed. Since I'm stuck with Linux for the time being (and don't really feel like changing OS), I may have to ask a co-worker if they let me check things out on their Macs...

Cheers,

Manuel

> Best regards,
> Kazunobu
>
>
>
>
> That said, I'm very happy for suggestions (or patches), and will try to have a draft ready soonish when suggestions do trickle in. :)
>
>
>
> Does this help?
>
>
>
> Cheers,
>
>
>
> Manuel
>
>
>
>
>
> --
>
> --
>
> You received this message from the "vim_dev" maillist.
>
> Do not top-post! Type your reply below the text you are replying to.
>
> For more information, visit http://www.vim.org/maillist.php
>
>
>
> ---
>
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
>

> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.

[Matěj Cepl]

On 2016-09-15, 12:33 GMT, manuelschiller.pimail via vim_dev wrote:
> Hi Matěj, Christian,
>
> I've added the value which is equivalent to the previous patch, i.e.
>
> " this should keep character 0-31 (control characters), and
> " [0-9A-Za-z] flowing through the glyph cache, and the rest
> " < 128 will bypass it
> let g:gtk_nocache=[0x00000000, 0xfc00ffff, 0xf8000001, 0x78000001]

I had to make some typo or something, because when copying this
line into my ~/.vimrc I got it working as before. Thank you!

> About the criticism that this is not the most user-friendly
> option to set: That's a very valid point. My reasoning was
> thus:

That was not my problem. Read
http://www.joelonsoftware.com/uibook/chapters/fog0000000059.html

# Every time you provide an option, you're asking the user
# to make a decision.

I would even say, “you force the user to make decision”. And he
will be grumpy for that. You should apologize every time you
force user to do anything else than why he started to use your
program in the first place.

Which is why I said, that if you need this during the
development, then I don't care. However, if normal user ever
sees it, she will get fits and she would be right.

> - The "user-facing part" depends very much on what people
> would like to have.

“Nothing” is the only possible answer, I think.

> That said, I'm very happy for suggestions (or patches), and
> will try to have a draft ready soonish when suggestions do
> trickle in. :)

Thank you very much for your work, despite my criticism.

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

[Kazunobu Kuriyama]

I didn't mention your patch at all.  So I don't understand why you were confused.

Anyway, I just wanted to share the-state-of-the-art about the ligature issue with you, suggesting that

use of g:gtk_nocache would not make the users happy.

Best,

Kazunobu

To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@googlegroups.com.

[manuelschi...@googlemail.com]

Ah. That was me assuming things from the context, and boy, was I wrong... Now I get it.

> Anyway, I just wanted to share the-state-of-the-art about the ligature issue with you, suggesting that
> use of g:gtk_nocache would not make the users happy.

I'll have a look when I get the chance on someone else's laptop - this pointer is definitely appreciated. From what you write in you last mail, that's pretty much the behaviour I get from the gtk3 version by now. :)

Cheers, and thanks,

Manuel

[manuelschi...@googlemail.com]

On Thursday, 15 September 2016 15:01:35 UTC+1, mcepl wrote:
> On 2016-09-15, 12:33 GMT, manuelschiller.pimail via vim_dev wrote:
> > Hi Matěj, Christian,
> >
> > I've added the value which is equivalent to the previous patch, i.e.
> >
> > " this should keep character 0-31 (control characters), and
> > " [0-9A-Za-z] flowing through the glyph cache, and the rest
> > " < 128 will bypass it
> > let g:gtk_nocache=[0x00000000, 0xfc00ffff, 0xf8000001, 0x78000001]

Hi,

> I had to make some typo or something, because when copying this
> line into my ~/.vimrc I got it working as before. Thank you!

glad that's sorted. :)

> > About the criticism that this is not the most user-friendly
> > option to set: That's a very valid point. My reasoning was
> > thus:
>
> That was not my problem. Read
> http://www.joelonsoftware.com/uibook/chapters/fog0000000059.html
>
> # Every time you provide an option, you're asking the user
> # to make a decision.
>
> I would even say, “you force the user to make decision”. And he
> will be grumpy for that. You should apologize every time you
> force user to do anything else than why he started to use your
> program in the first place.

Interesting read. Thanks for the pointer.

> Which is why I said, that if you need this during the
> development, then I don't care. However, if normal user ever
> sees it, she will get fits and she would be right.

Well, I was thinking along the lines that using these special ligature glyphs is definitely a matter of preference. (For example, Bram had a pretty strong opinion about ">=/<=" and how he would not like to see ligatures for these combinations, I seem to remember.)

I thought that not setting any option in .vimrc will produce usable output (without ligatures) which is fine for the majority of people who don't care about ligatures. No choice involved for the unsuspecting user, it'll just work, and produce unsurprising output for a hassle-free editing session (albeit without the ligature eye-candy... ;).

For those who want ligatures, they'll have to look at the patch at the moment, and set a variable in their .vimrc. In the long run, we should have a nice front end for that, and provide documentation, so it becomes easier to see what needs to be put into .vimrc.

I didn't know about the article when I wrote this, but the logic of the previous two paragraphs seems to tie in with the spirit of the article's last paragraph or so. Displaying ligatures (or not) is a matter of preference, and options should allow users to fine-tune what they would like to see. For people who don't care, they don't need to set anything, and get vim's old (ligature-less) behaviour that they're used to.

I'll do a more careful reading of the article tonight, in case I missed something important (which could well be - feel free to point it out, since I only skimmed the article so far).

> > - The "user-facing part" depends very much on what people
> > would like to have.
>
> “Nothing” is the only possible answer, I think.
>
> > That said, I'm very happy for suggestions (or patches), and
> > will try to have a draft ready soonish when suggestions do
> > trickle in. :)
>
> Thank you very much for your work, despite my criticism.

I appreciate the criticism! You haven't convinced me yet to change the default, but I'm willing to be convinced by the right argument. From what I've grasped from the article, I tend to lean more towards the current implementation (although I think we need something more user-friendly for the users who want ligatures).

But ultimately, what we come up with will have to convince the rest of the community, so it doesn't matter that much what I think, anyway. :) So keep the arguments coming...

Cheers, and thanks,

Manuel

[Kazunobu Kuriyama]

Manuel,

AFAICT, both iTerm2 and MacVim uses the class NSAttributedString and supports ligatures by tweaking the value of one of the attributes, called NSLigatureAttributeName [1].  (Of course, the reality is not that simple, though :)

The attribute itself is virtually of an integer type and has only three different values: 0 (essential), 1 (standard), and 2 (all available).

I feel this indicates that there's a possibility to narrow down overwhelmingly huge range of g:gtk_nocache to only three different predefined values, though I'm not sure if that will do or not.

[1] https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/AttributedStrings/Articles/standardAttributes.html

Best,

Kazunobu 

To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@googlegroups.com.

[Matěj Cepl]

On 2016-09-15, 14:52 GMT, manuelschiller.pimail via vim_dev wrote:
> Well, I was thinking along the lines that using these special
> ligature glyphs is definitely a matter of preference. (For
> example, Bram had a pretty strong opinion about ">=/<=" and
> how he would not like to see ligatures for these combinations,
> I seem to remember.)

That’s probably a good point: looking at issues on
https://github.com/tonsky/FiraCode/issues and
https://github.com/tonsky/FiraCode/wiki it seems there is truly
not The Right Way™ how to do it (especially, if Haskell people
get involved, who seem to have really weird requirements; well,
Haskell people ... enough said ;)). However, I still hold that
this variable should be seen by as few people as possible. So,
some sensible default should be provided, and only weird people
(like Bram ;)) should be sent to detailed instructions how to
fix it (or wiki pages, or something).

> I thought that not setting any option in .vimrc will produce
> usable output (without ligatures) which is fine for the
> majority of people who don't care about ligatures. No choice
> involved for the unsuspecting user, it'll just work, and
> produce unsurprising output for a hassle-free editing session
> (albeit without the ligature eye-candy... ;).

If you want to have this switched off, then I would prefer some
``ligatures_on`` Boolean preferences or something of that kind
which would be easy to use.

I would even argue that default off is a questionable choice,
because if somebody installs Fira Code, I assume, she wants
ligatures, why she would install this weird font otherwise (I
would call it “The Principle of the Least Atonishment”, but
somebody else used it)? But whatever, adults don’t argue about
defaults, so whichever way it is, I can switch it on in ~/.vimrc
myself.

Best,

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

If in desperation, read the documentation!
-- Brian D. Ripley, on R-help list

[Matěj Cepl]

On 2016-09-15, 16:42 GMT, Matěj Cepl wrote:
> That’s probably a good point: looking at issues on
> https://github.com/tonsky/FiraCode/issues and
> https://github.com/tonsky/FiraCode/wiki it seems there is
> truly not The Right Way™ how to do it (especially, if Haskell
> people get involved, who seem to have really weird
> requirements; well, Haskell people ... enough said ;)).
> However, I still hold that this variable should be seen by as
> few people as possible. So, some sensible default should be
> provided, and only weird people (like Bram ;)) should be sent
> to detailed instructions how to fix it (or wiki pages, or
> something).

Actually, it might be interesting to find out “default
ligatures” and “all ligatures” mean in
https://github.com/robertvojta/LigatureXcodePlugin. I guess,
they are trying to solve the same problem.

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

All parts should go together without forcing. You must remember
that the parts you are reassembling were disassembled by you.
Therefore, if you can't get them together again, there must be a
reason. By all means, do not use a hammer.
-- IBM maintenance manual, 1925

[Charles E Campbell]

Matěj Cepl wrote:
> So, for example how to make >= and <= be included so that they
> translate into (glyphs from :digraphs):
>
> =< ≤ 8804
> >= ≥ 8805
>
> Matěj
>

Hello:

If you're using utf-8, its easy to get the glyph transform with
mathmenu.vim (comes with
http://www.drchip.org/astronaut/vim/index.html#MATH): type >=, select
with visual-block (ctrl-v), then press the "&" key. Same sort of thing
for <=, too.
Lower case: select letter(s), press "_". Upper case: select letter(s),
press "^". Etc.

You'll also find an example with:
http://www.drchip.org/images/example_math.png . Enable it by typing
":Math".

Regards,
Chip Campbell

[Matěj Cepl]

On 2016-10-26, 17:26 GMT, Charles E Campbell wrote:
> If you're using utf-8, its easy to get the glyph transform
> with mathmenu.vim (comes with
> http://www.drchip.org/astronaut/vim/index.html#MATH): type >=, select
> with visual-block (ctrl-v), then press the "&" key. Same sort of thing
> for <=, too.
> Lower case: select letter(s), press "_". Upper case: select letter(s),
> press "^". Etc.

Thank you very much.

However, I have to say it. My first step was to create a git
repo https://gitlab.com/mcepl/math-vba and use it via
preferred pathogen-style package manager of my choice (vim-plug
in my case). I know that vimballs are you baby, but isn't the
time to admit defeat and give up on it and declare
pathogen-style repos as a preferred way of dealing with vim
plugins? It was an interesting idea, but it just didn't catch
with the vim population it seems.

Best,

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

The state is the great fictitious entity by which everyone seeks
to live at the expense of everyone else.
-- Frederick Bastiat

[Tony Mechelynck]

On Wed, Oct 26, 2016 at 8:32 PM, Matěj Cepl <mc...@cepl.eu> wrote:
> On 2016-10-26, 17:26 GMT, Charles E Campbell wrote:
>> If you're using utf-8, its easy to get the glyph transform
>> with mathmenu.vim (comes with
>> http://www.drchip.org/astronaut/vim/index.html#MATH): type >=, select
>> with visual-block (ctrl-v), then press the "&" key. Same sort of thing
>> for <=, too.
>> Lower case: select letter(s), press "_". Upper case: select letter(s),
>> press "^". Etc.
>
> Thank you very much.
>
> However, I have to say it. My first step was to create a git
> repo https://gitlab.com/mcepl/math-vba and use it via
> preferred pathogen-style package manager of my choice (vim-plug
> in my case). I know that vimballs are you baby, but isn't the
> time to admit defeat and give up on it and declare
> pathogen-style repos as a preferred way of dealing with vim
> plugins? It was an interesting idea, but it just didn't catch
> with the vim population it seems.
>
> Best,
>
> Matěj

Don't make the too frequent error to believe that everyone else, or at
least most of them, has the same preferences as you. The fact that we
hardly ever see a post about whether or not vimballs are the way to go
does not necessarily mean that everyone dislikes them the way you do.
On the contrary, I would intrpret that silence as meaning that the
current state of affairs is OK for most people, because if it weren't
they would complain. I hardly ever see such a complaint, yours is a
"once in a blue moon" thing.

Best regards,
Tony.

[Matěj Cepl]

On 2016-10-26, 22:23 GMT, Tony Mechelynck wrote:
> Don't make the too frequent error to believe that everyone
> else, or at least most of them, has the same preferences as
> you. The fact that we hardly ever see a post about whether or
> not vimballs are the way to go does not necessarily mean that
> everyone dislikes them the way you do. On the contrary,
> I would intrpret that silence as meaning that the current
> state of affairs is OK for most people, because if it weren't
> they would complain. I hardly ever see such a complaint, yours
> is a "once in a blue moon" thing.

I don’t think this is the case. I have been following a lot of
vim-related news and communications in the last year or more,
and I have found a lot of plugins which are just uploaded to
vim.org/scripts/ (mostly tarballs or zip archives), and a lot of
plugins which are in a git repo (99.9% cases in GitHub). This is
the first vimball I see in a long long time.

And you are wrong if you except complains ... featuretitis
doesn’t represent itself by complaints from users. Only by the
rotting code, and emerging (security and other) bugs.

[Tony Mechelynck]

On Thu, Oct 27, 2016 at 2:08 AM, Matěj Cepl <mc...@cepl.eu> wrote:
> On 2016-10-26, 22:23 GMT, Tony Mechelynck wrote:
>> Don't make the too frequent error to believe that everyone
>> else, or at least most of them, has the same preferences as
>> you. The fact that we hardly ever see a post about whether or
>> not vimballs are the way to go does not necessarily mean that
>> everyone dislikes them the way you do. On the contrary,
>> I would intrpret that silence as meaning that the current
>> state of affairs is OK for most people, because if it weren't
>> they would complain. I hardly ever see such a complaint, yours
>> is a "once in a blue moon" thing.
>
> I don’t think this is the case. I have been following a lot of
> vim-related news and communications in the last year or more,
> and I have found a lot of plugins which are just uploaded to
> vim.org/scripts/ (mostly tarballs or zip archives), and a lot of
> plugins which are in a git repo (99.9% cases in GitHub). This is
> the first vimball I see in a long long time.

Ohhh, so that's what you mean. Well, here is another try: Not being
imitated, or not being a copycat himself, don't necessarily mean that
Dr. Chip is wrong, especially with Vim, which prides himself about
allowing the users to obtain the same results by different means.

Regards,
Tony.

[Matěj Cepl]

On 2016-10-27, 11:58 GMT, Tony Mechelynck wrote:
> Ohhh, so that's what you mean. Well, here is another try: Not
> being imitated, or not being a copycat himself, don't
> necessarily mean that Dr. Chip is wrong, especially with Vim,
> which prides himself about allowing the users to obtain the
> same results by different means.

I don't even know whether Dr. Chip was wrong when he invented
vimballs. His crystal ball was obviously a bit murky so he
haven’t expected git (and particularly GitHub) typhoon. Either
because of that or for whatever other reason vimballs really are
not that widespread as was expected when they became part of
vim. I believe it is time to accept the reality and move on.

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

Never ascribe to malice that which is adequately explained by
stupidity.
-- Napoleon Bonaparte (or many other people to whom this
quote is ascribed)

[Charles E Campbell]

Matěj Cepl wrote:
> On 2016-10-27, 11:58 GMT, Tony Mechelynck wrote:
>> Ohhh, so that's what you mean. Well, here is another try: Not
>> being imitated, or not being a copycat himself, don't
>> necessarily mean that Dr. Chip is wrong, especially with Vim,
>> which prides himself about allowing the users to obtain the
>> same results by different means.
> I don't even know whether Dr. Chip was wrong when he invented
> vimballs. His crystal ball was obviously a bit murky so he
> haven’t expected git (and particularly GitHub) typhoon. Either
> because of that or for whatever other reason vimballs really are
> not that widespread as was expected when they became part of
> vim. I believe it is time to accept the reality and move on.
>

Well, I don't intend to spend hours re-inventing my scripts which do a
lot of housekeeping and are responsible for updating my website, and
which work with vimballs. Nor am I about to start keeping your and
other's git repositories up-to-date. That said, I don't mind your git
repositories, its just that I won't be using them.

Bram asked me to come up with a vim-based way years ago to handle vim
plugins; that is the provenance of vimball. Prior to that I myself used
tar and gzip.

Regards,
Chip Campbell

[Matěj Cepl]

On 2016-10-27, 21:01 GMT, Charles E Campbell wrote:
> Well, I don't intend to spend hours re-inventing my scripts
> which do a lot of housekeeping and are responsible for
> updating my website, and which work with vimballs. Nor am
> I about to start keeping your and other's git repositories
> up-to-date. That said, I don't mind your git repositories,
> its just that I won't be using them.

I haven’t expected you to do either of these.

> Bram asked me to come up with a vim-based way years ago to
> handle vim plugins; that is the provenance of vimball. Prior
> to that I myself used tar and gzip.

Right, 818 lines of unused vim code is probably more than
I would like to have, but it is not that bad (I was afraid, that
some parts of vimballs require C implementation, so that it is
part of vim C codebase). On the other hand, I am not sure why
vimball should not be kept in a separate external project as all
other package managers do, but it is probably not that big deal.

Matěj

--
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

I would like to die sleeping, like my father — rather than
screaming and helpless, like his passengers.

[Tony Brown]

On Sunday, April 20, 2014 at 11:55:22 AM UTC-4, François Gannaz wrote:
> Hello
>
> In a few words, here is a patch that makes gvim work better with ligatures
> in fonts, which can be useful even for programmers. Details follow.
>
> I tried to use a Hasklig[^1], a font with ligatures intended for the
> Haskell language. It serves the same objective as the Haskell Conceal
> script[^2], but with the added benefit that even a mouse copy-paste works
> as intended.
> [^1]: https://github.com/i-tu/hasklig
> [^2]: http://www.vim.org/scripts/script.php?script_id=3200
>
> Unfortunately, gvim doesn't support ligatures on ASCII characters. The
> following assertion fails at run-time:
>
> ascii_glyph_table_init: assertion 'gui.ascii_glyphs->num_glyphs ==
> sizeof(ascii_chars)' failed
>
> and many characters are displayed with the wrong glyphs.
> The attached patch limits the function ascii_glyph_table_init() to
> spaces and alphanumeric chars. It solves the problem here.
>
> Yet I wonder if the current hack with ASCII characters is really useful.
> Is there any performance test to check if a simpler behaviour wouldn't be
> suitable, at least for modern desktop installations?
> As the code comment mentions spaces, maybe it should be restricted to
> blank lines?
>
> Regards
> --
> François

all you have to do is put this in your `~/.gvimrc` file:
`set macligztures`
`set guifont=NameOfFont:h22`