How can I output ᾱ̔ (1FB1 0314) using dead keys?

[Ryan]

Hello. I'm using Ukelele software on my Mac OS for creating a Greek polytonic keyboard. I created a key for ᾱ (1FB1), which doesn't normally come with Greek polytonic keyboards. When I use the dead key for the rough breathing mark (Dead key, next state: 14. Terminator: U+1FFE GREEK DASIA) and click on my new key ᾱ (1FB1) I want it to output ᾱ̔ (1FB1 0314), but instead it is outputting ῾ᾱ (1FFE 1FB1). What am I doing wrong? Or is it not possible to output ᾱ̔ (1FB1 0314) using dead keys like I am trying?

[Sorin Paliga]

You do not have a precomposed character, you have a succesion of two chars encoded differently. So you must have a font and an application capable of doing that. 

If these two conditions are met, then the problem is with your keylayout. I have also created such a Greek Polytonic keylayout, I do not have time to check this now (there are several, so perhaps at least of them may also fit your needs). In your case, the succession is to define a dead key with all these specific chars, and then type the basic letter, in your case ᾱ. In the attached screen shot, I copy-pasted your basic char, and then played with adding diacritical marks above and below, as you may see it work without any problem. I used my keylayout for linguistic needs, but it may be the same with yours.

On 28 Jun 2021, at 06:35, Ryan <ry...@ryanmasterson.com> wrote:

Hello. I'm using Ukelele software on my Mac OS for creating a Greek polytonic keyboard. I created a key for ᾱ (1FB1), which doesn't normally comewith Greek polytonic keyboards. When I use the dead key for the rough breathing mark (Dead key, next state: 14. Terminator: U+1FFE GREEK DASIA) and click on my new key ᾱ (1FB1) I want it to output ᾱ̔ (1FB1 0314), but instead it is outputting ῾ᾱ (1FFE 1FB1). What am I doing wrong? Or is it not possible to output ᾱ̔ (1FB1 0314) using dead keys like I am trying?

--
You received this message because you are subscribed to the Google Groups "Ukelele Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ukelele-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/c7d12fef-2d78-440b-88a4-ed0d2ef76e05n%40googlegroups.com.

[Gé van Gasteren]

Hi Ryan,

The thing is: like characters have to be assigned to keys, also results of combinations using dead keys have to be assigned.

The output you see (῾ᾱ) when first entering the rough breathing mark, then ᾱ, is the "default" behavior for unassigned dead key sequences.

What you need to do in Ukelele is: enter the dead key state for the rough breathing mark and there add a two-character output (U+1fb1 U+0314) on the key to which you have assigned U+1fb1 in the normal state.

You have successfully added a character (U+1fb1) to your custom keyboard layout, so I’m confident you can also do this next step: add a character (or two, in this case) to a dead key state. Let us know how you manage :)

On Mon, Jun 28, 2021 at 5:35 AM Ryan <ry...@ryanmasterson.com> wrote:

Hello. I'm using Ukelele software on my Mac OS for creating a Greek polytonic keyboard. I created a key for ᾱ (1FB1), which doesn't normally come with Greek polytonic keyboards. When I use the dead key for the rough breathing mark (Dead key, next state: 14. Terminator: U+1FFE GREEK DASIA) and click on my new key ᾱ (1FB1) I want it to output ᾱ̔ (1FB1 0314), but instead it is outputting ῾ᾱ (1FFE 1FB1). What am I doing wrong? Or is it not possible to output ᾱ̔ (1FB1 0314) using dead keys like I am trying?

--

[Sorin Paliga]

He may add one or more combining diacritical marks, his only problem is to instruct UKELELE to do this properly. I showed this by a quite relevant screen shot. 

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/CAOH1hL9uVYENoZ%2BYAv79BMrDRPtqOL%2BThAT5PbGRrsDBsC-25Q%40mail.gmail.com.

[Gé van Gasteren]

Hi Sorin,

The main issue is probably the confusing situation with the two approaches to create accented characters.

One is the dead-key method, used in most standard keyboard layouts and practically always producing precomposed characters.

The other method is to type combining digital marks AFTER any base character, including after an accented character or a base character already having other combining digital marks "attached" to it.

The first method is limited but works in all applications, while the second is much more powerful and probably the method of the future, yet is still not supported everywhere, as you wrote. (I don’t have a good overview, but Microsoft apps come to mind, especially older versions.)

Fonts are not a big factor here, most fonts support both methods, although the placement of CDMs may not always be good-looking with the second method.

That Ryan wants to use a combining digital mark in a dead-key sequence may seem illogical to you – you may think it’s easier to just type the CDM after typing the base character – but if his entire layout and typing habits are built around dead keys, then this approach may work best for him.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/63DB6F8B-9377-4F69-A3BF-9ADADFB052F4%40gmail.com.

[Sorin Paliga]

Everybody has his/her style, but in his case it is a problem of sequence, I cannot figure out exactly what is wrong there, but what he wants is easy to achieve as shown in the screenshots. It should work outright, I cannot reproduce his problem, frankly do not have much time at the moment.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/CAOH1hL9_C-H%3DXSHTvggKQ4pxirVOAt4Kj62RQbHQSkfHX9OStw%40mail.gmail.com.

[Ryan]

Hi guys, thank you so much. 

Gevan, I think you are right; I think I am confused about the two approaches to create accented characters. I didn't think about using the second method for this. I'm also not worried about applications or fonts. I only need it to output on my Mac OS applications, and I think I understand fonts enough to know which to use.

Regarding method one ("One is the dead-key method, used in most standard keyboard layouts and practically always producing precomposed characters")

"What you need to do in Ukelele is: enter the dead key state for the rough breathing mark and there add a two-character output (U+1fb1 U+0314) on the key to which you have assigned U+1fb1 in the normal state."

I'm not sure how to do this. Can you point to the page in the Ukelele Tutorial? I have version 3.3.

Regarding method two ("The other method is to type combining digital marks AFTER any base character, including after an accented character or a base character already having other combining digital marks "attached" to it.")

If you think this is the way of the future, then maybe I should recreate my Greek Polytonic keyboard to use this method instead of dead keys. I notice that the app Hoplite https://philolog.us/hoplitekeyboard.html uses this method. However in the end it doesn't work right either. 

When using either the "precomposed" or the "combining" method in Hoplite, it outputs the Greek letter alpha with a macron and a rough breathing mark as 03B1 0304 0314 instead of 1FB1 0314. I have to have it as 1FB1 0314 because it renders nicely in the Google Suite in Safari while 03B1 0304 0314  does not. 

In this screenshot, see 03B1 0304 0314 above. It looks ugly with the rough breathing mark to the side. See  1FB1 0314 which is nice.

[David J. Perry]

Ryan,

Gé has given you good directions, but let me back up a little so that you can perhaps understand the issue better.

Unicode expects combining diacritical [not digital] marks to always be entered AFTER the the base character.  So you could enter alpha + combining rough breathing + combining macron (or alpha + combining macron + combining rough; doesn't matter) using any available method -- custom keyboard, Mac Hex Entry method, whatever.  Mac OS is quite good about trying to position the combining diacritics properly, although it is not always 100% successful at delivering the best appearance.  But it should work to some extent with any font that contains the characters.  If the font maker has supplied good mark-to-base and mark-to-mark lookups, the results should be perfect.

What happens if you enter the precomposed alpha+macron and then the combining rough will vary.  I don't have access to my Mac at the moment, but Mac OS will try; the results may depend on the font.  If the font developer has allowed for the possibility of someone entering a combining rough after the precomposed character, fine; if not, maybe.

If you typically create text using "ordinary" polytonic Greek (i.e., not using macra) that's one thing, and you can use deadkeys to generate the precomposed combinations as you are probably used to doing; macra will be handled as special cases, possibly in a way that is less easy to type.  But if you regularly create teaching materials or reference materials that need vowels with macra, I would suggest moving to using the combining diacritics typed after the vowels.  This may require some re-training yourself, but I think you get better results and you will be consistent with what is best practice for the future.  (Any Unicode text that you find online, such as at Perseus, almost certainly uses combining diacritical marks.)

Hope this helps.

David

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/750e8e88-a18c-4913-9643-6a7a3ae27c54n%40googlegroups.com.

[Gé van Gasteren]

Hi Ryan,

It shouldn’t matter for this job, but Ukelele’s current version is 3.5.2.

You can probably update through the Ukelele menu: Check for Updates, or else just download the new version. In that case, there’s no need to uninstall the old version, although that can avoid confusion.

Let’s focus on the dead-key method for now, so you’ll have a working keyboard layout. Your less-than-optimal result with the Hoplite app/layout already shows the possible issues with the method of inserting CDM (combining diacritical marks) after characters, so it’s good to know both approaches, anyway :)

(Also, although that method is in itself more straight-forward than dealing with dead keys, you’ll probably want to make a completely new keyboard layout for that approach, so it’ll take some more thinking and planning before you embark on that. David Perry’s thoughts here are a good start.)

If you haven’t found it yet, you can open Ukelele’s PDF manual from the Help menu.

There’s lots of stuff about dead keys, but you already have a dead key state, you just want to add one more key in there.

So check out section 6.7.5 "Editing an existing dead key" (page 34 in the manual, which is actually marked as page 30).

Specifically:
Ukelele 3.5 Manual

Simply double-click the dead key in the Ukelele window, when it is shown with the dead key colour (red in the default colour theme). You will see a dialog that asks what you want to do. Choose the "Enter State" tab and click OK to edit the dead key state.

After that:

Ukelele 3.5 Manual

What you see is the output of each key in that particular dead key state. A normal-looking key will have output in the dead key state, while the greyed-out keys show the output in state “none”, for your reference.

The key where you have ᾱ in the "normal" state should be looking greyed-out inside this dead key state, as you haven’t assigned any output to it yet.

You already know how to edit keys: One way is to double-click the key, after which a text field pops up.

There you can enter the two characters ᾱ̔, e.g. by pasting them in after copying them from any text.

Click "Done", then click the "Leave" button to Exit the Dead-key state, save the layout, and you’re done.

Good luck!

[Ryan]

Dear David and Gé,

This is awesome information. I'm learning a lot. Thank you. So you believe that the second method of typing combining diacritical marks AFTER any base character is more powerful and the way of the future? If so, then yes, I'll gladly recreate an entire new Greek Polytonic keyboard layout for myself like this. I want to move forward using the better solution, even if that means relearning how to type. One thing that's cool about Ukelele is the ability to create "New From Current Input Source." Is there already a Greek Polytonic keyboard for Mac that combines the digital marks AFTER the base characters that I can start with as a base?

Also, I'm curious, why did the Hoplite app/layout fail to insert the CDM (combining diacritical marks) for ᾱ̔ (1FB1 0314) correctly? It seems like it's on the right path, but failed in this particular case. I'm just wondering why.

[Gé van Gasteren]

Maybe Sorin can upload one of his keyboard layouts for you to use as a starting point?

Actually, he’ll probably give you ALL of them :)

Another keyboard layout you can have a look at is ABC Extended, which comes with MacOS and has tons of CDMs defined:

With "the way of the future" I meant, that I expect that the support for this method, typing CDMs after base letters, will become more solid over time.

In fact, although I can’t say for sure, I suspect that the Hoplite problem you faced has something to do with incomplete support.

[Sorin Paliga]

At a given moment I found that the link to them was discontinued, I have rechecked and both the page in Romanian and in English are OK, the link will lead to Dropbox folder for download.

https://unibuc.ro/user/sorin.paliga/

Check under 

Main fields of activity, prelast line includes a link. Perhaps the one you need is US Academic U6

If not, this link should work, I cannot figure whether anyone can download from that folder, the link to it is

https://www.dropbox.com/sh/ttp22x04rvsyyr9/AACFcXSVrI7vafRwL_7IfpwEa?dl=0

On 29 Jun 2021, at 20:38, Gé van Gasteren <gevang...@gmail.com> wrote:

Maybe Sorin can upload one of his keyboard layouts for you to use as a starting point?

Actually, he’ll probably give you ALL of them :)

Another keyboard layout you can have a look at is ABC Extended, which comes with MacOS and has tons of CDMs defined:

<Screen Shot 2021-06-29 at 19.35.30.png>

With "the way of the future" I meant, that I expect that the support for this method, typing CDMs after base letters, will become more solid over time.

In fact, although I can’t say for sure, I suspect that the Hoplite problem you faced has something to do with incomplete support.

On Tue, Jun 29, 2021 at 7:14 PM Ryan <ry...@ryanmasterson.com> wrote:

Dear David and Gé,

This is awesome information. I'm learning a lot. Thank you. So you believe that the second method of typing combining diacritical marks AFTER any base character is more powerful and the way of the future? If so, then yes, I'll gladly recreate an entire new Greek Polytonic keyboard layout for myself like this. I want to move forward using the better solution, even if that means relearning how to type. One thing that's cool about Ukelele is the ability to create "New From Current Input Source." Is there already a Greek Polytonic keyboard for Mac that combines the digital marks AFTER the base characters that I can start with as a base?

Also, I'm curious, why did the Hoplite app/layout fail to insert the CDM (combining diacritical marks) for ᾱ̔ (1FB1 0314) correctly? It seems like it's on the right path, but failed in this particular case. I'm just wondering why.

--
You received this message because you are subscribed to the Google Groups "Ukelele Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ukelele-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/CAOH1hL_brCQTZVRPuwcYNU983%2BZ-OwY0G4Gy8s9_uHkZW7C90Q%40mail.gmail.com.

[John Brownie]

I hate to throw a wet blanket, but many programs will seek to normalise
Unicode data, either to a precomposed (NFC) or a decomposed (NFD) form.
If it goes to NFD, which I think is the preferred form, then U+1FB1 will
be transformed to U+03B1 U+0304, which gets you back to the form that
doesn't appear to work correctly.

So my warning is that the data in the document may be changed under your
feet by a program which is doing the right thing, so you have to work
out whether it's going to be a stable solution for you. If it works in
the program you are using, and you aren't going to share the document
via another program, then go ahead. But this may be a source of further
frustration further down the line.

John
--
John Brownie
Mussau-Emira language, New Ireland Province, Papua New Guinea
Kouvola, Finland

[Gé van Gasteren]

Hi Sorin,

With your link, I was able to click through and download the keyboard layouts, all in one package or one by one.

So that works.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/A27284C5-849E-4C8E-A0FD-BDE97ECD5656%40gmail.com.

[Gé van Gasteren]

Also, I'm curious, why did the Hoplite app/layout fail to insert the CDM (combining diacritical marks) for ᾱ̔ (1FB1 0314) correctly? It seems like it's on the right path, but failed in this particular case. I'm just wondering why.

Here's a little check on how 8 different fonts behave in TextEdit in a relatively old version of MacOS: 10.11.

On the first line, I typed the three characters U+03b1 U+0304 U+0314 and on the second U+1fb1 U+0314.

Positions 3 and 4 in each line are the fonts Colibri and Cambria.

Cambria is the only font where I can see a difference in appearance between the two lines.

[Kamal Abdali]

This is awesome information. I'm learning a lot. Thank you. So you believe that the second method of typing combining diacritical marks AFTER any base character is more powerful and the way of the future?

IMHO the method of combining diacritical marks (CDMs) is more flexible. I use it exclusively now. For example, for a keyboard with math symbols and various accents, CDMs made my design attempt much easier than dead keys. The layout of this keyboard can be seen in  https://github.com/KamalAbdali/PolymathKeyboardMac/blob/master/distrib/US-PolymathKeyboardMac.pdf. The source is at https://github.com/KamalAbdali/PolymathKeyboardMac/tree/master/distrib.

Kamal Abdali  

--

You received this message because you are subscribed to the Google Groups "Ukelele Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ukelele-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/3ba3baf1-f7b4-427f-aedf-92e94ad2ef3en%40googlegroups.com.

[Sorin Paliga]

Good, so the link has been restored.

On 30 Jun 2021, at 00:25, Gé van Gasteren <gevang...@gmail.com> wrote:

Also, I'm curious, why did the Hoplite app/layout fail to insert the CDM (combining diacritical marks) for ᾱ̔ (1FB1 0314) correctly? It seems like it's on the right path, but failed in this particular case. I'm just wondering why.

Here's a little check on how 8 different fonts behave in TextEdit in a relatively old version of MacOS: 10.11.

On the first line, I typed the three characters U+03b1 U+0304 U+0314 and on the second U+1fb1 U+0314.

<Screen Shot 2021-06-29 at 23.14.40.png>

Positions 3 and 4 in each line are the fonts Colibri and Cambria.

Cambria is the only font where I can see a difference in appearance between the two lines.

--
You received this message because you are subscribed to the Google Groups "Ukelele Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ukelele-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/CAOH1hL9uLk0U_41eVeDwjvv8YbS3qOjBAPcDsjzcF7y5%2BHRyhQ%40mail.gmail.com.

[Sorin Paliga]

CDMs were added at a given moment when realizing that it is a lot simpler and practically covering all the needs if adding a diacritical mark to ANY basic char, above or below, or more such CDMs, above and below above/below the same char. 

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/CAHpX9ZfdHhyh%3DSsGcq%3DgPWsT03DfxLqOCwTtChdG4wss4tobaw%40mail.gmail.com.

[Ryan]

This is such a great forum with educated, active participants. Thank you. I have a few questions.

I want to make sure I understand the terms being used. What do the abbreviations NFC (precomposed) and NFD (decomposed) stand for? And is it true to say that CDM (combining diacritical marks) is basically NFD (decomposed)?

Here's a response to Gé regarding his comments "Here's a little check on how 8 different fonts behave in TextEdit in a relatively old version of MacOS: 10.11. On the first line, I typed the three characters U+03b1 U+0304 U+0314 and on the second U+1fb1 U+0314. Positions 3 and 4 in each line are the fonts Colibri and Cambria. Cambria is the only font where I can see a difference in appearance between the two lines."

Unfortunately good CDM support is not just dependent on fonts, I have discovered. It also depends on the software and/or web application. For example, the font Cardo is a very good font for classicists. It looks better than New Athena Unicode, in my opinion. 

Cardo, as well as other fonts, look good in the MacOS software Pages (and TextEdit). But when using Google Suite, I have only found good support in Safari. Macrons display poorly in Google Suite (Gmail, Google Docs, Google Spreadsheets, etc.) because these services are using Google Fonts, and at the time of this writing Google Fonts does not support a nice display of macrons. This is easily demonstrated here: https://fonts.google.com/?subset=greek-ext&preview.text=%E1%BF%A1%CC%94%CE%BC%CE%B5%E1%BF%96%CF%82&preview.text_type=custom. 

Before I started this conversation here I assumed that precomposed (NFC) was superior to decomposed (NFD)/combing (CDM). But now I understand that the latter is the way of the future and I should make my keyboard output that, even though there is limited support for what I want to use it in (in this case, if I'm using Google Suite I should use exclusively in Safari). 

[Gé van Gasteren]

Unfortunately good CDM support is not just dependent on fonts, I have discovered.

Yes, you’re absolutely right, the rendering software is also important.

Every OS has a "rendering engine" built in, but applications can install and use a different one.

Uniscribe, Harfbuzz, CoreText are a few names.

Even if a font contains the right "glyphs" (forms) and the right instructions (such as anchor points indicating where CDMs should be placed relative to base letters or other marks), if the rendering engine doesn’t understand the instructions – or doesn’t look at them – things turn out less-than-great.

Of course, this causes chaos and frustration: one never really knows how a font will look in a certain environment until one tries it.

Font designers have to pre-empt the quirks of the various rendering engines, sometimes building in workarounds to achieve acceptable results, but not all designers are able or willing to put in that much effort – and even maintenance, as software updates do take place.

This may all be a little disappointing, but that’s how it is. To sum it up:

As you want to make your keyboard layout now, maybe it’s more realistic to use not "the method of the future", but what works best now …

[Sorin Paliga]

DTP applications are not usually designed for linguists, who need such complicated things like at least one diacritical mark, often more than one. Or such situations combined with RTL scripts or...

Many word processors like Nisus or Mellel behave OK with these, but what happens when one sends a doc created with these apps to another platform or even opens the file thus created in another app? Well, the results are often bad. One must test how things are before taking a decision with the final form of the document.

--
You received this message because you are subscribed to the Google Groups "Ukelele Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ukelele-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/CAOH1hL-DaK5h2jCGWHVzCSkBxZA%2BWAwDojXopJVatcAL6Q0kqg%40mail.gmail.com.

[Tom Gewecke]

Regarding NFC, NFD and related, see

https://unicode.org/reports/tr15/

On Jul 3, 2021, at 12:31 PM, Ryan <ry...@ryanmasterson.com> wrote:

T

[Gé van Gasteren]

On Sat, Jul 3, 2021 at 7:24 PM Tom Gewecke <thge...@gmail.com> wrote:

Regarding NFC, NFD and related, see

https://unicode.org/reports/tr15/

Thanks for that, Tom, I had no idea!

Do I understand it correctly that this Canonical or Compatibility Decomposition, sometimes followed by Canonical Composition, is one of the jobs of the rendering engine, and that it happens before the glyphs are looked up in the font and the font’s OpenType features are applied?

[Tom Gewecke]

On Jul 3, 2021, at 2:25 PM, Gé van Gasteren <gevang...@gmail.com> wrote:

Do I understand it correctly that this Canonical or Compatibility Decomposition, sometimes followed by Canonical Composition, is one of the jobs of the rendering engine, and that it happens before the glyphs are looked up in the font and the font’s OpenType features are applied?

That is my understanding, with mark reordering also possibly taking place between decomposing and recomposing.   But my knowledge of rending engines is extremely limited.

[Gé van Gasteren]

Thanks, Tom! I’m going to take some time studying that page  …

[Gé van Gasteren]

Ryan: I just got an answer from Thomas Phinney, not quite what I expected, but still worth a try:

The one thing I would suggest, would be to change the “composer” (text layout engine). This can be done at the preference level (change the default composer) or on specific text. The default is “Adobe Paragraph Composer” and I would suggest trying one of the “World Ready” composers (either paragraph or single-line).

If that doesn’t work, I probably can’t be much more help. Since I left Adobe 12 years ago, InDesign development moved to India, so none of the text engineers I knew are there any more. I would just suggest reporting it as a bug to Adobe and see what happens.  :/

Ryan, do you know how to apply the World Ready composer, so you can test his suggestion?

In CS6 it’s done over the OpenType menu, which you get from the rightmost button on the ribbon after you have selected some text:

[Tom Gewecke]

> On Jul 3, 2021, at 8:16 PM, Gé van Gasteren <gevang...@gmail.com> wrote:
>
> Ryan: I just got an answer from Thomas Phinney, not quite what I expected, but still worth a try:
>
> The one thing I would suggest, would be to change the “composer” (text layout engine). This can be done at the preference level (change the default composer) or on specific text. The default is “Adobe Paragraph Composer” and I would suggest trying one of the “World Ready” composers (either paragraph or single-line).

I tried all the composer options on my trial copy of 16.2, but there was no impact on the problem.

[Gé van Gasteren]

I just realized I’ve continued the thread about the input problem in InDesign in the wrong place…