Custom Keyboard and InDesign

[Fred Jones]

Hi,

I have a custom keyboard where a single keystroke enters the primary glyph (for instance a "C") followed by diacriticals such as a "combining Caron" and then a "glottal". This works fine in all tested apps except InDesign (latest version of InDesign running on both Big Sur and High Sierra.) The result is that only the first glyph in the sequence is displayed when typed either directly or with the Mac onscreen keyboard. Also, I have certain glyphs that I access with a shifted keyboard (for instance a shift + 3 to enter a wsuperior.) In the latter example, the character does not appear in InDesign, either with a direct keystroke or using the Mac onscreen keyboard. 

It almost appears that InDesign is ignoring the Apple system keyboard.

Hoping someone can enlighten me.

Fred

[Tom Gewecke]

Does changing the font make any difference in this behavior?

What version of Indesign?

--
You received this message because you are subscribed to the Google Groups "Ukelele Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ukelele-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/f85aab26-3cb2-4748-bef4-7ee869b46282n%40googlegroups.com.

[Fred Jones]

v16.1 on both workstations.

The fonts are custom native language typefaces I've created (unicode/opentype compliant.) The fonts and Ukelele custom keyboards work fine in other common apps such as Word, Pages, Mellel, etc (including Mac OS finder). I have two types of custom keyboards. Type 1 you type each glyph and Opentype assembles them. Type 2 is a single keystroke that types multiple unicode code points, and Opentype assembles them. Neither keyboard works with InDesign.

Fred

[Fred Jones]

BTW, I can copy the characters (glyphs) from Word and paste them into InDesign, and they look fine (with the exception of certain instances where it looks like InDesign is doing font substitution....still working on that.)

Fred

On Wednesday, June 30, 2021 at 3:19:33 PM UTC-6 Tom wrote:

[Fred Jones]

Also, if I pull up the InDesign glyphs panel, and click on the character/glyph for custom fonts, they are entered into InDesign. That seems to indicate that there are not necessarily problems with the font, but with the ability to use a custom system keyboard.

Thoughts?

Fred

[Sorin Paliga]

Some apps cannot handle such complex combinations, and this refers to some DTP apps. Are you using the last version of InDesign? I may confirm that this regularly happended with its older versions, so if you are using the last one, this means this ‘feature’ is going on. 

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/F86B9ED1-0985-430F-A341-481159FBE56F%40gmail.com.

[Tom Gewecke]

So your custom keyboards also do not work  when you have set your text to  use ordinary fonts other than the custom ones you created?

[Fred Jones]

The keyboards don't make much sense for other fonts/typefaces as most other fonts/typefaces don't support the unicode/opentype features. That being said, I configured Ukelele to type standard qwerty characters when the caps lock is set. That works fine. I attached a screenshot of the keyboard. I also attached a screenshot of properly composed characters from Pages. The keyboard display of the glyphs is not precise as the keyboard is using a system font.

And, here I'm typing the characters using this keyboard into this message. Don't know if they will work in google groups interface.

acc̓čč̓ehikk͏ʷk̓͏ʷll̓

ḷłmm̓ṃnn̓ṇopp̓qq̓

q͏ʷq̓͏ʷsštt̓uww̓;'

x̣x͏ʷx̣͏ʷyy̓ƛ̓ə,./

[Fred Jones]

The primary problem, from my perspective, is that InDesign will only accept the first unicode code point in a custom keyboard keystroke.

Example:

"x + dotbelow combining + wsuperior" is assigned to the "c" key on a qwerty keyboard layout.

that is:

 U+0078 LATIN SMALL LETTER X  U+0323 COMBINING DOT BELOW; nang [IPA: closer variety of vowel; Americanist and Indo-Europeanist: retraction or retroflexion; Semiticist: velarization or pharyngealization; Vietnamese tone mark]   U+02B7 MODIFIER LETTER SMALL W [labialization]

When the custom keyboard is active in InDesign, and I press the "c" key, what appears is just "x", none of the following glyphs. It appears that InDesign is ignoring all but the first unicode code point.

Make sense?

[Tom Gewecke]

When I dissect this character here, I find there is also a U+034f combining grapheme joiner in the middle.  Is that intended?

[Tom Gewecke]

On Jul 1, 2021, at 10:43 AM, Fred Jones <fred...@gmail.com> wrote:

The keyboards don't make much sense for other fonts/typefaces as most other fonts/typefaces don't support the unicode/opentype features. 

I am puzzled by that.   Your 3 codepoint example

x̣ʷ

 should fully display an ordinary font like Arial.  I am wondering if when you set your font to Arial in InDesign whether all the three components show up or not.

[Gé van Gasteren]

Hi Fred,

When you asked for enlightenment, I didn’t feel up to the task, but now that you seem happy with just thoughts, I can oblige :-)

First, I don’t understand your statement:

"The keyboards don't make much sense for other fonts/typefaces as most other fonts/typefaces don't support the unicode/opentype features."

With Unicode, the worst-case scenario is that boxes or question marks are displayed instead of glyphs, so for the sake of the current troubleshooting, it doesn’t matter, I’d think.

Obviously, what Tom is trying to do is splitting the situation in two parts, to see if the fonts may be part of the problem or if it’s really only a keyboard layout issue.

When I create a custom keyboard layout in Ukelele with the mentioned character sequence assigned to a key:

U+0078 : LATIN SMALL LETTER X
U+0323 : COMBINING DOT BELOW {nang}

U+034F : COMBINING GRAPHEME JOINER [CGJ]

U+02B7 : MODIFIER LETTER SMALL W

I can install the layout and use it in InDesign CS6 on El Capitán to produce the correct rendering, as far as I can see:
(the typeface is Linux Biolinum)

I haven’t tried any other character sequences so far.

If you like, I can see how your layout works on my system, with or without one of your typefaces, although I’m not sure how much information that would give.

More thoughts:

As Adobe wants InDesign to work the same in MSWindows and MacOS, they often use their own routines instead of relying on MacOS resources.

Conclusion: there is certainly room for Mac things not being supported.

E.g. in the past, I’ve had problems using InDesign shortcuts when a keyboard layout was active that was connected to a different language than InDesign’s interface language.

I also seem to remember that custom keyboard layouts in MSWindows don’t support assigning multiple characters to one key.

But to make real headway, I think Thomas Phinney would be a good person to ask whether the described behavior in InDesign rings a bell:

http://www.thomasphinney.com/

[Fred Jones]

Thanks for the replies and thoughts.

Some additional notes:

1. If I copy/paste the glyphs (x + dot below + wsuperior) from Word or Pages into InDesign everything is fine. Character displays well and OpenType features are used

2. I have two different keyboard layouts. #1 is used by typing each individual glyph (as you've noted above) in InDesign. That also works okay for most instances.

3. #2 keyboard is a 'touchtype' layout (shown in the screenshot prior) where the user just has to press a single keystroke and all associated unicode glyphs are displayed (and formatted by OpenType). We did this because we created custom printed physical keyboards with each fully formed Salish character as separate keys (the qwerty key images was small and in the upper corner of each key.) The #2 keyboard, which is much faster for many people, works in all tested apps (including MacOs) except for InDesign. In testing, I've found that InDesign will only accept and present the first unicode codepoint.

4. I have created the same two types of keyboard layouts that I've used with various versions of Windows for probably over 20 years. I created the windows keyboards with MS Keyboard Layout Creator. Those keyboards also work with tested windows apps (and Windows explorer)...but I don't have Windows InDesign installed or accessible for testing. With MS KLC, I'm able to assign multiple unicode code points to a single key on the touch-type version of the keyboard.

5. With both the 'standard' and the 'touch-type' versions of keyboards (on both Mac and Windows) I use the Caps Lock key to instantiate standard QWERTY key layouts. The result is users can switch from the Salish touch-type layout to a standard qwerty layout just by toggling the Caps Lock key. No need to go to the language bar to switch layouts when typing mixed languages. 

To wrap up, we've been creating custom language fonts (Salish and Kootenai) and custom keyboards with both Mac and PCs for over 30 years. It has just recently come to light that the touch-type versions of the keyboards don't work with InDesign. Not a deal killer, but for people that are used to the touch-type version will have to learn multiple keystrokes (and usually type them in the proper order) to use InDesign.

[Tom Gewecke]

> On Jul 1, 2021, at 4:05 PM, Fred Jones <fred...@gmail.com> wrote:
>
> Thanks for the replies and thoughts.
>
>

> 3. #2 keyboard is a 'touchtype' layout (shown in the screenshot prior) where the user just has to press a single keystroke and all associated unicode glyphs are displayed (and formatted by OpenType).

Could you explain what you mean by “formatted by opentype”? Having a single keystroke produce a series of Unicode codepoints is should work ok with various fonts, like the Arial example I gave earlier. If you have a special font which uses opentype table to produce a special single glyph whenever the same sequence of codepoints is received, that should work too. The keyboard would be the same however. Or do you do something else?

[Fred Jones]

Maybe a better term is 'composed' rather than 'formatted'.

I have OpenType features that assemble a series of glyphs into a properly composed character using joining handles. For instance, the bottom handle for a "combining Caron" will automagically join with the top handle for a "C", and then the bottom handle for a combining glottal will automagically join with the top handle for a "combining Caron". But, I don't think that OpenType features have anything to do with the issue at hand since pasting the glyphs into InDesign produces a properly composed character. So, the font works and the OpenType features work.A custom keyboard with only single code points per key work fine. It's just Indesign doesn't like what I'm doing with the keyboard.

All I'm trying to determine if there is a pref in InDesign I can toggle to disable whatever InDesign is doing to intercept the keystrokes from my custom keyboard. Since the keyboards work with all the other apps I've tested (including typing file names as the finder level) I'm fairly certain it's an InDesign issue.

[Gé van Gasteren]

All right, some notes from my side:

1. If your custom fonts only use mark-to-base and mark-to-mark positioning, I believe you’ll find that current fonts are better than you think.

In other words: the big gap in typographic quality that once existed between your own fonts and current system fonts has largely disappeared, as illustrated by the sequence you posted earlier showing up perfectly, in my view:

acc̓čč̓ehikk͏ʷk̓͏ʷll̓

ḷłmm̓ṃnn̓ṇopp̓qq̓

q͏ʷq̓͏ʷsštt̓uww̓;'

x̣x͏ʷx̣͏ʷyy̓ƛ̓ə,./

2. Maybe I didn’t write it clearly enough in my previous post, but InDesign CS6 does exactly what you want, at least on my computer. So your next step may be to consult an InDesign expert what has changed since CS6.

3. Have you considered proposing these languages’ characters for inclusion in Unicode? It’s one thing to be able to compose the right forms so printed or on-screen texts look good, but for sorting and searching, it’s almost essential to have each character represented by just one glyph. E.g. now, when you search for x̣, you also find all instances of x̣͏ʷ – and it’s like that for many of your characters: c k l m n p q s t w y

--

You received this message because you are subscribed to the Google Groups "Ukelele Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ukelele-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/725db9d4-01d6-4175-9fd7-30f92f599e31n%40googlegroups.com.

[Fred Jones]

Thanks for the info.

I haven't received any bites from InDesign forums regarding this specific issue. That's why I tried here. Thought there might be someone who uses V16.1.

1. There are some seldom used unicode code points that are not in a lot of fonts, for instance a barred lambda or schwa. Even if the users used different fonts, the basic problem still exists, InDesign v16.1 will only accept the first unicode code point in an assigned keypress.

2. When we stack more than 1 combining character (combining Caron and combining glottal) they generally get mashed together and are unreadable with other folks fonts.

3. The users of the font are using InDesign V16.1. I understand what you're saying about CS6 (no complaints when they used that years ago) but they're not downgrading as their graphics consultants use v16.1 too.

4. I've been working with unicode/opentype folks for quite awhile and they tell me using the individual glyphs to compose a character is the standard for IPA based languages.  Some of the other folks who do these specific language fonts use the OpenType ligature feature to accomplish this, but I'm told that is now not acceptable. They also use the PUA, which is a big no-no. I was told years ago there's no way the consortium would consider adding any new characters where an IPA composed solution exists.

Thanks all, I'll just tell the users not to use the touch-type keyboard for InDesign. Just use the standard keyboard where there is a single code point per keypress.

[Gé van Gasteren]

OK, that clears up a lot.

Sorry for repeating my questions, but I wasn’t sure about the situation.

I’m curious why you don’t want to use the PUA, especially as you’re already using custom fonts?

If the Unicode consortium doesn’t want to comply, the PUA seems the best option.

Regarding the difference in behavior between InDesign versions, I would encourage you to contact Thomas Phinney. I think he’ll be happy to look into it.

And the last suggestion would be that you could resort to macros, in one way or another:

• InDesign has many automation modalities

• Keyboard Maestro can simulate key presses, so this would be another way to produce multi-character sequences with one keystroke. You can configure it to do this only in InDesign. I haven’t used it really, and of course, it’s one more thing to install and contend with for users, but failing other approaches, this could be a workaround.

[W.P.J. Rietbroek]

Just a thought: have you considered activating HarfBuzz as the OpenType shaper in InDesign? See:

https://community.adobe.com/t5/indesign/script-for-enabling-harfbuzz-workflow-in-indesign/m-p/11295166

The older ‘Lipika’ shaper is not without its bugs. HarfBuzz may perform better, you never know.

On fonts that support the characters and diacritics you need: give John Hudson’s ‘Brill’ typeface a try:

www.brill.com/brill-typeface.

It has very extensive OpenType feautures including the GPOS diacritic positioning needed to get the character rendering you need, multiple combining diacritics stacking included.

HTH,

Best,

Pim Rietbroek

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/8863e56e-8572-44fd-bc36-de22f9b865d5n%40googlegroups.com.

[Tom Gewecke]

Thanks for the additional info!

I downloaded a trial of InDesign 16.2.1 and verified the problem that you have described, with all fonts, which is certainly very strange.

There are lots of preference settings I have not explored, and will try to do some of that later...

[Gé van Gasteren]

Just a shot in the dark:

(This setting exists already in InDesign CS6.)

[Fred Jones]

Thanks for the InDesign pref hint. I tried changing the preference and it doesn't appear to alter the behavior of the custom keyboard in v16.1. Adobe InDesign will still only accept and display the first unicode code point from the custom keyboard keystroke. Again, the font/typeface don't have anything to do with this problem, as was replicated above with v16.1. It's an annoying problem, but the workaround is to use a custom keyboard with just single codepoints per keystroke.

Fred

[Tom Gewecke]

I also tried playing with every conceivable preference in v16.2.1 and have the same result as Fred no matter what.

--
You received this message because you are subscribed to the Google Groups "Ukelele Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ukelele-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/2ebd07c1-3cd1-460d-bb61-5eb4eb74659dn%40googlegroups.com.

[Gé van Gasteren]

(Continuing here i.s.o. in the other thread with OP Ryan)

I got an answer from Thomas Phinney, but unfortunately, he isn’t involved in InDesign development anymore and his suggestion of using the World-ready Composer didn’t help, as Tom verified.

Therefore, the only way seems to be to file a bug report with Adobe…

Just one more wild thought then to finish the day, for Fred to try out maybe:

Is the behavior the same when you type in the Story Editor?

[W.P.J. Rietbroek]

Dear all,

You may have missed this before:

Have you considered activating HarfBuzz as the OpenType shaper in InDesign? See:

https://community.adobe.com/t5/indesign/script-for-enabling-harfbuzz-workflow-in-indesign/m-p/11295166

The older ‘Lipika’ shaper is not without its bugs. HarfBuzz may perform better, you never know.

Best,

Pim

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/f33b2f70-4c5e-456e-bb41-31fd62221368n%40googlegroups.com.

[Tom Gewecke]

I did try this, without any impact.  But I am not totally sure that harfbuzz is being used.  Anyway to test that?

[Gé van Gasteren]

How I wish I could edit posted messages here …

Anyway, Thomas gave me a weblink for filing the bug report, writing:

[

What he is trying to do is definitely the correct way to do it, and it sounds likely to be an InDesign bug. I would suggest he report it, if he has not done so already.

https://blog.adobe.com/en/publish/2017/08/09/requesting-features-or-reporting-bugs-for-illustrator-indesign.html

Not that this helps in the near term, of course.   :(

]

Actually, I almost assume that Fred has tried that already, as I don’t get anywhere special through that weblink, just some kind of InDesign forum:

https://indesign.uservoice.com/forums/601180-adobe-indesign-bugs

[Fred Jones]

At one time we were working with the developers for crossword forge to support crossword puzzles and the Salish or Kootenai languages. Unfortunately the wsuperior was seen as a separate character and was placed into a following square. The solution was to use a grapheme joiner to indicate to the crossword app to include the wsuperior with the preceding character glyphs.

Fred

-- 
You received this message because you are subscribed to a topic in the Google Groups "Ukelele Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ukelele-users/nawHm5BO2vE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ukelele-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/4BA06BD5-B3BC-456D-83C9-C349AB884BA7%40gmail.com.

[W.P.J. Rietbroek]

InDesign gives no indication at all whether HarfBuzz is active or Lipika, the Adobe World-Ready Paragraph Composer. But there is a reliable bug in Lipika that HarfBuzz does not exhibit: the string

aʾ

(a+U+02BE)

is incorrectly converted by InDesign to ẚ (U+1E9A LATIN SMALL LETTER A WITH RIGHT HALF RING). That happens in the fonts that contain all three characters, including Times New Roman.

Harfbuzz correctly leaves aʾ well alone.

That is the litmus test I always perform to test whether a copy of InDesign has HarfBuzz active or not.

HTH,

Pim

--
You received this message because you are subscribed to the Google Groups "Ukelele Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ukelele-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ukelele-users/36920174-B26D-49D0-A972-090DB8441849%40gmail.com.

[Tom Gewecke]

Thanks! My Indesign does seem to have HarfBuzz active.

[Tom Gewecke]

Thanks for that explanation!