Presence of ZWJ doesn't cause distinct word

[B Kh]

In Sinhala there are some combinations of letters that always require the presence of a Zero Width Joiner to make them considered correct spelling. However there are other words where the presence or absence of a ZWJ is just considered an alternate spelling. 

The problem is that FLEx doesn't seem to recognize them as distinct forms, so they end up with erroneous homograph numbers. For example රක්ෂාව is the common form in every day writing, but the "proper" form in formal writing is රක්‍ෂාව. So I need to include both as this is  a learner's dictionary.

I have added both ZWJ and ZWNJ to the character list:

It's not a disaster, but it's confusing to the user. Is there a way to get Flex to fully treat the the ZWJ as a real letter character in the dictionary listing?

The complication is, unfortunately, that the ZWJ should be ignored (as it is currently) when it comes to searching. For example we see this when searching:

This is the correct behaviour for search because someone searching for one word form would always want to find the other form if it existed.

Basically the presence or absence of a ZWJ is a cosmetic difference but an important one. Hope that makes sense.

[David Rowe]

Have you included the ZWJ in your collation (sort order)? There should be options to allow you to ignore the ZWJ for purposes of collation. I don't know if this would address your issue.

--
"FLEx list" messages are public. Only members can post.
flex_d...@sil.org
http://groups.google.com/group/flex-list.
---
You received this message because you are subscribed to the Google Groups "FLEx list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flex-list+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/flex-list/485b6b11-f27f-4375-8ec8-a73bfef33968n%40googlegroups.com.

[B Kh]

I'm using the default search order. I tried doing a custom order previously (for different reasons) and couldn't get it to work. Quite complicated. So I'm hoping that isn't the fix for this issue.

[David Rowe]

Circling back to this in the new year. 

Have you resolved the problem (or found someone else to look at it)? If not, would you be willing to share your project with me?

Thanks,
David Rowe

To view this discussion visit https://groups.google.com/d/msgid/flex-list/bee952d8-2ef4-48c1-b495-bcd56f074d78n%40googlegroups.com.

[B Kh]

Thank you so much for checking back in.

I wasn't able to reproduce the problem consistently. So perhaps you could consider this solved for now and I'll come back and ask again if I can get it breaking more consistently.