Fieldworks Writing System setup - Valid Characters - What does 'From Data' do?

[David Rowbory]

I'm trying to specify valid characters in a vernacular writing system and saw the 'From Data' tab, which has the attractive 'Scan' button. I hoped this would allow me to specify a file containing valid characters but the dialog looks for .lds/.lng language files and I don't know what that format is. I tried using some .lds files I found (from Paratext) but these didn't work as it told me it couldn't find any characters in them. I also tried plain text (UTF-8) files with only the characters I wanted to add. That was also rejected.

The Help at this point simply says
> 1.In the Valid Characters dialog box, click the From Data tab.
> 2.Click the Scan button, and then click an option.
> •If you clicked File, in the Browse for Language File dialog box, select the language file which contains the characters you want to add, and then click Open.
> The upper-left pane displays characters, character codes, counts and check boxes in a table.....

But it doesn't explain what the Language File format is. Has anyone used this feature? Do you know what the language file format is? What is actually supposed to happen here?

[Hasso Pape]

Another dialog lists *.lds as a Paratext format and *.lng as a Toolbox format, so the .lds file should work. Could you open your .lds file with a text editor and check whether the characters you want are there? It is also possible that the Paratext .lds format has changed and FieldWorks hasn't been updated.

[David Rowbory]

I actually hand-crafted the .lds based on another language.lds file to include the characters I wanted so I know they are there, but suspect that something about the file isn't what Fieldworks wants if indeed Fieldworks knows what to do with it. Another reply off this list suggests that the whole Valid Characters area doesn't have much use any more for anything.

[Ken Zook]

In FieldWorks 8.3.12, the file to scan could be any UTF-8 text file. You could access this by choosing All Files (*.*) in the Browse for Language File dialog. Unfortunately, this functionality got lost in FW9. In FW9.1.18 it says no characters are in a text file. Even when giving it a Toolbox .lng file, it says No characters are available, and it won’t read an ldml file from Flex or Paratext. So it seems to be totally broken at this point.

 

The Valid Characters dialog is in bad need of replacement so that it works properly for modern Flex versions. The current dialog was designed for Translation Editor which no longer exists, but we still have limitations from that implementation. In general you can use Manual Entry Single Character or Unicode value to get characters into valid characters, but it will not accept any upper planes of Unicode, and you can’t include any digraphs,  or characters with overstriking diacritics, etc.

 

The Valid Character dialog has very little purpose in Flex. It fills in the exemplarCharacters element in the language ldml file, and it also loads from that file. It would be nice to be able to define exemplarCharacters accurately. But the only time you really need something in valid characters in Flex is if you have interlinear text that contains some code points that are not assigned as a Unicode letters (e.g., hyphen or apostrophe). By default, interlinear text will break these words on the non-letter code points. But if those letters are in Valid Characters, then it will treat them as word-forming characters. If you need some character for this purpose, but cannot get it into the Valid Characters dialog, the other option is to edit the ldml file directly and put those code points in the exemplarCharacters element. Flex will load these code points and use them as word-forming in interlinear baselines.

 

Ken

--
"FLEx list" messages are public. Only members can post.
flex_d...@sil.org
http://groups.google.com/group/flex-list.
---
You received this message because you are subscribed to the Google Groups "FLEx list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flex-list+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/10139d93-01f9-4002-9396-756621b1c1dan%40googlegroups.com.