Re: [FLEx] Spelling checker in Flex

[Bart Eenkhoorn]

Hello Afsar Ali,

The fieldworks help text has a lot of insights:

Basically, if your language does not already have a hunspell dictionary you build it one by one in flex by adding words. If you have a wordlist that is correct spelling you can create a .dic file based on those words as a start.

I wrote some advice on how to do this and distribute it among other users of the project. (see belows)

Once you have thus hunfiles set up, you can also use these with LibreOffice products and Indesign for example. If you want to create rules and build the .aff file as well, you may find this video helpfull  https://youtu.be/iLwuZi6Beeo?si=FyJPAcNgs-VXi6oc

I hope you find this helpfull.

Bart.

============================

Workflow :  vernacular spelling with hunspell in FLEX for multi-user projects

In addition to what is written in the FLEX documentation concerning the use of hunspell vernacular spelling, there are several considerations to take into account (see below) especially for a multi-user projects.

Firstly, if for a project a new orthography writing system was created, ie abc-x-ortho as in the example below, (abc = ISO language code)

even though abc-x-orth.dic will be created, in practice the abc.dic and abc.exc files will be used by FLEX. (I do not know if this is a missing feature or a bug ...). 

If your project language is not one of the major languages for which hunspell dictionary files already exist, the vernacular abc.dic file will be empty and all lines of abc text will have red lines underneath indicating erroneous or unknown spelling status. Once the .dic files becomes populated or if you have chosen to populate it with lists of known correct spelling, red lines will become less frequent and correct alternatives are offered more and more:

If in the example Zifahaan is correct and is added, it will be added to abc.exc (not to abc-x-ortho.exc). The next time a similar word is found, for instance Zzifahan, the alternatives Zifahan from abc.dic and Zifahaan from abc.exc will be offered.

  

So far so good. Thus, the .exc file will keep growing while you are adding words. It is important to note that the send/receive process does not merge or distribute these files among users. It is up to the users to do this manually and organise the distribution themselves.

How to merge, eliminate double words, and distribute .dic and .exc hunspell files.

When a checked wordlist already exists it is advantageous to create the vernacular abc.dic file based on such a wordlist. This could be the case if you have access to a paratext project in the same language and if in that project the wordlist has been checked. In that case the file spellingstatus.xml can become a basis for your abc.dic file. Copy and rename the file to a different location and with the help of your favourite editor, isolate the correct words with status “R” as well as words between <correction> </Correction>. Words marked "W" are to be disregarded.

To do the merge, collect all the abc.exc files as well as the abc.dic files from all the users of the FLEX project. (the are located at

) for each user

Copy all abc.exc entries of all received files into one file, sort the lines alphabetically and with your favourite editor apply the following regex:

^(.*)(\n\1)+$

replace with $1, thus eliminating any double entries. The regex will only work if double entries are adjacent. So make sure to sort before eliminating double entries. For further information on how to do this with for instance vscode, see this reference :

https://stackoverflow.com/questions/37992493/how-can-i-remove-duplicate-lines-in-visual-studio-code

With the double entries eliminated, locate the last line and note the line number (11869 in the example).

Add the number you have found before the first entry in the file -- probably not 11869 ;-)

and save the .exc file.

Do the same for the abc.dic files you have received from project users. Optionally, the abc.exc file could be incorporated into abc.dic, thus starting with empy .exc files for all users after re-distribution.

The last step is to send the new .dic and .exc files (two files per user) to all users and ask them to store these (overwrite the old files) in their respective

file locations.

Since the AppData directory is normally hidden in windows, some users may need additional guidance to copy these files.

For users to profit from each users’ spelling additions this process should be regularly repeated.

Please also note that for languages like English or Spanish, for which hunspell dictionaries already exist, additions (exceptions) will also need to be merged between users of a multi-user project.

# # # # #

On 13/02/2025 07:16:39, Afsar Ali <afsar...@gmail.com> wrote:

Hello everyone,

I want to know how to build a spell checker in flex. Deos anyone have any idea about this? especially for the right to left perso arabic script.

 

Your help would be appreciated.

 

Kind regards,

Afsar

Virus-free.www.avast.com