Tlex, Flex and Toolbox, Advantages and Disadvantages?

[Mustafā Maryāṇai]

I would like this coversation to be a point where advantages and disadvatages of the three software (Tlex, Flex, Toolbox) are discussed. 

people who have used two of the software (or all of them) can share thier experiences. 

why do you prefer one over the other? 

how does your prefered choice ease your workflow? 

which software is beginner freindly (easy to learn)? 

I will try to post the same question on Tlex's email list (the group seems to be private) and share my findings here.

[Alan Vogel]

Mustafá, I used Toolbox for many years both for interlinearizing texts and for building a dictionary. I liked Toolbox, although I was frustrated with how slow the interlinearizing went, because so many alternative analyses were presented to choose from. I eventually switched to Flex for interlinearizing texts. One thing that I appreciated is that it remembered previous parses and presented these as alternatives. When I have looked at Toolbox since then, it seems that the parsing mechanism has improved. In any case, after some years I also switched to Flex for the dictionary, because it is not possible with Toolbox to generate the kind of file that is necessary to use Dictionary App Builder to build an Android app. It was a lot of work transferring the dictionary from Toolbox to Flex, but I am satisfied with how Flex works both for interlinearizing texts and for the dictionary.

Alan

--
"FLEx list" messages are public. Only members can post.
flex_d...@sil.org
http://groups.google.com/group/flex-list.
---
You received this message because you are subscribed to the Google Groups "FLEx list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flex-list+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/a7e16dcd-7b26-47fe-90f8-c91d2738217cn%40googlegroups.com.

[Jonathan Dailey]

From my experience with Toolbox and FLEx I will make an observation.

Toolbox allows you to do anything without constraints or limits and this can be good. You are not constrained by where a field is located or how it is presented. Ordering can be changed at a whim etc.

Toolbox allows you to do anything without constraints or limits and this can be bad. The data can be entered inconsistently. It can give a lot of options that can be overwhelming.

FLEx does what it is designed to do. I use it and teach it a lot. It is being improved by the team in an ongoing manner. It can import data from Toolbox.

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/319e2ae3-9921-c6de-994c-d555ec8d3024%40sil.org.

[Ron Moe]

I still use Toolbox, but not for dictionaries. I only use it for simple databases that have limited structure. I do not try to print such databases. The print mechanism for Toolbox became so complicated that you could no longer edit it without damaging the change table that transformed the database into a printable dictionary. So other programs were developed specifically to print a dictionary. But the lack of constraints on a Toolbox database meant that the user had to clean up his data before running it through the print program. So FLEx was developed with the goal of forcing the user to use standardized structure and rules for what to do with your data. This standardization made FLEx far more powerful than Toolbox. The program knew what data to expect in each field, how the fields were related, how it all was structured, etc. This made it much easier to develop other tools such as the print process and parsing.

I used to clean up my own and other people's Toolbox data so they could print it. I had to learn to use the Consistent Changes program, which required basic "programming" skills. It was really hard because I'm not a programmer. That whole system is now so archaic that almost no one still uses it. So I will never go back to Toolbox to manage a dictionary.

Occasionally  people complain that FLEx is too restricted. For instance you can't use it to manage a comparative dictionary with multiple languages. But it was never designed for that. Toolbox can handle such things, but only if you know how to structure the data, and only if you are disciplined to put the data in the correct place, and only if you can develop a way to print the data.

Ron Moe

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/CA%2BKRmbYyt4-_Fhidj7eWpKcDiAPxEtLMbbfWC45PD5DJkDupag%40mail.gmail.com.

[KenK]

Hello, a lot of people (especially European linguists) like and use toolbox for creating lexicons, text collections, interlinearizations, etc. It’s very simple, but as others have mentioned, it is unconstrained. I know a few famous linguists who use it extensively, and prefer it. It is fast and simple to use if you use the MDF conventions and follow the rules. However, it can be a lot of work to make corrections. If you make a correction in FLEx, then it is applied globally.

 

All the best, Ken

 

Sent from Mail for Windows

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/CABMU4AyXUwdrfQ1YN-0BX3480viCeFNCJrOffX%2Bv_r0F5Ym8pg%40mail.gmail.com.

 

[Dennis W]

I basically agree with what has been said here, but I want to add a little explanation. As a Toolbox lexical database matures, the user inevitably wants to add more information, such as cross-references to other entries. This results in the same information being manually typed in two different places in the Toolbox database. If the data in either location changes, or is typed incorrectly, the link is broken. These inconsistencies can then be difficult and very time consuming to find and fix. This is just one example, but there are many others, and this is why a more extensive lexical database is more manageable in FLEx than in Toolbox. 

FLEx was designed on a robust database platform that keeps itself consistent, with the foundational intention that each piece of information would exist in just one place in the database. When that one piece of data is changed, the change is reflected in every part of FLEx that refers to that piece of data. For example, an entry in the lexicon area may be linked to other lexical entries as a variant or a cross reference. At the same time, it may be linked to various word analyses in the Texts & Words area, and these word analyses are related to interlinearized texts. When the user gets a better understanding of that lexical entry and adjusts the spelling of the entry or its gloss field, the new information will be immediately reflected in each of those other views of the FLEx data. This is not possible in Toolbox. 

This can be helpful to know so that you can choose your software tools according to your purpose, as Ron and others have suggested. 

Dennis

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/42738716-F99C-4FD7-817A-2CC05C3A2571%40hxcore.ol.

[Mustafā Maryāṇai]

thank you Dennis, your comment truly deserves the best comment award. 

you clearly pointed out a major disadvantage in ToolBox and how Flex is different from it. 

You received this message because you are subscribed to a topic in the Google Groups "FLEx list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/flex-list/LHcddb_ZEfc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to flex-list+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/CANcsFbrmT5XWj_L7PvSt4ivvdiGssC41deiRv8XLS8ppbF83kw%40mail.gmail.com.

[Kari Valkama]

Hi all, 

I would like to add to the conversation that Flex does not either keep data consistent.

In the texts, if I change the division into sentences, the number of paragraphs and the number of sentences changes and thus the references of the example sentences in the Lexicon Entry gets out of sync. I do not know how to repair this, other than doing find examples again and deleting the old examples. 

If one changes the number of senses in the lexicon, Flex can get out of sync. 

For example in the lexicon I have the word allo which has four meanings: 1. day, 2. sunlight, 3. noon, and 4. be exposed to sunlight.  But in the Texts & Words in the Word Analyses there are only three meanings: 1, 2, and 3. Thus, when I am interlinearizing texts, I cannot assign the fourth meaning to a word. I cannot figure out why this is so. 

These are only two examples when making changes to data in Flex causes need for manual synchronisation. 

Yours, 

Kari

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/CANcsFbrmT5XWj_L7PvSt4ivvdiGssC41deiRv8XLS8ppbF83kw%40mail.gmail.com.

[Roland Fumey]

I just want to confirm what Dennis wrote. I typeset two dictionaries that were done in Toolbox. In both cases it had been an enormous task for the linguists to keep the Toolbox databases consistent. In one case I started the typesetting all over after already having spent a lot of time on it as the changes became too complex to handle.

This convinced me to transfer my own Toolbox lexicon to FLEx, which actually can be an enormous task too! So I think it's much better to start with FLEx right-away!

Roland

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/CANcsFbrmT5XWj_L7PvSt4ivvdiGssC41deiRv8XLS8ppbF83kw%40mail.gmail.com.

[Dennis W]

Thank you Kari for pointing out that FLEx does not keep *everything* consistent. This is true, and I have my own examples of it. 

However, I think the underlying data model for FLEx, and the software engine that keeps things *substantially* consistent--these offer great advantages over Toolbox, or a simple spreadsheet solution, when working with a large and complex data set. 

Thanks for sharing your experience and checking my broad description. =) 

Dennis

p.s. Wouldn't it be great to see some reinforcements joining the FLEx dev team to make it even better? 

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/33606CCA-8735-4AEA-9EF3-2AAD74D3AD69%40mac.com.

[Kari Valkama]

Hi Dennis, 

Thanks for your email. 

You said:

"p.s. Wouldn't it be great to see some reinforcements joining the FLEx dev team to make it even better?”

My response: 

Yes, it would be really important. 

It seems to me that the L in SIL has been lost. We are no longer summer institute of Linguistics, we are just Summer Institute. 

I use Flex and hope to be able to write a grammar with its help and also publish a dictionary.

At the moment Flex is excellent if you already know the morphology and grammar, and just enter the data. But if you are in the middle of figuring out the morphology and grammar, making changes to the analysis of the database is a lot of work. 

1. For example, if one changes the part of speech of a word, it is not enough to make the change in the lexicon, one has also to make the change in the interlinearized texts. 

2. Analysing reduplication is tedious. One has to enter the CV pattern manually for each part of speech  like this: 

It is very easy to miss a pattern. 

3. Also, when adding example sentences to a word in the lexicon, it only finds example sentences when you are in the root. And then it finds examples that include complex forms. But when you are in the complex form, it does not find any example sentences. Then you have to manually transfer the examples sentences from the root form to the complex form. This is a lot of work. And you loose the link to the text in the process. 

4. I have not kept careful statistics, but I think in average Flex crashes once a day. 

These are just a few examples. 

SIL should use funds to develop Flex further. 

Yours, 

Kari 

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/CANcsFbpRCGaB2BPtvKSNwyC0pTA25HKLwOXKPngYK9UVMWK%3DhQ%40mail.gmail.com.

[Beth-docs Bryson]

Kari, using funds to pay for developers is certainly one way to do it—many of the software tools in SIL have begun to pay developers, though that hasn’t typically been the model in the past.

I am still hoping there would be “full time supported staff” who would want to join the FLEx development team.  There *are* such folks joining other software efforts; they just haven’t been finding their way to FLEx.  

If you know of good developers who would be candidates to work on FLEx, please do let us know!

-Beth

On Jun 14, 2023, at 11:43 PM, 'Kari Valkama' via FLEx list <flex...@googlegroups.com> wrote:

Hi Dennis, 

Thanks for your email. 

You said:

"p.s. Wouldn't it be great to see some reinforcements joining the FLEx dev team to make it even better?”

My response: 

Yes, it would be really important. 

It seems to me that the L in SIL has been lost. We are no longer summer institute of Linguistics, we are just Summer Institute. 

I use Flex and hope to be able to write a grammar with its help and also publish a dictionary.

At the moment Flex is excellent if you already know the morphology and grammar, and just enter the data. But if you are in the middle of figuring out the morphology and grammar, making changes to the analysis of the database is a lot of work. 

1. For example, if one changes the part of speech of a word, it is not enough to make the change in the lexicon, one has also to make the change in the interlinearized texts. 

2. Analysing reduplication is tedious. One has to enter the CV pattern manually for each part of speech  like this: 

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/D94ED82B-4858-4905-B3DE-C1863E97EACD%40mac.com.

[Kari Valkama]

Hi Beth, 

Thanks for your response. 

I also would like to see full time supported staff to develop Flex further. 

Recruiting is important and here are some questions I have concerning software development in LSDev. 

1. In the past in SIL there used to be a list of available jobs, which included software developers. Is such a list still available? If not, it would be a good thing to have. It is difficult for an ordinary SIL member to recruit other people, if such information is not available from official sources. One needs to know, what programming skills one needs to have, or are UI/HCI/UX experts needed etc.? There seems to be lack of information concerning the need of developers in LSDev. I believe better communication would help recruitment efforts. 

2. It is my understanding that SIL is not as interested as it used to be in putting resources on linguistic software, they prioritize translation software. Is that so, or am I mistaken? If priority is on translation software, the new recruits will be invited to work on translation software projects. 

3. In the past, if a Mac developer wanted to join LSDev, that person was told that LSDev does not do Mac programs. Is that still the case? Unfolding Word has a program called TranslationCore, which is multi-platform. I believe they have less resources for development than LSDev, yet they have been able to do a multi-platform program. It would be nice, if LSDev were interested in doing Mac programs. Potential Mac programmers would like to work on Mac software. Also, usually Mac programmers are good with UI. 

4. What kind of features would those new full time supported staff be working on? In the past I have heard ideas about adding new features to Flex. From my perspective it is better to fix the existing code. For example adding AI or something else to Flex would be nice, but only after the basic functionality and UI has been improved and the bugs repaired. Who decides what is going to be done? 

Yours, 

Kari 

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/6DB874DA-4E4F-4CA9-800F-FF39817E87FD%40sil.org.

[Kent Rasmussen]

I find this comment interesting:

On 6/21/23 02:40, 'Kari Valkama' via FLEx list wrote:

In the past I have heard ideas about adding new features to Flex. From my perspective it is better to fix the existing code. For example adding AI or something else to Flex would be nice, but only after the basic functionality and UI has been improved and the bugs repaired. Who decides what is going to be done? 

One of the first signs of a dying project, I understand, is the lack of ongoing development. That is, if one is just bug fixing, that is one foot in the coffin. Not that projects cannot continue in a "maintenance phase" for years, but that would seem to typically not be the case. So I'm not sure that the tradeoff between new features and bug fixing is a real thing; if we aren't doing both, we will soon be doing neither.

On the other hand, I am still waiting for key functionality for FLEx to be useful to my work.  I have supported FLEx as our corporate flagship software product, through formal and informal training, despite the fact that I never have gotten it to work for me (having made multiple serious attempts over multiple decades). That is, it seems to assume an established orthography, which I almost never have. So when we talk about it being a linguistics tool, I think we mean it is a grammar/text study tool —for S linguistics, not P linguistics. As a phonologist, I need to be able record primary phonetic data (including contrastive pitch) in a way I can store for posterity and use in analysis. Texts&Words may do that for strictly segmental data (even in IPA with diacritics), but there is no good way to record, store, or analyze data for the 90%+ of African languages which have tone. So until it moves beyond being a decent (buggy or not) tool for syntax and discourse analysis on languages with established orthographies, I will continue to recommend and support it for others.

Grace,

--
Kent Rasmussen, Ph.D.
SIL Linguistics Consultant / Conseiller en Linguistique de SIL
Orthographies for Francophone Africa / Orthographes pour l'Afrique francophone
Text/voice in Cameroon: +237 682417131
Text/voicemail/Signal: +1 541-357-7276

[Beth-docs Bryson]

I really shouldn’t have posted to the whole list.  It would be better to take this conversation off-list.

I’ll respond to just a couple things:

1. SIL is definitely interested in linguistic software, not just translation software.  The thing that is different is the funding landscape, but the priorities of SIL have not changed.

2. At the moment the priority for FLEx is fixing bugs and doing things that would have high impact for users, within constraints of keeping installation straightforward and maintaining compatibility between FLEx projects across new versions.

3. For the future, there is a plan to redo the architecture, with one result being that cross-platform would basically “come for free” with the new tech stack.  But that will be a significant undertaking.

4. At the time of such a rearchitecture, that will be a good time to revisit the model and explore ways to work with text data with phonetic representations, as Kent was discussing.  

5. There are plans for other new features, but certainly the area that Kent brought up is one clear significant area for advances.

-Beth

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/BC520346-652C-4926-A8CD-EB8BE41CBAD0%40mac.com.

[Kari Valkama]

Hi Kent, 

Thanks for you response. 

I agree with you that new features, in the sense you wrote, are important. I think I was unclear in my message. With new features I meant AI and things like that. At the moment Flex cannot find example sentences for words with suffixation, only for the roots. Fixing that is in my mind fixing a bug, not a new feature. Also repairing the issue of expecting an established orthography, is in my mind fixing a bug, not adding a new feature. But I understand that I may be inaccurate and they should be called new features. 

What I was trying to say is, that these basic features should be fixed/done before developing  some new features that require AI for example. 

What Beth wrote is encouraging:

3. For the future, there is a plan to redo the architecture, with one result being that cross-platform would basically “come for free” with the new tech stack. But that will be a significant undertaking.

About the philosophical issue of whether maintenance mode is a sign of a dying project, I have no firm opinion, just observations. There is a program called Pcalc, which is a calculator, that is may be 30 years old. It is debatable, whether that has been in maintenance mode or not. 

Also Shoebox/Toolbox has been in maintenance mode for a long time, and I assume that it is doing fine. I do no use it, so I have no first hand knowledge.

Yours, 

Kari

--
"FLEx list" messages are public. Only members can post.
flex_d...@sil.org
http://groups.google.com/group/flex-list.
---
You received this message because you are subscribed to the Google Groups "FLEx list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flex-list+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/flex-list/9b487742-fda3-673d-e37d-7279187bedac%40sil.org.