Notepad with Dictionary and Spell Check.

[Paul Bowers]

On the theme I have off pushing SailForms to see how far it will go, I have already determined that functionality wise it can be pushed and poked to delivery application level functionality - see my Programmable Calculator which has morphed into a IDE and Runtime with for variant of BASIC and inline Assembler. I have continued to extend it - with structured file handling, recursive searching and pop up keypads on INPUT etc. I will aim to drop a new release soon.

So now I wanted to just how many records it could handle and still remain responsive and stable. I have projects with 60,000 or so records in - I have also found images bloat the DB - so I avoid importing images and now reference them externally using URI intents - which works quite well.

N

So here's a project thats just mad. I have imported 466,500 records (words in the English language). - Yep really. and wrapped it with a notepad app, with word search and spell check - just for the sake of it.

Importing that volume of records does take time, but once its in its rather wizzy. Another impressive points is even though there is nearly half a million records it compresses to just over 3MB. 

Feel free to play - restore takes a few mins - so be patient it will get there.

A couple of challenges to over come with this project was the search command in Sailforms terminate the command if there is not found. To work around this I created a process called Search which uses a query with a group variable filter and a set found flag is the command finds a match. YOu set it to not found before to call the process. Then by nesting this process you have a search that return true or false.

Enjoy, and feel free to mail if you have any questions.

Paul.

[Paul Bowers]

some screen shots

[Paul Bowers]

some fixes and updates -

a) resequencing of note index following delete of a note - so that search in note will work - as native filter and search does not support "contains" or "like" I simulate this using the string INDEXOF command. This can be done in foreground by manually cycling through the records using search on a record number - or in a process setting a match flag on each record and then selecting in a remote table on the match flag.

b) fix to spell check to resolve all words to lower case in the word table and search patten as SailForms is case sensitive on the Search form command (but not in the List mode spyglass  search.)

On Monday, 2 October 2023 at 17:25:57 UTC+1 Paul Bowers wrote:

[davidche...@googlemail.com]

Tried your notepad and the spell check does not work does not indicate the word that is incorrect and when you press spell check then correct it shows you the word but does not let you replace that word even if you select it. 

[Paul Bowers]

David

thanks for the spot on the issue with case matching in spell check - this resulted from having to lower case all the words in the word table as search is case sensitive - which meant I also had to lower all words in the notepad to match them. Consequently when it came to tag the word as in error using replace it couldnt find it as it too is case sensitive. TO resolve this I save the original word to match against. and all look OK now.

[Paul Bowers]

David

I cant see any error in the correct screen functions - what device are you using and android version -

Ill test on my old Android 7 tablet with 1gb to see how it performs on that 

On Wednesday, 4 October 2023 at 06:25:39 UTC+1 davidche...@googlemail.com wrote:

[Paul Bowers]

I am being asked how I get procedures  and functions to work in foreground without having to call processes lists

I use a feature call set focus recursion. I have a field where all the processing is contained on the 'when gets focus'

to execute I have a typically a button which does some initialisation of variables and then sets focus to this field with the processing I want to do.

The first thing this process does it set focus toa dummy field with nothing on it. I then to the processing I want. To make to loop and do it again I set focus to itself - the last command is always a stop,

Don't forget to code an exit statement when you have done  - otherwise it will loop forever.

IN order for set focus to work the fields cant be set to hidden. So the trick if you don't want the user to see them is place them off the visible screen to the right

Set focus will still be able to find them and execute them - but the user doesn't see them flashing as they get focus

On Wednesday, 4 October 2023 at 06:25:39 UTC+1 davidche...@googlemail.com wrote:

[Paul Bowers]

I may have identified an issue with SailForms on machines with low ram - I am testing on an old android 7 machine with just 1GB of ram. it looks like when Sail Forms  runs out of heap space it just stops the current process. So for example when you press Spell Check nothing appears to happen. IF I flush memory by clearing all the other running apps - it then works.

I will see if I can optimise down or find a solution. ideas welcome......

Paul.

On Wednesday, 4 October 2023 at 06:25:39 UTC+1 davidche...@googlemail.com wrote:

[Paul Bowers]

I have confirmed - and managed to recreate a failure in SailForms on machines with low 1GB ram - when it runs out of memory its doesnt give any error, but just stops at the command in flight. By clearing any other apps open on the device - then this runs as it should do. o to my original challenge - how far can we take Sailforms - we now know that on a 1GB machine a table with half a million records runs it close to the memory limit.  Newer machines with more RAM and a later version of Android with improve virtual memory are OK.

Paul.

[Paul Bowers]

i have an idea for reducing the overhead of a single table with the 467K entries and the subsequent cache overhead it requires - device memory trace usage shows that SailForms is peaking at between 87 and 90MB heap usage. - I am going to play with splitting the table up into smaller sections - say A-K and L-Z - assuming at at what ever point around half the records is. This should I hope enable the table cache to be swapped in and out more effectively - it will need to enhancement to the search function to check which table to look at based on the first letter of the word. - and the word browser will need a tweak too.  But as a test to see how SailForms manages and if it reduces memory load on devices with a smaller RAM capacity.

[davidche...@googlemail.com]

What you need to do also is add a button called Delete Spell Check History and use run process on spell check table and delete list entries eith remote reversals and this would clear the spell check list which can get very large 

[Paul Bowers]

Well, this has been an interesting exercise. Splitting the table of 467,000 words into two didn't reduce the overall memory overhead - as we still had to cache the same total number across multiple tables. Infact it did increase the overhead. It did however, have a performance improvement for search on the low powered low RAM device - as the application only had to search smaller tables to locate a word. I did have to introduce and A-Z index to let the SEARCH process know which table to look in - based on the first letter of the word. An thus make the multiple tables transparent to the rest of the app. But on a high performance smart phone or tablet - you cant see any difference. 

During my experimentation I did discover some behaviours with how SailForms processes Group Variables. They are defined and initialised on group startup - which is fine normally. IF however, you wanted to jump to a group from another group - then this does not happen. Any group variables you want inthe child group need to be defined in the parent group. You can prove this by looking at the content of the variables and see which ones got changed. In this instance - the app failed as it thought the variables weren't defined until I added them to the parent group. Not directly related but a similar consequence of running out of memory. 

Therefore, as I saw no real benefit from the split version - ill just archive that away and not build on it further. 

I have taken some of the learnings back into the original main project version and optimised by removing unneeded variables and fields etc, and fixed some buglets etc.

New version attached, remember if running on a machine with low capacity RAM (1GB) then be sure to close all other open apps to ensure you free up as much memory as possible. On my old Android 7 tablet with 1GB RAM this version behaves itself when nothing else open.

[davidche...@googlemail.com]

Basically to make back up faster I loaded up two versions of the database in one I deleted everything then I moved every form except words to the empty database. The I changed both groups names. Backed both databases then I deleted both of them then I installed both of them again and the notepad worked with spell check.  Now you can back up everything quickly as you don't have to back up database with words in it. 

[Paul Bowers]

Hi David, I am glad you find a method that works for you. It's a technique

 I have also used to integrate the dictionary into other projects

Without the over head of the dictionary on each project.

It also means I can vary the note form for each project whist l whilst

Maintain the functionality I want.

The latest attached version I have managed to compress down to 1.9MB

And is quite stable on my old 1MB 1Ghz tablet. And fly's on my new stuff.

Regards

Paul

[Paul Bowers]

Final version I think - removed redundant processes and optimised the code around the new search method and fixed a couple of bugs.

On Wednesday, 11 October 2023 at 19:24:36 UTC+1 Paul Bowers wrote:

following my discovery of a method to do a search form using remote count field I have replaced all the Process LIST functions to do the spell check search with Remote count field - its quicker and slicker. 

[Paul Bowers]

updated and final version - removed redundant process and optimised code around new search method and a few bug fixes

Paul

[Paul Bowers]

Here is an update to the notepad app - I think volume wise I have pushed it as far as sensible - the app now has 1.2 million records

these are 466,000 words to drive the spell check, 199,000 definitions (dictionary) and 548,000 thesaurus entries.

The search dictionary screen lets you search for an work and it lists absolute matches in the dictionary and thesaurus  - as sailforms only has 

capability to to "contains" and "islike" search on the list form then there are buttons to that and use the spyglass search there - this also searches for all matches in the definition and terms - which is quite good - great for cross words I am finding.

the entire app just squeezes under 24MB. 

i would if you are wanting to create backups of your notes in the app you put that form in a separate group to keep the back up speedy - as suggested to me - thanks.

the updated app is in the next post as its just a tad to large to post in one with the screen shots

[Paul Bowers]

https://drive.google.com/file/d/1LrChTUkYROMDfYdz7bcNWAGD8HUiSOiz/view?usp=drive_link

use the download option to get the entire sfg file.

There is a max app size limit on this forum of 24MB - so here's a link to the file in my GoogleDrive

enjoy

Paul.