Question: Incorrect storage/display of FontAwesome characters after unordered lists

[Frederico Jeronimo]

Hi all,

This is a bit of an odd one. I've recently installed Font Awesome's (http://fontawesome.io/) TrueType font so I can use its custom icons in my Todolist Richtext entries.

Everything looked fine as I was typing or copy pasting, but then things got weird as soon as an unordered list was used and some of the unicode characters didn't get saved/displayed properly once the task/file was reloaded.

The attached screenshot (Broken_Unicode_Chars_After_Unordered_Lists) exemplifies the problem with 2 characters that got broken and two that remained fine, alongside with their unicode representation in parenthesis.

I confirmed that the tdl encoding was unicode (utf-16) and that this behaviour also happens on a new todolist and on the introduction.tdl.

The broken characters seem to have their font automatically replaced by 'Symbols' instead of 'FontAwesome'. If I change it to 'FontAwesome' again, they render fine but then they get transformed to a different character altogether when reloading (Character_Transformation)...

Thanks in advance for the help.

Kind regards,

Frederico

[.dan.g.]

Welcome Frederico and thanks for your detailed report.

Can you also copy the content displayed in the screenshots to the 'Introduction.tdl' file and post it here so I can experiment with it?

[Frederico Jeronimo]

Hi Dan,

Thanks for the quick reply. It's a pleasure to join the community (I've been using Todolist for many years, it's a great tool).

Here's the Introduction.tdl with the requested example (in the task named Ordered lists and FontAwesome characters example).

You will also need the FontAwesome Truetype font, which I've also attached for convenience.

Finally, I've upgraded and tested with the latest stable Todolist (7.1.4.2) and the issue still happens.

Best regards,

Frederico

[.dan.g.]

Many thanks for the supporting data Frederico.

As a bit of background, ToDoList uses the same RichText control as Windows' WordPad so my first test is always to see how WordPad handles things. 

And unfortunately WordPad seem to have the same problem at least as far as that test went.

What I did:

1. Copied the base 'good' content from the tasklist you supplied.

2. Pasted it into WordPad 3 times

3. Left the first instance as-is

4. Formatted the second instance as an ordered list

5. Formatted the third as an unordered list

6. Saved the file as Introduction.rtf

7. Closed WordPad

8. Re-opened WordPad

9. Re-opened Introduction.rtf

--> Exactly the same outcome as the supplied tasklist.

Some ideas:

1. Experiment a bit further with WordPad and see if you can find a way around it which we can also apply to TDL.

2. Posting a bug report to FontAwesome's forum to see if this might be the fault of the font.

3. Spelunk the RTF itself to try to make sense of how the Unicode characters are being stored

4. Experiment with every single character in the font to see if there is a pattern (is there a page where every character is displayed?)

Dan

[.dan.g.]

Hi Frederico

I found a page showing every icon in the font file and so I:

1. Copied the content to WordPad10 and saved it as 'FontAwesome_WordPad10_Original.rtf' 

2. Loaded 'FontAwesome_WordPad10_Original.rtf' into Word2007 and saved it as 'FontAwesome_Word2007.docx'

3. Loaded 'FontAwesome_Word2007.docx' into Word2007 and saved it as 'FontAwesome_Word2007.rtf'

4. Loaded 'FontAwesome_Word2007.rtf' into WordPad10 and saved it as 'FontAwesome_WordPad10_From_Word2007_RTF.rtf'

Then I tried loading these same files (attached as a zip) back into WordPad10 with the following results:

1. Reloading 'FontAwesome_WordPad10_Original.rtf' causes some icons to display incorrectly

2. All the other files displayed correctly

The next step (after some head-scratching) was to compare 'FontAwesome_WordPad10_Original.rtf' with 'FontAwesome_WordPad10_From_Word2007_RTF.rtf'

And this reveals the problem: For reasons unknown (at present) WordPad10 fails to save the correct unicode values for some characters. But once Word2007 corrects the problem, WordPad10 has no problems from then on.

Then I tried to see if this could also produce a positive outcome for your specific data but unfortunately not. But tomorrow is another day as they say!

[.dan.g.]

Hi again

I took the raw data file I created before and tabulated it, sorted it by Unicode value and then applied 'Unordered list' to the first column.

Then I loaded that file (attached) into WordPad10. If you do the same you will see that the entire range F021-F0FE gets substituted and the substitution seems to come verbatim from the 'Symbol' font as show below:

My suggestion would be to forward this file to FontAwesome and see what they have to say. I suspect they will blame WordPad10 which is probably fair, so then you could try submitting a bug report to Windows...

On Sunday, 10 September 2017 17:58:46 UTC+10, Frederico Jeronimo wrote:

[Frederico Jeronimo]

Hi Dan,

Thank you for your investigation and for the background information. Things make more sense now. :)

I will report this as you suggest, but in the meantime I found a workaround.

I've imported the FontAwesome TrueType font in the IcoMoon App (https://icomoon.io/), moved the glyph codes in the affected range (F021-F0FE) to a new range (F621-F6FE) and saved the updated font to a new TTF (FontAwesome-icomoon.ttf).

It's not ideal (e.g. I will have to repeat the operation whenever a FontAwesome update is released, I cannot use the cheatsheet reference directly for those icons, etc.) but it works in Todolist (and Wordpad) in all tested scenarios (including unordered lists).

Thanks again,

Frederico

[.dan.g.]

I'm glad you figured out a workaround of sorts and thanks for supplying links to the required tools.