Reviving TiddlyFox for Firefox > 57 (and Chrome)

[Riz]

Tiddlyfox addon for firefox browser stopped working for post 57 versions of firefox when mozilla switched over to sandbox model, restricting access to the file system. However webextension addons provide APIs which interface with filesystem. This is an addon using native messaging, essentially handing over the contents to a webextension host which does the actual saving. For the end user it means a couple of extra steps to install the addon but once installed, you can save the standalone TW from anywhere in your harddrive without any more interactions, like the original Tiddlyfox addon did.

The set up is currently complete for Linux and Mac users. Windows workflow is slightly different and since I do not have a windows machine to test, I cannot provide assistance. However core of the application remains the same, and the work to port it for chrome browser and windows platform would be minimal. The principle followed can be used to develop addons for Edge browser too, for those so inclined.

 Requirements:

* Firefox >57
* Python - usually installed by default in linux systems

 Installation:

#### Linux

Debian derivatives(Debian, Ubuntu, Linux Mint etc), Arch Derivatives (Arch, Antergos, Manjaro)

* Step 1: Run

sh -c "$(wget https://raw.githubusercontent.com/ibnishak/Timimi/master/InstallScript/linux.sh -O -)"

* Step 2 : In your home folder you will find timimi.xpi. Drag and drop it to Firefox addon manager page (about:addons).



If you do not have git and wget installed in your system, or if the install script is throwing errors, follow the steps given below.

* Download the repo from https://github.com/ibnishak/Timimi
* Unzip the downloaded file and open the folder.
* Open native-messaging-hosts/timimi.json and change the name `richie` in the path key to your username.  You can find your username by typing echo $USER in your terminal
* Copy native-messaging-hosts folder to ~/.mozilla. If you already have a native-messaging-hosts folder in ~/.mozilla, you can safely merge them.
* In Timimi/addons/web-ext-artifacts you will find timimi.xpi. Drag and drop it to Firefox addon manager page (about:addons).


### Mac - untested

* Download the repo from **https://github.com/ibnishak/Timimi**
* Unzip the downloaded file and open the folder.
* Open **native-messaging-hosts/timimi.json** and change the path to **~/Library/Application Support/Mozilla/NativeMessagingHosts/timimi.py**
* From **native-messaging-hosts** folder, copy timimi.json and timimi.py to **~/Library/Application Support/Mozilla/NativeMessagingHosts/**
* In **Timimi/addons/web-ext-artifacts** you will find **timimi.xpi**. Drag and drop it to Firefox addon manager page (about:addons).



#### Windows:
Need more time.



### Status

1. Presently it is a barebones addon, as in it just does one thing - saving. Other features of old tiddlyfox plugin, like disabling, showing save status etc can be easily added. I leave it to those who are interested and familiar with Tfox code.

2. The webextension host is currently written in python under the assumption that linux systems usually have it installed by default. It is possible to write the same in nodejs or other language for different platforms to minimize number of installations.


### Pros

1. Webextension host can be expanded for different purposes depending on personal preferences. For eg:
  * Saving time stamped backups instead of overwrting.
  * Monitor changes to a tiddler and enter it to a local relational database like sqlite.

2. It can be easily ported over to chrome
3. No need to run a server, does not require continued user interactions

### Cons
1. Cannot be used in android

[TonyM]

Riz,

Thats very exciting. Sounds like we are almost on the side of firefox appocolypse. It must be said that it has however spawned much innovation.

Sadly I am a windows user so I may need to wait a little, unless I can be given the info to help or test. I am a techie on windows.

Thanks
Tony

[BurningTreeC]

Awesome Riz, thanks SO MUCH!

Saving works great on FireFox under Linux!

This is huge :)

[Riz]

Hi Tony

Yeah I ll get to that soon. It is just a couple of steps according to MDN. Will outline the steps as soon as I get back to my computer.

[Riz]

Hi BTC

Thanks. If you do not mind, may I know which OS specifically and which version of firefox?

Sincerely
Riz

Make Tiddlyfox Save Again

[BurningTreeC]

Here you go: 

Arch Linux, kernel 4.17.14

FireFox 61.0.2 (64-bit)   - from the arch linux repositories (extra)

[TonyM]

Riz,

Edited: to attache file

I followed the above instructions, and created a .reg which can add the registry entry. I have bundled the part into a zip file attached.

I also installed Python from  https://www.python.org/ using the Windows x86-64 web-based installer

If extracted to C:\ it will create the correct folder matching the settings entered in the files - from where the xpi can also be dragged

C:\TiddlyWiki\timimihost

Of course you did not document the "use of the addon so I did this

Using a new TiddlyWiki from empty.html

• to copy to C:\Data\TW5\Development\Test\NewWiki.html
  
• then used C:/Data/TW5/Development/Test/NewWiki.html in the browser (firsfox) address bar to open
  
• I then customised it
  
• added some tiddlers 
  
• and it autosaves after creating new tiddler, editing or renaming it. with the Wiki Saved message
  

Unfortunately doing a reload all the changes are missing, but it "looks like" its working.

Is there another step, as in with TiddlyFox we needed to indicate to allow the save?

If this is how it should work, how may I debug it?

I checked I have the following in the "path"

C:\Users\antho\AppData\Local\Programs\Python\Python37\Scripts\;C:\Users\antho\AppData\Local\Programs\Python\Python37\;

I added C:\TiddlyWiki\timimihost to the path with no success.

I reloaded firefox in case

Thanks for this and any future assistance

Tony

On Monday, 13 August 2018 04:36:20 UTC+10, Riz wrote:

Here is the possible steps for windows. Untested, of course.

1. create a registry key with the following name:

HKEY_CURRENT_USER\SOFTWARE\Mozilla\NativeMessagingHosts\timimi



2. Add a folder (any convienient folder) as the value to the key

3. Copy timimi.json and timimi.py from timimi/native-messaging-hosts to the path you set above

4. In the same folder, create a timimi.bat file with the content

@echo off
python -u "c:\path\to\folder\timimi.py"


5. Now open the timimi.json and change the path to "C:\path\to\timimi.bat"

6. Finally drag and drop the xpi from timimi/addon/web-artifacts


Possibly a batch file could be made to make the whole process smoother if these instructions actually works. let me know

Reference: Instructions are based on MDN Link 1 and Link 2

[Riz]

Once you open firefox, open browser console using ctrl+shift+j. Then load the local tiddlywiki and try to save. Send me a screenshot of the errors in browser console.

[TonyM]

Done

Not a local Tiddlywiki file background.js:10:5
we are insidefile:///C:/Data/TW5/Development/Test/NewWiki.html logsimple.js:15:3
Initiate content-script.js:41:9
sendTimingEvent addSite start undefined undefined verboseLoggerHooks.js:1:1120
sendTimingEvent addSite start undefined undefined verboseLoggerHooks.js:1:1120
saver-handler: Saving wiki with method autosave through saver tiddlyfox $:/core/modules/utils/logger.js:33:10
Sending native message background.js:23:5
Win error 5 during operation open on file C:\TiddlyWiki\timimihost (Access is denied. ) NativeMessaging.jsm:99
Error: Error: Attempt to postMessage on disconnected port background.js:19:5
Content Security Policy: Ignoring ‘x-frame-options’ because of ‘frame-ancestors’ directive.
Saved successfully to C:\Data\TW5\Development\Test\NewWiki.html content-script.js:58:17
Message ID is tiddlywiki-save-file-response-1 content-script.js:59:17

I do have Bob and TiddlyServer running in other tabs, but this is between a reload, create tiddler and autosave.

Regards

Tony

[Riz]

This seems to be a permission problem. You have followed the steps correctly as far as I can see. Can you try to redo the steps but changing the location to a folder in your documents folder or desktop folder?
I linux I would have said the problem is we havent allowed the script to be executable as a program, or that the location by its virtue requires root permissions. I do not think either of that applies to windows.

sincerely
Riz

[TonyM]

Will Do,

Actually there was some hints this may be the case. As windows strengthens its security.

Regards

Tony 

[TonyM]

Ok,

I update the file path in the files and registry entry and put them in C:\Users\antho\Documents\timimihost and loaded C:\Data\TW5\Development\Test\TiddlyFox2nd.html

A second attempt also showed 

Win error 5 during operation open on file C:\Users\antho\Documents\timimihost (Access is denied. ) NativeMessaging.jsm:99 

I wonder why it is saying "not a Not a local Tiddlywiki file"

Should I be concerned about this in the python file? 

#!/usr/bin/env python

The Browser console shows

 we are insidefile:///C:/Data/TW5/Development/Test/TiddlyFox2nd.html logsimple.js:15:3

Not a local Tiddlywiki file background.js:10:5

Initiate content-script.js:41:9
sendTimingEvent addSite start undefined undefined verboseLoggerHooks.js:1:1120

syncer-browser-tiddlyweb: Retrieving skinny tiddler list $:/core/modules/utils/logger.js:33:10
syncer-browser-tiddlyweb: Retrieving skinny tiddler list $:/core/modules/utils/logger.js:33:10
sendTimingEvent addSite start undefined undefined verboseLoggerHooks.js:1:1120
saver-handler: Saving wiki with method save through saver tiddlyfox $:/core/modules/utils/logger.js:33:10

Sending native message background.js:23:5

Win error 5 during operation open on file C:\Users\antho\Documents\timimihost (Access is denied. ) NativeMessaging.jsm:99
Saved successfully to C:\Data\TW5\Development\Test\TiddlyFox2nd.html content-script.js:58:17

Message ID is tiddlywiki-save-file-response-1 content-script.js:59:17

Error: Error: Attempt to postMessage on disconnected port background.js:19:5

saver-handler: Saving wiki with method autosave through saver tiddlyfox $:/core/modules/utils/logger.js:33:10

Sending native message background.js:23:5

Win error 5 during operation open on file C:\Users\antho\Documents\timimihost (Access is denied. ) NativeMessaging.jsm:99

Error: Error: Attempt to postMessage on disconnected port background.js:19:5

Saved successfully to C:\Data\TW5\Development\Test\TiddlyFox2nd.html content-script.js:58:17
Message ID is tiddlywiki-save-file-response-2

This also failed so I moved the wiki file to file:///C:/Users/antho/Downloads/TiddlyFox2nd.html

saver-handler: Saving wiki with method save through saver tiddlyfox $:/core/modules/utils/logger.js:33:10

Sending native message background.js:23:5

Win error 5 during operation open on file C:\Users\antho\Documents\timimihost (Access is denied. ) NativeMessaging.jsm:99
Saved successfully to C:\Users\antho\Downloads\TiddlyFox2nd.html content-script.js:58:17

Message ID is tiddlywiki-save-file-response-1 content-script.js:59:17

 [Show/hide message details.] Unchecked lastError value: Error: Could not establish connection. Receiving end does not exist.

background.js:1

we are insidefile:///C:/Users/antho/Downloads/TiddlyFox2nd.html logsimple.js:15:3

Not a local Tiddlywiki file background.js:10:5

Initiate content-script.js:41:9

sendTimingEvent addSite start undefined undefined

verboseLoggerHooks.js:1:1120

saver-handler: Saving wiki with method autosave through saver tiddlyfox $:/core/modules/utils/logger.js:33:10

Sending native message background.js:23:5

[Show/hide message details.] Win error 5 during operation open on file C:\Users\antho\Downloads\timimihost (Access is denied.

) NativeMessaging.jsm:99

Error: Error: Attempt to postMessage on disconnected port background.js:19:5

Saved successfully to C:\Users\antho\Downloads\TiddlyFox2nd.html content-script.js:58:17

Message ID is tiddlywiki-save-file-response-1

My Email is behaving funny so I will save it with the diagnostics so far

Tony

[Riz]

One last attempt. Keeping everything as it is according to your last message, can you try changing the registry key nam to

HKEY_LOCAL_MACHINE\SOFTWARE\Mozilla\NativeMessagingHosts\timimi

Failing this I ll spin up a virtual machine and test the windows myself later today.

Thanks for the patience and perseverence

Riz

[Riz]

Do not worry about the initial "Not a local tiddlywiki file" message. I have set the plugin to check the url of the tabs in the browser and if it does not start with "file:///", do not
load the plugin at all. This way the plugin is more privacy friendly as it will be blind to all your internet browsing.
Initially when you open firefox, your url bar must be either empty or showing something like "about:blank". Since both of these fail to meet the conditions, the port is immediately disconnected. Later when you open the tiddlywiki file, the url scheme is rechecked. Once confirmed it is a local file, it checks if the file is actually a tiddlywiki. When both conditions are met, it opens the port and sends the message "Initiate"

From the console logs it is clear that plugin is loaded, and it attempts to send the changed content to python when you save it. The issue is either the plugin cannot access the python script or python script cannot access the file.

[Riz]

PS: Can you tell me your windows version and firefox version?

[Rizwan Ishak]

Also, mozilla docs for windows says "The registry key should have a single default value, which is the path to the manifest."

May be they meant you need to add the full path including the name of file itself - ie,  C:\\Users\\antho\\timimihost\\timimi.json

Sincerel

---

From: Riz
Sent: Monday 13 August 2018 1:15 PM
To: TiddlyWiki
Subject: [tw5] Re: Reviving TiddlyFox for Firefox > 57 (and Chrome)

PS: Can you tell me your windows version and firefox version?

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/_HToPNJnldw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/0f1dde92-f738-4844-9731-4013d74e768f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[TonyM]

Riz,

I( am sure I am responsible for the deleted messages in this thread, something is going wrong when I attempt to place the log entries in a Gmail. I will nor "sanitise" it before posting.

I am on the Latest FireFox Quantum 61.0.2 (64bit) and Windows 10 Home.

I tried some variations on the registry and still had errors. However I have another plugin installed which has similar entries

[HKEY_CURRENT_USER\Software\Mozilla\NativeMessagingHosts\timimi]
@="C:\\Users\\antho\\Downloads\\timimihost\\timimi.json"

Browser log

we are insidefile:///C:/Users/antho/Downloads/TiddlyFox2nd.html logsimple.js:15:3
Not a local Tiddlywiki file background.js:10:5
Initiate content-script.js:41:9
sendTimingEvent addSite start undefined undefined
verboseLoggerHooks.js:1:1120
saver-handler: Saving wiki with method autosave through saver tiddlyfox $:/core/modules/utils/logger.js:33:10
Sending native message background.js:23:5

[Show/hide message details.] Error parsing native manifest C:\Users\antho\Downloads\timimihost\timimi.json: JSON.parse: bad escaped character at line 4 column 15 of the JSON data NativeManifests.jsm:80
_tryPath/<
resource://gre/modules/NativeManifests.jsm:80:11
[Show/hide message details.] No such native application timimi NativeMessaging.jsm:99
NativeApp</this.startupPromise<
resource://gre/modules/NativeMessaging.jsm:99:9

Error: Error: Attempt to postMessage on disconnected port background.js:19:5
Saved successfully to C:\Users\antho\Downloads\TiddlyFox2nd.html content-script.js:58:17

Message ID is tiddlywiki-save-file-response-1 content-script.js:59:17

Because we see

[Show/hide message details.] Error parsing native manifest C:\Users\antho\Downloads\timimihost\timimi.json: JSON.parse: bad escaped character at line 4 column 15 of the JSON data NativeManifests.jsm:80
_tryPath/<

C:\Users\antho\Downloads\timimihost\timimi.json

{
  "name": "timimi",
  "description": "Native messaging host to save TW5",
  "path": "C:\Users\antho\Downloads\timimihost\timimi.bat",
  "type": "stdio",
  "allowed_extensions": [ "timimiATtesseract.com" ]
}

I hope this helps

Regards

Tony

[Rizwan Ishak]

eh, json in windows need to escape backslah characters. so the path in timimi.json has to be in the form C:\\user\\antho\\documents\\...

Sincerel

---

From: TonyM
Sent: Tuesday 14 August 2018 6:27 AM

To: TiddlyWiki
Subject: [tw5] Re: Reviving TiddlyFox for Firefox > 57 (and Chrome)

--

You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/_HToPNJnldw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/107cd87d-3feb-43cc-8a32-ffb420513371%40googlegroups.com.