Experimental Chromium native saver

[Dyllon Gagnier]

I wrote a small module for saving using the Chromium file system API (which is in the process of being standardized).

Example TiddlyWiki 5 Saver Using HTML 5 File System Access API (github.com)

The first time a save action is triggered, it prompts for a save location but future saves should go to the same location.

It's a bit dangerous because if saving fails, there is no way AFAIK to disable the saver after it loads. I would be interested to know if anyone knows a way to disable a saver after it has already loaded.

[Mohammad Rahmani]

Hi Dyllon,

 Interesting! How can I give a try?

Best wishes

Mohammad

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1fc43f38-1afc-49ab-95f3-7c40438a054cn%40googlegroups.com.

[Jeremy Ruston]

Hi Dyllon,

I wrote a small module for saving using the Chromium file system API (which is in the process of being standardized).

Example TiddlyWiki 5 Saver Using HTML 5 File System Access API (github.com)

Great stuff, I’ve been following the File System API with interest, and would be keen to get a saver in the core once the spec settles down.

Here’s a link for those unfamiliar:

https://web.dev/file-system-access/

I don’t know if any other browsers are planning to implement the API. It seems like an obvious requirement for ChromeOS, but perhaps other browsers will have less incentive to implement.

The first time a save action is triggered, it prompts for a save location but future saves should go to the same location.

It's a bit dangerous because if saving fails, there is no way AFAIK to disable the saver after it loads. I would be interested to know if anyone knows a way to disable a saver after it has already loaded.

I think you’re already following the correct approach: for the save() method to synchronously return false if the save cannot proceed so that the next saver in the cascade gets a chance.

Another thought with respect to the File System API is that it may be possible to write a syncadaptor module so that we can support TiddlyWiki folders containing individual .tid files in the browser.

Best wishes

Jeremy

[TW Tones]

Folks,

This may be the opportunity to simplify saving and give the local storage plugin or its equivalent safe access to the local system. Thus users can access a read only online resource and use it as if it were a local app, with their own data stored locally.

Any way to reduce the complexity of using, especially saving tiddlywiki(s) will increase adoption. If the logic and and setup can be included in the original wiki then they become eminently deployable with self documented setups. This could include backups, localisation or cloning, I can quickly do this now with Timimi already in place, if such facilities can be introduced without a two step, browser specific install, all the better. Yes, all the appropriate authorisation is needed for security.

Tones

[Dyllon Gagnier]

I would caution on being too hasty with this. There are some serious security implications with opening wikis hosted on the file system (i.e. file://, not talking about local servers or TiddlyDesktop).

Apparently permissions are shared pretty broadly for anything open on file://. The code I shared is probably ok, but I have reported one issue to the Chromium team that I would like to hear

their feedback on since it has some potential security implications for TiddlyWiki. I know there are people who store very valuable data on their wikis which is why I want to put a big caveat

on all this.

Even with my current concerns, I believe that the API is very safe so long as you serve the wiki from a local server (any local server, doesn't need to be TiddlyWiki specific). For example,

launching a web server in a directory with the wiki via "python3 -m http.server". There is less of a benefit over the tiddlywiki node server, but I do think there is still a benefit. Since there

is almost no coupling between the server and TiddlyWiki, you can debug things entirely in the browser and synchronization of state is easier to think about. There is only the state on disk

and the state of the browser tab, no hidden state/watch issues with the web server.

I think expanding the code to support wiki folders would be great and would help solve some of the issues around having images embedded in portable wikis.

I am planning on releasing this as an actual plugin, but in addition to security and versioning, I want the plugin to:

• Support versioning/backup like TiddlyDesktop.
• Have settings for the plugin to control stuff like logging from the component itself without modifying code.

Thanks,

Dyllon

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/IczqdIdC3lE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6d3bc90f-54eb-4512-a56d-3a62931b75b4n%40googlegroups.com.

[TW Tones]

Dyllon,

Thanks for working on this project. I agree security is paramount, and failure to address it may simply result in in braking with the next security update to a browser. 

However, I want to point out there are different use cases for tiddlywiki, where there is only one user, on a LAN or a trusted team, as opposed to Internet publishing. There are cases where adding a plugin that exposes a security issue is not an issue in many cases. Sadly Security is the "Tail that wags the dog" today and stops a lot of possibilities, but all it takes is accounting for context, such as only if on local host etc.... 

Regards

Tones

[Mohammad Rahmani]

On Mon, Apr 26, 2021 at 11:58 PM Dyllon Gagnier <dyllon...@gmail.com> wrote:

I would caution on being too hasty with this. There are some serious security implications with opening wikis hosted on the file system (i.e. file://, not talking about local servers or TiddlyDesktop).

Apparently permissions are shared pretty broadly for anything open on file://. The code I shared is probably ok, but I have reported one issue to the Chromium team that I would like to hear

their feedback on since it has some potential security implications for TiddlyWiki. I know there are people who store very valuable data on their wikis which is why I want to put a big caveat

on all this.

Even with my current concerns, I believe that the API is very safe so long as you serve the wiki from a local server (any local server, doesn't need to be TiddlyWiki specific). For example,

launching a web server in a directory with the wiki via "python3 -m http.server". There is less of a benefit over the tiddlywiki node server, but I do think there is still a benefit. Since there

is almost no coupling between the server and TiddlyWiki, you can debug things entirely in the browser and synchronization of state is easier to think about. There is only the state on disk

and the state of the browser tab, no hidden state/watch issues with the web server.

I think expanding the code to support wiki folders would be great and would help solve some of the issues around having images embedded in portable wikis.

I am planning on releasing this as an actual plugin, but in addition to security and versioning, I want the plugin to:

• Support versioning/backup like TiddlyDesktop.
• Have settings for the plugin to control stuff like logging from the component itself without modifying code.

This looks very promising! Having a simple method for saving is very essential in many use cases! While I understand your concern for security issues, I think still this approach of saving TW is very useful and demanding!

 

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/CAJsXKEMU2-R3asA6jOZOB35NQca7mpaxnT%3D%3D6sZiev6XVvWG-g%40mail.gmail.com.

[Dyllon Gagnier]

@mohammad.rahmani the security issue is actually relevant to TiddlyWiki even if the file itself is secured.

Google decided this was a duplicate issue so I can disclose the issue in full. 1202597 - Security: File Access API Permission Bypass - chromium

You can save file/directory handles to IndexedDB (like localStorage, but more confusing) so that you don't need to select the TiddlyWiki

file from the file picker each time you refresh the tab/open it. However, Chromium has a bug/design which considers everything from file://

to be from the same origin/domain. This means if you download and double click on some "malicious.html" it could just scan IndexedDB

and use the same descriptors TiddlyWiki saved for convenience.

Here is a potential outline of this attack:

1. You save teaminfo.html to a shared drive which includes passwords/valuable docs/etc.
2. You are writing some Rust code and have the docs open (which the rustdoc tool conveniently generates for you locally).
3. Someone, NSA/Russia/whoever, compromised some libraries docs and inserted malicious JS.
4. This malicious code gets the file descriptor by scanning IndexedDB and uploads it to some remote location.
5. If you happen to have said wiki open in another tab (as I personally often do), it can also overwrite this file and then be ransomware.

How to avoid this:

• Do not save file descriptors to IndexedDB
• The code I posted doesn't save to IndexedDB, so this is for people modifying the code.
• I am working on a proper plugin for this and my plan is to only save descriptors if not served from file://.
• Use a static server like "pythom -m http.server"
• Firefox doesn't support this API yet, but they seem to be more strict with same origin than Chromium.
• Wait for Chromium to actually fix this issue and enforce a better origin policy for IndexedDB.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/CAAV1gMCuej%2Bg6Aup7WXRdQco4yo5OXyiBeZqb4BMkVa%3D%2BBXOhw%40mail.gmail.com.

[Mohammad Rahmani]

Hi Dyllon!

Many thanks for the detailed answer and clarification! 

I hope someday we could have this native saving feature! It would be very convenient to be able work with Tiddlywiki without any extra requirements!

Best wishes

Mohammad

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/CAJsXKEMvmHaNG-uwkJAyuHOJd8ZpSkkoimx-qEhPmdM%3DW%3D8TqA%40mail.gmail.com.

[Jeremy Ruston]

Hi Dyllon

We actually have the same issue with the GitHub saver module: it saves the access token to local storage, which is subject to the same peculiarity of all file: URIs counting as a single origin. Thus, if an attacker can induce a user to download an HTML file and open it in the browser then they can exfiltrate all the access tokens.

I think Google’s answer would be “don’t use file URIs to manage sensitive data”, but it makes me sad that there’s so much erosion of the users power to manage their own data locally. I think the alternative is to run TiddlyWiki as a PWA, but that would subject us to browser compatibility issues, and give end users a much more obscure mental model.

Best wishes

Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/CAJsXKEMvmHaNG-uwkJAyuHOJd8ZpSkkoimx-qEhPmdM%3DW%3D8TqA%40mail.gmail.com.

[TW Tones]

Jeremy et al,

I think if we could at least make it look like a PWA functionally. Please bare with my level of technical knowledge, but I believe I can see something appearing in the fog of these issues.

Given initial saving to IndexDB is workable if not reliable in the long term, I wonder if we provided a tool in Tiddlywiki to save the Wikis index db, or the wiki itself out to file as a backup? In fact as a start we could use this to trigger local backup (See Mario's and others "Savers"). Noting the backups would not be html or other "executable", thus stopping them subsequently being loaded with all local permissions etc...

I have always thought that a mechaisium to logout of tiddlywiki would be a nice option. We could autosave to the backup then logout, which would trigger a save to file (even if we must get a dialogue) A little code or cookie may be able to detect if the tiddlywiki no longer holds a matching last file save it would prompt to load the backup from a known folder. This method may have the added advantage that all backups can be files under the browser downloads folder thus have the necessary browser rights.

The above method also means you will find your tiddlywiki at the URL where you hosted it, no need to address files unless you have a failure or during the "occasional" backup to file. Such addresses can be local dns or IP address that is only hosting the original (read only wiki), the content is off in IndexDB and backups to the downloads folder. Of import here is you must continue to have access to the original, but all subsequent content is localised (owned by the user).

Finally we could host dozens of empty.html files on a static server (by other names / domains), perhaps some preloaded editions, then tiddlywiki users can find one of these as a starting wiki and design and add content on top of there. eg todo.tiddlywiki.com sandbox.tiddlywiki.com mywiki.tiddlywiki.com with smart server management these could all be the same html file plus installable packages.

Once more experienced, once they have a server or saver Timimi they can export their wiki to a specific local location , until then just use them at the address, but bound within a browser.

I truly hope to make the use and design of tiddlywiki so seamless then we can surely attract more people with lower level skills so they can start on the journey.

Tones

[Dyllon Gagnier]

For those interested, I've wrapped this code into a proper plugin at  slaymaker1907/TW5-browser-nativesaver (github.com).

The main additional features I added were:

• A settings page (Control Panel/Saving/HTML Native File System Saver)
• Ability to disable saver from said settings page (should be able to disable it and then use another saver if desired)
• Ability to turn on/off logging to dev console
• If served via a webserver like "python3 -m http.server", it saves the file location to IndexedDB so you don't have to constantly reselect the file to save to when you reload the page. Note that for enhanced security, I recommend using a specific port for the webserver and/or "yourwiki.localhost:8000" (any *.localhost domain should just resolve to localhost, but the browser should treat it as a separate domain).

I'm currently working on adding support for automatic backups like TiddlyDesktop. In the future, I may also support saving as a wiki folder which should increase performance as well as allowing.

I highly recommend using a backup solution like Dropbox, GoogleDrive, OneDrive, etc. since versioning is not supported out of the box. Just something which allows you to recover the document from a specific time period. The plugin itself should either work or fail gracefully, this recommendation is more for situations where the wiki gets corrupted due to a disk issue or because of some other faulty tiddlers/plugins. You'll thank me later.

[Mark S.]

Could the file descriptors be encrypted before storage? Then when there's a save the plugin could fetch the encrypted descriptor, decrypt it, and use it for the save?

[Dyllon Gagnier]

Unfortunately not at the present time. These descriptors are very opaque and need to be saved essentially as is to IndexedDB.

However, I did come up with a proposal to the browser vendors that would allow this Allow for serializing and deserializing a handle as a string · Issue #295 · WICG/file-system-access (github.com). If this were implemented, I could do that or even just save the handle to the wiki itself in a secure way.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/3a052774-a404-4759-a668-1a7576be194an%40googlegroups.com.

[PMario]

On Thursday, May 13, 2021 at 8:55:59 PM UTC+2 dyllon...@gmail.com wrote:

Unfortunately not at the present time. These descriptors are very opaque and need to be saved essentially as is to IndexedDB.

I think there should be a possibility to "reuse" read/write permissions. It's just not implemented yet. see: https://wicg.github.io/file-system-access/#api-filesystemhandle-querypermission 

-mario

[PMario]

On Thursday, May 13, 2021 at 8:27:53 PM UTC+2 dyllon...@gmail.com wrote:

• If served via a webserver like "python3 -m http.server", it saves the file location to IndexedDB so you don't have to constantly reselect the file to save to when you reload the page. Note that for enhanced security, I recommend using a specific port for the webserver and/or "yourwiki.localhost:8000" (any *.localhost domain should just resolve to localhost, but the browser should treat it as a separate domain).

If I need a web server to use the file system api, wouldn't it be better to use the nodejs version, which already does save back, without any permission problems.

I'm just curious...

-m

[Dyllon Gagnier]

I think the biggest advantage is that you can run any old static webserver rather than one specific to TiddlyWiki. This enables things like more easily handling multiple wikis (just stick them all in a common directory and serve that directory, no config necessary). On Linux, no extra software has to be installed, just download the wiki file and python is already installed which has a webserver.

The plugin doesn't require a webserver to work though, you just have to pick the file in the filesaver once per reload. Personally, I don't refresh my wiki page very often so for me this is probably once every couple hours at most unless I'm installing new JS plugins or something. Once Chromium fixes their SOP policy for file://, these can be safely stored in IndexedDB and have parity with running it from a webserver. Another possibility is if FireFox implements the File System Access API since they have a more restrictive origin policy for file:// domains.

--

You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/IczqdIdC3lE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/896df457-9ac0-4e9d-b187-2a8c43e5b533n%40googlegroups.com.

[PMario]

On Friday, May 14, 2021 at 1:56:08 AM UTC+2 dyllon...@gmail.com wrote:

I think the biggest advantage is that you can run any old static webserver rather than one specific to TiddlyWiki. This enables things like more easily handling multiple wikis (just stick them all in a common directory and serve that directory, no config necessary). On Linux, no extra software has to be installed, just download the wiki file and python is already installed which has a webserver.

OK. I'm running IIS on windows with a WebDav server and a nodejs at localhost on port 80. So no port is needed in the URL. I personally consider this as simple as running a python static server on linux. ... Anyway, that's a different topic.

 

The plugin doesn't require a webserver to work though, you just have to pick the file in the filesaver once per reload.

Yea, and exactly that seems to be a "convenience" problem.

 

Personally, I don't refresh my wiki page very often so for me this is probably once every couple hours at most unless I'm installing new JS plugins or something. Once Chromium fixes their SOP policy for file://, these can be safely stored in IndexedDB and have parity with running it from a webserver.

OK. We will see, with what they come up with.

 

Another possibility is if FireFox implements the File System Access API since they have a more restrictive origin policy for file:// domains.

IMO not really. They have the same problem, that file:// is only 1 domain for any of the local storage variants. ...

I'm following the FireFox "file API" discussion very closely. ... IMO at the moment they don't intend to implement anything similar, to what chrome does. ... There is an open "bug", for a file API for web-extensions, that also has a "permission dialogue" for the first time you access a directory or a file. ... But then the setting should be permanent as long as the user doesn't revoke it. ...

The advantage is, that a webpage has no access to any data store, that is available for a browser AddOn. ... So there shouldn't be a security problem with file:// URLs.

On the other hand, there is a high chance, that the web-extension API for file:// URLs will be crippled. ... I don't hope so, but "We will see" :)

-mario

[Mohammad Rahmani]

Hi Dyllon,

This is absolutely useful! Thank you for the update!

Tested on Chrome/Edge 90 on Windows 10

1. Using the plugin alone, works in both browsers like a charm
2. Using the node.js static server works great [1]
3. Using the python http server works great [2]

So, it works like a hub for me! very nice! 

Two comments:

Comment 1, when you load a wiki, for the first save it asks you to give a file name! One expects he/she is saving on wiki opened in the browser, but the plugin actually saves in the file just selected and not in the location you see in the address bar!!, this may cause some confusion! Of course if you use the same file name, it would be fine!

Comment 2, when you load a wiki, for the first save it asks for a file name, when the browser opens file save dialog, cancel it (e.g press Esc). Browser shows a error message as below

Click ok and create another tiddler so Tiddlywiki triggers another save, this time Browser will not ask you for a file name and it will save into the temp folder automatically!

For example on Windows 10

.../AppData/Local/Temp/MicrosoftEdgeDownloads/9911cf23-4ce9-4a8c-bb30-9135e161d280/example-wiki.html

This is not important as it only occurs if you cancel the first save!

Best wishes

Mohammad

[1]. Node.JS static server

With Node.JS I use http-server - npm (npmjs.com)
Then from the command line I just fire up http-server --port 8083 and then I get access to my wikis for example from TV or on mobile or other device (home network)

[2]. Python http server

Using python 3.9 I on Windows 10 I use the below command at the prompt

python3 -m http.server and the I have access to 

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d519dc2c-43ae-4c31-b423-37c062bcf70fn%40googlegroups.com.

[Nolan Darilek]

I tried exporting this plugin from the download page and importing it into a new wiki. It shows up in my plugins list and is enabled both as a plugin and in its settings page, but whenever I click Save, it still seems like the page just downloads. I don't get any sort of prompt to select a file.

Do I have to configure a default saver somewhere in addition to enabling this one?

Running in Brave, which may be doing something to protect me despite being Chrome-based.

Thanks.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d519dc2c-43ae-4c31-b423-37c062bcf70fn%40googlegroups.com.

[Nolan Darilek]

Never mind, works perfectly in Chrome. Apparently the filesystem API was recently disabled in Brave, which is what confused me. Tested it a couple months ago and it worked fine.

Sorry for the noise.

[PMario]

On Friday, May 14, 2021 at 3:49:30 PM UTC+2 ndar...@gmail.com wrote:

Never mind, works perfectly in Chrome. Apparently the filesystem API was recently disabled in Brave, which is what confused me. Tested it a couple months ago and it worked fine.

I think it's good to know, where it works and where it doesn't. .. So I'd call it an "experience report"!

-m

[TW Tones]

Folks,

Is there a simple set of instructions to make use of this. I cant see them and don't have a lot of time available to read through the discussion?

Personally I am interested in a php implementation.

Thanks in advance 

Tones

[Mark S.]

If you're not saving to indexedDB, then how is the path saved? Do your earlier caveats about file:// still apply?

If I understand correctly, when using a static server, you're basically using the server path name as a safe way to save the actual file system pathway? So you could maybe also post to github or other site as a basis for your local save? That would have some interesting uses, especially if it also works on Android, where the existing browser-save mechanisms are mostly broken.

It does seem like an interesting alternative to existing savers.

Thanks!

On Tuesday, April 27, 2021 at 3:59:23 PM UTC-7 dyllon...@gmail.com wrote:

[Mark S.]

It's pretty straight forward. You just need 

1. A chromium-based browser

2. Dyllon's plugin from above

3. Any way to serve up the file including Tiddlywiki on node (it serves up things in the /files dir) and python. 

Python comes with a simple server where you can just run 

python3 -m http.server 9000

(where 9000 is the port) and everything is the directory will be served up as static. 

If I understand correctly, the static server is just providing a "safe" pathway that can be used as an index to save the physical (on-disk) path.

You put the plugin in your TW file and save it via your conventional way (perhaps as a download) to it's home. Then serve up the file with your web server. Then make changes and save. The first time it saves it will ask for a path. Specify the path to where you want to save. After that all saves are automatically saved to the same path.

Remember when you're testing to either turn off Timimi or to use an account without Timimi installed. 

Alas, it doesn't seem to quite work on Android. It wants to do that file (1), file (2) .... thing. 

[Nolan Darilek]

FWIW, I'm not using the separate server here and things work fine on Chrome. I imagine it helps with security concerns, but I've chosen to not worry about any as they relate to this plugin for now, particularly since I only use Chrome for this.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/cdd36141-ced4-4d6d-a8ee-03ec77fdc298n%40googlegroups.com.

[Dyllon Gagnier]

Nolan, the security should be fine even if opening the HTML file from the file system (i.e. file://path/to/yourwiki.html) unless you modify the JS in the plugin. If you use a web server, then I save the file handle between page reloads so you don't have to select it from the file picker. This is unsafe using file:// since file:// is one origin and thus any HTML file you open in Chrome from file:// can access that file handle.

I detect when the plugin is being used from file:// and do not attempt to save the file handle in that case to avoid this security issue. It's not really a huge deal, you just have to select your wiki from the file picker each time you reload the wiki (I personally only do this if I'm installing new plugins or if my computer reboots).

In terms of practical usage, I would recommend making sure you backup your wiki regularly including multiple previous versions. That isn't really for security, but more to help if you accidentally make a recursive tiddler of doom or something.

You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/IczqdIdC3lE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/b9ff337b-8725-8cd2-92a9-1fae98be92aa%40thewordnerd.info.

[TW Tones]

As has being said;

file:// is one origin

This seems like a gap to me on browsers in general. Perhaps a side effect of recent years security changes.

If we could add into the security permissions to allow write to file://C:/scratch file://C:/data/tiddlywiki etc... to a particular site or address, and perhaps local host etc... a lot of our savings issues could be eliminated. Of course an interactive dialogue requesting such permissions ONCE would be fine. With a reminder when relying on it.

Regards

Tones