Saving: TiddlyFox, supporting TWC and TW5 + documentation

[Yakov]

Hey everybody,

in the latest release of TiddlyWiki Classic I've fixed the upgrading engine and now it's time to deal with saving more consistently. That's where I'd like not only to ask about some dev aspects but also discuss some conventions and create docs for consistent support of saving both TWC and TW5.

I'll start with TiddlyFox since it is still used by many users (with either FireFox 56- or Pale Moon, WaterFox and other FF forks). One thing about it is building the xpi: I've read about building it a little and the crucial quesion is: is it still possible to create signed xpi, or updating TiddlyFox is impossible in principle?

If it's not, I'd only like to ask Jeremy to add 2.x tags to the repo and attach xpi files to the github releases; or add me to the repo and I'll do that myself. Otherwise, I'd like to contribute a bit more and have some more questions.

Next and the main point of this thread is: both TWC and TW5 have many supporting solutions (native apps, servers, browser plugins etc) for saving but I'm not aware of docs that describe how actually one supports saving (and may be more functionality) of both TWC and TW5. For instance, MainTiddlyServer could be extended a bit so that it fits the php server gap, but I (as a developer of TWC extensions, MTS and maintainer of the core) am not really aware of how to support TW5 correctly; likewise, the aTW saver for Android supports only TW5 (as far as I know), but I'd like to participate and support TWC as well, yet I still didn't even get an answer about the app status (and I suspect Simon may be reluctant to support TWC without nice docs describing that). Finally, it seems every time one tries to support saving in their software, they invent some new implementation which (this diversity) is difficult to follow and the ecosystem is quite difficult to maintain.

Thus, it seems to me that we really need 2 things:

• nice docs describing how to support saving of TWC, TW5 (and, in perpective, more features like image uploading and more)
• core libraries in different languages (js for browser plugins? js for node, PHP, Java/Android, Swift/iOS, may be more) which one can use for developing new solutions

Now this thread is mostly about the first part, the docs (and by the way saving is not the only thing which would be nice to document; think, for example, of the .tid format which is not described anywhere, at least Jeremy wrote so some months ago).

My proposal consists of 2 parts:

1. create a repo in the TiddlyWiki org @github (I'd like to be a member to contribute)
   
2. start building docs

For the second part, I now have a draft based on what I've learned from the TiddlyFox sources, see it below the main post. I invite you to comment it, correct where I'm wrong, fill the gaps, especially about TW5.

Best regards,

Yakov.

== first draft of "how to support TW saving" docs ==

To support TW saving, it is to be desided:

1. how to recognize a TW?
2. when to inject new JS?
3. what JS to inject (and where)?

(and also the "protocol" and the "back-end logic" stuff)

How to recognize a TW?

First, let's note that the simplest implementation can omit TW recognition: just ask whether to inject JS or inject it into every html "served" with the saver app.

Looking at TiddlyFox code, it seems the following procedure is ok:

• to recognize TWC, check whether there's #storeArea element [TF searches DOM, not source], #versionArea element and the latter contains the word "TiddlyWiki"
• to recognize TW5, check whether among <meta> elements there's one with name "application-name" and content "TiddlyWiki"

This surely can give false positives which seems to be ok; this also doesn't support "pure store" format.

When (and where) to inject new JS?

this section is to be written yet. Notes:

• MainTiddlyServer injects JS bits into the TWC code before serving it and strips those back when saves "the whole thing" (it can also send save increments); some modifications (like support of incremental saving) are made as hijacks which are done in loadPlugins so that the store is ready
• server-side implementations (that inject JS into TW so that user doesn't have to install a plugin into TW) also need to decide where to inject JS, see next section
  
• not sure whether TiddlyFox adds JS on load or eariler (probably the latter)
• neither I am sure of the exact moment tiddly-node-saver uses although the approach looks interesting
• same for TiddlyDesktop overwriting
  

What JS to inject (and where)?

For now, I'm describing some existing approaches instead of a "canonical" one to be proposed.

TiddlyFox overwrites window.mozillaSaveFile and window.mozillaLoadFile with new implementations and also sets window.convertUriToUTF8 and window.convertUnicodeToFileFormat to plain s => s functions (needed for TWc before 2.9.2). This works for TWC and seems to be enough for bare loading/saving functionality. One thing to add here is overwriting the config.options.chkHttpReadOnly option if TW is served via http(s): instead of file: scheme.

A similar set of changes is implemented in tiddly-node-saver, it's simpler (window.saveFile is -overwritten-, loading is not implemented at all, chkHttpReadOnly is set to false). Actually window.saveFile is not overwritten but rather defined and I suspect it relies on the following definition in TWC core:

window.saveFile = window.saveFile || function(fileUrl,content)

which substituted this:

function saveFile(fileUrl,content)

in 2.7.0 which is a quite sane limitation.

I haven't quite figured the approach of TiddlyDesktop yet.

Unlike most solutions for local usages only, MTS is supposed to be used on servers as well as locally, that's why it is more strict and currently doesn't support saving arbitrary files. Instead, it injects "return saveOnlineChanges();" into saveChanges (and also inject the definitions before "function saveMain(" and changes the default value of chkHttpReadOnly, but now I see this should be refactored since there are better approaches).

As for TW5, I'm not really sure about the exact stack, but it seems for saving, a handler of the "tiddlywiki-save-file" custom event should be added. The handler should expect the "event" (first argument) to have .path and .content properties which should hold (local?) path to the file to save and (UTF-8?) content string of the file. Not sure if some sort of "file-save-success" event is supported to notify the core that the file was actually saved (and TiddlyFox actually has a bug to always report that TW was saved, even if it was on a USB storage which was removed before saving, because the injected saver always returns true).

Further considerations

There's a number of topics that should be covered as well, mostly after those above are, but some of them will affect architecture and hence should be considered anyway (can affect the core, especially in case of TWC; I'm ready to adapt it to better architecture):

• sync vs async saving
• full saving vs sending changes, shortcuts for backuping (sending full file to backup vs sending "copy file A into B" message)
• protocol layer (shouldn't necessarily be uniform, but recommendations can facilitate development of new tools)
  
• streaming changes via websockets and sync editing
• security consideratinos
• server logic (authorization & roles, ..)
• federation (including content from other TWs, saving it back)
• advanced features (image uploading, integration with git etc)
  

====

Looks like that's all I've gathered for now. Again, contributions are very much welcome.

[TonyM]

Yakov,

I do not have the skills to help you but would point out how Timimi is a Tiddlyfox replacement, that perhaps only works with TW5 however it shows the path to such saving in modern browsers.

Regards

Tony

[Yakov]

Hi Tony,

thanks for sharing. Did I get the description right: is Timimi able to save TW5 in any folder, unlike file-backups or SaveTiddler which only allow to save only TWs in the browser download folder? That sounds very intriguing and if that's the case, I'll test it out and have a closer look at the source.

However, I really hope somebody among TW5 ecosystem developers will join me to make simple and clear docs for this (also AFAIK I can't create a new repo in https://github.com/TiddlyWiki/ , I only have access to https://github.com/TiddlyWiki/TiddlyWiki/ to maintain the core).

Best regards,
Yakov.

четверг, 28 марта 2019 г., 5:17:43 UTC+3 пользователь TonyM написал:

[TonyM]

Yakov,

Timimi, Is better than TiddlyFox. Once the "FireFox" plugin and local piece is installed, and firefox is your default browser double clicking a html file (aka Tiddlywiki) anywhere on your desktop will open in the browser and can save back to that file with no other action. 

To save, it just saves it in place. If you download any internet hosted file wiki to any location using the default download file saver then open that file you can edit and save it in firefox.

Mario has some youtube videos on using github that may help https://www.youtube.com/user/pmariojo/search?query=github

Regards

Tony

[Jeremy Ruston]

Hi Yakov

I'll start with TiddlyFox since it is still used by many users (with either FireFox 56- or Pale Moon, WaterFox and other FF forks). One thing about it is building the xpi: I've read about building it a little and the crucial quesion is: is it still possible to create signed xpi, or updating TiddlyFox is impossible in principle?

Mozilla was responsible for signing the extension as part of the process of uploading it; I don’t think there’s ever been a way for anyone else to sign it and have it accepted by recent versions of Firefox. But I don’t know much about it, it may be worth asking on the Mozilla dev mailing lists.

If it's not, I'd only like to ask Jeremy to add 2.x tags to the repo and attach xpi files to the github releases; or add me to the repo and I'll do that myself. Otherwise, I'd like to contribute a bit more and have some more questions.

No problem with adding you to the repo.

Next and the main point of this thread is: both TWC and TW5 have many supporting solutions (native apps, servers, browser plugins etc) for saving but I'm not aware of docs that describe how actually one supports saving (and may be more functionality) of both TWC and TW5. For instance, MainTiddlyServer could be extended a bit so that it fits the php server gap, but I (as a developer of TWC extensions, MTS and maintainer of the core) am not really aware of how to support TW5 correctly; likewise, the aTW saver for Android supports only TW5 (as far as I know), but I'd like to participate and support TWC as well, yet I still didn't even get an answer about the app status (and I suspect Simon may be reluctant to support TWC without nice docs describing that). Finally, it seems every time one tries to support saving in their software, they invent some new implementation which (this diversity) is difficult to follow and the ecosystem is quite difficult to maintain.

TW5 supports saving in two completely different ways: saving the entire HTML file versus syncing individual tiddlers to a server.

Syncing is the easier to explain of the two: TWC has plugins that let it sync changes to TiddlyWeb, a venerable Python-based TiddlyWiki serverside we developed at Osmosoft. I’ve not tested it recently, but those plugins should also enable TWC to save back to a TW5 server.

In TW5, saving is modular: there are eleven built into the core:

• andtidwiki - for saving within the AndTidWiki Android app
• beaker - for saving within Beaker Browser
• download - for the fallback HTML5 compatible download saver
• fsosaver - for the Internet Explorer ActiveX saver
• manualdownload - a version of the download saver for older browsers that don’t support the necessary HTML5 features
• msdownload - a version of the download saver for older Microsoft browsers
• put - for saving to WebDav servers
• tiddlyfox - for saving under TiddlyFox
• tiddlyie - for saving to the Internet Explorer TiddlyIE extension
• twedit - for saving within TWEdit, an old iPad/iPhone app for TiddlyWiki
• upload - for saving to TiddlySpot, or a PHP server running the upload script

The TiddlyFox saver talks to TiddlyFox via a hidden DIV and custom events.

TiddlyFox itself has different handling for TW5 and TWC. For TW5, it uses the hidden DIV and custom events, and things are reasonably elegant. For TWC, TiddlyFox has to inject overrides to the saving functions which then set up the hidden DIV and handle the custom events.

Thus, it seems to me that we really need 2 things:

• nice docs describing how to support saving of TWC, TW5 (and, in perpective, more features like image uploading and more)

• core libraries in different languages (js for browser plugins? js for node, PHP, Java/Android, Swift/iOS, may be more) which one can use for developing new solutions

Do you mean docs for writing an app for handling saving of TiddlyWikis? Or for building a serverside for TiddlyWikis? Or do you mean docs for how the saving mechanisms work?

Now this thread is mostly about the first part, the docs (and by the way saving is not the only thing which would be nice to document; think, for example, of the .tid format which is not described anywhere, at least Jeremy wrote so some months ago).

I think we are indeed missing docs for the .tid file format, but probably only because it’s so basic. Are you running into a problem with it?

My proposal consists of 2 parts:

1. create a repo in the TiddlyWiki org @github (I'd like to be a member to contribute)
   
2. start building docs

For the second part, I now have a draft based on what I've learned from the TiddlyFox sources, see it below the main post. I invite you to comment it, correct where I'm wrong, fill the gaps, especially about TW5.

Good stuff. I think you’ve captured the behaviour of the TiddlyFox plugin. I’ve added comments to your questions.

This surely can give false positives which seems to be ok; this also doesn't support "pure store" format.

I do remember the “pure store” format being discussed, but don’t recall the details. If it reduces the HTML file to the storeArea, then the heuristics above should work.

• MainTiddlyServer injects JS bits into the TWC code before serving it and strips those back when saves "the whole thing" (it can also send save increments); some modifications (like support of incremental saving) are made as hijacks which are done in loadPlugins so that the store is ready
• server-side implementations (that inject JS into TW so that user doesn't have to install a plugin into TW) also need to decide where to inject JS, see next section
  

I’m not familiar with the details of MTS, but it sounds like these modifications would be done on the HTML file, rather than in the DOM. Somewhat different techniques need to be used in those situations.

• not sure whether TiddlyFox adds JS on load or eariler (probably the latter)

I think that the code gets injected after the page has loaded:

https://github.com/TiddlyWiki/TiddlyFox/blob/master/index.js#L30

• neither I am sure of the exact moment tiddly-node-saver uses although the approach looks interesting
• same for TiddlyDesktop overwriting
  

TiddlyDesktop just implements the TiddlyFox protocol of the hidden DIV and the custom events.

As for TW5, I'm not really sure about the exact stack, but it seems for saving, a handler of the "tiddlywiki-save-file" custom event should be added. The handler should expect the "event" (first argument) to have .path and .content properties which should hold (local?) path to the file to save and (UTF-8?) content string of the file. Not sure if some sort of "file-save-success" event is supported to notify the core that the file was actually saved (and TiddlyFox actually has a bug to always report that TW was saved, even if it was on a USB storage which was removed before saving, because the injected saver always returns true).

As discussed above, TW5 has pluggable save handlers so the two approaches to implementing saving are to either re-use one of the existing core savers (e.g. as does TiddlyDesktop) or to add a brand new saver module (e.g. as we do for Beaker Browser).

Best wishes

Jeremy

Further considerations

There's a number of topics that should be covered as well, mostly after those above are, but some of them will affect architecture and hence should be considered anyway (can affect the core, especially in case of TWC; I'm ready to adapt it to better architecture):

• sync vs async saving
• full saving vs sending changes, shortcuts for backuping (sending full file to backup vs sending "copy file A into B" message)
• protocol layer (shouldn't necessarily be uniform, but recommendations can facilitate development of new tools)
  
• streaming changes via websockets and sync editing
• security consideratinos
• server logic (authorization & roles, ..)
• federation (including content from other TWs, saving it back)
• advanced features (image uploading, integration with git etc)
  

====

Looks like that's all I've gathered for now. Again, contributions are very much welcome.

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/e4526996-ad42-45fd-af38-caac660cc07c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yakov]

Hi Jeremy, thanks for stepping in,

Thus, it seems to me that we really need 2 things:

• nice docs describing how to support saving of TWC, TW5 (and, in perpective, more features like image uploading and more)

• core libraries in different languages (js for browser plugins? js for node, PHP, Java/Android, Swift/iOS, may be more) which one can use for developing new solutions

Do you mean docs for writing an app for handling saving of TiddlyWikis? Or for building a serverside for TiddlyWikis? Or do you mean docs for how the saving mechanisms work?

Well, I mean docs for (writing an app/backend for) handling saving in the first place. But later those can cover supporting more features and I don't really see the boundary where one can call those "docs for building serverside" (a primitive serverside is a script which just saves TW). Of'course the docs may later cover topics like authentication, or uploading images etc, but the MVP is "how to support saving TWs". Another point about such docs/repo is those can be a more structured way to discuss and understand current and future architecture of TWC/5, so one can think of them also as [infrastructure for] a process, not a product. (for instance, I never really understood what adaptors are exactly)

That said, also think that saving engine of TWC should be updated as well and I'm even considering refactor it for easier support of async and granulated saving (already supported by MTS, see implementOnlineSaving in the injected JS), may be by making it similar to TW5. By the way, I'm a bit confused with TF's handling of TW5 saving (or rather the saving mechanism itself) in a couple of aspects, but let me go into details below.

I think we are indeed missing docs for the .tid file format, but probably only because it’s so basic. Are you running into a problem with it?

Well, I have at least one question about it. As far as I understand, it looks like:

field: value
field2: value2
title: some tiddler

tiddler text goes here, after 2 linebreaks (watch out for CRLF!)

so is there support for multiline fields? I have a bunch of tiddlers in my TWs which have multiline field values, so it's not purely theoretical (I believe using fields for database-like approach is generally somewhat natural as well as multiline content in those).

Now to the TW5 and TF aspects.

Like I said, I understand that the "tiddlywiki-save-file" custom event (with .path and .content properties) is dispatched when saving is needed. What I don't understand is how TW5 is gets back the confirmation of changes being saved (and which set of changes was saved), because there can be "disconnections" – in case of TF it can be ejected USB-storage with TW, in case of a server – absence of network connection.

I also wonder if the current architecture of granulated saving in TW5 is compatible with undoing changes (is there a saved queue of changes?). This is not of major importance, but it's better to think about it to avoid another refactoring in the future.

As for TWC, TF actually introduces a bug: since the overwritten mozillaSaveFile always returns true, with newer (2.8.1+) versions saveMain reports changes being saved even if USB-storage with TW was ejected (it wasn't the case in earlier versions of TWC because loadOriginal always loaded original from file system, and saveMain stopped before calling mozillaSaveFile if TW is not currently available).

I'll start with TiddlyFox since it is still used by many users (with either FireFox 56- or Pale Moon, WaterFox and other FF forks). One thing about it is building the xpi: I've read about building it a little and the crucial quesion is: is it still possible to create signed xpi, or updating TiddlyFox is impossible in principle?

Mozilla was responsible for signing the extension as part of the process of uploading it; I don’t think there’s ever been a way for anyone else to sign it and have it accepted by recent versions of Firefox. But I don’t know much about it, it may be worth asking on the Mozilla dev mailing lists.

I'll be thankful if you suggest one [mailing list] to begin with. I can explore the topic myself, but since you have some experience, may be it's easier for you to point where to look. (hmm.. I wonder if Pale Moon requires same singing as FF or they have built another infrastructure, may be worth researching)

No problem with adding you to the repo.

Right, would be nice.

Ok, let me discuss only a fraction of technical details in this reply to answer sooner and to focus on organization aspects first (and actually technical details will be easier to discuss in the saving docs repo). I'll reply to some points which I can do quickly:

Syncing is the easier to explain of the two: TWC has plugins that let it sync changes to TiddlyWeb, a venerable Python-based TiddlyWiki serverside we developed at Osmosoft. I’ve not tested it recently, but those plugins should also enable TWC to save back to a TW5 server.

Sounds interesting. This is namely two-way syncing or one-way saving of separate tiddlers? In other words, can it be used for simultaneous editing of different tiddlers by different users so that they see the changes made by others without reloading TW? Is syncing tiddler-wise? Meaning that it can be used for what I've just described but cannot be used for simulteneous editing one same tiddler by different users?

By the way, why saving the whole HTML is not easier to explain: is it very different from what TWC uses for ordinary saving?

TW5 supports saving in two completely different ways: saving the entire HTML file versus syncing individual tiddlers to a server.

Same question here: TW5 supports granulated (tiddler-wise) saving or two-way syncing? (which would be very cool)

In TW5, saving is modular: there are eleven built into the core:

So each of them implements both "full-TW-saving" and "syncing"?

Ok, I'll stop here for now. The other things have more nuances to describe/learn/discuss, I'll just leave unanswered quotes below.

Best regards,
Yakov.

Next and the main point of this thread is: both TWC and TW5 have many supporting solutions (native apps, servers, browser plugins etc) for saving but I'm not aware of docs that describe how actually one supports saving (and may be more functionality) of both TWC and TW5. For instance, MainTiddlyServer could be extended a bit so that it fits the php server gap, but I (as a developer of TWC extensions, MTS and maintainer of the core) am not really aware of how to support TW5 correctly; likewise, the aTW saver for Android supports only TW5 (as far as I know), but I'd like to participate and support TWC as well, yet I still didn't even get an answer about the app status (and I suspect Simon may be reluctant to support TWC without nice docs describing that). Finally, it seems every time one tries to support saving in their software, they invent some new implementation which (this diversity) is difficult to follow and the ecosystem is quite difficult to maintain.

[TW5 2 saving ways]

In TW5, saving is modular: there are eleven built into the core:

• andtidwiki - for saving within the AndTidWiki Android app
• beaker - for saving within Beaker Browser
• download - for the fallback HTML5 compatible download saver
• fsosaver - for the Internet Explorer ActiveX saver
• manualdownload - a version of the download saver for older browsers that don’t support the necessary HTML5 features
• msdownload - a version of the download saver for older Microsoft browsers
• put - for saving to WebDav servers
• tiddlyfox - for saving under TiddlyFox
• tiddlyie - for saving to the Internet Explorer TiddlyIE extension
• twedit - for saving within TWEdit, an old iPad/iPhone app for TiddlyWiki
• upload - for saving to TiddlySpot, or a PHP server running the upload script

The TiddlyFox saver talks to TiddlyFox via a hidden DIV and custom events.

TiddlyFox itself has different handling for TW5 and TWC. For TW5, it uses the hidden DIV and custom events, and things are reasonably elegant. For TWC, TiddlyFox has to inject overrides to the saving functions which then set up the hidden DIV and handle the custom events.

[PMario]

On Tuesday, April 9, 2019 at 11:26:43 PM UTC+2, Yakov wrote:

In TW5, saving is modular: there are eleven built into the core:

So each of them implements both "full-TW-saving" and "syncing"?

TW savers just save the whole TW file. No syncing.

adaptors are able to create a 2 way syncing mechanism.

-mario

[PMario]

On Wednesday, April 3, 2019 at 10:05:38 AM UTC+2, Jeremy Ruston wrote:

Hi Yakov

I'll start with TiddlyFox since it is still used by many users (with either FireFox 56- or Pale Moon, WaterFox and other FF forks). One thing about it is building the xpi: I've read about building it a little and the crucial quesion is: is it still possible to create signed xpi, or updating TiddlyFox is impossible in principle?

Mozilla was responsible for signing the extension as part of the process of uploading it; I don’t think there’s ever been a way for anyone else to sign it and have it accepted by recent versions of Firefox. But I don’t know much about it, it may be worth asking on the Mozilla dev mailing lists.

It's NOT possible to sign old XPI addOns with the existing Mozilla AMO. ... Addons.mozilla.org (AMO)

see: 

 - About Firefox AddOns  ... some basic info to start with

 - Signing and dirstributing AddOns ... more Developer related info.

 - OUTDATED info about XPI signing

If you present a XPI to AMO it will be rejected.

have fun!

mario

[PMario]

Hi,

There are 2 ways to distribute an AddOn

 - Self distribution with eg: github. I use this process for betas. manual update needed.

 - Distribution with addons.mozilla.org ... I use this mechanism for stable versions. If AddOns are updated there, they will be automatically installed by the browser.

The important thing is: Both ways need AMO to sign the AddOn.

-mario

[PMario]

Some info about TiddlyWeb.

Terms:

tiddler ... contains the minimal useful content to work with

bag ... A bag is a container of uniquely named tiddlers. A bag can have rules for who can edit, delete or read the tiddlers in the bag. These rules are defined in a policy.

policy ... A policy is the TiddlyWeb concept for controlling access to resources on the system. That is: it is the mechanism by which authorization is managed. There are policies associated with each bag and each recipe.

recipes ... A recipe is an ordered list of bags and filters that generates a list of tiddlers.

    /bags/system/tiddlers?select=tag:systemConfig;sort=-modified
    /bags/{{ user }}/tiddlers

space ... A user friendly name for a 2 recipes. A space was bound to 1 user (by default). It did contain a "private" and a "public" recipe. If you where authenticated and the owner, you could see the "private" space. If you where a guest, you could only see the "public" space.

More info can be found at tiddlyweb.com .. Info from above can be found at the Abstract Overview.

On Tuesday, April 9, 2019 at 11:26:43 PM UTC+2, Yakov wrote:

....

Syncing is the easier to explain of the two: TWC has plugins that let it sync changes to TiddlyWeb, a venerable Python-based TiddlyWiki serverside we developed at Osmosoft. I’ve not tested it recently, but those plugins should also enable TWC to save back to a TW5 server.

Sounds interesting. This is namely two-way syncing or one-way saving of separate tiddlers?

2 way syncing was possible, but (I think) it wasn't implemented in the client UI. Every tiddler had it's own history. The UI had a simple mechanism to compare 2 version. So there was an easy fix.

The "Space" philosophy worked with "stacked" recipes.

eg:

  /bags/system/tiddlers?select=tag:systemConfig;sort=-modified
  /bags/{{ userBBB }}/tiddlers

  /bags/{{ userME }}/tiddlers

All bags are combined into 1 TW, which was served to the client.

While content of userBBB can be seen, it can't be changed. ... If I want to change a tiddler, I had to clone and publish them in my userME bag.

The last bag in the recipe list WINS! This concept is unprecedented and it is extremely powerful. Our future goals shouldn't be less than that!

 

In other words, can it be used for simultaneous editing of different tiddlers by different users so that they see the changes made by others without reloading TW?

Would have been possible, but I think the sync-adaptor pulled a "skinny list" of all tiddlers and

 

Is syncing tiddler-wise?

yes

 

Meaning that it can be used for what I've just described but cannot be used for simulteneous editing one same tiddler by different users?

With TiddlyWeb, we had so-called spaces. A space was bound to a user. Users only have write access to their own spaces.

Personal Note!

Nobody except me, has the right to write to a space that I own. .. period! If you want to change my content, clone it, modify it and distribute it on your own space. (Assumed the right license is in place.)

 

By the way, why saving the whole HTML is not easier to explain: is it very different from what TWC uses for ordinary saving?

TW savers are much, much simpler than adaptors.

have fun!

mario

[PMario]

On Sunday, March 24, 2019 at 3:58:40 PM UTC+1, Yakov wrote:

...

in the latest release of TiddlyWiki Classic I've fixed the upgrading engine and now it's time to deal with saving more consistently. That's where I'd like not only to ask about some dev aspects but also discuss some conventions and create docs for consistent support of saving both TWC and TW5.

Some basic info about savers can be found here: https://tiddlywiki.com/dev/#Saver

If you have a closer look at the implementations

 

 - tiddlyfox.js ... outdated, but the mechanism is the same.

 - beaker.js ... new saver using the dat-project

 - put.js ... WebDav PUT saver.

IMO this info should be enough to create your own simple saver. ... I think the upload.js used a php server side.

If a "save action" is triggered, the core uses the saver.canSave() info to check if the saver can be used at all. saver.info() returns the saver capabilities and the priority.

If several savers are able to provide the needed save action, the saver with the highest priority wins. ... ATM  "There can be only one!"

have fun!

mario

[Yakov]

Hi Mario and Jeremy,

thanks for your responses. Now that I've adapted Timimi for TWC via plugin and also created a PR to support it "natively" by Timimi, I've got more specific questions. First, Mario, could you clarify this bit:

tiddlyfox.js ... outdated, but the mechanism is the same

what you consider outdated? Do you refer to it being named "tiddlyfox.js" while it's used for Timimi and TiddlyDesktop which are more up-to-date? Or you mean that the saver design itself is outdated in some aspect?

Second question is: why all the savers call the callback with null as the first (and only) argument? I guess, it's a placeholder for some specific design, could somebody clarify it?

Finally, and that's my main current concern, I'd like to discuss the events system. The thing is, as far as I can see, "tiddlyfox" saver only relies on the tiddlyfox-have-saved-file event fired on successful saving, it doesn't listen to any ~tiddlyfox-have-failed-to-save-file event. And hence Timimi doesn't implement firing such events. Which is not nice in some cases (say, you have a TW on a removable USB storage device, you remove it, then press save, and don't get any notification about saving actually failed) and for some applications (well, that's needed at least in TWC upgrading engine which displays the stage/status of the procedure). So I'd like to propose introducing such an event. But since Timimi is designed for TW5 in the first place, I'd like to discuss the design of the event with you so that Rizwan have a reference and can participate the discussion too.

Now, tiddlyfox-have-failed-to-save-file is definitely not a nice name, because TiddlyFox is both outdated and doesn't fire such an event. So it should be something more abstract, I think. Like tiddlywiki-saver-have-failed-to-save-file or something shorter like tiddlywiki-saver-fail. Ideally, such an event should hold some values that help identify the cause of a fail (folder doesn't exist, permission denied, .., unknown), presumably in a form of some codes (unknown = 0, folder doesn't exist = 1, ...) so that localization is possible. A second thought is: if such new event is introduced, perhaps new events should be used for saving-request and saving-succeeded too (both old and new events should be fired for backward compability): I'd propose tiddlyfox-have-saved-file → tiddlywiki-saver-success and tiddlyfox-save-file → tiddlywiki-saver-request. So that we have:

init saving: tiddlywiki-saver-request (+ tiddlyfox-save-file for backward compatibility)
saving succeeded: tiddlywiki-saver-success (+ tiddlyfox-have-saved-file for backward compatibility)
saving failed: tiddlywiki-saver-fail

Perhaps an even better choise would be to give names for arbitrary IO operations (saving, loading, saving binaries etc) which could be:

init saving: tiddlywiki-backend-save-request (+ tiddlyfox-save-file for backward compatibility)
saving succeeded: tiddlywiki-backend-save-success (+ tiddlyfox-have-saved-file for backward compatibility)
saving failed: tiddlywiki-backend-save-fail
and can further be extended to things like
tiddlywiki-backend-load-request, -success, -fail
etc

What do you think?

Best regards,
Yakov.

PS1 I think the scope of docs which I proposed in this thread is actually "TiddlyWiki interoperability and data formats".

PS2 Besides multiline fields, there's another issue about .tid format, namely convention regarding filenames (say, tiddler name contains * which is not allowed in filenames – what should be the .tid filename? Obviously, the convention should describe a reversible convertion (without ambiguities).

PPS To test: how does TiddlyDesktop handle such fields and filenames in .tid files?

среда, 10 апреля 2019 г., 12:37:38 UTC+3 пользователь PMario написал:

[Yakov]

By the way, I've found a fix for TiddlyFox's injectedLoadFile so that it now works with paths containing non-ASCII chars and proposed to apply it to Timimi (along with supporting TWC). I've took a look at injectedLoadFile implementation of TiddlyDesktop and found that it, like Tiddloid, doesn't load any files except for TW itself. Jeremy, is there any particular reason for such a restrictive implementation? Are xhr forbidden in that environment? This is even more important since I presume that Tiddloid's implementation is actually a copy of TD implementation.

понедельник, 17 июня 2019 г., 15:36:39 UTC+3 пользователь Yakov написал:

[Jeremy Ruston]

Hi Yakov

Second question is: why all the savers call the callback with null as the first (and only) argument? I guess, it's a placeholder for some specific design, could somebody clarify it?

Invoking the callback with null indicates that there was no error. An error is indicated by passing a string. You can see the callback that is used when the savers are invoked here:

https://github.com/Jermolene/TiddlyWiki5/blob/master/core/modules/saver-handler.js#L163-L177

Finally, and that's my main current concern, I'd like to discuss the events system. The thing is, as far as I can see, "tiddlyfox" saver only relies on the tiddlyfox-have-saved-file event fired on successful saving, it doesn't listen to any ~tiddlyfox-have-failed-to-save-file event.

That’s correct. The TiddlyFox design assumes that the save succeeded if TiddlyFox accepts it. (The reason is that originally TiddlyFox was implemented for TWC where the save functions are synchronous, but we were actually triggering an asynchronous operation. Thus there was nowhere to deliver the asynchronous error return).

And hence Timimi doesn't implement firing such events. Which is not nice in some cases (say, you have a TW on a removable USB storage device, you remove it, then press save, and don't get any notification about saving actually failed) and for some applications (well, that's needed at least in TWC upgrading engine which displays the stage/status of the procedure). So I'd like to propose introducing such an event. But since Timimi is designed for TW5 in the first place, I'd like to discuss the design of the event with you so that Rizwan have a reference and can participate the discussion too.

Now, tiddlyfox-have-failed-to-save-file is definitely not a nice name, because TiddlyFox is both outdated and doesn't fire such an event. So it should be something more abstract, I think. Like tiddlywiki-saver-have-failed-to-save-file or something shorter like tiddlywiki-saver-fail. Ideally, such an event should hold some values that help identify the cause of a fail (folder doesn't exist, permission denied, .., unknown), presumably in a form of some codes (unknown = 0, folder doesn't exist = 1, ...) so that localization is possible. A second thought is: if such new event is introduced, perhaps new events should be used for saving-request and saving-succeeded too (both old and new events should be fired for backward compability): I'd propose tiddlyfox-have-saved-file → tiddlywiki-saver-success and tiddlyfox-save-file → tiddlywiki-saver-request. So that we have:

init saving: tiddlywiki-saver-request (+ tiddlyfox-save-file for backward compatibility)
saving succeeded: tiddlywiki-saver-success (+ tiddlyfox-have-saved-file for backward compatibility)
saving failed: tiddlywiki-saver-fail

Perhaps an even better choise would be to give names for arbitrary IO operations (saving, loading, saving binaries etc) which could be:

init saving: tiddlywiki-backend-save-request (+ tiddlyfox-save-file for backward compatibility)
saving succeeded: tiddlywiki-backend-save-success (+ tiddlyfox-have-saved-file for backward compatibility)
saving failed: tiddlywiki-backend-save-fail
and can further be extended to things like
tiddlywiki-backend-load-request, -success, -fail
etc

What do you think?

We could perhaps stick with “tiddlyfox” in the names, with the idea that we’re re-assigning the name to refer to the protocol rather than the original Firefox extension. In the docs we’d talk about the “TiddlyFox protocol”.

Perhaps the next step is to formally document the current V1 TiddlyFox protocol at tiddlywiki.com/dev, alongside the proposal for the V2 protocol.

Most important is to retain as much backwards compatibiltiy as possible. A v1 tiddlyfox saver in TW5.1.x should work with a V2 tiddlyfox host, and vice versa.

I’m happy to update TiddlyDesktop to the new protocol.

Best wishes

jeremy

PS1 I think the scope of docs which I proposed in this thread is actually "TiddlyWiki interoperability and data formats".

PS2 Besides multiline fields, there's another issue about .tid format, namely convention regarding filenames (say, tiddler name contains * which is not allowed in filenames – what should be the .tid filename? Obviously, the convention should describe a reversible convertion (without ambiguities).

The filename of .tid files is ignored; the system uses the title field within the file as the tiddler title

среда, 10 апреля 2019 г., 12:37:38 UTC+3 пользователь PMario написал:

On Sunday, March 24, 2019 at 3:58:40 PM UTC+1, Yakov wrote:

...

in the latest release of TiddlyWiki Classic I've fixed the upgrading engine and now it's time to deal with saving more consistently. That's where I'd like not only to ask about some dev aspects but also discuss some conventions and create docs for consistent support of saving both TWC and TW5.

Some basic info about savers can be found here: https://tiddlywiki.com/dev/#Saver

If you have a closer look at the implementations

 

 - tiddlyfox.js ... outdated, but the mechanism is the same.

 - beaker.js ... new saver using the dat-project

 - put.js ... WebDav PUT saver. 

IMO this info should be enough to create your own simple saver. ... I think the upload.js used a php server side. 

If a "save action" is triggered, the core uses the saver.canSave() info to check if the saver can be used at all. saver.info() returns the saver capabilities and the priority. 

If several savers are able to provide the needed save action, the saver with the highest priority wins. ... ATM  "There can be only one!"

have fun!

mario

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/4592137f-a8e6-442f-82fe-f96ab88599e9%40googlegroups.com.