[TW5] Is there a way to import TW5 engine through <script source> tag dynamically and left in my html page only set of payload tiddlers?

[Martian]

I'd like to import TW5 core and etc dynamically like any other JS libraries and to have just small html page with set of my tiddlers.

[Jeremy Ruston]

Hi Martian

I'd like to import TW5 core and etc dynamically like any other JS libraries and to have just small html page with set of my tiddlers.

Thanks for the question, it has cropped up before but it hasn’t really had much discussion.

Firstly, it is eminently possible to structure TW5 as an HTML file containing the tiddler store area that includes a JavaScript file containing the code; it’s just a matter of rearranging some of the existing templates. I’ve made a very quick lash-up to demonstrate how it can be done in this branch:

https://github.com/Jermolene/TiddlyWiki5/tree/externalise-js

The changes from the master branch can be seen here:

https://github.com/Jermolene/TiddlyWiki5/compare/externalise-js

To try it out, download the branch and then on the command line execute:

tiddlywiki.js editions/tw5.com --build external-js

And then in the `editions/tw5.com/output` folder you’ll find:

index.html - 3,889,031bytes

tiddlywiki.js - 1,878,319 bytes

(Compared to the standalone.html size of 4,844,364 bytes)

TiddlyWiki is functional in this arrangement, even with the default download saver, as long as you make sure that the tiddlywiki.js file is in the same folder as the index.html file. It’s currently configured so that clicking the ‘save changes’ button will save just the HTML portion of the split format. We could also make it switchable between the self contained vs. split format, and we could add a button to download the tiddlywiki.js file.

Now, there are some undeniable advantages to this arrangement:

* The HTML file is smaller

* Browsers can cache the tiddlywiki.js file, improving online performance

* Lots of TiddlyWikis can share the same tiddlywiki.js file, saving a couple of megabytes each time

* One could upgrade all the TW HTML files in a directory just by updating the tiddlywiki.js file

But I wouldn’t be inclined to adopt this arrangement as the default because it also introduces a bunch of significant problems that stem from the fact that the TiddlyWiki file is no longer self contained:

* Users need to remember to copy the tiddlywiki.js file around with it as well, adding to the cognitive load of using TiddlyWiki

* The documentation for getting started (which is already pretty hair-raising) would get significantly more complex

Nonetheless, I’m interested to see people experiment with the idea, and learn more about the pros and cons. If there’s any interest I might restructure this branch as a plugin, and add some user interface bits.

Another thought that occurs to be is that this mechanism could be used to share tiddlers between several wikis: it’s possible to pack arbitrary tiddlers into the tiddlywiki.js file.

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/8eea3bd9-b898-4c1e-a984-5f1f072e318e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[PMario]

On Wednesday, July 4, 2018 at 1:59:56 PM UTC+2, Jeremy Ruston wrote:

tiddlywiki.js editions/tw5.com --build external-js

And then in the `editions/tw5.com/output` folder you’ll find:

index.html - 3,889,031bytes

tiddlywiki.js - 1,878,319 bytes

For me the interesting one is empty.html which is 85 kByte with the new branch.

That's very interesting for WebDav, if the js-part can be cached. ... BUT it seems to mess up the existing file-saving extensions. ...

I need to make more tests, but I wouldn't recomment to use it for production atm.

have fun!

mario

[Martian]

That's great! Thank you! I will definitely play with this externalize branch.

I have another crazy idea - to integrate ClojureScript compiler into TW5 to use it in a line with JS. And think such externalization could be helpful.

[Jed Carty]

I would like this too. I have been trying to find ways to make Bob load wikis faster and I already have it set up so plugins that are specific to the node side aren't pushed to the browser, being able to cache tiddlywiki.js in the browser would greatly reduce the load times for when Bob is served on a vps, currently they are much longer than I would like.

Jeremy,

I will look at the branch you have already made and see if there is anything I can help with.

[PMario]

On Wednesday, July 4, 2018 at 1:59:56 PM UTC+2, Jeremy Ruston wrote:

...

Another thought that occurs to be is that this mechanism could be used to share tiddlers between several wikis: it’s possible to pack arbitrary tiddlers into the tiddlywiki.js file.

I'm not very happy with this idea. ...

Splitting content and the core already creates a versioning compatibility probelm. ... Not now, with the first approach, but in the future.

There should be only 1 core per version, so it is possible to run a "fingerprint" hash-check, to verify it the core file plays nice!

Just some thoughts.

-mario

[PMario]

Hi,

I did play a little bit with the WebDav idea. ...

My conclusion

 - I'll switch on server side compression for static content

   - Which reduces the file empty.html size that is transfered from 2.000 kByte to about 400 kByte

 - I'll activate the caching mechanism

   - So the whole file will be loaded from cache most of the time.

I think it's not worth the problems for my WebDav setting.

THE PROBLEM!

 - I need to use a modified WebDav PUT saver, which will be a plugin. ..... Soon!

have fun!

mario

[TonyM]

Jeremy et al..

I was going to raise this idea separately, but since it is somewhat related I am adding here, I hope it is not "off topic", I don't think it is.

A TWX version as they say

In part reacting to your comments, an my morning coffees, here is my related idea that this externalise approach may facilitate, the idea of "compiling" a TiddlyWiki, perhaps compiling is not the correct word.

• Such that if such an externalised scripts method, could include some user customised system tiddlers, it may be possible to place settings and encryption keys into the tiddlers and external file,
  such that it is effectively unreadable until it loads in the specific wikis context (or group of wikis). Perhaps there is a public private key pair between the shell wiki and the external scripts file. "You need one to get the other, to get the first"
  
• A key could also be pulled from a user ID password file, so unless logged in, they can not use the wiki. 
  
• if the wiki is secured with a key in the user ID password File, they would be prohibited from downloading a single file version they could read without being given the key.
• This would allow secure documents and propriety information to be published without concern of bulk theft.
• It may also allow people to sell end user licences copies of tiddlywiki editions/solutions that are activated by saving a key, but which once accessed largely remain an open source solution that can be changed.
• Editions or plugins could be distributed that generate the encrypted script file and the shell wiki, or a wiki from the current file/folder wiki which can then be placed online or network shares (in the multi-file form).
  
• An (encrypted) security feature could determine whether save as (effectively generate a single file) is possible or not from a milti-file form)
• With node online every wiki can share this single external file, which can be cached once for any network of wikis. This may get around the core versioning issues and single file issues by making the split into multiple files the last thing you do.
  
• This may also help reduce the save size of an online single file wiki as implied elsewhere in this thread?, without resorting to a server.

I understand your desire to keep the single file model primary to tiddlywiki, but as you can see with all the server solutions people are trying to build more advanced versions to meet various requirements, If such wikis can be exported as single file versions, whilst allowing them to be secured sometimes, it is even more powerful. 

In the future

• There may be a way to split out key macros to another encrypted file so that some key algorithms can be exported, shared amongst wikis and not visible in the main wiki file (An example could be a payment processing module), Yes these things will be visible in memory but you are 99% of the way of securing a TiddlyWiki from users, if they do not have a way to edit key tiddlers and 90% of the way from securing it against hacking (un-trusted users at least)
  
• For really large wikis online perhaps additional division of externalised core or user static data could be done to further optimise performance, like lazy load a products database.

The more secure TiddlyWiki is against hacking the easier it is to open update access on the internet, for a higher level of interaction.

To me this ability for tiddlywiki to operate securely and at high performance on the Internet is possibly the last border for tiddlywiki to cross, to become truly a universal solution, I have at least three startup ideas that could leverage this.

Regards

Tony

[TonyM]

Mario,

See my prior response where I suggest 

Editions or plugins could be distributed that generate the encrypted script file and the shell wiki, or a wiki from the current file/folder wiki which can then be placed online or network shares (in the multi-file form).

"This may get around the core versioning issues and single file issues by making the split into multiple files the last thing you do" and you must do again to upgrade the core.

Also the save as single file (by Authorised users) would allow the whole wiki to be saved back to the developer, upgraded, "tested" and republished.

Regards

Tony

[TonyM]

Jed,

In my above suggestion I am cognisant it should not break such solutions as Bobs multi-user, so I would also appreciate your consideration of how to maintain this in the suggested approach of mine.

Regards

Tony

[Jed Carty]

Tony,

A lot of what you describe is either already working or planned for my public facing Bob server, but in a different way than what you describe. The first two points are largely covered by having a secure login and fine-grained access control to wikis and tiddlers. So far I have fine grained control to wikis, I don't have the tiddler level set up yet but it will not be too hard to add it in.

As far as the fourth item is concerned (preventing downloads of the wiki file), it is very hard to serve content that you don't want copied in a tiddlywiki. The technologies that allow that sort of thing are, in my opinion, very bad ideas and are a big part of how botnets are formed. It is doable, but browser drm leads to bad things.

Having the base tiddlywiki cached separately is a good idea I think and I would like to find a good way to do that.

None of these pieces would necessarily break the single file wikis. They seem to mostly be based around the idea of selectively sending tiddlers to a wiki similarly to what Bob currently does but with more control over individual tiddlers.

And you make it sound like I should see about getting you a login on my Bob server online. There are many little details that I am working on fixing to make it polished but I have been using it for my work. I want to make it more polished before I start advertising it but it is available now.

[TonyM]

Jed,

Thanks, what you are doing is exciting. My main issue was if we do divide tiddlywiki in the hosting world, that those changes do not break Bob's essential techniques, via the messaging etc... I value what Bob is doing too much, and I want others to know they should do nothing that breaks it. I also wanted you to be across this so that you can participate as the Bob representative.

I look forward to your future solutions

Regards

Tony

[Martian]

--//  some time passed...  //--

Hi guys! 

Just want to say thank you and  just in case to direct to the place where this thread have resulted :)

https://tiddlywiki.com/#Using%20the%20external%20JavaScript%20template

(cause there is no more that https://github.com/Jermolene/TiddlyWiki5/tree/externalise-js Mr.Ruston mentioned )

Maybe it's time to eat these fruits