How does TW5 load modules/plugins?

[TJ Walker]

In my exposure to software architecture (not extensive), I've seen nothing like whatTW5 seems to me to be, a fully interactive browser based application, featuring a user facing domain-specific language (WikiText), and a module/plugin system for extensibility, all self-contained in a single HTML file. I wonder about how much of it is designed and implemented. To start, though, I'm curious as to how the app manages to load the strings (are they stored as strings?) representing modules into itself? I'd love to understand this on every level; abstract, are there design patterns concerned with this, general programming notions, et cetera; down to the concrete, is eval() used, what files/lines in the git repo are responsible for turning module text into executable JS? Hope this makes sense. Thanks.

[TonyM]

TJ,

Have you reviewed https://tiddlywiki.com/dev/ ? especially the architecture etc...

You wonder "about how much of it is designed and implemented" - all of it I would think. TW5 is Built on top of the earlier tw classic which has now being around 10+ years and is quite mature. 

Of critical note is almost, if not all, that TW does is visible inside every copy of TW itself. If you look into the $:/core and its subtiddlers, the tiddler idea is used from near the very beginning.

A word of warning, TiddlyWiki is so adaptable it easily becomes an obsession, along with presenting almost infinite possibilities, and as a result novel and expansive possibilities are often discussed and many implemented. It is a firmament of innovation and engages user level through to deep coder input.  

You question will no doubt be answered by someone with a deeper coding experience to me but my point is the answer may already be sitting in your empty copy.

Regards

Tony

[TJ Walker]

Hi, again, Tony 👋

I'd not seen the dev specific docs. Thanks for sharing, I'll give that a look.

Yeah, I'd seen the $:/core jawn in my copy. In fact, it is in looking at that that I pondered "man, how is this turned into executable code; how is this plain-text made evaluate-able?". I inspected the index.html that is my TW wiki and was instantly overwhelmed.

You wonder "about how much of it is designed and implemented" - all of it I would think.

LOL agreed [facepalm]. I realize now that that phrasing reads ambiguously. I meant "there is much about the design and implementation of TW that I am curious about".

[Jeremy Ruston]

Hi TJ

Great questions!

In my exposure to software architecture (not extensive), I've seen nothing like whatTW5 seems to me to be, a fully interactive browser based application, featuring a user facing domain-specific language (WikiText), and a module/plugin system for extensibility, all self-contained in a single HTML file.

Indeed, TiddlyWiki is rather an outlier in the software world. The ability to run as a self contained HTML file requires some unorthodox design choices. It’s nature as a wiki and not a database also sometimes confounds expectations (that’s easy to explain: TiddlyWiki seeks to help you work with semi-structured data where the structure changes over time, and doesn’t try to achieve the things that we can already achieve with Microsoft Access and its descendents).

Anyhow, some of the principles of TiddlyWiki:

* Information is more useful when it is re-usable, and information is more re-usable if it is cut up into the smallest semantic units and threaded back together into multiple interactive hypernarratives

* Everything is a tiddler. Literally everything that could be a tiddler is represented as one: the motivation is so that we can aggressively re-use the tiddler handling mechanisms for everything else

* Everything is a parse-render-refresh cycle. The heart of TW5 is an engine that parses wikitext tiddlers to produce a render tree that can be efficiently updated when the tiddler store changes. In the browser, the render tree can be directly attached to the DOM. On the server, we extract HTML from the render tree. Again, we reuse this process endlessly: it’s how tiddlers are rendered, but it’s also the mechanism used to generate the standalone HTML file when saving changes in the browser

* Everything is wikitext. The entire user interface of TW5 is implemented as wikitext, and as such is easy for anyone to customise as deeply as they need

The weird architecture of not requiring a server has ended up being more useful than I thought. By being aggressively self contained TiddlyWiki is very well suited to emerging JavaScript platforms such as AWS Lambda and cryptographic storage mechanisms like Least Authority Filing System or Beaker browser.

I might add that the highest calling of TiddlyWiki is to give special powers people who are too busy being something else to learn to be a software developer. This is important because we live in a world mediated by software and yet for most of the population software is an inscrutable thing requiring absurd investment to understand and exploit. So most people end up using mass market software that was designed for somebody else, and never for you. TiddlyWiki, and the community of sharing around it, gives everyone the opportunity to evolve their own custom system, whatever their level of technical skill.

I wonder about how much of it is designed and implemented. To start, though, I'm curious as to how the app manages to load the strings (are they stored as strings?) representing modules into itself? I'd love to understand this on every level; abstract, are there design patterns concerned with this, general programming nations, et cetera; down to the concrete, is eval() used, what files/lines in the git repo are responsible for turning module text into executable JS? Hope this makes sense. Thanks.

TiddlyWiki 5 incorporates a custom implementation of the CommonJS JavaScript module standard (http://wiki.commonjs.org/wiki/Modules/1.1) that is based on tiddlers instead of files, and works in the browser and under Node.js. In the browser it uses eval(), but under Node.js it uses vm.runInThisContext() which has (slightly) better sandboxing.

The top level structure of TW5 is that there is a boot kernel of (~2,000 lines) that sets up the CommonJS module environment and then invokes the main core plugin ($:/core) that contains JS modules and wikitext tiddlers that make up the core of TiddlyWiki itself.

A good path for exploring TW5 code is:

* Review the standalone HTML file in a text editor, and locate the various parts (the boot kernel, the tiddler store, the boot CSS etc)

* Review the source of the boot kernel: https://github.com/Jermolene/TiddlyWiki5/blob/master/boot/bootprefix.js and https://github.com/Jermolene/TiddlyWiki5/blob/master/boot/boot.js

* Review the core plugin content by systematically looking through here: https://github.com/Jermolene/TiddlyWiki5/tree/master/core

Very happy to answer any further questions that I can,

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/2ae8a3bb-e5e6-47f0-aa8e-156d4f766e73%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[TJ Walker]

Just the sort of clarification and direction I'd hoped for. Thanks, Jeremy.

[jologsd...@gmail.com]

I might add that the highest calling of TiddlyWiki is to give special powers people who are too busy being something else to learn to be a software developer. This is important because we live in a world mediated by software and yet for most of the population software is an inscrutable thing requiring absurd investment to understand and exploit. So most people end up using mass market software that was designed for somebody else, and never for you. TiddlyWiki, and the community of sharing around it, gives everyone the opportunity to evolve their own custom system, whatever their level of technical skill.

This is absolutely how TW works for me. When there's a kind of cross-linking or transclusion style I need done, TW can do it. And when I want to know how TW does something, I can just read the available documentation or lurk around the community. That's how I learned it's possible to style the heck out of it, which really was the thing that made me stick with the system. I'm a sucker for customizable minimalist design, and powerful engines. Other text editors don't have that combination, or maybe there's one out there that could, but maybe it's not less than 2mb in size.