Tiddlywiki Empty: The size empty.html

[Mohammad]

Tiddlywiki is rapidly improving and you can see great features in the recent releases. 

From 2.1.17+ amazing features have been added to Tiddlywiki! One question is about the size of empty.html (the virgin Tiddlywiki). See the below table

Release   Size (kb)

5.1.12       1820

5.1.15       2025

5.1.17       2033

5.1.19       2153

5.1.21       2235

5.1.22       2197

5.1.12       2282

While I love the new features, but, how big a virgin Tiddlywiki  can be? Assuming many users work with single file mode and using different Tiddlywiki for different purposes, I think we should set a maximum size, before going to have bigger empty.html

Suggestions

A. Use official plugins

1. Keep the core as light as possible, go down to 1MB size (strip everything extra)

2. Put extra features in official plugins

B. Start a new generation 

1.Release TW 5.2 with minimum size core (include only essential part)

2. Leave all backward compatibility to Tiddlywiki 5.1.xx

3. Stop developing 5.1.xx and only release bug fixes

If Jeremy can go for a new generation of Tiddlywiki, I may also suggest

1. Sweep the core from all duplicated codes and those retained backward compatibility and leave them for 5.1.xx

2. Rethink about filters and implement using the more versatile approach, like those are going on in GitHub (e.g. Saq proposal for multi input filters)

3. Use flexible switchable page layout

4. Think for a cleaner scripting (remove all duplication, improved grammar, ... there is a lot of good discussions in the forum and GitHub)

5. Think of a new name (re-branding)

6. ...

I am sure people can suggest more here

Best wishes

Mohammad

TW-Scripts codes, macros, and solutions in Tiddlywiki

TW-Commander bulk operations on tiddlers

TW-Trashbin a Tiddlywiki trashbin tool

TW-Favorites a favorites and bookmarking tool

TW-Todolist organize, prioritize, and plan your work

https://github.com/kookma

[Charlie Veniot]

I'm an instant fan of your thoughts.

Your post makes me think of Linux  (Tiny Core Linux and, even more so, the stripped-down version: Micro Core Linux.)  It would be neat for TiddlyWiki to have a Debian-like repository and something akin to a package manager.

Yeah, I dream kind of grandiose...

[Ste]

Welcome back  Mohammad!

Diving in with some heavyweight thoughts!

Fit tiddlywiki on a floppy!...(I found some in an old lap top bag the other day....none of my students knew what it was.)

I've not been a party to the dev/ github discussions but as much as new and shiny appeals, losing say, tidgraph...NOOOOO.  But then my existing wiki isn't going anywhere I guess (at least now I have a backup from tiddlyspot!).

Or would it be a case of SOME things breaking, SOME things still working?  How radical an overhaul is it?

Tiddlywiki classic, 5.1-classic and shiny...hmmm

Ste

[TW Tones]

Mohammad et al,

I am all for a minimum tiddlywiki being freely available and empty.html has provided this, perhaps some of you suggestions can be moved into it. However as I have stated a number of times empty.html is for people who know what they are doing, building from a base. It is not a good starting place for new users but it is the first download available to new users. 

Especially in functionality was reduced for space in empty.html but already I think we should have a "standard edition", with a little more functionality for users to help them be productive sooner.

It should be quite easy to document, even have a plugin that "reduces" an empty or standard wiki to a minimalist one so perhaps that is what we should use not another edition?

Regards

Tones

[Mohammad]

Hi Ste,

 Many thanks for your kind words. 

I like to give a real example. Fortran is a very old language and still in progress in 2020 and its latest standard 202x was released ( first draft ) a few months ago ago.

Fortran is very popular in the scientific community when it comes to high performance computation and number crunching. Its 2020 compiler (support Fortran standard 2018) can compile and run the code back to 1970. There are many deprecated features, by default compiler uses the recent standards but there are compiler switches to tell compiler to compile a legacy code with many deprecated rules.

What I propose for Tiddlywiki is like Fortran. The core can support most of the legacy tiddlers in TW5 but the TW 5.2.xx by default works based on new clean and light core and does not support deprecated rules. So what people can do if they have legacy tiddlers, plugins, ... in new TW 5.2.xx?

Like Fortran which has compiler switches, TW 5.2.xx can have an official plugin e.g. backwardcompatibility plugin which can be installed on demand and support all TW 5.x.yy

So while TW 5.2.xx is kept light, clean and stripped out and push users to use the new core which support good scripting styles (we can say new TW standards) , it has options (plugin) to simply used and support all legacy things (tiddlers, plugins, themes,...)

So by default TW 5.2.xx is based on stripped out, clean, minimal core BUT users have option to install official backwardcompatibility plugin(s) to work with all old things.

Best wishes

Mohammad

TW-Scripts codes, macros, and solutions in Tiddlywiki

TW-Commander bulk operations on tiddlers

TW-Trashbin a Tiddlywiki trashbin tool

TW-Favorites a favorites and bookmarking tool

TW-Todolist organize, prioritize, and plan your work

https://github.com/kookma

[Mohammad]

Hi Tones,

 That is quite true and I see your efforts in this regard. I totally agree with you, having some starter edition with enough plugins, table of content, themes, palettes, ...

 The empty.html as described by Josiah (the virgin edition) should be available for experienced users and for creating other editions.

I myself distribute a customized edition to my graduate students which contains: empty.html + commander + shiraz + utility + favorites + trashbin + tinyTodo + relink + jighlight.js + katex + codemirror.

They are happy with that. We use tag very carefully and try to have one line at the end of tiddler body as (keywords: list of related keywords)

e use heavily from 

- sidebar table of content

- tiddler menu bar info (for references, fields, ...)

- tag pill to find other tiddlers in that category

Best wishes

Mohammad

TW-Scripts codes, macros, and solutions in Tiddlywiki

TW-Commander bulk operations on tiddlers

TW-Trashbin a Tiddlywiki trashbin tool

TW-Favorites a favorites and bookmarking tool

TW-Todolist organize, prioritize, and plan your work

https://github.com/kookma

[Mohammad]

Hi Charlie,

Thank you. Well having a stripped out core as a clean light empty.html gives more flexibility to developer to create custom editions with many customization and keep it still light for sending it through email.

Best wishes

Mohammad

[David Gifford]

One thing that takes up space is the set of language tiddlers for translating to other languages. English-speakers don't need those, usually. What if those could be a plugin? Or else an English-only version stripped of those tiddlers? Feels arrogant and colonial even mentioning it. I only mention it as one area where TiddlyWiki could be at least a little lighter.

[Mark S.]

What makes you think that TW can be stripped down?

TW was born overweight, compared to TWC.

My guess is that the reason TW5 is so much bigger, is that so much of the core is written in wikitext instead of javascript. This makes TW5 extensible, but also bulkier.

[Mohammad]

Hi David,

 I assume other language shall be installed on demand!

I have downloaded the latest empty.html from https://tiddlywiki.com/prerelease/

These are some facts

$:/core

has 2047 tiddlers and occupies around 1781 kb

Listing all tiddlers with language word in their title using advanced search [all[shadows]prefix[$:/]search:title[language]]   results in 

965 tiddlers and they are 115 kb (I am not sure how part of these tiddlers can be distributed separately as language pack)

There is two themes distributed with empty.html

1. Vanilla: 73.5 kb

2. Snowwhite: 3 kb 

There are 16 palettes in $:/core which occupies 66 kb

Best wishes

Mohammad

[Mohammad]

Hi Mark,

On Wednesday, November 4, 2020 at 6:20:13 PM UTC+3:30 Mark S. wrote:

What makes you think that TW can be stripped down?

I am not quite familiar with  JS code behind the scene, but I see some of backward compatibility, some of non essential features can be stripped out and distributed in form of official plugin! Even filters and widgets can be cleaned and selected based on good programming styles and extra ones stripped out. But yes someone knows the Tiddlywiki internals can give a much more accurate measure of what really can be stripped out!

TW was born overweight, compared to TWC.

Yep! 

My guess is that the reason TW5 is so much bigger, is that so much of the core is written in wikitext instead of javascript. This makes TW5 extensible, but also bulkier.

I think so!

[odin...@gmail.com]

Pardon my ignorance. But what are the benefits of a smaller empty TW5? It seemed to be implied in this thread that this is the case.

Op woensdag 4 november 2020 om 16:00:09 UTC+1 schreef Mohammad:

[Mohammad]

With a little customization it gets 3 to 4 MB in size and it is a bit heavy for single html!

[Mark S.]

On a desktop with a local file, it doesn't make much difference. But on a small, older  device  over a slow data feed larger sizes will mean longer loading times and slower operation.

[Mark S.]

On Wednesday, November 4, 2020 at 7:00:09 AM UTC-8, Mohammad wrote:

Listing all tiddlers with language word in their title using advanced search [all[shadows]prefix[$:/]search:title[language]]   results in 

965 tiddlers and they are 115 kb (I am not sure how part of these tiddlers can be distributed separately as language pack)

You could hard-code the expressions into the system. However, it appears that there is only about 60 bytes overhead per expression. So you would only save 95kb (965 * 60) and you would lose a lot of flexibility.

I don't think there's  enough cruft in TW to pare it down to 1 M. The only way to get it down would be to rewrite major sections in Javascript.

[Mohammad]

I have not gone deep in TW core, but considering organic growth of TW it should be true!

[TW Tones]

Mohammad,

I think you will find empty.html is near its smallest, and as speeds and storage increases this size becomes even less important, but yes we hope tiddlywiki is universal, and there are very different circumstances the world over.

• On the server side there is a way to externalise javascript and of course make use of skinny tiddlers so arguably you could reduce the size "served" at least initially.
• Also since tiddlywiki is a single file there are arguments that its performance after initial load is higher and caching and CDN's may work better.
• Always use a splash screen to accommodate slow loading tiddlywikis.

Have you some pressing issues that makes you to take this path to minimise?

To me the same efforts would be better targeted.

Regards

Tones

[dieg...@gmail.com]

All,

Out of curiosity, when we say TW is big, what do we mean? relative to what? TWC? Anything else besides that?

What is the incentive to make it smaller and smaller? Something like: "The smaller TW gets, the more we _____"?

[Mark S.]

If TW5 could be made to run like javascript, then you could separate the core from the working files. So you could have a local core library which all your tiddlywiki files could resource, without having to include them in every download/upload/save.

On Monday, November 2, 2020 at 11:00:21 PM UTC-8, Mohammad wrote:

[TW Tones]

Mark,

As I understand it this is already possible. External Javascript.

I also wonder if one could selectively remove code not used, however this would need an analysis process. And an exclusion on save, perhaps an alternate core plugin.

The dynamic range of application of tiddlywiki would possibly be enhanced with a lower size however in todays world we most often have sufficient overheads that reduction has less benefit than being comprehensive or self documented. 

Regards

Tony

[Jeremy Ruston]

The idea of stripping TiddlyWiki down to a microkernel and making everything be a plugin comes up fairly frequently, and seems to be an idea with some appeal.

One factor that hasn’t been mentioned so far is that the microkernel architecture would make testing and support a good deal more complicated. Core developers and plugin authors would have to test against all the common combinations of core plugins. Every single support enquiry would have to be prefaced by a discussion to find out exactly what combination of plugins the user has.

Of course, to some extent we have this problem already because we already have plugins. But right now the goal for the core is for it to contain not the bare minimum possible functionality, but rather the functionality that has the potential to be universally useful. That broader criteria means that the core itself is sufficient for a lot of work with TiddlyWiki, making the life of users and developers a lot easier. 

A consideration that has already been mentioned is whether it really matters that the core gets big, especially now that we’re in a world where a simple news story weighs 20MB. There are valid concerns about slow networks but the best solution there is to set the server up with GZip compression because it will benefit the content in the wiki too.

As to why TW5 is larger than TWC, it’s really just because TW5 has so much more functionality beyond the basic user experience that both systems share.

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/849b14d2-cfa3-46b8-be83-3c68ce74c306o%40googlegroups.com.

[Mohammad]

Hi Jeremy,

 Many thanks for clarification! So, there are obstacles and limitations for having a microkernel Tiddlywiki.

Best wishes

Mohammad

[Jeremy Ruston]

Hi Mohammad

 Many thanks for clarification! So, there are obstacles and limitations for having a microkernel Tiddlywiki.

Just factors that hadn’t been discussed so far, it’s a good discussion.

I meant to add to my earlier reply that I do plan to do something similar for v5.2.x in that I’d like to move obsolete and deprecated components into an optional plugin. It won’t necessarily have a huge impact on the size of the core, but it will make it simpler and easier to understand.

Best wishes

Jeremy.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/eb50d5f0-598f-43b3-88b1-a6293bd22869n%40googlegroups.com.

[Mohammad]

Thank you Jeremy!

That is really good! I think 5.2.x will be an elegant new generation as I follow discussions, ideas and proposals on GitHub.

There are good experiences and huge amount of feedback in forum and GitHub on TW 5.1.xx.

Stay healthy!

Mohammad

p.s: I really like to have a set of clean and flexible filters! I think important core widgets like $list may need to be carefully evaluated and get proper improvement for a

simple and flexible scripting in TW 5.2.x

[TiddlyTweeter]

Jeremy Ruston wrote:

(My emphasis)

 

... right now the goal for the core is for it to contain not the bare minimum possible functionality, but rather the functionality that has the potential to be universally useful. That broader criteria means that the core itself is sufficient for a lot of work with TiddlyWiki, making the life of users and developers a lot easier. 

Practically I agree. It has been my experience in churning out work for quick purposes its got a lot easier  as the core has expanded. It avoids having to manage a vast shopping list of plugins.

Of course the question of what "universally useful" (i.e. widely relevant) functions are can be argued over. But my overall impression is that choices over extensions of the core have been very well measured--if anything slightly too "conservative".

A consideration ... is whether it really matters that the core gets big, especially now that we’re in a world where a simple news story weighs 20MB. There are valid concerns about slow networks but the best solution there is to set the server up with GZip compression ...

Right. I think a few tests showing the benefit of GZippery on this might help show a larger core is not, in itself, a major issue for most use cases.

What performance issues I've had have never been to do with core size per se; rather they have been about in-efficiency in some design decision I took (handling large numbers of tags being one of them; screen refresh implications another).

Best wishes

TT 

[Jeremy Ruston]

Hi TT

On 7 Nov 2020, at 09:53, TiddlyTweeter <Tiddly...@assays.tv> wrote:

Right. I think a few tests showing the benefit of GZippery on this might help show a larger core is not, in itself, a major issue for most use cases.

GZip is enabled for tiddlywiki.com, so an interesting test case is to look at https://tiddlywiki.com/upgrade.html; it’s the largest html file on tiddlywiki.com because it includes a copy of every plugin/theme/language for upgrade purposes.

Using the “Network” tab of developer tools, the HTML file shows as taking 16.5MB on disc, but only 4.3MB when it is transmitted by the server in its compressed form.

Best wishes

Jeremy.

[TiddlyTweeter]

TiddlyTweeter <Tiddly...@assays.tv> suggested:

 

I think a few tests showing the benefit of GZippery on this might help show a larger core is not, in itself, a major issue for most use cases.

 

Jeremy Ruston replied:

 

GZip is enabled for tiddlywiki.com, so an interesting test case is to look at https://tiddlywiki.com/upgrade.html; it’s the largest html file on tiddlywiki.com because it includes a copy of every plugin/theme/language for upgrade purposes.

Using the “Network” tab of developer tools, the HTML file shows as taking 16.5MB on disc, but only 4.3MB when it is transmitted by the server in its compressed form.

Ciao Jeremy

Good example! 

Basically proves the point that for online performance, particularly for the important user experience of "load time" , it is not core size per se that is often the issue. It is simply about knowing how to leverage servers to optimally use commonly available methods for reduced bandwidth delivery.

Best wishes

TT

[Mark S.]

There's more than server storage size that's important.

Imagine you were an instructor who wanted to distribute and receive lesson plans via TW to remote students with limited data rate accounts. Having 2megs overhead for each transfer would make the whole process expensive, in addition to slow. 

But, as they say, it is what it is. 

[TW Tones]

Using a CDN like cloud flare made an otherwise almost unusable wiki fast, add a splash screen to ensure people wait for it to load, advertise they are loading the whole site for fast performance.

Now when distributing content consider sharing JSON packAges of tiddler, or via a library, then the user need only the delta after the first save.

Regards

Tony