optimising performance (searching vs saving tiddlers)

[haroulito]

Hi all,

I've been pondering on the best approach for maintaining TW performance as the size of single-file wikis grows. This has been bothering me for a while so I thought to share my thoughts and get any insights / advice.

I have considered to go the node.js or TiddlyServer way but have not taken the plunge yet. (I have only done some early tests with TiddlyServer.)

I primarily use 3-4 wikis. The main TW I use is between 7 and 8 mb and when it reaches around 8mb I start to export/archive some content to maintain the performance. (I only store text, no images or embedded files). I use this wiki for my workflow (GTD-style) and for quick entry of content in my inbox. 

So performance is crucial as I am creating, editing and tagging many small tiddlers in succession and often on occcasions where there is time pressure.

What I need to improve is the delay when clicking to save a tiddler. For my 8mb wiki it takes about 4 seconds to save each tiddler. On a blank TW with all the plugins I need installed it takes ~2 seconds. This difference is really noticeable under time pressure and big volumes of work.

(I use TiddlyDesktop with backups enabled. Disabling backups does not produce much noticeable difference but I do disable when I bulk-edit tiddlers to speed things up)

Searching

Searching is fast for the size of wikis I use. If I start splitting my content into more wikis then my problem would be searching all the wikis. 

Obviously single-file wikis have various advantages and I also heard Jeremy say in one of the hangouts that not all the plugins would work well in a node.js implementation.

If I decide to stick to single-file wikis, I've been thinking to merge all my wikis just for the purposes of searching. E.g. export all tiddlers from all wikis using "[!is[system]]" and import to a single wiki. I would then search the single wiki if I can't tell which of the individual wikis I should search. Obviously not an elegant way to do things even if the  process of merging all my wikis could be automated.

 

Any ideas on improving performance of saving tiddlers after editing and also ways to search content across multiple single-file wikis?

Many thanks 

H.

p.s.1 My computer setup:

Win7

intel i7, 8gb of ram, magnetic HDD

TW 5.1.19

TiddlyDesktop with backups enabled

p.s.2 I have found this post useful. It seems that there will be improvements in indexing but the issue I am facing is about saving tiddlers after editing rather than fast searching.

https://groups.google.com/forum/#!topic/tiddlywiki/BXQNiAoJQSE

[PMario]

On Friday, July 19, 2019 at 7:45:14 AM UTC+2, haroulito wrote:

...

What I need to improve is the delay when clicking to save a tiddler. For my 8mb wiki it takes about 4 seconds to save each tiddler. On a blank TW with all the plugins I need installed it takes ~2 seconds. This difference is really noticeable under time pressure and big volumes of work.

Hmmm, It shouldn't need 2 seconds to save an empty tw, with only plugins installed.

Which plugins do you use?

-m

[Watt]

Hi haroulito
If you haven't already seen it there is another useful discussion here;

https://groups.google.com/forum/m/#!topic/tiddlywiki/Aheyeybfydg

Bimlas and others comments towards the end offer some good tips.
Good luck!

[PMario]

Hi,

There is a setting in the control panel: Saving: General tab

You can switch auto-saving off. BUT you need to hit the save-button on your own, from time to time!!

see: https://tiddlywiki.com/#%24%3A%2Fcore%2Fui%2FControlPanel%2FSaving%2FGeneral

-m

[haroulito]

Thanks Watt,

I went through most of the discussion and will go through it again more thoroughly to get ideas.

Best wishes 

H.

[haroulito]

Hi Mario,

Many thanks for your posts.

I will be disabling autosave for short periods of intensive editing.

I often have the side pane showing the "Open" or "Recent" tab. I should make a point to remember to use a static tab instead although I don't see much improvement if I do so.

In terms of plugins, I don't use a lot; basically TiddlyMap, mermaid and the supporting plugins for these.

I used to have TiddlyMap always on but then switched to having it disabled most of the time and enable it only when I need to use it. It seems to slow things down for me probably because it has to create an id field for each new tiddler created.

I realise now that I have been disabling TiddlyMap but not the related plugins  (HotZone & TopStoryView). I will try disabling these as well to see if things speed up.

Using mermaid involves three pluging that are always enabled.

Thanks again,

H.