TW too slow to use at only 8 MB

[S]

TW has gradually become slower and more unusable the more I add to it. It is text only, no added plugins or other extras, entirely structured and organized through tags, with three tiddlers of tags given their own Table of Contents type arrangement in the main row of TW tabs.

Tags are used in tree format to organize tiddlers by time (year tiddlers and month-years within them), but I also use tags in the more traditional sense for tagging tiddlers by topics. Aside from the tiddlers used to form this structure, meaning all of my tiddlers that have actual content in them and not just a line of code, those tend to have on average 2-10 tags each. Some tags for subject matter rarely get used, while a few have been used 100+ times.

Many tiddlers have a lot of text but hopefully that's not the problem.

I do not keep the expanding lists of tags in tree structure all expanded at the same time, only what I need to reference.

I was running an older version of TW5 when this issue began, and then updated to 5.1.13 which was the new version at the time. This made no difference.

Whole TW file is 8 MB with 800 tiddlers (that I've made), and 140 tags.

Opening, saving, and editing tiddlers and the TW are all slow. The worst is typing text in a tiddler as oftentimes the characters appear very delayed after I type them. All of the lag is a problem, but this is most problematic of all. These performance issues persist through various high spec computers, different OSes, multiple web browsers, TiddlyDesktop, running off disk vs portable media, as the only thing running vs running other programs or browser tabs, etc.

TW is just miserable to use right now which is a shame considering how much I love it when it works and how much time I put into it.

I did read through an earlier thread which maybe identifies my problem, possibly tags being used too many times.

https://groups.google.com/forum/#!searchin/tiddlywiki/number$20of$20tiddlers%7Csort:date/tiddlywiki/U1ZKK8UIk94/q2DSlN_TAwAJ

This plugin was mentioned towards the end of that thread that I think utilizes fields instead of tags.

https://groups.google.com/d/msg/tiddlywiki/U1ZKK8UIk94/o3SH1PrQAwAJ

Do you suspect my issue is tags rather than some other issue? If so, is there an easy way to convert some or all of my tags to fields or some other better-performing format while also maintaining the structure of tiddlers created by tags? I haven't done coding in a long time and so most of the coding in TW especially goes over my head. Tags make sense to me and I know how to use them, whereas I have no idea how I'd turn fields into the same structure I have if that's even possible and it looks like maybe there are a lot of rules with fields? I don't know if there's a solution, or if there is, if it's something I understand enough to do.

Short of switching to a different platform, the only non-complex but tedious and inefficient solution that might work is to make all of those table of contents/expandable/lists of tiddlers in tiddlers (ruled by tags) into ordinary static tiddlers without code, that I update by hand with links to other tiddlers. So no longer would the tags let these lists automatically generate, but I would by hand add links to tiddlers in all of these places. For obvious reasons, I would rather not do this.

Thanks!

[TonyM]

S

There is much that can be done to help. I assume its a single file wiki and it has evolved over time?

tiddlywiki is open to massive numbers of tiddlees, relationships and is flexible out of the box. This dose however allow methods that are less than optimal once you have large numbers. This is good in some ways because in most use cases its very flexible and you do not need to consider optimisation.

my first recommendations are;
hide the sidebar does performance improve?
Upgrade even just as a test to the latest release it has optimisations
Investigate if your browser is running out of its own space or your pc ram.
Turn off autosave to see if saving is slowing you down

Is your wiki very private? Can it be shared? even just to a few people?

If you can't share list size, browser, saving method any thing else that may help us diagnose it.

Regards
Tony

[Mark S.]

Reading your post, it sounds like you're still on 5.1.13. There were improvements in 5.1.20. I would suggest making a copy

of your file and upgrading it to 5.1.21.

It's really hard to diagnose at a distance. But you shouldn't be having serious problems with only 800 tiddlers and 8megs on *all* platforms.

Good luck!

[S]

Thank you for your response! Closing the sidebar did not make a difference, RAM is aplenty to begin with but I always turn off and turn computer on again with nothing else running if I want the best performance in general. Autosave is always turned off. Browsers, have primarily used Chrome, Safari, Firefox. Usually just the browser's ability to download files, though TiddlyDesktop is far more convenient with saves. Saving method nor where I work with the file affects the performance--it's bad everywhere. Wiki is private or otherwise I would consider sharing.

I just went ahead with the upgrade to 5.1.21 and WOW that is a lot better! I just finished reading all of the past release notes to see what's new. I'll have to do some more testing to make sure it's for real, haha. But it is definitely much faster than before.

So clearly the newest version of TW has some significant performance improvements (saw tag improvements mentioned in multiple releases), and I might be okay for now, but will I run into this slowdown again? Anything I could or should change about my TW file so that TW performance is ahead of the data?

Thanks, Tony!

[S]

I just upgraded to 5.1.21 and it is much better! Have to experiment with the new version more to be sure it will be usable for my purposes, so in the meantime, I might be okay. I still have concerns of a slowdown like this in the future though, especially if I add larger tiddlers or greater numbers of them than usual. Do you have ideas for how I could prevent a slowdown in the future, such as existing aspects of my TW and its structure I could change, or just in the future?

Thank you Mark!

[Arlen Beiler]

Definitely upgrade -- alot of improvements have been made recently. 

Also, the tag filter starts having performance problems when you start getting into tens of thousands of tiddlers (like a dictionary). It's a known bug and we're working on a solution. 

You can turn on performance instrumentation in the control panel and open developer tools to see the results. I think it tells you to run a certain command to see the results. 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/8c809e95-3a2e-4d5f-b87e-2972faf0a56a%40googlegroups.com.

[Michael Wiktowy]

I have noticed, even in 5.1.21, that with modestly sized tiddlers containing html content, if you set the mime-type to be text/html, it renders quickly (but in a slider window with some limitations). But if you set it to text/vnd.tiddlywiki or leave the mime-type blank in order to blend in some Wikitext, things will get *very* slow on loading that particular tiddler ... at least the first time you load it and/or edit it.

I am not sure if that is what makes up the content of your wiki but I have been forced to import html pages as plain text (by cutting and pasting the text into an editing window rather than using the drag and drop or cut and paste into the main window) route and re-applying analogous wikitext formatting to make it resemble the original.

/Mike

[TonyM]

S

Anything I could or should change about my TW file so that TW performance is ahead of the data?

Without looking at your wiki specifically, this is a very open question, We would have to speculate, however you have given some clues, substantial use of tags. What I suggest is try to gain a little more understanding about the way tiddlywiki works and the performance cost of different approaches. Keep in mind you can copy you wiki and change/remove something to see if it has an impact on performance, then you can use another copy and see if you can improve the performance relative to that particular thing. If you identify a bottleneck ask in this forum and someone can help.

• You are benefiting from performance improvements now with 5.1.21, for more details read this https://tiddlywiki.com/prerelease/#Performance  take particular not of the starting forms of the filters, you could locate all your key filters and ensure they comply with this format so that automatic indexing takes place.
• Make sure if you use the performance instrumentation you turn it off, I think 5.1.20 accidently left it on
• If you over use the tag system (perhaps you are) you can create complexity in there, try and learn how to move tags into other fields, how depends on the nature of the tag such as does a tiddler either have it or not, then perhaps it can be moved to a field of tagname with a value of yes or no (default no), and more examples. Remember any tag can have a list field generated containing a list of everything it tags (This is a little overhead that may cost you size and performance).
• Remember it is often possible to automate retagging, field creation etc.. to multiple tiddlers at once (List inside a button with actions)
• Usually if you can hide something, even temporarily such that it need not be displayed the recurring refresh process can go a lot quicker.
  

Regards

Tony

[HC Haase]

torsdag den 31. oktober 2019 kl. 01.17.21 UTC+1 skrev Michael Wiktowy:

I am not sure if that is what makes up the content of your wiki but I have been forced to import html pages as plain text (by cutting and pasting the text into an editing window rather than using the drag and drop or cut and paste into the main window) route and re-applying analogous wikitext formatting to make it resemble the original.

FYI. thers a great plugin that converts html selected in the text field to wikitext. colud save you some key strokes

http://skplugins.tiddlyspot.com/

[HC Haase]

torsdag den 31. oktober 2019 kl. 02.00.33 UTC+1 skrev TonyM:

• You are benefiting from performance improvements now with 5.1.21, for more details read this https://tiddlywiki.com/prerelease/#Performance  take particular not of the starting forms of the filters, you could locate all your key filters and ensure they comply with this format so that automatic indexing takes place.

I am not sure if I understand correctly.





do [all[tiddlers]tag[x]... run a lot faster than [tag[x]... ??





thanks


[Michael Wiktowy]

On Thursday, October 31, 2019 at 5:30:42 AM UTC-3, HC Haase wrote:

FYI. thers a great plugin that converts html selected in the text field to wikitext. colud save you some key strokes

http://skplugins.tiddlyspot.com/

 

Thank you for the pointer to that plugin! It is built on an older version of TW but worked quite well still. Not flawless but certainly super helpful and any fix-ups needed post conversion were minor. I'll keep it around in my toolbox wiki.

/Mike

[TonyM]

HC,

Yes, As I understand there is a behind the scenes generation of indexes which is searched rather than all tiddlers. It is an intentional performance mechanism.

When such indexes are "very big" it may not operate on all, but I believe in most use cases it is a sold improvement.

Something I only touched on in the first reply is if you are a big tiddlywiki user and have a lot of or large wikis open in your browser you could consume the memory allocations the browser sets to avoid too much of consuming the host computers resources. Personally I set FireFox and Chrome settings higher given my 16GB RAM and almost exclusive use of tiddlywiki. I believe using large tiddlywikis this is a less than intuitive performance enhancer. 

Regards

Tony