The Performance At Scale Bummer ... Initial Notes

[@TiddlyTweeter]

Recently over on the DesignWriteStudio Google Group I witnessed very clearly the issue around limits of TW scalability (read below).

Its kinda depressing in that TW offers all the tools for innovative organisation but often fails once the volume of Tiddlers goes beyond modest numbers (a few thousand).

That said, wardjh's 85K Tiddler wiki does function, albeit at the speed of a snail with a heavy stone on its back.

Such performance is unviable for anything public online. BUT its perhaps okay for using TW to PREPARE content for transport to another system that will do the final publish.

I don't see much discussion, well, any, discussion of TW as a very useful PREPARATION tool. Its flexibility seems to me ideal for positioning between inputs for re-arrangement and outputs optimally structured for
other systems.

I thought some of the comments by Mark S. about performance (from another thread) in general were interesting & worth noting too.

Best wishes
Josiah

wardjh wrote ...

Since my original project of 85,000 tweet records ran too slow to make it pretty much worthless I scaled down the project. 8-(
Still have some work to do in it.

You can find the new project - http://wardjhtweets.tiddlyspot.com/ (2,800 Tiddlers)

Old project can be found - http://wardjhtrumptweets.tiddlyspot.com/ (85,000 Tiddlers)

Mark S. wrote ...

There's no indexing system in TW, and thus no scalability. Indexed databases were able to handle hundreds of thousands of records even with 1980s hardware.

There's a way to send empty tiddlers via the server that only get populated on demand. The problem then is that your search will only see the titles and tags (fields ??) for searching. But if what you wanted was link and tag based (without depending on text searches) then it should work OK. You would need a separate (or enhanced) search box to fetch tiddlers from the server directly.

Or, maybe someone could write a system that would store an index to the current TW's text context. Then a special search would look through the index and use that for creating a list of links of tiddlers. The tiddlers would become populated when clicked on. The index would be updated on saving (harder) or on demand.

[BurningTreeC]

Hi @TiddlyTweeter 

For a good comparison I would like to see that big one using the latest prerelease version, where some performance tweaks have been done

BTC

[@TiddlyTweeter]

BTC

Good thought. I'll point the author to your suggestion.

TT

[war...@sunyit.edu]

I will have the larger project ported over to the newest version of TW this weekend and run it off my personal server.
For the smaller project I am  going to keep adding 100 more records at a time in the spreadsheet until I notice that it starts to slow down.

Thanks,

[Mark S.]

My first attempt to import the tiddlers into a pre-release version crashed the browser tab. So I thought that maybe I needed to use node to render out the tiddlers as tids. Now we hit another scalability wall. The process has been running 30 minutes and isn't even a quarter of the way through. Looks like converting to tids will take two hours!

Next question, is the entire master branch on github considered the pre-release?

-- Mark

[Jeremy Ruston]

Hi Mark

My first attempt to import the tiddlers into a pre-release version crashed the browser tab. So I thought that maybe I needed to use node to render out the tiddlers as tids. Now we hit another scalability wall. The process has been running 30 minutes and isn't even a quarter of the way through. Looks like converting to tids will take two hours!

The approach that worked on my ancient laptop was to open James’ original TiddlySpot wiki in Chrome and then export a JSON file of all tiddlers. Then on the command line I built the new wiki with:

tiddlywiki editions/empty/ --load ~/Downloads/wardjhtrumptweets.json --build index

However, I found that performance was drastically *worse* in 5.1.16, which I am now investigating...

Next question, is the entire master branch on github considered the pre-release?

Yes, the prerelease is just the Git master.

Best wishes

Jeremy.

-- Mark

On Friday, April 6, 2018 at 7:37:19 AM UTC-7, @TiddlyTweeter wrote:

BTC

Good thought. I'll point the author to your suggestion.

TT

BurningTreeC wrote:

For a good comparison I would like to see that big one using the latest prerelease version, where some performance tweaks have been done

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/5e037fd3-3d96-49c8-b006-8072e34f7382%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[BurningTreeC]

I found two interesting pages that are quite up-to-date and talk about javascript performance on nodejs (v8) and the second on firefox:

• https://www.nearform.com/blog/node-js-is-getting-a-new-v8-with-turbofan/
  
• https://developer.mozilla.org/en-US/Firefox/Performance_best_practices_for_Firefox_fe_engineers
  

From the second link, requestIdleCallback() appears to be very interesting - if the browser support is given. It reschedules workloads to idle-time like requestAnimationFrame()

This page shows a shim if there's no support for it: https://developers.google.com/web/updates/2015/08/using-requestidlecallback

I tested my latest chrome, firefox, brave browser and beaker browser and they're all supporting it

I just put this in the developer console: 

if ('requestIdleCallback' in window) {

  console.log("yes");

} else {

  console.log("no");

}

This benchmark https://jsperf.com/object-keys-vs-for-in-with-closure/10  tells me that Object.keys with a for loop - and for (key in object) are pretty much the fastest cross-browser

It also suggests that Object.keys().forEach ... should be avoided, it's 44% slower than the fastest in my benchmark

Simon

[Jed Carty]

Oh, that is unfortunate. I use Object.keys.forEach a lot in my plugins because it is convenient for me to type. Also reading about how the performance of for (property in object) {..} type loops is changing is interesting. It is going to be less efficient than the object.keys().foreach option in node 8 due to some serious performance hits to it and then in the next version it should go back to being much faster (in node, in the browser from what I can see it is just faster).

It isn't a difficult change so I guess I am going to be spending some time today updating plugins.

Another thing that I have been looking at is not sending plugins to the browser if they only affect the node process. For the core tiddlywiki plugins I think that this is just the filesystem plugin but for what I have been working on there have been more. This probably won't make a difference in the normal performance of a wiki but it should make the file loaded in the browser smaller and hopefully give a shorter load time.

[BurningTreeC]

In the first link I posted it shows something notable about using "delete" for deleting object properties: 

In V8 6.0 and 6.1 (not yet used in any Node releases), deleting the last property added to an object hits a fast path in V8, and thus it is faster, even, than setting to undefined. This is excellent news, as it shows that the V8 team is working towards improving the performance of delete. However, the delete operator will still result in a significant performance hit on property access if a property other than the most recently added property is deleted from the object. So overall we have to continue to recommend against using delete.

I've seen that it's used within the tiddlywiki core and the mentioned discovery appears to be a recent one

@Jed Carty

It's a bit annoying not knowing exactly where it goes and which tweaks made now are future-proof

I hope that we can filter those tweaks that make sense so that we won't need to change again in half a year

Simon

[Jed Carty]

Looking through my things, despite using the Object.keys().forEach structure many times there is only one time I have found that may lead to any sort of performance improvement. Most of the uses are over a very small list (only one or two items generally) and only happen rarely (like when a tiddler is saved). It is the same with using delete. It is used to remove entries about tiddlers when they are deleted, which happens relatively rarely and is in a context where the alternative (setting the value to undefined instead of deleting it) isn't a viable solution because it would cause more noticeable problems elsewhere. Checking to see if a property is listed is quicker than checking to make sure that the property is listed with a value other than undefined, and checking for the existence of a tiddler happens much more often than deleting a tiddler.

I always get excited when I see things like this that sound like they can give a big performance boost but almost every time it turns out to be optimisations that would just be lost in the other processing. Even if there is a 99% performance increase in the parts that are affected by these (the object.keys loops and delete) I don't think that the overall performance of the wiki would be noticeably changed, neither one is directly related to the changes in performance as the number of tiddlers in a wiki increases.

I think to fix the problem of scalability we probably need to look at a plugin that moves the tiddler store into an in-browser database and do filtering based on queries to that instead of having everything held in memory as a json object. But there is almost certainly going to be a trade-off there, for normal use where a big wiki has a few hundred tiddlers than adding on a database engine would probably make the wiki slower. I have been considering trying to use TaffyDB (http://taffydb.com) with tiddlywiki but I haven't had a chance to try it out yet and I have no idea if it would actually help at all. Another option is to use an in-browser sqlite database and maybe web workers.

I suspect that using a database this way may require a re-write of all the existing filter operators. I am not sure, this is all speculation.

[PMario]

On Saturday, April 7, 2018 at 9:21:49 AM UTC+2, BurningTreeC wrote:

From the second link, requestIdleCallback() appears to be very interesting - if the browser support is given. It reschedules workloads to idle-time like requestAnimationFrame()

https://caniuse.com/#search=requestIdleCallback

seems to be a no-go for me.

-m

[PMario]

On Saturday, April 7, 2018 at 11:48:55 AM UTC+2, Jed Carty wrote:

....

 

I have been considering trying to use TaffyDB (http://taffydb.com) with tiddlywiki but I haven't had a chance to try it out yet and I have no idea if it would actually help at all. Another option is to use an in-browser sqlite database and maybe web workers.

Have you had a look at the code. ... For me it seems to be a wrapper for jsavascript object handling. ... I doubt it would make anything fast.

TiddlyWiki's utils.js does the same things and is compatible with TW.

-m

[BurningTreeC]

also when only using it when available, so that someone that wants to do workload-heavy things could use one of the compatible browsers (latest chrome/chromium/tiddlydesktop/firefox/brave/beakerbrowser ... )?

Simon

[@TiddlyTweeter]

The discussion in this thread is interesting to me.

At a practical level its, perhaps, worth noting that in the last year I put three projects "on hold" because TW testing indicated it could not cope.

FWIW, they are ...

1 - 50,000 simple sections from the Perry Mason TV series scripts. The idea here is to display 100 tiddlers of quotes at a time. And let readers favourite what grabs their interest.

2 - A complex project for Dickens' novel Great Expectations that is chunked down to paragraphs, with annotations at paragraph level, cross linking to another wiki for clips from thirteen versions of GE that have been filmed, and a Glossary of terms used.

3 - Chaucer's Canterbury Tales in middle English with line by line commentary.

All three TW can do what is needed very well, easily. Except that NONE of them can cope with the SCALE. In practice I can't get it responsive enough to make any of them public.

Just saying
Josiah

[TonyM]

Josiah/Jed,

Has anyone tried to see if noteself with its in browser counchDB can perform faster with large wikis?

Tony

[TonyM]

Josiah,

I have given this some thought, and have a lot to add. Perhaps we should start a discussion specifically on settings and tools for large wikis?

I just loaded the thousands of standard tiddlers (mostly tweets) from the large TiddlyWiki exhibiting performance problems. When it come to scale there are a number of performance issue's, one of which NoteSelf solved by showing the  "Loading" message before TW can respond at all.  What I quickly realised is when you have a lot of tiddlers, quite a few assumptions made in a regular tiddlywiki's, for features and responsiveness are no longer valid. Such as a search that responds keystroke by keystroke.

There is no reason why we can't supply some tools to support such large data-sets. I believe it is not necessary to rebuild tiddlywiki but to empower it and designers to handle larger data-sets.

What I suggest is we bundle a range of settings together, perhaps in a plugin designed to reconfigure TiddlyWiki for larger data sets. And some tools, This would allow the community to bundle a Range of settings and possibly customised system tiddler overwrites as needed.

For the specific example given, Curiously if the tweet content is the main thing you search, perhaps the tweet content should be the title (guaranteed unique). Then the Skinny tiddler search is all that is needed.

Regards

Tony

[Mark S.]

> Such as a search that responds keystroke by keystroke.

I would love to turn that feature off. You don't need 85,000 tiddlers to have it slow things down -- even a few hundred is enough on a tablet. I'd much rather type and enter without the performance and interference hit.

-- Mark

[@TiddlyTweeter]

Ciao TonyM & all interested

I do think that a "Rules Of Thumb" note on SCALABILITY could be very useful.

The issue is expressing it correctly.

So much of performance is an EMPIRICAL issue, that without testing variant use cases, you can make TW look worse than it is. That is not a good idea.

To give an example: Whilst 8,000 Tiddlers of Tweets with tags that rely on filtering gets very slow, a TW with addresses for 8,000 external images of which the next displayed is calculated via a random number it is fine. In the first case you got a lot of processing & rendering. In the second you don't.

To be useful guidance on Scalability I think needs informing by testing by USE CASE.

Best wishes
Josiah

TonyM wrote:

... Perhaps we should start a discussion specifically on settings and tools for large wikis?

... When it come to scale there are a number of performance issue's, one of which NoteSelf solved by showing the  "Loading" message before TW can respond at all.  What I quickly realised is when you have a lot of tiddlers, quite a few assumptions made in a regular tiddlywiki's, for features and responsiveness are no longer valid. Such as a search that responds keystroke by keystroke.

There is no reason why we can't supply some tools to support such large data-sets.

... What I suggest is we bundle a range of settings together, perhaps in a plugin designed to reconfigure TiddlyWiki for larger data sets...

 

[@TiddlyTweeter]

Ciao Mark S.

I guess part of the issue is that "usability" developments can have consequences unforeseen at the time???

J.

[Mark S.]

I perceive the results-as-you-type thing as more of a toy -- not a usability development. Not just TW, but anywhere it's used. It may make sense when it's attached to  a powerful back-end server database, but not so much when you have a local, non-indexed data source. In the case of TW, it interferes with usability particularly on small devices. Re-rendering with each keystroke is another related problem that affects all TW's performance. I can only assume most users are not touch typists, have very powerful machines, have small TW's, or are using screen keyboards.

-- Mark

[Mat]

Apropos results-as-you-type, I just (re-)stumbled over this, for anyone is interested. Who knows, maybe it has some superior algorithm for big data sets or some other useful aspect that we can use: Awesomplete

<:-)

[@TiddlyTweeter]

How about a "don't do it till you click" button?

[PMario]

On Monday, April 9, 2018 at 5:21:21 PM UTC+2, Mark S. wrote:

I perceive the results-as-you-type thing as more of a toy -- not a usability development.

You can change it. There is a HiddenSetting that defaults to 400ms. ... That means: If you stop typing for longer than 400ms the draft tiddler will be stored. .. which may trigger a refresh of the visible tiddlers involved. Typing a new character will reset the timeout.

You can set it to 10000 (= 10 seconds) or even higher. ....
 

Not just TW, but anywhere it's used. It may make sense when it's attached to  a powerful back-end server database, but not so much when you have a local, non-indexed data source.

In the contrary. ... Every time a tiddler is saved, it will be synced with the backend. .. If the backend is connected with a slow connection, you don't want this behaviour at all.
 

In the case of TW, it interferes with usability particularly on small devices. Re-rendering with each keystroke is another related problem that affects all TW's performance. I can only assume most users are not touch typists, have very powerful machines, have small TW's, or are using screen keyboards.

As I wrote. You can change the parameter. 

Just my thoughts
-mario

[@TiddlyTweeter]

PMario

VERY useful info! Who knew but you? :-) the problem with Hidden Settings is they are ... Hidden :-).

[PMario]

On Tuesday, April 10, 2018 at 2:42:40 PM UTC+2, @TiddlyTweeter wrote:

VERY useful info! Who knew but you? :-) the problem with Hidden Settings is they are ... Hidden :-).

For a reason. ... They have the ability to cause "strange behaviour" if used wrong.

-m

[PMario]

The alternative to "hidden" would be "not implemented" -> removed.

-m

[Jeremy Ruston]

On 10 Apr 2018, at 14:09, PMario <pmar...@gmail.com> wrote:

VERY useful info! Who knew but you? :-) the problem with Hidden Settings is they are ... Hidden :-).

https://tiddlywiki.com/#Hidden%20Settings

Best wishes

Jeremy

[Mark S.]

In the past when I've tried the type-ahead setting, I couldn't find that it made any difference one way or the other. I'll try again.

What I mean about the database on the backend, is that big companies like Google and Amazon use some streaming/buffering Ajax technology where search key-by-key actually works. But with large datasets or on small devices, key-by-key search doesn't work efficiently with TiddlyWiki.

Since there's a hidden setting to control minimum key length, it must be possible internally to change the mechanism so that it only searches upon enter.  Maybe there could be a hidden setting to turn off key-by-key searching?

Thanks!
-- Mark

[PMario]

On Tuesday, April 10, 2018 at 5:35:07 PM UTC+2, Mark S. wrote:

Since there's a hidden setting to control minimum key length, it must be possible internally to change the mechanism so that it only searches upon enter.  Maybe there could be a hidden setting to turn off key-by-key searching?

https://tiddlywiki.com/#Hidden%20Settings:%5B%5BHidden%20Settings%5D%5D%20%5B%5BHidden%20Setting%3A%20Search%20Minimum%20Length%5D%5D

Search Minimum length defaults to: 3 .... since v5.1.14

-m

[Mark S.]

Yes -- that's what I said. But is there a setting to turn off key-by-key searching and replace it with traditional type-and-enter ?

Thanks!
Mark

[BurningTreeC]

Yes -- that's what I said. But is there a setting to turn off key-by-key searching and replace it with traditional type-and-enter ?

Thanks!
Mark

Hi Mark, couldn't that be done with a <$keyboard key=yourkey actions="set statetiddler on"> before the search input field

... and this in the search results popup: 

<$reveal statetiddler=on>

searchresults

</$reveal>

Simon

[Mark S.]

Could subvert the minimum length field. Put this below first div in $:/core/ui/SearchResults:

<$reveal tag="div" state="$:/config/Search/MinLength" type="nomatch" text="50" default="">
<$button>Hide<$action-setfield $tiddler="$:/config/Search/MinLength" text="50"/></$button>
</$reveal>

and replace contents of $:/language/Search/Search/TooShort with

<$reveal tag="div" state="$:/config/Search/MinLength" type="match" text="50" default="">
<$button >
<$action-setfield $tiddler="$:/config/Search/MinLength" text="3"/>
Show
</$button>
</$reveal>

This is clumsy I know, since effectively there are now two ways to dismiss the results, and the user no longer has a way to set a true minimum (other than hacking the hack) .

The problem with keystrokes is that the specialty (non-ASCII) keystrokes are difficult to make on a tablet or phone where this kind of feature is most needed.

Thanks!
Mark

[TonyM]

Josiah,

As you say "Whilst 8,000 Tiddlers of Tweets with tags that rely on filtering gets very slow" when looking at the specific case I realised the main thing searched for was the tweet, so in a tiddlywiki for this purpose why is it a secondary fields when it could be the title. Then in None single file implementation most searching will be the title of a "skinny tiddler" only.

In TWC the goto plugin was my favourite as first you search only titles then click to search content with the advanced search.

I would like to add back a title only search tool in TW5

Regards

Tony