[tw5] Bob and BobEXE version 1.6.0, Mopbucket, backup your data

[Jed Carty]

Bob 1.6.0 is finally released. It fixes some large bugs from 1.5.2.

The BobEXE version is available here: https://github.com/OokTech/TW5-BobEXE/releases/tag/1.6.0

The plugin is available here: https://github.com/OokTech/TW5-Bob

Highlights:

- General stability improvements for syncing

- Backups should no longer give the RSOE when you open the html files

- Typing into fields that directly edit tiddlers, like changing the wiki title, should no longer cause the server to choke

- Video files should be streamable from the server using _canonical_uri tiddlers

Unfortunately the built-in syncer and Bobs queue system still don't play too well together so saving large numbers of tiddlers at once can take a long time.

Full change log:

Special thanks to @joshuafontany for some much requested stability improvements.

Note: Federation is disabled in the code because it doesn't have proper security yet

* (BobEXE) switched to using nexe instead of pkg.

* Fixed uploading media to the RootWiki so it has the correct _canonical_uri

* improved the normalise tiddler function so tiddlers with no text fields are handled consistently

* Fixed a regression where draft tiddlers were saved to the server

* partially fixed a regression caused by switching to the built-in syncer, tiddlers you edit directly (like `$:/SiteTitle`) try to save on every keystroke and it chokes, before there was a throttling system in place for saving tiddlers that used the message queue. Instead of keeping it from saving, it is just slow to respond.

* Added a `$tiddler` attribute to `action-websocketmessage` to simplify a few things.

* Single file wikis that contain the Bob plugin shouldn't show the RSOE anymore

* Fixed some rare edge cases that could cause Bob to crash

* Moved more messages to use the Bob logger instead of console.log

* The file server now correctly supports video files

* Hopefully removed the need for the `proxyprefix` setting.

* All around better handling of draft tiddlers on the node side (thanks @joshuafontany)

* Add some error handling specific to windows (thanks @joshupfontany)

* Fix a race condition that could lead to ignoring file system paths filters. (thanks @joshuafontany)

* Fix a problem where canceling an edit on a shadow tiddler would leave the tiddler locked

* Federation - udp multicast works for discovering other Bob servers on the local network.

* Federation - pulling tiddlers from other Bob servers on the local network works

* Federation - federated chat on the local network works

* Federation - fixed a problem where some network configurations would prevent servers from finding each other using UDP multicast.

* Federation - there is a bug with dates when fetching tiddlers, sometimes dates come out as NaNaNa.... in the receiving wiki.

[Stobot]

Love this update Jed - I've been using for a while now and it's the most stable yet!

[Dave Parker]

FYI, the new BobExe doesn't seem to play well with https://saqimtiaz.github.io/sq-tw/streams.html

I.e. when I start typing a new line in streams the first letter is displayed and then the whole thing chokes until you kill bob and restart, or when the "a web page is slowing down your browser - what would you like to do: stop it or wait" message comes up if you click "stop it" you can continue typing, but the Bob instance stops working normally.

[Jed Carty]

Until I work out some issues with using the core syncer anything that tries to save on every keystroke is going to have this problem.

[Saq Imtiaz]

Dave: here is one thing to try which may help.

• Edit $:/config/sq/streams/new-node-tiddler-template, for the text field type in the name of a template tiddler that you are going to create, like mytemplate
• Create the template tiddler with the name you used in the step above, add a field throttle.refresh with value true.

Now try creating new nodes with Streams and see if that works better with Bob.

[Dave Parker]

It helped a bit, i.e. didn't break as badly, ha ha

see attached for screenshot

[Saq Imtiaz]

If that helped at all, you could also try increasing the refresh delay in $:/config/Drafts/TypingTimeout

[Saq Imtiaz]

PS: I don't quite remember if these settings impact just the refresh, or the dirty state handling and saving as well. But it was worth a try as a potential easy fix.

[Saq Imtiaz]

@Jed

Does Bob set a value for $:/config/SyncThrottleInterval or pass a value for the sync throttle in the options to the syncer?

See https://github.com/Jermolene/TiddlyWiki5/blob/81f07cdf852d8dd7c8e8e41984a6193c5d9a9c15/core/modules/syncer.js#L49

A value around 1000 (i.e. 1s) might help.

I might be mistaken but it seems to be set to 1ms here: https://github.com/OokTech/TW5-Bob/blob/6434efd0ed413ed9bb7c897bab1178d70e98da19/CoreSettingsOverride/syncThrottleInterval.tid

[Dave Parker]

That didn't affect it any further

[Saq Imtiaz]

@Dave I have never used Bob before but just did a quick test where I set  $:/config/SyncThrottleInterval to 1000, and didn't have any problems with using Streams on Bob.

Note this is without any of the other changes I asked you to make (though those can't hurt)

[Saq Imtiaz]

PS: well it doesn't crash but I am getting some weird artifacts where some letters go missing from nodes being edited, likely some kind of a weird refresh issue.

[Jed Carty]

What you are describing shouldn't make Bob crash on the non-beta version of 1.6.0 that I released with this announcement, the beta version, and 1.5.2, could crash when the server got confused.

Changing the saving delay isn't the problem, the delay should be set as low as possible because Bob uses acknowledgement signals from the server to try and ensure that the messages aren't sent faster than the server can handle. The problem is that editing a tidldler that gets synced outside of draft mode tries to save on every keystroke.

I used to have some complex logic there to prevent race conditions, switching to the internal syncer broke that logic so there are race conditions that cause very rapid updates to get received and processed out of order, which causes strange behaviour when typing rapidly in a tiddler where updates are being saved that quickly.

It is one of the highest priority fixes at the moment, but the problem should be limited to the possibility of the last few keystrokes being incorrectly saved if you are typing more than about 5 characters a second and saving on every keystroke.

If you are getting crashes and you are certain that you are using the newest version there is some other bug I need to track down.

[Saq Imtiaz]

Jed: I just checked again. On Windows 10/Chrome, with Bob 1.6.0 with the streams plugin I do not get any crashes. However, I do get missing characters and if typing really fast, it can end up in the situation where it keeps refreshing, once with the missing character and once without. So overall it seems like your understanding of the root of the problem is correct.

[Dave Parker]

other than the Streams thing, I do definitely find this version is more stable :)