[tw5] offline-first approach to TW5?

[jayfresh]

Hello,

I've set up TiddlyWiki5 on Heroku (for how, see my post at https://groups.google.com/forum/#!topic/tiddlywiki/XDFW_tcSbf0) because I'm interested in having an online notebook to jot ideas down, and this fits the bill.

The main shortcoming for me is the large download size you get on page load, which (1) makes it less fun to access through a mobile; and (2) increases the "time to writing" by several seconds compared to opening up a local text editor (which is long enough to end up preferring the local tool).

A few things have come up on earlier threads or in conversations, which might be helpful, so I'm posting to get some feedback on whether anyone has been working on these or related areas and has any thoughts.

1) caching

I've seen that NGINX has been mentioned as a cache, so it seems like it might be possible to tell the browser that the resource (the tw5 file) has not changed, which would prevent it from re-downloading it. That seems like it would be quicker as the page would load from the local cache.

2) app manifest

The browser's app manifest feature could possibly be used to cache the tw5 file locally, but it's not clear to prompt the browser to download a new copy when the tiddlers update. However, if some JS requested tiddlers from the server, it could sync the local version.

3) lazy loading

Rather than load all the tiddlers, load an empty tw and get the contents afterwards. This would perhaps be a good setup for (2) or (1), but by itself doesn't buy much time saving as the empty tw5 file would still need to be downloaded each time.

Any thoughts much appreciated.

Jonathan

[Jeremy Ruston]

Hi Jonathan

I've set up TiddlyWiki5 on Heroku (for how, see my post at https://groups.google.com/forum/#!topic/tiddlywiki/XDFW_tcSbf0) because I'm interested in having an online notebook to jot ideas down, and this fits the bill.

Great. It's a process I'd like to automate; really it should be one click from tiddlywiki.com to getting set up with your own Heroku instance.

 

The main shortcoming for me is the large download size you get on page load, which (1) makes it less fun to access through a mobile; and (2) increases the "time to writing" by several seconds compared to opening up a local text editor (which is long enough to end up preferring the local tool).

There's quite a lot of scope for improvement by reconfiguring things to take advantage of caching and lazy loading, as discussed in the responses to your points below.

In terms of the file size, we don't currently minimise the JS, and there's a lot of it. I did a quick experiment: there is 592705 bytes of .JS in the core. Minimising it with the first online minifier I found reduces that to 277066 bytes. That's not a huge problem with static hosting that automatically provides gzip compression (as with tiddlywiki.com), but it might be a problem when serving the HTML file direct with TW5 (which doesn't implement gzip compression at all). Possibly another reason to cache with NGINX

The nuclear option is of course that we could add a dedicated browser app for creating new tiddlers, as we explored with TiddlySpace.

 

A few things have come up on earlier threads or in conversations, which might be helpful, so I'm posting to get some feedback on whether anyone has been working on these or related areas and has any thoughts.

1) caching

I've seen that NGINX has been mentioned as a cache, so it seems like it might be possible to tell the browser that the resource (the tw5 file) has not changed, which would prevent it from re-downloading it. That seems like it would be quicker as the page would load from the local cache.

Yes, I think this is the obvious first step. As I mentioned on the other thread, when TW5 starts up in the browser it requests the latest skinny tiddlers list from the server, and lazily load any new ones (ie, the text won't be retrieved until the tiddler is displayed or accessed directly). So that mechanism is no good for delivering plugins, but it works fine for content tiddlers. So you should be able to cache the HTML file for days at a time without problems.

 

2) app manifest

The browser's app manifest feature could possibly be used to cache the tw5 file locally, but it's not clear to prompt the browser to download a new copy when the tiddlers update. However, if some JS requested tiddlers from the server, it could sync the local version.

I've not had much experience with app manifests. It does seem as though service workers promise to be more reliable and useful.

 

3) lazy loading

Rather than load all the tiddlers, load an empty tw and get the contents afterwards. This would perhaps be a good setup for (2) or (1), but by itself doesn't buy much time saving as the empty tw5 file would still need to be downloaded each time.

Lazy loading is supported at the moment. You just use a different template in the server command. See:

https://github.com/Jermolene/TiddlyWiki5/blob/master/bin%2Flazy.sh

It uses this template that simply omits non-system image tiddlers to build the main HTML file:

https://github.com/Jermolene/TiddlyWiki5/blob/master/core%2Ftemplates%2Fsave-lazy-images.tid

Then when the file starts up in the browser it will lazily load the images.

 

Any thoughts much appreciated.

I hope that helps,

Best wishes

Jeremy

 

Jonathan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

--
Jeremy Ruston
mailto:jeremy...@gmail.com

[jayfresh]

Thanks Jeremy, will have a think.

In light of the recent thread about couchdb[1], I wondered whether PouchDB[2] might be an interesting adaptor for TW5. It assumes that the client is offline and syncs data when it can. It's very much inspired by couchdb so feels like a compatible place to start.

J.

[1] https://groups.google.com/forum/#!topic/tiddlywiki/XDFW_tcSbf0

[2] http://pouchdb.com/

[Jeremy Ruston]

Hi Jonathan

John Udell worked on a PouchDB adaptor back in February (I'm on my phone and so can't find the link at the moment). It'll likely need a couple of tweaks to bring it up to date.

Best wishes

Jeremy

—
jeremy...@gmail.com

--

[Danielo Rodríguez]

Having a PouchDB adaptor for TW is a thing I dreamed soo many times... I was really excited when I first heard about it, but the project seems to be stuck. 

Hopefully you can go ahead with it.

Would be nice to have an android application (for each TW) and have the ability to edit on mobile and on computer with sync. 

[jayfresh]

Ah great, thanks Jeremy. And I agree with Danielo, that would be very handy.

I've pinged Jon on Twitter to see if he has the code for the PouchDB adaptor anywhere, I couldn't find it.

[Jon Richter]

I managed to get the CouchDB version deployed, but fear the amount of data transmitted each time something is saved, as it uses the self-contained HTML renderer, which creates a huge file sent up- and downstream upon each sync.

Thus I have great interest in a native replication of individual tiddlers, i.e. via PouchDB. As I couldn't find the mentioned proof of concept either, getting offline wikis maybe the simplest solution via that part?

Reference: https://twitter.com/judell/status/520711934048350208