How to continuously build static HTML?

[Edgaras]

I am trying to optimise the workflow of building static sites, now I am trying to use Node.js TW version.

Is there a way to continuously or periodically build/export static site respecting defined filters (I only want to export tagged tiddlers).

I am planning to share my optimised website building process when I am done! 😊 

[TonyM]

Edgaras

That would be very interesting do share.

Some possible leads. There is a core plugins called save trails which continuously saves changed tiddlers, perhaps you could get it or a version to use a static tiddler template.

There is also a file system plugin for node.

I would think there would be value in saving localy and periodically publishing online.

If you were keen to operate on single file wikis perhaps a HTA file or tiddlydesktop hosted wiki would allow you to trigger local batches or file management.

The recent zip to static plugin may help.

Regards
Tony

Just food for thought.

[Edgaras]

Thanks for the reply Tony!

I think "Save trails" is a good idea, I caught a glimpse of it sometime. I have to look into it!

Recently I switched from single file TW to Node.js version (which is in theory still one file). Still learning the process, but I like the automatic saving feature and also clean

folder, where new tiddler .tid appear only when I change something. That gives a good overview. In one file wiki I keep loosing a track what did I upload or change in the sytem.

Also I stopped using TiddlyDesktop as many browser features are missing like a simple cmd+F search.

But it's good idea with local file management that TiddlyDesktop allows.

HTA seems to be only the Windows thing, I am on Mac 😊 

Zip plugin I was using so far, but the process become a bit cumbersome.

I have good hopes for "Save trails"!

[Saq Imtiaz]

@Edgaras A few ways to go about this. I favour a more unix inspired modular approach, small pieces of the overall workflow that do one thing and do it reliably.

My personal approach would be something like this:

1) use config/FileSystemPaths based on a tag to save the tiddlers that should be exported to a separate sub directory. See https://tiddlywiki.com/#Customising%20Tiddler%20File%20Naming

2) a cron job (a scheduled task, both windows and osx have something equivalent too) set to run how every frequently suits you? Every 5 minutes? Every 15 minutes or an hour?

3) have the cron job run a script that runs the node js tiddlywiki commands to export your static site

4) I would probably also use a Makefile to handle the static build process, so that the export static commands only run when dependencies (tiddlers) have changed. 

5) You could also add a step that pushes the exported files to your hosting service.

Alternatively, I believe the tiddlywiki.com build process uses Travis for continuous integration: 

https://travis-ci.org/

Hope this helps,
Saq

[Soren Bjornstad]

I've been working on this with the goal of publishing only part of my personal wiki. I'm just using a real straightforward shell script. Here's a snippet:

PUB_FOLDER="public-wiki"
PRIV_FOLDER="zk-wiki"
FILT="[is[system]] [tag[Public]] -[[$:/plugins/tiddlywiki/tiddlyweb]] -[[$:/plugins/tiddlywiki/filesystem]]"

pub_wiki="${PUB_FOLDER:?oops}/wiki"

echo "Exporting public tiddlers..."
rm -rf "$pub_wiki"
"$(npm bin)/tiddlywiki" "$PRIV_FOLDER" --savewikifolder "$pub_wiki" "$FILT"

This will create a new wiki in the $pub_wiki location containing only the tiddlers matching your filter, which you can then modify further if needed (e.g., I adjust the starting tiddlers and some metadata) and build via whatever method you're currently using.

You could run this periodically or maybe find some way to watch for changes (but since Node saves continuously, that might just cause a lot of thrashing).

[TonyM]

To avoid thrashing if you can trigger this process on save wiki (without autosave) it may be an appropriate regular and appropriate interval.

Tony

[Edgaras]

Seems a bit complex with all extra script modules everywhere.

I just wish simply that TiddlyWiki Node.js would export my recently edited tiddlers into html or markdown files with selected settings (filters).

That can happen automatically on every auto-save, just like new tiddler files are created in "tiddlers" folder. I need recently edited and filtered tiddlers to go to another "static" or "exports" folder.

Then I will sync this folder with dropbox and fast.co automatically will serve my website files. The process would be flawless.

I hope to achieve that with "Save trails". I haven't tried yet.

[Edgaras]

Basically I just need three updates for the new $:/plugins/tiddlywiki/savetrail plugin

1. Let me export not only JSON but also .html files, and ideally also .md

2. Let me define a filter, so I can use tags and export only those tiddlers, instead of currently, all modified tiddlers.

3. Let me define a folder on my TW project where to export files.  

I posted now the feature update request.

[TonyM]

Edgaras,

My suggestion of the savetrails plugin was it has all the code to do the saving. But yes perhaps extending its features could be an approach. Only with item 3 I expect you could find limitations on folders outside the server or browsers folder.

Regards

Tony

[Scott Kingery]

I've recently started using static tiddlers for a couple of projects. The way I'm doing it, I just generate the files and have a scheduled task that runs an FTP process to get the files up to my web host.

The key to it is an addition to the tiddlywiki.info file that sits in the folder with all your wiki files. Add a comma after the last ] in the static section then add:

"last2": [

"--rendertiddler","$:/core/templates/static.template.html","static.html","text/plain",

"--rendertiddler","$:/core/templates/alltiddlers.template.html","alltiddlers.html","text/plain",

"--rendertiddlers","[!is[system]days[-2]]","$:/core/templates/static.tiddler.html","static","text/plain",

"--rendertiddler","$:/core/templates/static.template.css","static/static.css","text/plain"]

Then instead of "build --static" you "build --last2"

This will then build only tiddlers modified in the last 2 days. That may be overkill since the files don't change that much but if for some reason a schedule got missed It would still pick up the files.

The whole process is:

1. Build static and ftp to get everything up to the site

2. Task schedule to build last2 and ftp only changes to site..

- the main caveat here is that a build last2 will leave ONLY that last 2 days of tiddlers in your static folder so you have to be sure your FTP command doesn't erase files that are already up on the server.

Scott