Building static sites in the browser

[Jeremy Ruston]

A disadvantage for some users of TiddlyWiki’s static site generation capabilities is the requirement to use Node.js. I’m therefore pleased to announce an update to the JSZip plugin that makes it practical to render medium sized static sites in the browser, without needing to use Node.js.

https://github.com/Jermolene/TiddlyWiki5/commit/6a0ff7db1807f45b73061ced82f5a85f1a529bbf

The updated plugin adds a handful of messages that can be used in the browser to:

* create a blank ZIP file as a tiddler of type "application/zip”

* add text files to a ZIP tiddler

* download a ZIP tiddler as a .zip file

To try it out, visit the prerelease and review the docs:

https://tiddlywiki.com/prerelease/#%24%3A%2Fplugins%2Ftiddlywiki%2Fjszip

The complete code to save a static site looks like this:

\define actions-render-static-site()

<$action-sendmessage $message="tm-zip-create" $param="$:/temp/_ZipTiddler"/>

<$list filter="[all[tiddlers]!is[system]limit[100]]">

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename={{{ [<currentTiddler>encodeuricomponent[]addsuffix[.html]] }}} tiddler=<<currentTiddler>> template="$:/core/templates/static.tiddler.html"/>

</$list>

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename="static.css" template="$:/core/templates/static.template.css"/>

<$action-sendmessage $message="tm-zip-download" $param="$:/temp/_ZipTiddler" filename="myzip.zip"/>

\end

<$button actions=<<actions-render-static-site>>>

Render site

</$button>

Performance is a bit sluggish with large zip files (>2MB on my desktop machine) but otherwise it seems to work well. I’ve got an idea for improving performance that I’ll work on if I have time.

Questions and comments welcome.

Best wishes

Jeremy

[Mat]

To try it out, visit the prerelease and review the docs:

https://tiddlywiki.com/prerelease/#%24%3A%2Fplugins%2Ftiddlywiki%2Fjszip

For some reason that link doesn't work. I think this one does (even if it looks identical to me):

https://tiddlywiki.com/prerelease/#%24%3A%2Fplugins%2Ftiddlywiki%2Fjszip

<:-)

[Mat]

Having now tried it - good stuff!!! Thank you Jeremy!

It would make sense to access this functionality via the usual AdvancedSearch > Filter export button.

<:-)

[Mark S.]

That's exactly what I was thinking -- a way to download individual tiddlers en masse.

[Jeremy Ruston]

Hi Mat

> Having now tried it - good stuff!!! Thank you Jeremy!
>
> It would make sense to access this functionality via the usual AdvancedSearch > Filter export button.

Yes, perhaps for some simple cases, but for publication one would normally want to pack a bunch of rendered tiddlers and a stylesheet all together into the ZIP, not to mention control over the template.

Best wishes

Jeremy.

[Jeremy Ruston]

Ah, I see what you mean (and presumably Mat too): to export a bunch of tiddlers as a ZIP full of .tids or .jsons etc rather than as a static site. That would indeed make sense.

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/28c0f992-5f35-4927-8bf3-e59b0351bc7b%40googlegroups.com.

[Mark S.]

I wonder how many steps would be left to render a complete epub? I think it's basically a zipped up group of html pages.

[Jeremy Ruston]

> I wonder how many steps would be left to render a complete epub? I think it's basically a zipped up group of html pages.

Yes indeed, that's correct, they're pretty simple (you can rename a random EPUB to .ZIP and unzip it to see).

Best wishes

Jeremy

[Mark S.]

In the doc examples, I think it would be better if a real template were used. "The Content" doesn't exist, and suggests a tiddler with content, not a template. I found it confusing, so maybe others will too ;-)

For "Create zip file" it might be helpful to indicate that this has to be run before the other messages can be run.

Thanks!

On Saturday, May 2, 2020 at 1:36:01 PM UTC-7, Jeremy Ruston wrote:

[TonyM]

Jeremy,

This sounds wonderful, and to achieve it you have introduced additional functionality. It looks like much of this below is possible already, I am just voicing my desires.

Your output continues to astound me.

• I hope these can be used independently such as the zip method discussed here.
• Other outputs such as Generating the files for a Library into a zip as well.
• EPUB and other zipped exports as a smarter alternative to publishing

I would hope a little guidance to allow the following would exist

• Generate the zip and save it in the wiki for subsequent drag and drop
• Generate a zip containing binary files
• This could be away to distribute low size software to install say timimi local host in a wiki.
• Clone or modify the output template and make minor changes
• I am keen to export static where all the links refer to the actual wiki file
• This would allow search SEO to find the HTML tiddlers but user interaction would load the full interactive wiki
• I think this should be a static template made available already

Could we build a little helper plugin to assist in these various uses of such a process? This would release the functionality to more users, sooner in their tiddlywiki learning.

[Mohammad]

Hi Jeremy,

 This is really amazing and this way one can publish small website, blogs, ...

  Thank you a million!

  Few comments:

   1. The tiddler contains nasty characters in their titles like (space /\;  ) results in uggly filename! Like --render the message tm-zip-render-file can have a filename-filter to avoid this.

   2. If commend 1 is implemented then we need a variable to let correct the links of tiddlers their filename corrected as step 1 above

I see alto of potential here and this plugin can be simple and powerful tool for publishing not only static pages but also epub and like that

Best wishes

Mohammad

[Anne-Laure Le Cunff]

Wow, this is amazing! Thanks so much, Jeremy. Very useful and much easier for less technical users.

[Jeremy Ruston]

Hi Mohammad

   1. The tiddler contains nasty characters in their titles like (space /\;  ) results in uggly filename! Like --render the message tm-zip-render-file can have a filename-filter to avoid this.

Just like under Node.js, we’re currently “uri encoding” the tiddler titles. It’s a simple way to guarantee that we don’t generate illegal filenames.

With tm-zip-render-file one can use a filtered attribute to apply this processing as in the examples:

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename={{{ [<currentTiddler>encodeuricomponent[]addsuffix[.html]] }}} tiddler=<<currentTiddler>> template="$:/core/templates/static.tiddler.html"/>

   2. If commend 1 is implemented then we need a variable to let correct the links of tiddlers their filename corrected as step 1 above

The static templates already include a definition of tv-wikilink-template that allows manipulation of the generated tiddler links.

I see alto of potential here and this plugin can be simple and powerful tool for publishing not only static pages but also epub and like that

Great, it’s something I’ve been meaning to do for a few years.

Best wishes

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f586d3ed-07e5-4491-b3dc-08f4b1d4acf5%40googlegroups.com.

[Jeremy Ruston]

Hi Anne-Laure

Wow, this is amazing! Thanks so much, Jeremy. Very useful and much easier for less technical users.

Thanks! It’s entirely thanks to your interest in static site generation that there has been this revival that prompted me to put this together.

Best wishes

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/a1820e61-7ab1-4863-8e03-7750d4e072f7%40googlegroups.com.

[Mohammad]

Hi Jeremy,

Many thanks for clarification!

Best wishes

Mohammad

Hi Mohammad

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

[Thomas Elmiger]

This sounds very promising, thanks a lot. I am looking forward to try it when I find the time.

Am Samstag, 2. Mai 2020 22:36:01 UTC+2 schrieb Jeremy Ruston:

To try it out, visit the prerelease and review the docs:

Do we need the prerelease or only the plugin and TW 5.1.22 ?

Cheers,

Thomas

[Anne-Laure Le Cunff]

I don't use the prerelease and it works perfectly fine for me just with the plugin :)

Here is a quick tutorial and here is my implementation if you want to download it straight away.

Thanks so much again, Jeremy!

[Yoni Balkind]

Hi @Jeremy

This plugin is a great help, thank you.

I notice that the permalink it generates contains funny characters 

/About%2520this%2520site.html instead of /About-this-site.html

Is there a way I can change this? 

I think you answered this question above when you say 

The static templates already include a definition of tv-wikilink-template that allows manipulation of the generated tiddler links.

I'm not sure what that entails. I'm a relatively non-technical user but I'm happy to change templates if someone can point me in the right direction? 

Another question, the export automatically saves to my download folder even though I've my browser settings are such that I should get asked where to save each download. Is there a way I can configure the export to save to a specified folder? 

And finally does it have to be zipped?

The last two questions are to enable me to optimise my publish process. 

[Suzanne McHale]

Yay! This is what I have long wanted in TW - an inbuilt choice to export to separate files, rather than have to use the convoluted Node.js method. The only issue is that I am getting the Javascript freeze-up Unresponsive error when using this for a large TW (in Firefox ESR 52.9) - 

A script on this page may be busy, or it may have stopped responding. You can stop the script now, open the script in the debugger, or let the script continue.

Script: file:///F:/PERSONAL/Tiddlywiki…ogram%20tips line 18025 > eval:13

This happens a few times (the "eval:" changes numbers each time); I just keep clicking the continue button and the file eventually exports.

[Siniy-Kit]

Hi,  Jeremy. Why you put files to zip? All old Tiddlywiki versions can generate static multipage sites without node.js  and JSZip and any macros. And browsers can download many files at one time.

For example open this page https://heeg.ru/shop2.html?id=11f1IzEfXaPZuuVnMya7_50oHOz6kVqK_9-nIYojOHz4#static_cut and press buttons to generate css or js or all pages.

 

<$action-sendmessage

 $message="tm-download-file" 

$param="$:/core/templates/HEEG/static.tiddler.html" 

exportFilter=<<quotedCurrent>>

filename=<<filename>>/>

суббота, 2 мая 2020 г., 23:36:01 UTC+3 пользователь Jeremy Ruston написал:

[Edgaras]

This is very helpful Jeremy! Makes TW an even more competitive Static Site Generator and CMS.

I have a suggestion that removes the need to export the whole site every time:

What if we check the "Last modified" field of every tiddler and export ONLY those that were modified after the latest export?

That would be brilliant, because it would not need to export the whole site everytime!

[Saq Imtiaz]

@Edgaras You can do that already actually. As always, TW is very flexible when it comes to these things. The code Jeremy has provided is just an example of how to leverage the new capabilities. Ohter users may have different use cases and needs, so the flexibility is key. What would be useful is perhaps documentation of other ways to use the new capabilities.

In fact setting it up the way you describe, would be an excellent exercise for an interested beginner like yourself, seeking to better understand how filters and actions in TW work. 

The steps involved would be to customize the button to:

• save the last export date to a tiddler, see docs for ActionSetField: https://tiddlywiki.com/#ActionSetFieldWidget
• customize the list filter around tm-zip-render-file to only include tiddlers since that last export date. See docs for compare operator https://tiddlywiki.com/#compare%20Operator (hint: you may need a nested list)

Regards,

Saq

[Edgaras]

Saq Imatiaz thank you so much for the clarification and the encouragement to develop my own use case! I think I could do it, I will let you know if I succeed :D

–––––––––––––––

Another thing, is there a way to publish static files directly to GitHub Pages? I know it's already possible to publish the site .html file to GitHub, but I would like to only publish static generated files, upload them at every export. 

I know Publii has this feature and it works wonders. I attached the screenshot bellow:

[Mohammad]

Hello Siniy-Kit!

To not hijack this thread, I use another one and I appreciate to give instruction there!

--Mohammad

[Yoni Balkind]

@Edgaras I'd also like a way to export directly to Github pages, but in the interim I've been using to fast.io to host my site directly from Google Drive. So now I export the static pages to a Google Drive folder on my desktop and fast.io has access to that folder and automatically updates my site. Works beautifully, and its free. 

The only problem is that the export automatically goes to my download folder and then I have to unzip it to the correct Google Drive folder. Not a big deal. But if I could configure to the export button to go to a specified folder and for the export to not be zipped then I would literally have a 1 click publish process which would be amazing. 

[Edgaras]

Yoni Balkind funny, I am also using fast.io following Anne-Laure's recommendations, and it works great! I am using Dropbox instead of Google Drive.

Exactly, it would be amazing if it could save to the specified location. Also TiddlyDesktop does not seem to work for exporting at the moment.

Also, I would love to be able to change script so it generates folders named by tiddler names and inside of each – index.html file. So instead of "MyNote.html" I could get "MyNote/index.html" (that's a file inside a folder).

This way the static website using fast.io could display mycustomdomain.com/mynote instead of mycustomdomain.com/mynote.html

[<currentTiddler>encodeuricomponent[]addsuffix[.html]]

I have tried to change this part to achieve that, but I could not figure out how.

[Anne-Laure Le Cunff]

Edgaras - I used this to fix my URLs!

[Yoni Balkind]

@Edgaras regarding TiddlyDesktop not exporting nicely - I'm just using the normal TW file and saving with Timimi, works nicely. 

@Anne-Laure I was just about to comment on the fact that I'm still struggling with ugly permalinks like this:
www.mysite.com/About%2520this%2520site.html

Those scripts that you linked to - can they be used with the new export method? If so, do we put them into the "export" tiddly together with the other code? 

[Anne-Laure Le Cunff]

Hi Yoni, see walkthrough attached.

This also should be an easy solution to the URL issue.

[Anne-Laure Le Cunff]

By the way the files are called something "nurselog" because his website is called Nurse Log, I used "nesslabs" for my files - doesn't really matter but just in case you wonder what "nurselog" is about :)

[Edgaras]

Yoni Balkind sadly Timimi does not have a Safari plugin...

Anne-Laure wow, nice thank you for sharing! It seems a bit complex, I put those two macros in the system, but I am not sure how to reference them in the export button code.

[Anne-Laure Le Cunff]

I only tested these with the Node.js version of TiddlyWiki, and I didn't have to do anything else beside creating these two files and making sure to set their type properly. When I exported again, the URLs had hyphens instead of spaces between the words.

Honestly not sure if there's an extra step to make it work with the in-browser export tool, maybe someone can chime in!

On Friday, May 8, 2020 at 7:51:27 PM UTC+1, Edgaras wrote:

Yoni Balkind sadly Timmini does not have a Safari plugin...

[C. Wess Daniels]

HI all,

I've got a hiccup in my setup as well.

I am using the exporter and uploading the static site to neocities.org.

I've setup my custom domain http://nurselog.online

But only the index page is working. All the other ones are broken and it seems like Neocities is adding a couple extra numbers.

I followed the instructions here (thanks to Anne-Laure's help): https://www.didaxy.com/exporting-static-sites-from-tiddlywiki-part-5

And then exported again but I'm still having the same problem.

Here are a few screenshots of what I entered and what I got.

Also note, I moved my tiddlywiki from the downloads folder to this path a few days ago and I think I have it all redirected but not sure if that would create an issue or not:

file:///Users/wessair/Dropbox/4.%20Archives/TiddlyWiki%20Backups/twBackups/empty.html/empty.html

Does anyone know what I need to do to fix this?

Thank you!

Wess

[Yoni Balkind]

Thank you Anne-Laure!

I gave this a quick test run and it did not work.. Put perhaps this gets us part of the way there and someone else can help with the additional steps

[Yoni Balkind]

Mind you I'm not sure if I'm correctly setting the type and module type? Anne-Laure I used the 3 fields as show in the screenshot. Is that right? 

[Anne-Laure Le Cunff]

Hi Yoni,

Here is a screenshot of what mine looks like. I have no idea what exactly is going wrong but I suspect the browser-based exporter doesn't work exactly like the node-based exporter I'm using.

[TonyM]

A Quick Tip;

For those using dropbox and drive remember you can synchronise these or part of such as a file or folder locally. If your workflow keeps and saves the output to the local drive it can be automatically synchronised. 
The only caveat is to make sure the sync finishes before turning off your computer. 

In this thread there are a couple of tools mentioned for publishing drive or dropbox files on the internet such as fast.io

A lot of web hosting services also allow you to configure WedDav or online Web drive equivalents for your own folders, and automated ftp is also possible. These synchronisation methods are particularly use full for read only or static sites, but as long as you retain the "source of truth" the wiki copy you edit things in, it should all go smoothly.

I think all this static file activity has inspired a larger number of internet publishing, Please continue to acknowledge tiddlywiki where practical and do share back how you did it, as well as publicising your own published content.

Regards

Tony

[C. Wess Daniels]

It looks like the only solution at this moment (that I'm able to figure out anyway) is to do the "kebab" approach referenced here: https://www.didaxy.com/exporting-static-sites-from-tiddlywiki-part-5

If you look at my page nurselog.online I've fixed the title of one entry to "Mind-Garden" removing the space and now it works. Not a real ideal way to go but I'm happy to have something that works. I'll update the rest of the pages soon.

Wess

[Edgaras]

C. Wess Daniels

I think the idea is to insert "kebab" dashes "-" during the export of your static site, rather than changing the original titles. The titles should stay as they are and the export process should take care of file names and therefore url slugs.

[Anne-Laure Le Cunff]

I have indeed seen people do the kebab casing for their titles to get around the issue:

https://tacticaltypos.net/

And yes always good to give credits to TiddlyWiki! I personally have links to tutorials as well on mine: https://www.mentalnodes.com/about

[Saq Imtiaz]

Regarding file names, it pays off to look at what your code is trying to do ;)

Pay attention to the highlighted line below, this is the parameter to the tm-zip-render-file message that provides the file name.

\define actions-render-static-site()
<$action-sendmessage $message="tm-zip-create" $param="$:/temp/_ZipTiddler"/>
<$list filter="[all[tiddlers]!is[system]limit[100]]">
<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename={{{ [<currentTiddler>encodeuricomponent[]addsuffix[.html]] }}} tiddler=<<currentTiddler>> template="$:/core/templates/static.tiddler.html"/>
</$list>
<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename="static.css" template="$:/core/templates/static.template.css"/>
<$action-sendmessage $message="tm-zip-download" $param="$:/temp/_ZipTiddler" filename="myzip.zip"/>
\end


<$button actions=<<actions-render-static-site>>>
Render site
</$button>

 currentTiddler is the tiddler title and in triple braces you have a filter manipulating the tiddler title. For example,

[<currentTiddler>split[ ]join[-]addsuffix[.html]]

will replace spaces with dashes in the tiddler titles. You may also wish to encodeuricomponent after that as well. See https://tiddlywiki.com/#Filter%20Operators

Now you need to get your links to match, for which you need to set the tv-filter-export-link variable. See https://tiddlywiki.com/#tv-filter-export-link%20Variable

Create a tiddler and give it the tag $:/tags/Macro and the content:

\define tv-filter-export-link() [split[ ]join[-]encodeuricomponent[]addsuffix[.html]]

The logic in this filter assumes it is being provided with the tiddler title as input. With that exception only, it should match the filter you set for filenames.

That should do the trick. Do check the syntax for potential errors if it doesn't, as this is untested code.
Regards,
Saq

[Edgaras]

@Saq Imtiaz  it works and it's very useful!

Is there a way to render each file in a dedicated folder? Instead of "My-Tiddly-Note.html" to "My-Tiddly-Note/index.html"?

[Saq Imtiaz]

@Edgaras, yes you can.

Just tweak the filter to get the structure you want:

[<currentTiddler>split[ ]join[-]addsuffix[.html]]

Hint: it is all about the addsuffix operator in this case, and the Filter tab of Advanced Search is a good place to test your filters

[Anne-Laure Le Cunff]

@Edgaras: it's very bad practice to do that. I assume you want to get rid of the .html in the file name to clean the URL? All the sites that don't have a file extension at the end of their URLs do that on the server with a .htaccess file for instance, not by doing folder/index.html for each page. You could look at Fast's documentation to see if you can upload a .htaccess file.

(apologies if that's not what you're trying to achieve here!)

[Edgaras]

Anne-Laure yes, but they don't support .htaccess for the time being. Therefore, I want to tweak my TW export for now to export to folders and index.html, that's the only way for now.

[Anne-Laure Le Cunff]

Ha that's unfortunate! Good Saq gave you a solution then. I'll also edit the tutorial at some point to add the URL encoding. Thank you!

[Edgaras]

Saq Imtiaz it worked! And the Advance Search filter tip is super helpful:) 

BUT, there is one problem! Is it a bug?

\define tv-wikilink-template() $uri_doubleencoded$.html

the tv-wikilink-template() should let you define the custom url link for your site. However, whatever I write I still get extra "./" characters in front. For example if I define:

\define tv-wikilink-template() /$uri_doubleencoded$/index.html

I still get ".//My-Note/index.html" instead of "/My-Note/index.html"

any ideas why?

[Saq Imtiaz]

@Edgaras 

"The tv-wikilink-template variable controls the value of the href attribute on the HTML a element generated by the $link widget. The tv-filter-export-link, if defined, it takes precedence over the tv-wikilink-template variable." from https://tiddlywiki.com/#tv-wikilink-template%20Variable

Once you have set tv-filter-export-link, tv-wikilink-template is ignored. Whatever customization you want to make should happen in tv-filter-export-link

[Saq Imtiaz]

You probably want something like this for tv-filter-export-link:

[<currentTiddler>split[ ]join[-]addsuffix[/index.html]]

[Edgaras]

@Saq Imtiaz Yes, I read that documentation and I was thinking about it, but I could not find where is that tv-filter-export-link is actually used?? 

[Edgaras]

@Saq Imtiaz oh no no, the exporting works fine now, creates folders and right file names. The problem now are the links within my files are messed  up with "./" in front, and I need just "/" because I need to point to root directory.

[Saq Imtiaz]

@Edgaras read carefully the second part of my post from 9:55CET in this thread, starting with the words "Now you need to get your links to match" :)

[Saq Imtiaz]

When you create macro in a tiddler with the tag $:/tags/Macro, you are creating a globally available macro/variable

[Mohammad]

Nice solution Saq,

It would be nice if someone summarize the final solution as a step wize procedure!

Cheers

Mohammad

[Edgaras]

ahhhh weird my message got deleted right away when posting...

Saq Imtiaz yes, I skipped that part you mentioned, or at least i did not understand the problem I will run into at that point ;D

I've tried to create the macro as you said, but it did not help. The filter should be correct I checked with AdvancedSearch:

Screenshot-2020-05-09-at-13-25-13.png

[Saq Imtiaz]

Edgaras:

Try this:

restore tv-wikilink-template to it's original value in $:/core/templates/server/static.tiddler.html (or your own template)

\define tv-wikilink-template() $uri_encoded$

Use this for tv-filter-export-link:

\define tv-filter-export-link() [split[ ]join[-]encodeuricomponent[]addsuffix[/index.html]addprefix[/]]

I just tested this on tiddlywiki.com and it works. Sample file attached.

[Edgaras]

Hmm... I did exactly like that, but still does not work, adds "./" to links

I added my who site here if you bother to see: http://whatif.space/whatifspace.html

3 key tiddlers:

- Export

- $:/whatifspace/Macros/exportlink

- $:/whatifspace/templates/static.tiddler.html

[Saq Imtiaz]

Delete $:/whatifspace/static/export/get-export-link.js

It tries to do the same thing as tv-filter-export-link but in javascript and the two are interefering with each other. 

Having the javascript option helps if you want to do something really complex, but using tv-filter-export-link is more user friendly in my opinion.

[Edgaras]

Saq Imtiaz amazing! It works now! This should be definitely documented! 

Anne-Laure you probably could use this to update your post?

Next thing will be to follow Saq your recommendations and to build filters on what to export ;) 

• Like we talked, export only the modified tiddlers after the last export.
• And probably only tiddlers marked with "Publish" tag, so personal notes can stay not exported.
• Also I will try to make it so the tiddler that is marked with "indexPage" tag will be named as index.html and exported to root directory, rather than custom directory.
  

[Saq Imtiaz]

In summary, if anyone wants to write a guide, to adjust filenames and links (for single file wikis) I recommend this process as per my current understanding:

1. create a tiddler called $:/config/static/export-link-filter and place here the filter that will operate on the tiddler title to transform it as you like, this filter will receive the tiddler title as input. For example:
   
   [split[ ]join[-]encodeuricomponent[]addsuffix[.html]]
   
   
2. use this filter in the code for your export button:

\define actions-render-static-site()
<$action-sendmessage $message="tm-zip-create" $param="$:/temp/_ZipTiddler"/>
<$list filter="[all[tiddlers]!is[system]limit[100]]">

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename={{{ [<currentTiddler>subfilter{$:/config/static/export-link-filter}] }}} tiddler=<<currentTiddler>> template="$:/core/templates/static.tiddler.html"/>

</$list>
<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename="static.css" template="$:/core/templates/static.template.css"/>
<$action-sendmessage $message="tm-zip-download" $param="$:/temp/_ZipTiddler" filename="myzip.zip"/>
\end

<$button actions=<<actions-render-static-site>>>
Render site
</$button>

3. Create a tiddler called $:/config/static/tv-filter-export-link, with the tag $:/tags/Macro and the content:

\define tv-filter-export-link() [subfilter{$:/config/static/export-link-filter}]

This is the same process as what I outlined earlier except, you write the filter once and refer to it in both places, reducing the chances of error.

Hope this helps,

Saq

[Saq Imtiaz]

@Edgaras one thing to note about only exporting tiddlers that were modified since the last export: it will work great if you have simple text content. But if you are using transclusions and widgets, the previously exported tiddlers might not have been modified since the last export, but may need to be exported again anyway since transcluded content may have changed.

[Edgaras]

Saq Imtiaz good explanation. It works well with your new update too!

What my OCD brain asks: how do you keep track of all those extra small config files that we scatter all over the place? I try at least to list all additional ones I add under my project name. So in the future I can tell someone how to achieve the same result from brand new empty.html TiddlyWiki. Anyways, this is offtopic😊

Yeah, you are right about the latest modified and problems with transclusions. I think I would have two buttons:

1. One simple one when I just want to publish few recently added or edited notes.
   
2. Second one full export, that I would use only once in a while, or when problem occurs to make sure all notes function as they should.

[TonyM]

Folks

I am keen to make use of your great work here and the tutorials to match but there has being a lot of water under the bridge. If one or more of can isolate the key changes and package it the instructions will become simpler and even easier to document and teach.

Its easy to do with a good understanding of tiddlywiki which I have but I do not have a strong understanding in this particular subject.

Please give it some thought.

Regards
Tony

[Saq Imtiaz]

@Edgaras it is a valid point, especially when I look back at things I setup a year or two ago!

I too tend to:

a) place things under a name, for example $:/sq/stories or $:/sq/static and then a filter can be be handy [prefix[$:/sq/static/]]

b) when it makes more sense for the tiddler to be in $:/config or if its a shadow tiddler I need to tweak etc, I usually have a listing page per "project" that lists all the tiddlers required for it, usually generated by the above filter with a few tiddlers added on by name.

With a filter for all the tiddlers you need for a certain "project" it is easy to see a list or create a button to drag and drop them into another wiki. For example:

<$button dragFilter=<<myfilter>> > Static export tiddlers </$button>

Regards,

Saq

[Mohammad]

Many thanks Saq, Briliant!

 This is really good! I will add this to TW-Scripts!

  I just missed the part you explained on storing tiddlers in different folders!

--Mohammad

[Mohammad]

@Saq,

 I will change the filter to 

[lowercase[]split[ ]join[-]encodeuricomponent[]addsuffix[.html]]

--Mohammad

[Saq Imtiaz]

@Mohammad if the filter returns a path with an "/" in the file name, you create a directory structure. The filter I gave is just an example and users should customize according to their use case.

Also if you can add a link back to the tiddler on TW scripts that would be very helpful.
Regards,

Saq

[Mohammad Rahmani]

Hi Saq,

 I got it! A million thanks!

Best wishes

Mohammad

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/166113f6-6502-4693-b891-16f259c96a69%40googlegroups.com.

[Jeremy Ruston]

I’ve just pushed an update that adds a new “slugify” operator to transform arbitrary tiddler titles into human readable form for use in filenames or URLs. The transformations applied are:

• If there is a tiddler with the same title that has a ''slug'' field, then return that field instead of running the following steps
• Replace uppercase letters with lowercase equivalents
• Transliterate diacritics to their basic lowercase ASCII equivalents (for example, "Æ" is transliterated to "AE")
• Replace spaces with dashes
• Remove all non-word characters except dashes and periods
• Replace multiple sequential dashes with a single dash
• Trim dashes from start and end
• If the result is the empty string then character codes are used instead (eg. "&£@" transforms to "38-163-64")

There is also a new duplicateslugs operator to detect when multiple tiddlers resolve to the same slug.

I’ve also updated the JSZip example to use the slugify operator.

You can try out the update at https://tiddlywiki.com/prerelease

Best wishes

Jeremy.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/CAAV1gMB5CEK0w2B1J%2B-HW4cGLdtTFBSDxSFmX00pYSoJpmGqKw%40mail.gmail.com.

[Saq Imtiaz]

Very welcome update Jeremy!

Everyone following my previous suggestions: Please note that the updated example replaces the 3 different steps in one go. Those steps will still work but are no longer necessary. See:
https://tiddlywiki.com/prerelease/#%24%3A%2Fplugins%2Ftiddlywiki%2Fjszip

The only potential improvement I would recommend is to use the subfilter trick as I outlined earlier, so you are writing the filter only once, for file names and links. This will make it easier to tweak the links and remove the need to make the change in two places.

[Edgaras]

Jeremy great update! 

I think I am getting a bug:

Screenshot-2020-05-09-at-18-59-36.png

Says that my note slugs are duplicate and exports all of them in an old way.

[C. Wess Daniels]

Hi all,

I am following along and the export part is working:

But the site won’t load with the links:

Saq - I think I followed the steps correctly from your earlier post.

1. Edited the zip-render-file

2. Created a tiddler titled $:/nurselog/Macro/exportlink, added the $:/tags/Macro  and code. 

3. I went in and restored $:/core/templates/server/static.tiddler.htm to \define tv-wikilink-template() $uri_encoded$

4. I deleted $:/nurselog/static/export/get-export-link.js

(I didn’t use the new code for the tv-fiter-export-link because it was customized for Edgaras.

But I’m still getting a dead site. 

Any idea what I’m doing wrong?

Thanks!

Wess

On Saturday, May 2, 2020 at 4:36:01 PM UTC-4, Jeremy Ruston wrote:

A disadvantage for some users of TiddlyWiki’s static site generation capabilities is the requirement to use Node.js. I’m therefore pleased to announce an update to the JSZip plugin that makes it practical to render medium sized static sites in the browser, without needing to use Node.js.

https://github.com/Jermolene/TiddlyWiki5/commit/6a0ff7db1807f45b73061ced82f5a85f1a529bbf

The updated plugin adds a handful of messages that can be used in the browser to:

* create a blank ZIP file as a tiddler of type "application/zip”

* add text files to a ZIP tiddler

* download a ZIP tiddler as a .zip file

To try it out, visit the prerelease and review the docs:

https://tiddlywiki.com/prerelease/#%24%3A%2Fplugins%2Ftiddlywiki%2Fjszip

The complete code to save a static site looks like this:

\define actions-render-static-site()

<$action-sendmessage $message="tm-zip-create" $param="$:/temp/_ZipTiddler"/>

<$list filter="[all[tiddlers]!is[system]limit[100]]">

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename={{{ [<currentTiddler>encodeuricomponent[]addsuffix[.html]] }}} tiddler=<<currentTiddler>> template="$:/core/templates/static.tiddler.html"/>

</$list>

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename="static.css" template="$:/core/templates/static.template.css"/>

<$action-sendmessage $message="tm-zip-download" $param="$:/temp/_ZipTiddler" filename="myzip.zip"/>

\end

<$button actions=<<actions-render-static-site>>>

Render site

</$button>

Performance is a bit sluggish with large zip files (>2MB on my desktop machine) but otherwise it seems to work well. I’ve got an idea for improving performance that I’ll work on if I have time.

Questions and comments welcome.

Best wishes

Jeremy

[Saq Imtiaz]

Wess:

Two issues:

1. Capitalization
   https://nurselog.online/Mind-Garden.html and not https://nurselog.online/mind-garden.html
   Avoid this by adding lowercase[] to both filters, eg
    [lowercase[]split[ ]join[-]encodeuricomponent[]addsuffix[.html]]
   Currently it looks like you have the lowercase operator in the file name filter but not the one for links.
   
   
2. You need the .html at the end of the URL's unless you set up an .htaccess file on your host to get around that.
   

I am also seeing some different handling of the . character in tiddler titles. I highly suggest you change the new slugify operator that Jeremy has posted.

Otherwise make sure that your filters for the file name and the link are doing the same thing. 

If still in doubt, post your exact code for the static export.

[Saq Imtiaz]

@Edgaras I assume that warning is just for some of your tiddlers? If so, you can give those tiddlers a field called 'slug' and specify what to use for the slug/filename

[Mohammad]

Hi Jeremy,

This is absolutely amazing feature! Now Tiddlywiki is a powerful static multi-pages website generator

and real killer of many rival software out there.

--Mohammad

On Saturday, May 9, 2020 at 7:44:50 PM UTC+4:30, Jeremy Ruston wrote:

I’ve just pushed an update that adds a new “slugify” operator to transform arbitrary tiddler titles into human readable form for use in filenames or URLs. The transformations applied are:

• If there is a tiddler with the same title that has a ''slug'' field, then return that field instead of running the following steps
• Replace uppercase letters with lowercase equivalents
• Transliterate diacritics to their basic lowercase ASCII equivalents (for example, "Æ" is transliterated to "AE")
• Replace spaces with dashes
• Remove all non-word characters except dashes and periods
• Replace multiple sequential dashes with a single dash
• Trim dashes from start and end
• If the result is the empty string then character codes are used instead (eg. "&£@" transforms to "38-163-64")

There is also a new duplicateslugs operator to detect when multiple tiddlers resolve to the same slug.

I’ve also updated the JSZip example to use the slugify operator.

You can try out the update at https://tiddlywiki.com/prerelease

Best wishes

Jeremy.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/166113f6-6502-4693-b891-16f259c96a69%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

[Yoni Balkind]

Sounds perfect Jeremy, thank you. 

Excuse my ignorance, but how do I make use of this update? Do I just follow identical steps as per the original post above (same link and same code)? 

[Edgaras]

Saq Imtiaz unfortunately, it was happening for all of my tiddlers, even new ones I create. I reverted back to old version. It works if I install plugin on a brand new site.

Yoni Balkind just drag and drop the plugin (puzzle icon) to your TW site window. Then go to More > Plugins and you can try the JSZip library plugin there. Use the examples tab in the plugin location and press "Render site" button. Later you can create your own new tiddler, paste the code from example and make your own changes 😊 

[Yoni Balkind]

Thanks Edgaras. I had the old one installed already. Just wasn't sure how to go about updating to the new version. 

Anyway I've done so now and I have the same issue as you, I get the warning that all my tiddlers have duplicate slugs. Nothing happens when I try to export. 

I even tried @Saq's suggestion about changing a slug on one of the Tiddlers but it was still listed as having a duplicate slug.. 

[Saq Imtiaz]

@Yoni @Edgaras just to be clear, you are both using the latest pre-release version of TiddlyWiki? the slugify operator is cutting edge and only available in the pre-release

[Yoni Balkind]

@Saq I did understand that the that plugin was pre-release, but are you saying that regardless of the state of the plugin it might have compatibility issues with version 5.1.22 of TW? 

Also, is there a "typical" timeline as to when this might reach get promoted to main release? (I fully understand of course that this is a volunteer open-source project)

[Saq Imtiaz]

The slugify operator is going to be a part of the TiddlyWiki core from the next release, it is not a part of the plugin.

So your choices for now are:

- use the steps I outlined earlier, which will work for TW 5.1.22. The only thing not available to you in 5.1.22 is the slugify operator. The zip plugin will work just fine.

- use the pre-release instead. I use it on a daily basis and haven't encountered any issues.

There is no release schedule for TW. Releases tend not to be too often, to ensure time for testing new features, backwards compatibility, and because updating too frequently can be cumbersome for non-technical users and not worthwhile for minor changes. This is entirely my opinion on the release schedule though, so take it with a grain of salt.

Regards,

Saq

[Yoni Balkind]

Gotcha. Okay I''ll try above version first, but in case I decide to upgrade to latest version, can you point me in right direction (for standalone file)? 
I've found the upgrade link (tiddlywiki.com/upgrade.html) but that presumably doesn't help for upgrading to a pre-release version 

[Saq Imtiaz]

Pre-release: https://tiddlywiki.com/prerelease/empty.html

Open it, drag and drop your old file on it, choose what to import.

[Edgaras]

Saq Imtiaz yes, that was the problem, thank you again for the support here!

[C. Wess Daniels]

Just to close this loop. I was able to get it to work. I started from scratch using the pre-release!

Thank you for all the help. I really appreciate.

https://nurselog.online/

On Saturday, May 2, 2020 at 4:36:01 PM UTC-4, Jeremy Ruston wrote:

A disadvantage for some users of TiddlyWiki’s static site generation capabilities is the requirement to use Node.js. I’m therefore pleased to announce an update to the JSZip plugin that makes it practical to render medium sized static sites in the browser, without needing to use Node.js.

https://github.com/Jermolene/TiddlyWiki5/commit/6a0ff7db1807f45b73061ced82f5a85f1a529bbf

The updated plugin adds a handful of messages that can be used in the browser to:

* create a blank ZIP file as a tiddler of type "application/zip”

* add text files to a ZIP tiddler

* download a ZIP tiddler as a .zip file

To try it out, visit the prerelease and review the docs:

https://tiddlywiki.com/prerelease/#%24%3A%2Fplugins%2Ftiddlywiki%2Fjszip

The complete code to save a static site looks like this:

\define actions-render-static-site()

<$action-sendmessage $message="tm-zip-create" $param="$:/temp/_ZipTiddler"/>

<$list filter="[all[tiddlers]!is[system]limit[100]]">

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename={{{ [<currentTiddler>encodeuricomponent[]addsuffix[.html]] }}} tiddler=<<currentTiddler>> template="$:/core/templates/static.tiddler.html"/>

</$list>

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename="static.css" template="$:/core/templates/static.template.css"/>

<$action-sendmessage $message="tm-zip-download" $param="$:/temp/_ZipTiddler" filename="myzip.zip"/>

\end

<$button actions=<<actions-render-static-site>>>

Render site

</$button>

[Celz Alejandro]

@Saq Imtiaz thanks for this! I've been trying to find a way to do this and I got it to work thanks to your instructions!

I'd like to add that step-by-step instructions like these are super helpful for people like me who aren't well-versed in coding

On Saturday, May 9, 2020 at 8:35:59 PM UTC+8, Saq Imtiaz wrote:

In summary, if anyone wants to write a guide, to adjust filenames and links (for single file wikis) I recommend this process as per my current understanding:

1. create a tiddler called $:/config/static/export-link-filter and place here the filter that will operate on the tiddler title to transform it as you like, this filter will receive the tiddler title as input. For example:
   
   [split[ ]join[-]encodeuricomponent[]addsuffix[.html]]
   
   
2. use this filter in the code for your export button:

\define actions-render-static-site()
<$action-sendmessage $message="tm-zip-create" $param="$:/temp/_ZipTiddler"/>
<$list filter="[all[tiddlers]!is[system]limit[100]]">

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename={{{ [<currentTiddler>subfilter{$:/config/static/export-link-filter}] }}} tiddler=<<currentTiddler>> template="$:/core/templates/static.tiddler.html"/>

</$list>
<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename="static.css" template="$:/core/templates/static.template.css"/>
<$action-sendmessage $message="tm-zip-download" $param="$:/temp/_ZipTiddler" filename="myzip.zip"/>
\end

<$button actions=<<actions-render-static-site>>>
Render site
</$button>

[C. Wess Daniels]

Hi all - I'm running into a new issue with the pre-release:

I went export the site this morning using the new plugin in the pre-released version (installed a few days ago) and it's only exporting the original set of notes I uploaded on Monday. It's not including the 6 new ones I created today. I'm using the JSZip Plugin "render file" button.

Any ideas about why this would be happening?

Thanks!

On Saturday, May 2, 2020 at 4:36:01 PM UTC-4, Jeremy Ruston wrote:

A disadvantage for some users of TiddlyWiki’s static site generation capabilities is the requirement to use Node.js. I’m therefore pleased to announce an update to the JSZip plugin that makes it practical to render medium sized static sites in the browser, without needing to use Node.js.

https://github.com/Jermolene/TiddlyWiki5/commit/6a0ff7db1807f45b73061ced82f5a85f1a529bbf

The updated plugin adds a handful of messages that can be used in the browser to:

* create a blank ZIP file as a tiddler of type "application/zip”

* add text files to a ZIP tiddler

* download a ZIP tiddler as a .zip file

To try it out, visit the prerelease and review the docs:

https://tiddlywiki.com/prerelease/#%24%3A%2Fplugins%2Ftiddlywiki%2Fjszip

The complete code to save a static site looks like this:

\define actions-render-static-site()

<$action-sendmessage $message="tm-zip-create" $param="$:/temp/_ZipTiddler"/>

<$list filter="[all[tiddlers]!is[system]limit[100]]">

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename={{{ [<currentTiddler>encodeuricomponent[]addsuffix[.html]] }}} tiddler=<<currentTiddler>> template="$:/core/templates/static.tiddler.html"/>

</$list>

<$action-sendmessage $message="tm-zip-render-file" $param="$:/temp/_ZipTiddler" filename="static.css" template="$:/core/templates/static.template.css"/>

<$action-sendmessage $message="tm-zip-download" $param="$:/temp/_ZipTiddler" filename="myzip.zip"/>

\end

<$button actions=<<actions-render-static-site>>>

Render site

</$button>

[Edgaras]

One reason could be, if you look closely into the code of that button there is a filter, I think it limits the export to 25 notes max. You can increase that I guess [all[tiddlers]!is[system]limit[25]]

[C. Wess Daniels]

Edgaras, yes, that seem to do the trick!

[Edgaras]

I am glad it helped ;) I removed the limit[25] part completely. I still have an idea to build a custom Export button, where only recently edited tiddlers are exported, so I don't need to reexport everything every time.

I would still have another button for global re-export. That one I would use once in a while to make sure everything connects well.

[C. Wess Daniels]

That sounds like it would be really useful. I'd like the functionality of excluding certain tags from an export. I'm sure it's possible but I'm not sure where/or what to enter. Any thoughts on that?

[Edgaras]

That's pretty easy actually and I am doing that already. I tag notes that I wish to be exported with "published" tag. And then change export button script, this part:

"<$list filter="[tag[published]!is[system]]">"