TW + Github pages = replace Jekyll and also replace tiddlyspot/space?

[Peter Merel]

I was desperately in love with Tiddlywiki back in the day, only just stumbled over TW5 and have been immersing for the past 24 hours.

My intended usage shouldn't be that unusual, I think, but I don't seem to be able to find anyone else who's doing things this way. As I'm not a javascript dude I'm basically asking for advice on how to go about this, or ideally where I can find something somebody's already done that works like this:

1) I have a small community of people working on documenting advances in Agile working. See agiletng.org for last year's incarnation of the group. It's all Octopress and I'm running screaming away from that kind of thing. I want a damned wiki and I want it to be damned tiddly. Intertwingling static web pages is too damned hard.

2) My little community of people would all run the tiddly node server locally on their machines to edit tiddlers as files locally on their machines. These tiddlers would all be backed by git and originate in some github pages repo similarly to the way Jeremy does his. From what I can see that's already working and documented on youtube.

3) When a tiddler saves, it automatically commits and pushes the respective tiddler file. If it turns out git has to pull a tiddler because someone else has edited it, it gives up and lets the user sort the situation out using git and human communication with the other dude. I think this happens on a per tiddler basis rather than some larger set of edits because that way people won't have so much rework to do and RecentChanges can save them most of the time. Anyway the point is tiddly does not need to pretend to be git.

4) There must be some npm tiddly server command that will generate a single-page wiki from the node server's files. I don't know what it is but it stands to reason - surely? Anyway the point would be to save that as the index.html of the github pages site and push it whenever a tiddler is successfully edited and pushed.  

5) It would be a lot better for load times if the one-page version could source the tiddlers and any images from the github pages repo on demand. Git could handle all the relationships between tiddler bases without doing tiddlyspace style inclusions.

This seems really simple and highly desirable to me. Which parts are wrong/unimplemented-at-this-time? Or maybe that's how it's supposed to work now anyway? Otherwise any criticism/advice would be very gratefully received!

[PMario]

Hi Peter,

With TiddlyWiki you can do both. .. you can create a big index.html file that contains the whole content and is served by github page. Same as tiddlywiki.com and all the existing editions.

It is also possible, to create one plane html page, without any javascript. see eg: http://tiddlywiki.com/static/HelloThere.html
All tiddlers, linked from this page are also plain html pages. ... So this is very similar to Octopress. TW uses the same page layout, but with enough TW knowledge, you can change this layout. .. It will be quite some work, but it should be doable.

Reading your post, it seems your users / editors are able to use git pull, push, merge, branch ... commands. So they can do some conflict handling. .. right?

IMO it would need an automated build process, that creates and publishes a new version of all pages, if someone pushes to the "master" branch.

 - So "master" is the "source of truth".
 - If a user wants to create new content, they pull master into their local tw repo.
 - They create a new feature branch
   - They can use the tiddlywiki local server to do the editing.
   - see: tiddlyiwki ..... --server command: http://tiddlywiki.com/#ServerCommand and http://tiddlywiki.com/#Installing%20TiddlyWiki%20on%20Node.js
 - if finished, they push the new content to the feature branch and create a pull request.
 - if the pull request can be discussed, changed  and merged
    - So you can use the full power of github.
 - If the PR is merged, this will trigger an automated build, that publishes the new content.

The workflow is the same, if you use a big tiddlywiki.html file or a single page index.html
If you have a lot of images or/and binary data, that need to be cached client side, I'd go with the single file approach.
The advantage of a TW file, that contains everything, is, that it is searchable, shareable ....

just some ideas.
mario

[Peter Merel]

Hi Mario,

Thanks for your thoughts. I was aware of the static generation feature but really see tiddly style intertwingling as ideal for this project. I think it will also make heavy use of TiddlyMap - really TW has become so beautiful these days ...

I guess to begin with all I'm going to do is hand generate the one-page version using the ServerCommand and a browser, then automate that and the pushing. Still I'm not quite sure how I can get the node server not to lazyload tiddlers nor link back to the original TW server. I want it to generate just a self-contained one-page-wiki once it's on github pages where there can be no server. Maybe lazyloading the images. My initial experiments don't seem to do this properly so I'd be much obliged if you can think of a formula that achieves it.

I like topic branches for code, of course, and I can see the PR workflow as being good for project documentation, but I question its wisdom for collaborative development of the more conceptual collateral my group is creating. I see a simple single-edit-push workflow without branching as closer to the WikiNature (http://c2.com/cgi/wiki/wiki?WikiNature) and easier for folk without much git-lore to grok so long as they're conceptually aligned. If the community became large I'd agree it would want to go to pull requests to avoid a c2 style heat-death.

Thanks for your ideas!
Pete.

[Peter Merel]

Whoops, my initial experiments were stupid, of course it's easy to save the served wiki as a flat file. D'oh!

[PMario]

On Thursday, August 6, 2015 at 12:10:28 AM UTC+2, Peter Merel wrote:

Thanks for your thoughts. I was aware of the static generation feature but really see tiddly style intertwingling as ideal for this project. I think it will also make heavy use of TiddlyMap - really TW has become so beautiful these days ...

I see. Yes. I also like TM a lot.

I guess to begin with all I'm going to do is hand generate the one-page version using the ServerCommand and a browser, then automate that and the pushing.

ok
 

Still I'm not quite sure how I can get the node server not to lazyload tiddlers nor link back to the original TW server.

I don't understand this question.

 - If you run a local nodejs server you need the source tiddlers as single *.tid files.
   - Those .tid files are source controlled. ..
 - An automated script builds and pushes the 1 file index.html TW to github pages repository.
- Github pages repo is separated from the tiddler source repo.

 - image can be image.jpg with an image.meta file. where the imgage.meta file contains the tiddler meta data. eg: tags. see: https://github.com/Jermolene/TiddlyWiki5/tree/master/editions/tw5.com/tiddlers/images

 - the .meta file can also contain a _cononical_uri. You just need to take care, that your images are available at this address.
 

I want it to generate just a self-contained one-page-wiki once it's on github pages where there can be no server. Maybe lazyloading the images. My initial experiments don't seem to do this properly so I'd be much obliged if you can think of a formula that achieves it.

For "lazy" external images, have a look here: http://tiddlywiki.com/#ExternalImages 
 

I like topic branches for code, of course, and I can see the PR workflow as being good for project documentation, but I question its wisdom for collaborative development of the more conceptual collateral my group is creating.

ok
 

I see a simple single-edit-push workflow without branching as closer to the WikiNature (http://c2.com/cgi/wiki/wiki?WikiNature) and easier for folk without much git-lore to grok so long as they're conceptually aligned.

The problem I see, if everyone wants to push to master, you will create merge conflicts. And those are much much harder to resolve in a sane way, than PRs for a branch. ... but anyway. That's your decision.
 

If the community became large I'd agree it would want to go to pull requests to avoid a c2 style heat-death.

hmmmm. IMO it's much easier to introduce "new tricks" to a small community, than to a big one with existing habits ;)


-------------

One question. Does your community use linux based or windows based OS?

have fun!
mario

[Jeremy Ruston]

Hi Peter

Great project. The GitHub pages workflow I use for tiddlywiki.com works very well for me, and I really like the idea of some automation magic to automatically update the static rendering.

One detail that might be worth noting if you haven't come across it already is that when using the --server command you need two extra plugins $:/plugins/tiddlywiki/tiddlyweb and $:/plugins/tiddlywiki/filesystem. The problem is that these plugins mustn't be included in the single file version. The workaround is to have a separate wiki for use with the --server command that includes the main wiki. You can see this arrangement with the tw5.com and tw.5com-server editions in the TW5 repo.

As an aside, I've been looking at Amazon's Lambda for tasks like this. It's a rather neat way of running a "single shot" JavaScript task in response to external triggers. There's a limit on execution time which might be a problem for very large wikis, but otherwise it's quite neat and avoids having to spin up a server or container.

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWeb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlyweb+...@googlegroups.com.
To post to this group, send email to tidd...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlyweb.
For more options, visit https://groups.google.com/d/optout.

--

Jeremy Ruston
mailto:jeremy...@gmail.com

[Peter Merel]

Hi Jeremy and Mario,

I love the idea of github as server-side because it focuses TW on what it's the best at - rendering, filtering and UX for tiddlers - while using github for what it's fantastic at - workflow decentralization. I take Mario's point that there could be a nasty merge conflicts if people were editing the same tiddlers at the same time per my naive paradigm. But then that's where I think human communication can come in - the conflicted folk would talk and decide to go make topic branches and do the usual git strategies to sort things out. Maybe I'm underestimating how often this would become a problem ... maybe best to just give it a go and get an experience while I only have a handful of people to be troubled by it.

But I am completely unworthy to suggest the right way to automate github-as-server-side to such TiddlyLuminaries as yourselves. It's not for want of will to take time to learn JS and TW properly but because right now my focus is the simplest thing that can get the xscale.wiki collateral mature and its community aligned. Having a bodgy but not very hard manual workflow in place for now I suspect you two would get around to doing github automation properly before I would see time to hack something up. If you don't I guess I will eventually but then I have occasionally been accused of taking on too many projects at once ...

On the thought about lazyloading tiddlers, I was imagining something ajaxy between the single-file .html and the tiddler store that would be accessible to it from github. Again, however, since tiddlers are so small this is just silly. I should expect images and blobs are all that are really worth lazyloading.

Cheers,

Pete.

[Jeremy Ruston]

Hi Peter

I love the idea of github as server-side because it focuses TW on what it's the best at - rendering, filtering and UX for tiddlers - while using github for what it's fantastic at - workflow decentralization.

Yes, absolutely. That's why the serverside for TW5 is so minimal in terms of authentication and static file serving; it seemed more sensible to offload "commodity" operations to components like nginx or apache that people already understand and trust. 

I take Mario's point that there could be a nasty merge conflicts if people were editing the same tiddlers at the same time per my naive paradigm. But then that's where I think human communication can come in - the conflicted folk would talk and decide to go make topic branches and do the usual git strategies to sort things out. Maybe I'm underestimating how often this would become a problem ... maybe best to just give it a go and get an experience while I only have a handful of people to be troubled by it.

Yes indeed. One of the benefits of cutting content up into small chunks is that it reduces the chances of collision when an edit is made. More subtly, it also increases the chance that a collision is actually semantic and deserves escalation for human intervention.

 

But I am completely unworthy to suggest the right way to automate github-as-server-side to such TiddlyLuminaries as yourselves.

Not at all! Speaking for myself, my expertise is tightly focused on TiddlyWiki and the technologies that it uses; I'm very much the noob as far as git and GitHub is concerned. 

It's not for want of will to take time to learn JS and TW properly but because right now my focus is the simplest thing that can get the xscale.wiki collateral mature and its community aligned. Having a bodgy but not very hard manual workflow in place for now I suspect you two would get around to doing github automation properly before I would see time to hack something up. If you don't I guess I will eventually but then I have occasionally been accused of taking on too many projects at once ...

To be honest, I'm unlikely to focus too much on GitHub integration, because I'm generally trying to focus more on the experience for casual users. So I'd suggest bodging away; I'd be very interested to follow the work.

On the thought about lazyloading tiddlers, I was imagining something ajaxy between the single-file .html and the tiddler store that would be accessible to it from github. Again, however, since tiddlers are so small this is just silly. I should expect images and blobs are all that are really worth lazyloading.

I agree, but I think there's plenty of good approaches for lazily loaded ordinary tiddlers, too.

Best wishes

Jeremy

 

Cheers,

Pete.