Keep Wikis settings and plugins in sync

[Tony Woode]

Hi all, much respect and adoration etc

My use of TiddlyWiki is via TiddlyDesktop for personal (non-shared) information only. I used to have only one wiki which I sectioned off into different topics, but I have found reasons over time to split these wikis up, so I now have 4 separate wikis (They aren't all located in a common folder, but all 4 are accessible at the same time from a single machine)

Every so often, I take a look at new plugins or tweak my settings in one of these wikis, and I find amazing and helpful things that become part of the wiki

You can see where this is going...right: the new plugins and features I use are meant to be globally-evolutionary across all of my 4 wikis, two examples are: that I started using the codemirror Vim keymap, or that I tweaked my theme sizes to be all % based rather than absolute.

My 4 wikis should all be essentially the 'same' in terms of settings, the only things that aren't 'data' ie: content tiddlers, that should be different between them is things like the wiki name, or the  favicon logo i've set for the wiki that I can see in tiddly desktop

So....can anyone see a process I could use to keep my settings and plugins 'synced' between these 4 near-identically-configured wikis? I guess, as with any sync operation, there may be instances where both sides have changed (or at least it isn't clear whether a change on one side is evolutionary), so some crude visual 'diff' to see what i'm going to update would be beneficial (although that isn't crucial I guess, the process could just overwrite - its usually just one of these 4 that get's upgraded and I want to make the other 3 mirror its settings, but because I haven't had a good way to 'sync' settings i'm sure i've ended up in a mess where some newer settings/plugins now exist across all 4 wikis)

thanks enourmously for everything in advance

[TW Tones]

Tony,

With only 4 wikis (I have over 100), I would just do any change you wish to make to one, and manually apply to others. There are neat and easy ways to do this.

• If in your master wiki you create a tiddler that lists (with links of all the things that changed you flag for the other wikis) you can just drag and drop them on your other wikis.
• You could export all the changes as a json file and drop that file (or import it) to your other three wikis. You can then collect a set of changes to drop on a new wiki if you get a 5th.
• One trick I like is to use an iframe in the child wikis, that opens the master wikis updates tiddler in an iframe,  you can actually drag items from in the iframe window and drop them on the wiki that the iframe is in.
• Mario has a bundler plugin you may like, one option allows you to select if you which to overwrite on import or not. This helps avoiding overwitting config tiddlers.
• I have created dragable packages installed in a master wiki that I can drag an drop as needed, and another collects all the plugins I come across, and I make sure I record the source. I often "disable" the plugin in this plugin repository to keep it clean. After dragging a disable plugin to a new wiki it becomes enabled in the new wiki.

With a more complex situation like mine I have established workflow practices. I try and make sure my changes are comparable where ever I install them and have a quick and easy way to install as I need them. If I am in wiki 2 and I want to use my smart-code view I install it. I have some nice easy ways to find such tiddler packages on windows I can share. 

The most sophisticated processes would be to bundle changes into a plugin and add them to a library not unlike the plugins add new plugins. The library facility allows you to trigger reinstalls. But this is still a little painful to achieve, and you need to serve the library via a server.

I do not know who skins cats but there are many ways to do anything in tiddlywiki, and a vast majority need only wiki text, widget and macros. I suggest developing your own solution that you can evolve over time.

Tones

[Lin Onetwo]

Hi, Tony and Tones,

Why multiple wikis? I haven't reached that point, so I wonder the reason.

Wouldn't that make transclusion and linking harder?

I'm now putting different content in different Github repo, and clone then, symlink them into a "main" wiki, and only main wiki have plugins, other "sub-wiki" just have tiddlers.

Sincerely

LinOnetwo

[TW Tones]

Lin,

I build for purpose wikis and I know which to use for which purpose. I have done a lot of design effort, few have yet seen, because I am building platform tools rather than specific functional wikis, however I am soon to get the economies of scale and you will start seeing solutions/editions from me as well as developer and UI tools.

I have a massive resource library. So I have already learned dozens of inter wiki solutions, that someone else may be forced to address if they push too much into one tiddlywiki.

However like mobile apps, it is easy proliferate for purpose wikis and build a logical layer on top to manage anything.

Limited for purpose wikis, or larger consolidated wikis each have their place.

I basically package components for distribution as needed, use custom editions if needed and I have started data manipulation which I am building another set of generic tools for.

Mario's bundler plugin, Mohamad's utility Wikis and plugin libraries.

Mohammads Searchwikis plugin uses index (dataTiddler) to search multiple external Tiddlywikis. It has two parts

1. An indexer, to build an index of all tiddlers in an external wiki
2. A search tool to search indexes and display a link to a tiddler found in an external wiki

I build ways to register and transfer project made up of multiple tiddler and indexes of wikis and their functions.

With TiddlyWiki there are few if any limits.

Regards

Tones

[Charlie Veniot]

That's an excellent question which, unfortunately, likely has answers that start with "It depends ..."

Sometimes, it makes sense for a wiki to handle a bunch of things that have a lot of commonality/reuse-opportunity.  For example, I have one TiddlyWiki that has content that is shared in various contexts/purposes/personalities:

• Product Reviews
  
• Urban Off Gridding for Laypersons
  
• Hydro Bill Cutting for Laypersons
  
• Chromebook: Beyond Web Browsing
  

What originally started with a "how would I create a Product Reviews TiddlyWiki" opened up reuse of info about those products into info about using those products for various hobby projects.  All kinds of transclusion delight going on there.

Other times, the subject is so focused, so unrelated to other writings/content with no commonality/reuse-opportunity, that it just seems to make more sense to have a narrow-focused/independent TiddlyWiki.  For example:

• ADHD Slice'n Dice
  

To me, wedging ADHD-related content/writing doesn't make any sense thrown into my "Product Reviews (etc.)" TiddlyWiki.  In a somewhat "distraction-free" writing spirit, I wanted that ADHD stuff neatly isolated.

Cheers !

[TW Tones]

Charlie/Tone and Lin

Carlie, I think you have touched on a key differentiator between one wiki and the other, when you do a search do you want the other content to come up?

• In other words you can store in one wiki all that you would like to find under than wiki name. 

As I said early you can search multiple wikis through an index system of Mohammads, You could even have an iframe open to the other wiki where you can refresh the index and drop it on the index wiki without leaving the tab. Or a tiddler open in the iframe with all the dragable tools listed.

My practice

I say I make a custom tag tool, after I make it work, I always package it in a JSON so it can be installed in other wikis, this includes a reference to the wiki in which it originates in, if I decide I want to change it. Every time I change it I save it under a folder on my LAN (with a set filename), every time I want to install something I open the LAN folder and type in a keyword and a list of JSON files appear, click and import.

I used to have a TW5 Reference, it included references and resources, this year I duplicated it and deleted the resources in one and the references in the other, Now I have two wikis. I index both an places the index in another index wiki where I can search both at once if I loose something, however I do not usually because I know what I consider a TW5 reference, and what I consider a TW5 resource, I even have another with all collected TW5 plugins.

Now I have so many wikis and Projects I have built a method of collecting finished projects in another projects wiki, and I can drag a special project tiddler to my Index Wiki, and a special wiki tiddler to my index wiki. So it I loose something I can find it but I rarely loose anything.

Say I have a set of data, I try and make sure I can select and save the data independently from the wiki (and my tools). So it can be exported and imported elsewhere if desired. This can be as simple of all the tiddler having a particular unique tag.

I love TiddlyWiki

Tones

[Tony Woode]

Hi all

Thanks everso for your replies, particularly thanks TW Tones for answering my question with all those options, that's very good of you

I was very much hoping we could not dicuss here WHY some of us need to split our wikis up and not just have one master wiki, I'm sure even if you can't see a need to do this, you can concieve of there being needs to do this, just think of factors around security or privacy or sharing and so on. I really wanted to discuss the HOW not the WHY

TW Tones, I think you're overestimating my skills with TiddlyWiki (and probably the skills of anyone finding this thread in future seeking to do similar?).  I can see how there are many options to achieve what I want to achive, but just as one instance: I tried to look at Mario's bundler plugin (perhaps it is this? https://groups.google.com/g/tiddlywiki/c/_Uqbg08Pjow/m/lmYj2ETxCAAJ) but my lack of domain knolwedge means I simply don't really understand what it fundamentally does, despite trying to read what is available, I don't have a clear view of what itch it even scratches? I have to say I'm not sure I could do any of the options you suggest without at least a list of steps to perform

So perhaps my lack of abilities can actually help here to narrow down on an options which might be relevant to me, I'm certainly not going to be creating my own TW plugins for instance. It would be good if someone would care at all to write me down a simple process I can follow, in as much detail as you can bear, to do any or all of the following:

1) determine WHAT the changes are between two wikis, to allow me to work out which setting might be 'newer'

2) apply settings changes from one wiki to another, just as TW Tones you suggest: in a manner where you're asked for confirmation or some other way you don't write over the most basic settings that shouldn't be overwritten eg: wiki name

3) apply plugins from one wiki to another, so you have certainty that if you compare two wikis, you know when both have the same plugin set

I think another key difference you guys might be describing is where you have a 'master' wiki and a large number of sub-wikis. This is not the case for me, I have a small number of peer wikis and I never see it growing into a large number. But if your use of a 'master' wiki is ONLY for a master set of settings/plugins then perhaps I should do that? But I guess the same applies: i'd need to know how to achieve the same 3 things

Thanks again for taking the time....

[Tony Woode]

So i've achieved a base-level method to sync >1 wiki's settings, thought i'd reply to my own post in case this helps someone:

The fundamental thing about TiddlyWiki Tones assumed I completely understood above was that dragging a link to a tiddler from one TiddlyWiki into another (such that you get the import bar popup at the top of the screen), causes the contents of the linked tiddler to be imported (after a confirmation dialog). It doesn't matter whether the linked tiddler is text/images/settings/code/plugins. This is a fundamental and astonishing magic of TiddlyWiki, and though an inexperienced user might use and see this in action from instructions etc, it can be difficult to appreciate just how all-ecompassing this ability is.

So what's a 'link' to a tiddler? Well it can just be as simple as the name of the tiddler from your 'open' or 'recent' sidebar tabs, just drag any link from there from one TiddlyWiki into another (this works regardless of whether one of these TiddlyWikis is on the internet and the other is on your filesystem, or if both TiddlyWikis are using TiddlyDesktop, and so on...). But Tones was specifically talking about, for ease of collecting settings together, making a link to a tiddler IN ANOTHER TIDDLER: by either just stating the name of a system tiddler like $:/themes/TiddlyWiki/vanilla/metrics/storyright or linking to another tiddler name using the [[double square brackets]] notation.

Another thing I guess Tones assumed I knew there was that every setting as you see them in a TiddlyWiki's control panel is stored each in an individual tiddler whose name is availble by just clicking on the setting name - this is an axiom of TiddlyWiki: we atomise individual things to individual tiddlers; so where I was worrying that i'd have to mass-copy all the setting in a TiddlyWiki's control panel (and that would include things like the wiki title which I don't of course want to copy), instead we come from the opposite trajectory: we look at the individual things that have changed from the standard settings and just drag the link to those (for instance the example I just gave: $:/themes/TiddlyWiki/vanilla/metrics/storyright - I changed the setting for 'how far the left margin of the sidebar is from the left of the page' from an absolute value to a percentage). So its easy, once you know what you changed, to make a list of links.

Once I grokked those things, the first option Tones gave me above makes sense: 'make a tiddler with just links to other tiddlers and drag the links' - well really the level of automation i've got so far is pretty much that, though its a bit flowery as I've made a kind of diary of what settings I found I needed for what I consider my 'standard' style (and where I got the ideas from etc), and in each section, when I worked out for each setting what tiddler it relates to, I've put a link to the setting tiddler(s).

So i've ended up with something a bit like the 'master-slave' relationship Tones was envisaging there, since I have a tiddler in a wiki that lists my 'master' settings, and i'm using that tiddler for each new wiki I make. Since i'm not intending to create the industrial number of wikis he has, I'm happy with just that really: although I'd point out my 'diary' tiddler of settings is also a symptom of me not really finding a good answer to my other original question of how I can diff changes already made to >1 wiki: I couldn't do this programatically and just had to start again with a standard TiddlyWiki and customise it whilst documenting settings I changed - time consuming, but worth doing if only to realise that I hadn't actually changed that much

In terms of plugins, I didn't really list standard TiddlyWiki plugins in my 'diary' tiddler, because they're just listed so very nicely in TiddlyWiki's plugins section already - so I found as long as I considered these things, I can just look at any of my existing TiddlyWiki's plugins pages and just copy it:
  * I had overwritten some 'shadow' tiddlers for a plugin or two, but I decided to just try to live without those customisations for ease of replicating plugins (its easier to know i'm using the stock plugin than potentially using something that includes prior customisations i've long-since forgotten I made)
  * there were some plugins I just decided I didn't need, just keep the list of installed plugins small
  * with codemirror, I just installed everything since I use a lot of things and installing everything is easier to remember than excluding the few things I don't need

As you can see this is all a very basic solution that reflects where i'm at with TiddlyWiki, in particular if my plugin needs were more complex it would be time to continue exploring Tones' options for automating synchonisation using bundlers/packages and so on, but I thought it worth documenting the basics. Tones' other options are all further enhancements of the 'list of links' functionality, but that functionality in itself is very powerful to begin with, and may be enough for you if you don't have 100 wikis

[PMario]

Hi Tony,

I did _only_ read this post. 

I think there are some elements in TW that can help you to be more effective.

In the ControlPanel : Info : Basics tab you can find this:

The "Number of overridden shadow tiddlers" can give you information about some "shadows" you have changed. eg: $:/SiteTitle, $:/SiteSubtitle, $:/DefaultTiddlers ... and so on. There are some $:/temp tiddlers, which you can ignore ;)

"Number of System tiddlers" will show them again and a lot more. Especially important are $:/config/... and may be $:/themes/tiddlywiki/..../options/...

If you click the "search icon" the tiddlers are shown in AdvancedSearch : Filter tab. There you can export those elements. .. Since it's tedious to redefine those lists all the time by hand, I did create a plugin named: bundler. Which can help you to manager your settings.

It will allow you to store your configurations as so called bundels. There is a link to the discussion about the new functions, which are also shown in the video.

The $:/Import tiddler allows you to see the differences between existing tiddlers and imported tiddlers. .. This is already part of the core TW functions.

1: Select the "Diff" option

2: Show the "diff - text" ... green is new text, red is removed text

have fun!

mario

[PMario]

Hi,

This may be interesting too! The "haschanged" operator, will show you the tiddlers that you have changed in "this" session.

-mario

[Tony Woode]

Thanks for the response!

Yes that really helps - I had indeed missed the overridden and system tiddlers advanced search, and the haschanged filter will prove very helpful, so thanks for taking the time

> Since it's tedious to redefine those lists all the time by hand

I'd already looked at and watched your video for your very impressive bundler plugin, but I think I'm stuck at first principles again: I've proved to myself that if I have a (standard) tiddler that lists other (standard) tiddlers, and I drag that between TiddlyWikis, its a shallow copy so doesn't copy the referenced tiddlers (though this must be different for plugin tiddlers since dragging a single tiddler does precicely this, that's precicely a factor that defines these as 'plugins'). So that's what I understand the principle of your bundler is: it essentially creates a deep clone of the children when supplied with a tiddler-consisting-of-links, am I close?

So, I would understand the process of using your bundler to capture settings like so: i'd use the output of the overridden/system advanced-search-filter tiddlers that you mentioned to manually create a tiddler-of-links based on them, and then use that tiddler as the input to the bundler, to deep-clone them to a bundle for import. But from watching your video I don't seem to have that process quite right: your plugin seems more than that because, as you say, it seems to facilitate making a custom filter to automatically remove the temp and other tiddlers you mention above from the search results each time its run - to whit: the input that drives your bundler is not a tiddler-of-link, but is instead a filter expression. Am I right? But since some of these tiddlers-to-filter-out seem to just need fully-specifying, I imagine the filter expression gets quite long, Maybe I've misunderstood, I will investigate further.

I had actually seen the diff functionality on import, what I wanted at the time though was a retrospective way to find the difference between 2 wikis that had been seperately created over a long period of time. I haven't investigated the structure of a tiddlywiki file but i'm sure it would be possible to write an external tool to inspect each tiddler object for comparison given an html, or I guess I could have exported an html-style wiki into the node-style .tid format and just used a file diffing tool...too late now ;-)

thanks again

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/zl0AoweAVYs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/af02f258-9768-4c84-a926-7f7c0a6e785dn%40googlegroups.com.

[PMario]

On Friday, March 26, 2021 at 4:47:41 PM UTC+1 tony...@gmail.com wrote:

...

I'd already looked at and watched your video for your very impressive bundler plugin, but I think I'm stuck at first principles again: I've proved to myself that if I have a (standard) tiddler that lists other (standard) tiddlers, and I drag that between TiddlyWikis, its a shallow copy so doesn't copy the referenced tiddlers

That's the same for a test.bundle tiddler. It only contains links. _But_ it also contains an interface in the tiddler options menu, that provides a slightly different view. There is the possibility to delete single tiddlers and there is an "export button", that will save a test.bundle.json file that contains all the tiddlers listed.

So if you drag and drop the .bundle tiddler to a different wiki, it does the exact same thing as TW does. ... Only if you export to JSON and then import that json file, you'll import all the exported tiddlers.

There is also a new config "bundles" overview in the right sidebar.

(though this must be different for plugin tiddlers since dragging a single tiddler does precicely this, that's precicely a factor that defines these as 'plugins'). So that's what I understand the principle of your bundler is: it essentially creates a deep clone of the children when supplied with a tiddler-consisting-of-links, am I close?

Only if you export the configuration. So .bundle is only a config tiddler.

 

So, I would understand the process of using your bundler to capture settings like so: i'd use the output of the overridden/system advanced-search-filter tiddlers that you mentioned to manually create a tiddler-of-links based on them, and then use that tiddler as the input to the bundler, to deep-clone them to a bundle for import.

Yes. As written above. The .json file contains the bundle. ... The advantage is, that this bundle can also contain plugins. ... So it's an easy way to "Kickstart" an empty wiki, with all the settings and plugins you normally use. ...

From time to time, you only need to take care, that your plugins are up to date. ... After you made backups and tested the new plugins ;)

 

But from watching your video I don't seem to have that process quite right: your plugin seems more than that because, as you say, it seems to facilitate making a custom filter to automatically remove the temp and other tiddlers you mention above from the search results each time its run

That's not right. If you use the AdvancedSearch : Filter tab and you include eg: "$:/StoryList" "$:/state/tab--1498284803" "$:/temp/search" You can export them. .. BUT if you drag & drop this tiddlers.json into tiddlywiki.com you'll see, that you can't import those tiddlers. None of them.

The bundler contains a new import interface, that will allow you to import those tiddlers. ... That's the new functionality. Blocking is default for standard TW.

 

- to whit: the input that drives your bundler is not a tiddler-of-link, but is instead a filter expression.

The video only talks about the "tiddler-of-link" configuration. ... There is a second possibility, but that will be a new video, which doesn't exist atm.

 

Am I right? But since some of these tiddlers-to-filter-out seem to just need fully-specifying, I imagine the filter expression gets quite long, Maybe I've misunderstood, I will investigate further.

Filtered bundles are for advanced users and for myself. There are usecases, where they are extremely useful and short. If they start to be complicated, the links-bundle is probably the better choice. ... But the important thing here is "choice" ;)

hope that helps

-mario

[PMario]

On Saturday, March 27, 2021 at 12:30:11 PM UTC+1 PMario wrote:

On Friday, March 26, 2021 at 4:47:41 PM UTC+1 tony...@gmail.com wrote:

...

I'd already looked at and watched your video for your very impressive bundler plugin, but I think I'm stuck at first principles again: I've proved to myself that if I have a (standard) tiddler that lists other (standard) tiddlers, and I drag that between TiddlyWikis, its a shallow copy so doesn't copy the referenced tiddlers

That's the same for a test.bundle tiddler. It only contains links. _But_ it also contains an interface in the tiddler info menu, that provides a slightly different view.

It should be the "tiddler info  menu" and not options.

-m

[Tony Woode]

great thanks for clearing all that up!  I now have a much better idea of the ideas behind bundling. TBH I generally always need to go back to first principles to pick something up, so in order to understand the above I found it very helpful to go back to here:

https://groups.google.com/g/tiddlywiki/c/WzwNpnxycDQ

(which I found via a link in the original discussion I first found about your bundler here: https://groups.google.com/g/tiddlywiki/c/_Uqbg08Pjow/m/lmYj2ETxCAAJ)

In that conversation we get introduced to some primordial topics like what is common between a plugin and a bundle, the importance of filtering to a bundle and so on -  so up-front discussion of some of what you've been trying to explain to me above. As and when I get to the stage of using your bundler plugin to sync my settings (a fantastic idea!), I'd be happy to do some work to collect/collate various salient parts of these discussions somewhere by way of documenation, as it seems to me there's quite some background required here

thanks for taking the time

[PMario]

On Tuesday, March 30, 2021 at 11:32:00 PM UTC+2 tony...@gmail.com wrote:

great thanks for clearing all that up!  I now have a much better idea of the ideas behind bundling. TBH I generally always need to go back to first principles to pick something up, so in order to understand the above I found it very helpful to go back to here:

https://groups.google.com/g/tiddlywiki/c/WzwNpnxycDQ

Yea, This was the initial thread, where we developed the idea and started to develop definitions. Bundels are similar to plugins, but bundled tiddlers are able to be deleted....

Plugins create "shadow" tiddlers, that can be overwritten, to become system tiddlers. If those system tiddlers are deleted, the "shadows" take over again. ... This is one of the powerful basic TW principles.

 

(which I found via a link in the original discussion I first found about your bundler here: https://groups.google.com/g/tiddlywiki/c/_Uqbg08Pjow/m/lmYj2ETxCAAJ)

In that conversation we get introduced to some primordial topics like what is common between a plugin and a bundle, the importance of filtering to a bundle and so on -  so up-front discussion of some of what you've been trying to explain to me above. As and when I get to the stage of using your bundler plugin to sync my settings (a fantastic idea!), I'd be happy to do some work to collect/collate various salient parts of these discussions somewhere by way of documenation, as it seems to me there's quite some background required here

I did create an issue at github about ... more docs ;)

have fun!

mario

[Tony Woode]

Great thanks, when I'm on my next iteration i'll return to that issue, speak soon