[TW5] A working prototype multi-user wiki

[Jed Carty]

BIG SCARY DISCLAIMER: I am the only one who has tested this so far, and only on linux mint. This may be full of bugs and could possibly make your computer gain self awareness and take over the world. Or maybe just not work very well.

I am close to having a working multi-user wiki. I currently have it so that the wiki in the browser (almost) immediately reflects any changes to the .tid files on the file system and you can set it to ignore a list of tiddlers. If you have multiple browsers connected to the same server than changes made to the wiki in one browser are pushed to the other open instances.

To prevent two people from simultaneously editing the same tiddler and overwriting each others edits I have it set up so that it sees when a draft tiddler is created and disables the edit button for that tiddler.

The current problem is that while the browser reflects changes on the file system the syncers don't act quickly enough so when you edit a tiddler in the browser it takes a few seconds to reflect that on the file system and pass the changes to the other open wikis. So I am going to write a new syncadapter that uses the websockets to get a faster response for preventing multiple people from editing the same tiddler at the same time.

The repo is here with a readme that explains how to set it up. Hopefully it works, I have only tested it on linux.

https://github.com/OokTech/TW5-Gatekeeper

[Riz]

Hi Jed,

You are my new hero. Been wanting this ever since "It's all text" add-on turned over and died in the firefox add-on-calypse.

Thank you. I hope this finds its way into core.

[Diego Mesa]

Wow!! This is great! Thanks for setting this up and I look forward to seeing it develop! 

[Lost Admin]

This is interesting. I don't run node. Any chance of this making it into the Webdav saver? I doubt it since TW uses a single file with DAV.

[Jed Carty]

This is only for node. the closest equivalent for a single file wiki is twederation which can be used to make multi-user wikis but doesn't have the live updates that this does. 

You can have the websockets plugin in a single file wiki but you will still need a server to connect it to so it wouldn't do anything for you.

[Sylvain Naudin]

Bonjour Jed,

That's sound promising !
I've try it today but not succeed at the moment.
I'll try later and have a closer look at your howto.

(in fact I'm not a node.js version export, just beginner).

Sylvain

[Jed Carty]

I have been having trouble making the new syncadapter using websockets, so if there is anyone who can help with that I would appreciate it.

Sylvain,

Most of the problems I have run into setting it up have been caused by not installing the npm modules correctly. If you have any specific questions let me know.

[Sylvain Naudin]

Well, I was trying globally (with -g install npm option) without success.

Just retry locally on a home folder on my Fedora, and it's seems to work well. It take about 20 seconds to see push change if I edit remotely on my phone a tiddler.

Hope you will be able to work on it to reduce this time as you said.

Cheers,
Sylvain

[Arlen Beiler]

I do, but I don't feel like replying. :) Because I can see how big of a project it will be. The server command is given a wiki object as an option. That wiki object should have somewhere you can listen for changes. I forget where but I know its there. From there it should be quite simple. Set up the connection, and set up the message handlers. 

Let me know what you come up with, and I will try to help where I can. I can also help answer questions. This is something that I see useful for TiddlyServer. 

Arlen

That being said, we really do need a WebSocket based sync adapter solution. 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/0046d266-31d1-43d9-9095-dccef5dafc7d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jed Carty]

I have the message handlers and the connection, the part that I am lacking right now is whatever gets the save hook from the wiki so that I can make it send the message to save the files on the file system. I can manually send things to be saved on the file system and when I do that it changes propagate quickly enough to be usable, but I don't want to reinvent the wheel because there is already a framework in place for it.

[Arlen Beiler]

If you are communicating between the server command and its client instance, then ignore the file system sync adapter and watch for changes directly on the wiki. 

Aren't you using the file system adapter to sync changes from the NodeJS server to the file system?

I am imagining that you want to watch the wiki on the server for changes and immediately send a message to the client as soon as there is a change. In that case, I would add a startup module that calls wiki.addEventListener("change", function(changes) {}). Where you get your wiki variable, I'm not sure, but look at the other startup modules to see how it works. You might even be able to push a list of the changed tiddlers to the client, where you can trigger a refresh of those tiddlers directly I believe.

Hope that helps.

Arlen

On Sat, Nov 4, 2017 at 5:49 PM, Jed Carty <inmy...@gmail.com> wrote:

I have the message handlers and the connection, the part that I am lacking right now is whatever gets the save hook from the wiki so that I can make it send the message to save the files on the file system. I can manually send things to be saved on the file system and when I do that it changes propagate quickly enough to be usable, but I don't want to reinvent the wheel because there is already a framework in place for it.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ad9b39f2-6023-41bb-b460-48b53d74a35b%40googlegroups.com.

[Arlen Beiler]

For watching other wikis for changes, you would probably need to manually load those changes into the server wiki using a separate script from the one above, which would then trigger that script to send the changes to the client. 

If you are watching your own folder, I think it would work basically the same way. Don't forget to take folder prefixes into account. Check the file system adapter to see where and how it saves files and load the files using the same settings. It might not matter though since you want to load all changed files anyway. How do you have it setup if the metadata and actual data files are two separate files?

Hope that helps. 

Arlen

[Jed Carty]

Watching for changes is relatively simple and I have that working, I have just been watching the file system for changes. I need to set everything up to have it watch all the appropriate folders and let you modify that while the wiki is running, but that shouldn't be difficult.

The only part that I don't have right now is hooking into what the wiki does when you click on the 'confirm changes to this tiddler' button. The sync adapters are set up to do this and can be configured in the ways that I need to configure them, but I haven't been successful writing my own yet. Pushing to or monitoring multiple wikis isn't the problem.

[Arlen Beiler]

I guess you understand that the client and server are two separate instances of TiddlyWiki?

When the client clicks save changes, the tiddler is saved to the wiki store on the client and a change event is queued. The sync adapter gets called with the changes allowing it to save them to where it needs to. 

The sync adapter calls the server URL with the updated Tiddler data. The server command is behind this and handles the request. The server command updates the wiki store on the server, which "dirties" the server wiki store and queues a change event. Once everything is saved to the server wiki store, the wiki is marked clean and the checkbox turns grey. 

On the server side, the file system sync adapter is the one that gets called with the changes, which it saves to the file system. 

The wiki object has an addEventListener handler that you can listen for that change event on. You would want to run the startup module on the node platform only if you are pushing change notifications to the client as I described earlier. 

Am I understanding you correctly?

Hope that helps, 

Arlen

On Sat, Nov 4, 2017 at 6:24 PM, Jed Carty <inmy...@gmail.com> wrote:

Watching for changes is relatively simple and I have that working, I have just been watching the file system for changes. I need to set everything up to have it watch all the appropriate folders and let you modify that while the wiki is running, but that shouldn't be difficult.

The only part that I don't have right now is hooking into what the wiki does when you click on the 'confirm changes to this tiddler' button. The sync adapters are set up to do this and can be configured in the ways that I need to configure them, but I haven't been successful writing my own yet. Pushing to or monitoring multiple wikis isn't the problem.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1fc0a38d-c415-44c4-b716-1346b560f173%40googlegroups.com.

[Jed Carty]

Yes, I understand the structure of the program. I have it working and I can do what you are describing without trouble. That isn't the problem. I will try and figure this out since I am apparently not describing it well.

[Arlen Beiler]

the part that I am lacking right now is whatever gets the save hook from the wiki so that I can make it send the message to save the files on the file system.

Listen for the change event on the client is what I think you want. 

In the gatekeeper, why don't you just save the updated tiddler to the server tiddler store instead of pushing it directly to the client?

If I am understanding you correctly, you want to send a message to the server with the updated tiddler which will then get pushed to the other browser instances and also gets pushed to the file system directly. Isn't that a sync adapter? :D  

If all I wanted to do was load changes into the browser, I would hook onto the server change event and send a message to all browsers to fire the "tm-server-refresh" message which would cause the sync adapter on the client to pull the skinny tiddler list from the server immediately. That is a feature we also need, but apparently you are trying to accomplish something more complex. 

I hope you get it figured out :)

Arlen

On Sat, Nov 4, 2017 at 6:50 PM, Jed Carty <inmy...@gmail.com> wrote:

Yes, I understand the structure of the program. I have it working and I can do what you are describing without trouble. That isn't the problem. I will try and figure this out since I am apparently not describing it well.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/07ac45c9-dd52-485a-8e17-708297a8d074%40googlegroups.com.

[Jed Carty]

Yes, a new sync adapter is exactly what I am trying to make. That is why when I said I was having trouble I said I was having trouble getting a new sync adapter to work. Because everything else works.

[Jed Carty]

Ok, I am going to try this again. As I said before, I have this working, the only component that I am having trouble with is the sync adapter.

Is there someone who has time to help and isn't just going to describe the structure of the tiddlywiki for 5 posts before concluding that I may need to make a sync adapter?

[Jed Carty]

Well, I never figured out how to make a proper sync adapter, but there are almost enough hooks to just use those to trigger sending the web socket messages without using a normal sync adapter. It works but I had to make some pretty hacky things to make everything fit the way it should and I had to add some extra hooks to the navigator widget.

Right now updates propagate almost immediately but I haven't been able to find where to add a hook to trigger actions when a tiddler is created by something other than the 'create new tiddler' button, so if you create a tiddler using something like the action-setfield widget than they aren't detected and propagated to the other wikis.

I need to clean up the code a bit before I push the new changes and it is bed time now, so expect the update tomorrow sometime.

[Jed Carty]

WOO!!

It works. So try it out if you know how to install npm modules.

Follow the instructions in the readme in the repo here https://github.com/OokTech/TW5-Gatekeeper and you should get a working multi-user wiki. It pushed changes almost immediately and when you are editing a tiddler on one computer it disables the edit button in all the other tiddlers. It also immediately pushes any changes on the file system to all the wikis but it only sees changes in the tiddlers folder and not in any sub-folders.

The code is kind of a mess right now, I will hopefully clean it up and write a better description tomorrow.

[TonyM]

.Jed,

I have not yet experimented with this, even advanced applications of node JS wikis, there is not enough in a lifetime to do everything that's possible with TiddlyWiki. This solution however is very promising. I will get into it further.

I think all the activity around saving has dominated a little recently.

I was thinking about a name, as I always love a naming competition and for some reason Transcendental tiddlers came to mind, but its too long for a name.

A Couple of other thoughts

teletiddlers - like telegraphe, but reminiscent of teletubbies
multi-wiki tiddlers
tiddlermessages
transwiki (tiddlers)

On the uses of such a solution there is I believe a long way to take this. I have an observation though that may help.

In TiddlyWiki and other environments that are feature rich, there seems to be a bit of polarisation, between how far you can go with tinkering and what is core or "standard uses".
I think the real advances come when features are made more accessible, and this starts with elegant solutions to somewhat simple problems on the surface, but which offer a whole world of opportunity.
You solution could be one of these, but I think it needs a powerful yet simple use case, to encourage adoption and then it will fly.

An example I am working on is to make fields more accessible, and usable. It should not be complicated to edit fields from the view template, when developing tiddler content. The idea of fields as tiddlers (like tags as tiddlers), an easy edit solution and the ability to set possible values is how I am hopping to develop a  powerful yet simple use case
.
 Have you any ideas about this for TW-Gatekeeper you can share? I would love to hear them!

I think wrapping some good practices around transferring tiddlers via TW-Gatekeeper, would help adoption.

Regards
Tony

[Jed Carty]

Tony,

That wouldn't really be part of the gatekeeper part. All the gatekeeper part does is act like the file system adaptors that tiddlywiki normally uses, it just works faster and lets you have multiple connected instances of the same wiki. You can define a set of tiddlers that are not synchronised but unless we redefine how the node version of tiddlywiki defines wikis this isn't really for inter-wiki communication.

We can use the websockets part to create an inter-wiki communication plugin though. That could be used to send tiddlers between wikis. I am hoping to use this to extend twederation but I haven't made any progress on that.