A brewing idea: TiddlyWiki on node.js: check for changes

[Charlie Veniot]

Say I have a TiddlyWiki running on node.js, and two people currently viewing the same TiddlyWiki in their browsers.

Say User 1 is editing a tiddler.  I'm thinking it would be cool if  User 2 could know that User 1 is editing the tiddler (or that anybody is editing the tiddler.)

And/or, it would be cool if User 2 can get a notification that a particular tiddler has changed (because of User 1's editing), and that User 2 should refresh the browser to get that change if he/she wants it.

Something like that.

I'm not convinced it is something I can do, but I'm kind of interested in trying IF it isn't something that's already been done.

[Finn Lancaster]

Charlie, 

One way to do this (since you are using node), is to implement Socket.io into the Wiki. Each time the edit button is pressed, tell Socket.io via 

socket.emit to in turn socket.emit to each client (the first socket.emit would contain JSON containing the name of the editor and the tiddler). With socket.io, too, on each client instance, just add a listener for a specific event using socket.on... , and have it display a simple message such as "name is editing tiddler such-and-such"

[Finn Lancaster]

I'm not sure how familiar you are with Node/Socket.io, so if this is a bit confusing for you, just ask me for clarification. I am also always willing to collaborate on great ideas, if you would like to build something like this and need help, I'd be happy to help via GitHub Collaboration!

[Charlie Veniot]

Good stuff for later.  Thank-you !

I'm still very new to node.js and TiddlyWiki on node.js (let's say "white belt") and I consider myself maybe an "orange belt" re TiddlyWiki.

Just to prototype, I'm going to go with some old-school thinking, just as a way to familiarise myself (a bit of a "brain-age exercise") with just the basics and see how far I can push those basics with a little help from copy/paste/tweak javascript code.  Without typing any javascript code (since I'm no fan of the stuff.)

Then I can pass the end-result onward for you to socket.io-cize the thing.

[Finn Lancaster]

Sounds good! You’ll probably need to add code to the TW Node server file, and can handle the rest through a plug-in. I have a few NodeJS projects that may help you, if you’d like, too, I can send you their links. 

If we are going to collaborate for this, it’s probably easiest to make a GitHub repo for it. Would you like to create the repo, or I’d be happy too, as well. 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d8f3a50a-c26b-49bc-9bdf-509793970391n%40googlegroups.com.

[Saq Imtiaz]

Hi Charlie,

On Tuesday, August 17, 2021 at 8:18:27 PM UTC+2 cj.v...@gmail.com wrote:

Say I have a TiddlyWiki running on node.js, and two people currently viewing the same TiddlyWiki in their browsers.

Say User 1 is editing a tiddler.  I'm thinking it would be cool if  User 2 could know that User 1 is editing the tiddler (or that anybody is editing the tiddler.)

I think most of the basics for this are in place and just needs some cobbling together.

Note that the nodejs server does synchronize tiddlers from client to server,  and from server to client. I believe the default refresh interval to get new changes from the server is about a minute if my memory serves me well.

 If User 1 is editing a tiddler, that creates a Draft tiddler that will get sync'ed to User 2's wiki. This would appear as a red draft indicator at the bottom of the screen, though it could easily be used to display a message on the tiddler in question as well.

 

And/or, it would be cool if User 2 can get a notification that a particular tiddler has changed (because of User 1's editing), and that User 2 should refresh the browser to get that change if he/she wants it.

The new tiddler should automatically get sync'ed from the server to User 2's wiki without need for a reload.

Note that there is work underway on making the server to client sync instantaneous:

- https://github.com/Jermolene/TiddlyWiki5/pull/5279

- https://github.com/Jermolene/TiddlyWiki5/pull/5882

Also, Joshua is doing some interesting work on implementing a real time multi-user feature.

Regards,
Saq

[PMario]

On Tuesday, August 17, 2021 at 8:18:27 PM UTC+2 cj.v...@gmail.com wrote:

Say I have a TiddlyWiki running on node.js, and two people currently viewing the same TiddlyWiki in their browsers.

Say User 1 is editing a tiddler.  I'm thinking it would be cool if  User 2 could know that User 1 is editing the tiddler (or that anybody is editing the tiddler.)

And/or, it would be cool if User 2 can get a notification that a particular tiddler has changed (because of User 1's editing), and that User 2 should refresh the browser to get that change if he/she wants it.

There is a "server sent events" plugin from Arlen https://github.com/twcloud/tiddlyweb-sse which can do exactly that. It's much easier to implement than sockets and it works well.

There are some hickups, because you can see tiddlers that are edited by others. They are shown as a "draft of ... by <name>" at the bottom of the screen. You can also open them and you will see live editing ... BUT you are not allowed to change something on your side :/ ... AND you have to close the draft with the Open sidebar X button. Otherwise you will disturb the other user.

-mario

[Charlie Veniot]

I had not noticed cycled refreshing.  Thanks!  I must watch those TiddlyWikis again.

[Joshua Fontany]

As Saq mentioned, I am very close to a public beta of a real-time "live multiplayer" syncer build on top of TiddlyWeb/TiddlyWiki5 Server + Websockets + Yjs (Conflict Free Replicated Datatype library).

I took a lot of inspiration from Jed Carty's Bob Server setup, but so many of the internal changed during the Yjs integration that it has become its own thing. I need to get to a stable point in the refactoring to start sharing my progress.

Aside: I also noticed that if you "Close" an open draft tiddler with the `X` button, that it deletes the draft from the wiki, and this really disturbs drafts from other users that are being edited live.... note to self to file a bug report on that.

Best,
Joshua Fontany

[PMario]

On Wednesday, August 18, 2021 at 4:27:18 AM UTC+2 joshua....@gmail.com wrote:

Aside: I also noticed that if you "Close" an open draft tiddler with the `X` button, that it deletes the draft from the wiki, and this really disturbs drafts from other users that are being edited live.... note to self to file a bug report on that.

The (x) button top right at the edit tiddler toolbar is the "discard changes" button. It needs to delete the draft tiddler. This behaviour can't be changed.

If you click the x-button in the right sidebar : Open tab it only removes the tiddler from the story river.

So plugins that want to implement a different behaviour have to replace the cancel button depending on the user name.

So if the user-name in the wiki is the same as the user-name shown in the draft tiddler they need to see the "core cancel button". ...

If it is different they need to see the plugin-cancel-button.

Replacing the core button, without modifying it is easy: Set $:/config/EditToolbarButtons/Visibility/$:/core/ui/Buttons/cancel to "hidden" and it will be gone. 

The new "Remove draft from story river" button can have a similar icon but only removes the tiddler from the story river.

The same function from "sidebar : Open" can be used. Creating a new plugin-cancel-button is simple just tag it $:/tags/EditToolbar

-mario

[Stobot]

Ok, think my post went somewhere else (sorry if someone got that directly)

Newbie question for Charlie or others: How did you get Node.js version to serve to multiple people? I use Node.js for my personal stuff and BOB for multi-user but I have a lot of issues with BOB during disconnects/reconnects that I don't have with "normal" node. 

Joshua: I eagerly wait to hear about your multi-user solution coming soon! As I mentioned to Boris and others when the open collective first started, I was happy to support the collective generally, but I would be most inclined to donate around future multi-user development, as that opens up a lot of things for me. Let me know if I can help in any way!

[PMario]

On Wednesday, August 18, 2021 at 1:30:01 PM UTC+2 Stobot wrote:

Ok, think my post went somewhere else (sorry if someone got that directly)

It did land in my PM inbox. .. No problem :)

 

Newbie question for Charlie or others: How did you get Node.js version to serve to multiple people? I use Node.js for my personal stuff and BOB for multi-user but I have a lot of issues with BOB during disconnects/reconnects that I don't have with "normal" node. 

I'm not mentioned directly, but I'll post anyway. ... The nodejs server could always serve the wiki to multiple people. ... BUT you shouldn't modify it at the same time.

Arlen did create a "server sent events" plugin, that doesn't need web-sockets, but it is "unidirectional". So it allows the server to send messages to the client in 1 way only. The advantage is, that it is simpler to implement. ..

If TW runs with nodejs, there is a function in the client that reads all tiddler-titles from the server every minute. If there was a change that the server knows about, it will load "all-tiddlers" from the server, including new ones.

The function is called "syncFromServer". So if you open 1 wiki in 2 tabs of a browser you will informed, if someone opened a tiddler. A _red_ "draft of ... by <name>" button will be shown in the bottom of the wiki window. The name will be only there if the username is set in the ConfigTiddler!

The maximum delay is 1 minute. This function is in there for quite some time already.

The problem is, that 1 minute delay can be way to long, if 2 persons want to edit at the same time. ... Server-sent-events works almost instantly. ... BUT it has some flaws at the moment, that I did describe in the other post.

-mario

[PMario]

Followup,

On Wednesday, August 18, 2021 at 2:03:53 PM UTC+2 PMario wrote:

...

The function is called "syncFromServer". So if you open 1 wiki in 2 tabs of a browser you will informed, if someone opened a tiddler. A _red_ "draft of ... by <name>" button will be shown in the bottom of the wiki window. The name will be only there if the username is set in the ConfigTiddler!

The server sent event plugin triggers the exact same mechanism. So it's possible to create instant feedback without the need to modify core functions.

The disadvantage here is, that there is quite some overhead over the wire. The message flow is like this:

 1) Server informs the client about new data

 2) Client reads all the tiddler meta-data without the text field

 3) Client compares all titles to see if something changed.

 4) If something changed the client reads ALL tiddlers including text content

 5) Client saves new data

 6) TW UI shows new data.

This workflow is OK for the current client implementation, since it runs on a client-side timer. But the with SSE it could be like this:

 - 1) Server sends the changed data to the client

 - 2) Client saves them

  3) TW UI shows new data

As you can see, there is a lot of room for improvements. Especially the "Conflict Free Replicated Datatype" that Joshua mentioned would be an interesting option here.

-mario

[Stobot]

Interesting - I greatly appreciate the input/detail @pmario! 

In many (though not all) of my intended use cases, restricting others to *only add* operations (no changes) via UI trickery would be doable. My use-cases usually involve *me* setting something up a list of business decisions for review, and have others comment / approve / rate etc, which I'd usually store in the background as system tiddlers incorporating user names. In such a worflow the "conflicts" are not a real issue, and I can even handle the timing lag of it being propogated to others in most cases, though reducing that would be ideal.

I start node each morning and it gives me the standard Serving on http://127.0.0.1:8080 message, so I'd have to figure out how to get that to a 10.1.X.X situation for LAN usage to multi-serve right? In BOB there's some buttons to press, so the actual method of how to do this in base-node is something I'm not aware of.  

[PMario]

Hi,

Which OS do you have?

-mario

[Stobot]

Windows 10

[PMario]

In powershell you can try

ipconfig

or

Get-NetIPAddress | Format-Table

Your PC address will be the one with 10.1.x.x  in it. Mainly the one which comes form a Dhcp server

-m

[PMario]

On Wednesday, August 18, 2021 at 4:01:11 PM UTC+2 Stobot wrote:

I start node each morning and it gives me the standard Serving on http://127.0.0.1:8080 message, so I'd have to figure out how to get that to a 10.1.X.X situation for LAN usage to multi-serve right? In BOB there's some buttons to press, so the actual method of how to do this in base-node is something I'm not aware of.  

On windows there are even better options. You can use IIS Internet Information Service to run your node server as a service on your computer. So as soon as it is switched on the server will be active.

BUT ... In an enterprise environment you will probably _not_ be allowed to start IIS as a service on your PC ...

If you do have admin rights on your PC you can also define a "local domain name" in the windows hosts file, if you have a fixed IP address.

eg: http://<anyName>.lan:8080  ..

The port will need to be there :/ ... BUT if you would have access to IIS you could start it at port 80. So the address would be http://<anyName>.lan ... Which imo is much cooler than http:/10.1.0.10:8080 

just my thoughts

-mario

[PMario]

On Wednesday, August 18, 2021 at 4:45:31 PM UTC+2 PMario wrote:

If you do have admin rights on your PC you can also define a "local domain name" in the windows hosts file, if you have a fixed IP address.

eg: http://<anyName>.lan:8080  ..

Oh ... I did forget. If others also need to know the local domain name, they will also have to adjust their hosts file. Which needs admin rights.

-m

[Stobot]

Thanks for the continued conversation @pmario,

So I'm aware of my IP and am fine with the ip:port need (as that's how it's done in BOB also), but the piece I'm stuck on is how to get node to serve it on my local IP address (though this is really not my area of expertise, so could be way over-complicating it). For example, to start each day I run this from the command line:

tiddlywiki "C:\Users\...\wikiname" --listen

and then it says "Serving on http://127.0.0.1:8080" in the console. How do I get it / change the command I'm running to put it at a 10.1.X.X address so that others on my LAN can get to it. It's not like they can enter my 127.0.0.1:8080 address from their computer and get to my wiki - right?

note that I don't have admin access at this time (for IIS etc), can sometimes get it temporarily, but let's just call it a "difficult" path, though maybe not impossible. I like the domain name vs. IP though :)

[PMario]

HI,

The TiddlyWiki --listen command has a lot of additional parameters. See: https://tiddlywiki.com/#ListenCommand

tiddlywiki "path/to/wiki" --listen host=10.1.x.x   should do the trick.

-m

[Stobot]

Ah! I knew it must be something easy - thanks for that! Lots to read up on :)

[TW Tones]

If node is set to 0.0.0.0 the IP address becomes server on the LAN at the current (or all) IP address(s) of the device.

On windows you can actually configure additional IP addresses on your desktop, and then rather than use 0.0.0.0 use the specific IP address. I have done this on bob but got it mixed up a few occasions. The smart trick is to assign an additional IP address outside the dhcp range and host your server on that but from your desktop. One day you may move your server and the additional IP address to another computer, say onto a NAS, server or always on computer - even a raspberry Pi.

Another nice tip is you can use any 127.X.X.X address on the local host as well so you keep 127.0.0.1 free for other services.

I can elaborate, I am just hinting here.

Regards

Tones

[Charlie Veniot]

Brewed idea has fermented into a different prototyping project.

Scenario:

Say I have two sales reps, each with his/her own TiddlyWiki running on node.js

They each maintain their own customer lists, but I want each sales rep to have view-only privs on the other sales rep's customer list.  So that each Sales rep can be a backup to the other one when a Sales rep in on vacation or whatever.

Each Sales rep should have a directory of his/her own customers, but also have available a global directory of all customers (so merging of the directories.)

And those directories should provide a way to see customer details, regardless of the source (Sales rep's own wiki or other sales rep's wiki.)

And a way to know when something new has happened in the other wiki.

All without having to learn new technical things, other than familiarizing myself with basic TiddlyWiki and node.js stuff.

The current scope of naughty bits

Playing around a little, here are some of the things that will be involved:

• WebServer Basic Authentication, WebServer Authorization, WebServer Parameter: credentials
• Simple ways to write protect tiddlers
• Using the read-only single tiddler view
• Symbolic Links
  
• Maybe:  This goofy approach to dynamic HTML/javascript creation in an iFrame
  • A way to know about "what's new" (re customer info) in other Sales reps' tiddlywikis
• Enhanced tiddler saving actions for a customer to update a "customers list" data tiddler
  • Each sales rep has a "customers list" tiddler, and each sales rep has a symbolic link to all other sales reps' "customers list" tiddlers; so a "list of customers" tiddler brings in, formats, sorts, all of these data items from all of these data tiddlers
    • won't be super-perfect because each tiddlywiki needs to be restarted to get the latest data tiddlers from the other tiddlywikis

Something like that.  No fuss, no muss.  Keeping it simple.

[Hans Wobbe]

Based on a cursory review of scenario, I think it can be generalized to Real Estate applications.  This opinion is based on my discovery that Agents frequently share their listings with other Agents ( even ones from competing Brokerages ).

It could be even more general if a Seller and several potential Buyers were inclined to share some of their property data.  Since my views are mostly Canadian, I can say that this has considerable potential in competing with CREA ( the Canadian Real Estate Agency that runs virtually all of the Canadian MLS = Multiple Listing System )

Best,

Hans