An Idea for syncing TiddlyWiki

[Jonnan]

This is an idea that's been percolating in my head for awhile - if there's a theory why it can't be done I haven't thought of it yet, though I can think of a dozen practical reason it might not work, mostly due to the computing overhead becoming intolerable inside a browser or edits on the file causing versioning issues. I can imagine either simultaneous edits being a problem, or edits being made in isolation and needing to be correlated after the fact.

That said - I've been using BTsync for various cloud backup/syncing applications for about eight months now, and I can't help but wonder if something conceptually similar could be used in TiddlyWiki.

For the uninitiated, BTsync uses the bittorrent protocol to advertise/transmit the contents of a folder on the internet with a (IIRC) 40Byte Encrpytion key - anyone with a matching key can have a read write or read-only (in effect) copy of the folder automatically synced. Put one on a free AWS server and you've got your own dropbox account (I put all my ebooks there).

It seems like something that *could* work with a single file as opposed to a folder. It would require implementing the same kind of btsync concept in Javascript of course, a non-trivial obstacle I'm going to just glide on by thank you very much.

Given that that can be done, I see two obvious problems, both involving versioning. One would be that in order guarantee updates at least one other copy would have to be open at any given moment - which one should be unimportant, as long as there is a 'chain of provenance' at any given moment - when a new tiddly wiki is opened it would update. The other would involve two or more people simultaneously editing a single tiddler; Some variation of 'diff'' logic would seem suitable.

It would be . . . awfully neat . . . if the first item could somehow  be managed completely within the confines of tiddlywiki, but I concede I'm seeing no strategy for doing that doesn't involve someone magically making sure there is at least one copy open somehow. I can however see it being integrated with BTsync in some way that a copy in a BTsync folder with the same key is somehow set as a master copy, with BTsync keeping it updated when not in use. It's not entirely lacking in elegance to be sure.

The second is probably more difficult than it looks, but doesn't seems terribly hard at first glance.

The result be a bit of html the would be (within certain limitations) genuinely 'in the cloud' as it were - a website without any pure client server relationship.

Does this strike people as something that could actually be done, or is it a neat idea that wouldn't work in practice?

[Daniel Baird]

On 9 May 2014 04:28, Jonnan <jonna...@gmail.com> wrote:

 

It seems like something that *could* work with a single file as opposed to a folder. It would require implementing the same kind of btsync concept in Javascript of course, a non-trivial obstacle I'm going to just glide on by thank you very much.

Not sure if Atwood's Law will help you here, but these guys have a sort of JS wrapper that handles asking the user to install their protocol hander, or something: http://btappjs.com/

Given that that can be done, I see two obvious problems, both involving versioning. One would be that in order guarantee updates at least one other copy would have to be open at any given moment - which one should be unimportant, as long as there is a 'chain of provenance' at any given moment - when a new tiddly wiki is opened it would update. The other would involve two or more people simultaneously editing a single tiddler; Some variation of 'diff'' logic would seem suitable.

Years ago I wrote a bunch of diff-ing code for comparing tiddlers, if anyone picks up this project lemme know and I'll send it all to you.

Cheers

;Daniel

--

Daniel Baird
objoke: I had a problem and decided to solve it with threading. Now, have problems. two I

[PMario]

Hi Jonnan,

There are several file syncing services, that may work well with TW.

I did discover seafile [1] service lately.

They provide a file based sync, and are able to discover conflicts, if 2 files have been edited at the same time.
The mechanism doesn't do any automatic merging ... The latest file wins, but the conflicting file is synced to both sides too, so it is possible to see, that there is something wrong. Resolution needs manual intervention. ...

I did show the TW node based possibilities in hangout#33 [2].

The cool thing about this particular service is, that they did include a "social discussion" component.
So it is possible to easily create groups and file related discussions. So discussions about a single file are linked to this file and can be easily discovered. ...

I didn't do much more testing, than shown in the video. I also forgot to mention the social aspect in the video.
I think this service could have great potential, because it is possible to discuss and share single tiddlers from a TW ...

The software is open source. So it is possible to run your own server / service, if you want.

Conclusion:
I think the file based syncing approach is interesting if you have the right setting.
eg: One or a view "writers" and many "readers", so the confilct handling doesn't cause too much trouble. Especially, if you can discuss TW changes prior to publishing them ...

If you have a very dynamic setting, with several people editing at the same time, imo a file based approach is broken by design.

have fun!
mario

[1] http://www.seafile.com/en/home/
[2] https://www.youtube.com/watch?feature=player_detailpage&v=GEqXu6xay7M#t=1989

[NT]

Hey Mario,
I stumbled upon your post on the search for a possibility to realize a shared/distributed work on a tiddlywiki in the cloud - not in the web - with automatic syncing between teampartners who should be able to edit the wiki at the same time (more or less). I think the problems are on hand for this case. I also watched the hangout with presentation of seafile. Some questions remained, as I had no time time try tw5 with node.js and seafile on my own, hopefully you could answer them??

Regarding to the context of your seafile presentation:
1. Looks like you worked with tiddlywiki on node.js in seafile, since I saw multiple separated files for each tiddler - did you (I did not yet, only have read about)?
2. That the inconsistent (conflict) file is brought up by seafile is nice, but do you get noticed about that?
2.1 Say you have 300 tiddlers aka 300 files - you'll not be able to discover on your own when a new conflict file will appear among them. So do you get a notice of the conflict?
3. Another question regarding node.js and multiple files - is that working on ios and android as well?
4. Is multiuser editing considered at all in the design of tw5? Because I only saw that you can enter the username in the properties of tw5, but whoever edits the wiki afterwards does it with the pre-entered user name if she's paying no attention - that's not relyable.


Thanks for your time, NT from germany.

[Tobias Beer]

I think with all these shared / collaboration concepts one thing is key:

separating "the wiki(s)" from "the store(s)"

For collaboration around TiddlyWiki, the single-file paradigm is essentially untenable,

not just for the simple reason that to currently push 1 byte of "changed tiddler"

you my need to push +2mb of "changed wiki"... unless you're using a pretty vulnerable node server.

So, imho, we're looking for solutions with "prebuild" / "cooked" / "baked" TiddlyWiki wikis that are configured to CRUD individual tiddlers to whichever stores said wiki is configured to talk to.

So, making TiddlyWiki use Seafile or BTSync, you have two problems...

1. reading, writing individual tiddlers to stores
• perhaps different adapters / syncer modules to talk to either
• it should be possible to tell TiddlyWiki to which store out of x to add tiddler y
• perhaps even to which subfolder
2. reading, writing the wiki (application) itself
• which could be the same as 1. if there was
• some skelleton TiddlyWiki with a basic configuration for some "wiki-store"
• all customizations to the wiki written to said store
• in other words, separating content from presentation / the application
• insofar that's needed for a proper startup
• the point being, that the wiki file itself is only modified when absolutely necessary
• so as make use of caching and not clogging communication with reading and writing an entire wiki rather than tiddlers
  
• of course, you don't want to make a bazillion requests per tiddler either,
• so, talk to stores and read "bags", e.g. meaningful collections of tiddlers
• while writing tiddlers back to it individually

Best wishes, Tobias.

[PMario]

On Tuesday, January 27, 2015 at 2:06:48 AM UTC+1, NT wrote:

Regarding to the context of your seafile presentation:
1. Looks like you worked with tiddlywiki on node.js in seafile, since I saw multiple separated files for each tiddler - did you (I did not yet, only have read about)?

yes, it was a nodejs version, where every tiddler is a single file. The TW server has a very very basic "store" handling, which is not designed for multi users.
 

2. That the inconsistent (conflict) file is brought up by seafile is nice, but do you get noticed about that?

You don't. If you reload the page, you'll see, that there are conflicting files. ... But TW doesn't have an easy mechanism to see the differences. ... So some manual work needs to be done, resolving the conflicts.
 

2.1 Say you have 300 tiddlers aka 300 files - you'll not be able to discover on your own when a new conflict file will appear among them. So do you get a notice of the conflict?

Page reload.
 

3. Another question regarding node.js and multiple files - is that working on ios and android as well?

I don't think, that nodejs works on IOS. I'm not sure about android. ... IMO If you need mobile access, you can use TW apps. There are some of them, but I don't know the links. ... There is no server side atm, to sync and resolve conflicts.
 

4. Is multiuser editing considered at all in the design of tw5?

This is not a design decision on the client. .. The client has so called sync adaptors, that can work with "any" backend. ... We need a __real__ server side backend + store.  The problem is, that most users want to use a file based backend, because they are familiar with it. ... But file systems are not very good if you need to edit them at the same time.
 

Because I only saw that you can enter the username in the properties of tw5, but whoever edits the wiki afterwards does it with the pre-entered user name if she's paying no attention - that's not relyable.

That would be solvable with some UI changes.

Hope that helps.

I'll need to read Tobias post and may post some more thoughts there.

have fun!
mario

[NT]

Of course, separating data & logic is a meaningful thing in most scenarios, but that does not answer my current questions.

Ok, to be a bit more constructive some minds on sync from my side...

If we speak about distribution on file level (in the cloud) the question is if it's neccessary to do that separation of data & logic on file level as well. One would think so.
But I like the 1 file approach of tw5 very much and would not like to give it up if it's not really neccessary - it makes things very handy at all.

If somebody would ask me for a blueprint of file level syncing of TW5 in a efficient but also simple way I would suggest the following:
- keep things simple with the single file approach - over disadvantages it has many advantages

- on sync side A:
-- use the TW5 autosave capabilities to do not a save but an export of a file (call it syncStatusA) that has the following contents
--- the logic to say what has changed (tiddler updated)
--- the data that changed (tiddler data)
-- the exported file will be automatically synced (if thats possible) by the cloud service client you are using (it's possible in mine)
- on sync side B:
-- use the TW5 autosave logic to do an import (read) instead of an export (if that's possible???)
-- look for a exported file that has also been synced in the current file space (the syncStatusA file) and import it
-- maybe a file naming convention for the syncStatus file is neccessary & sufficient here, but a property inside tiddlywiki describing the exact name of the import file for the current session would do it

- if you don't want to think about the implications by working with both wikis on sides A & B on one and the same syncStatus file, then you have to choose a different name for the import and the export file of one wiki - or respectively, choose a different name of the syncStatus import/export file on both sides A & B - then you have a handshake, don't you?!
- naming the import & export file different would be meaningful to make the sync robust for the unknown conditions of the cloud service (sometimes there's automatic file versioning, etc.)
- so finally, the names of the files could be syncStatusA and syncStatusB and they would be synced by the cloud service and carry the information and data what has changed and will be imported automatically
- and with minimal manual interaction of an handshake for a session you could have a synced tw5
- a session would mean here that you start with a synced wiki (copies) and do handshakes by determining the name (convention) of the syncStatus file considering the current cloud conditions
- after the sync the syncfile of A has to be deleted by B and A is not allowed to write a new syncfile while the old syncfile exists
-- that would work for a little number (10?) of team members, at least for 2, wouldn't it?
- if you are more than 2 - say 10 - and you need for technical tw reasons not the naming convention but the exact name of the syncStatus file, then you would need the possibility to tell your tw not only one import/export naming convention of the syncfile, but 10
- in this case after updating a tiddler on side A nine (9) syncfiles will be created for the team members
- so sync should be possible this way for small teams and I think you could put that functionality into a plugin
- this would keep things quite handy for a 3 person team and would be a large benefit in workflow

Perhaps Jeremy has a another comment on that?
Regards, NT

[PMario]

On Tuesday, January 27, 2015 at 3:53:50 AM UTC+1, Tobias Beer wrote:

I think with all these shared / collaboration concepts one thing is key:

separating "the wiki(s)" from "the store(s)"

right.
 

For collaboration around TiddlyWiki, the single-file paradigm is essentially untenable,

The point here is, that our users want it. They refuse to use server backends, because managing them seems to be to complicated.
 

not just for the simple reason that to currently push 1 byte of "changed tiddler"

you my need to push +2mb of "changed wiki"... unless you're using a pretty vulnerable node server.

That's not really a problem. Most servers use gzip before the send a file. So you have about 400kByte over the wire. ... One photo on a decent mobile phone has about 3MByte atm. Since it is jpeg it can't be compressed anymore. ... So imo file size isn't the main problem.

But you are right. Sending 400k to change 1 byte is kind of an overkill :)

I still think the idea that Jeremy had with a decent export / import mechanism + UI is a good one which would fit for many users.
eg:
You have a local TW copy, where you made some changes.
A friend gives you access to his modified version.
You drag & drop import it to your TW
-> new ->
You get an UI, that tells you 80% of the tiddlers are the same. They are not listed.

You get a list of tiddlers, that you modified
You get a list of tiddlers, that he modified .... + UI described below.
You get a list of conflicting tiddlers, because both modified them. .... Here we need a better UI, that let's us see the differences, and let us resolve them.


I'm sure there are quite some users, that would be happy with this mechanism, because it has no dependencies. It's extremely simple. An if you only did eg: 10 small changes in the text. Resolving will be fast too.

part 1 may be more to follow
-m

[NT]

Am Dienstag, 27. Januar 2015 11:09:29 UTC+1 schrieb PMario:

2. That the inconsistent (conflict) file is brought up by seafile is nice, but do you get noticed about that?

You don't. If you reload the page, you'll see, that there are conflicting files. ... But TW doesn't have an easy mechanism to see the differences. ... So some manual work needs to be done, resolving the conflicts.
 

2.1 Say you have 300 tiddlers aka 300 files - you'll not be able to discover on your own when a new conflict file will appear among them. So do you get a notice of the conflict?

Page reload.

Sorry, I was speaking here about the seafile service and not about tw. But I think you mean a page reload inside seafile as well. That's what I mean is a problem.
1. you have to do a page reload
2. i don't think that you recognize one more file (the conflict file) among 300 other files in meaningful time without to get noticed by seafile.
Do you get noticed be seafile??

 

3. Another question regarding node.js and multiple files - is that working on ios and android as well?

I don't think, that nodejs works on IOS. I'm not sure about android. ... IMO If you need mobile access, you can use TW apps. There are some of them, but I don't know the links. ... There is no server side atm, to sync and resolve conflicts.

 
Then is mobile use not possible on side B when on side A there is the node.js version running with multiple files getting synced to A??
I am using TW apps for mobile access right now, but i am working with the single file tw at the moment - and it's quite handy.

[NT]

Mario, please have a look at my comment to Tobias post as well. Thanks :)

[PMario]

On Tuesday, January 27, 2015 at 11:32:08 AM UTC+1, NT wrote:

If somebody would ask me for a blueprint of file level syncing of TW5 in a efficient but also simple way I would suggest the following:
- keep things simple with the single file approach - over disadvantages it has many advantages

Your description is OK, on a very high level, where every thing works, but the problem is in the detail. Like: "In theory there is no difference between theory and reality. In reality there is!"

I'm just picking one "simple" detail: handshake

- a session would mean here that you start with a synced wiki (copies) and do handshakes by determining the name (convention) of the syncStatus file considering the current cloud conditions

If you use your file based handshake, you need to sync the handshake = "lock file". ... But this is a "chicken / egg" problem.
To trigger the "3rd party syncing action" you need to delete the lock file on side A. -> syncing is triggered, but doesn't work -> So you are out of sync.

A thinks I'm ok
B thinks A is still locked ... The problem here is the 3rd party transport. Because you can't trust it. So the locking mechanism needs to be more sophisticated. ...

- after the sync the syncfile of A has to be deleted by B and A is not allowed to write a new syncfile while the old syncfile exists

This is a typical condition for a "dead lock", where both sides are locked, because the "delete" action didn't work, for some unknown reason.

-m

[PMario]

On Tuesday, January 27, 2015 at 11:50:24 AM UTC+1, NT wrote:

Am Dienstag, 27. Januar 2015 11:09:29 UTC+1 schrieb PMario:

Page reload.

Sorry, I was speaking here about the seafile service and not about tw. But I think you mean a page reload inside seafile as well. That's what I mean is a problem.
1. you have to do a page reload
2. i don't think that you recognize one more file (the conflict file) among 300 other files in meaningful time without to get noticed by seafile.
Do you get noticed be seafile??

No seafile syncs, but the nodejs server doesn't push this info to the client. So the client needs to do a page reload, to see the changes. 

How should seafile send a notice to the browser client. It doesn't know about it. It's a file sync service.

-m

[PMario]

On Tuesday, January 27, 2015 at 11:50:24 AM UTC+1, NT wrote:

Then is mobile use not possible on side B when on side A there is the node.js version running with multiple files getting synced to A??

IMO not really possible
 

I am using TW apps for mobile access right now, but i am working with the single file tw at the moment - and it's quite handy.

Yes. The single file approach has some really big advantages and its on thing, which makes TW unique.

-m

[Danielo Rodríguez]

2.1 Say you have 300 tiddlers aka 300 files - you'll not be able to discover on your own when a new conflict file will appear among them. So do you get a notice of the conflict?

Page reload.

No, that does not work. 

[Tobias Beer]

If we speak about distribution on file level (in the cloud) the question is if it's neccessary to do that separation of data & logic on file level as well. One would think so.

Yes. You would have tiddlers...

• in a desired file-format:
• for that purpose, TiddlyWeb implements serializers that return different formats
• e.g. txt (=tid), json, html, wiki
• at a desired storage
• a file system with folders (or bags in TiddlyWeb / TiddlySpace)
• a database, possibly with (even typed) tables
• proper interfaces on both ends that handle all that browser <> server talky-talky

But I like the 1 file approach of tw5 very much and would not like to give it up if it's not really neccessary - it makes things very handy at all.

It sure makes it easy to share and host a wiki (for an individual),

but is entirely counter-productive for anything collaboration.

If somebody would ask me for a blueprint of file level syncing of TW5 in a efficient but also simple way I would suggest the following:
- keep things simple with the single file approach - over disadvantages it has many advantages

...which don't pay out with multi-user, in fact, are in stark contrast to it.

- on sync side A:
-- use the TW5 autosave capabilities to do not a save but an export of a file (call it syncStatusA) that has the following contents
--- the logic to say what has changed (tiddler updated)
--- the data that changed (tiddler data)
-- the exported file will be automatically synced (if thats possible) by the cloud service client you are using (it's possible in mine)
- on sync side B:
-- use the TW5 autosave logic to do an import (read) instead of an export (if that's possible???)
-- look for a exported file that has also been synced in the current file space (the syncStatusA file) and import it
-- maybe a file naming convention for the syncStatus file is neccessary & sufficient here, but a property inside tiddlywiki describing the exact name of the import file for the current session would do it

Yes, but at the end of the day you want one storage location, not two, and so you need to save an entire wiki and make sure entire wiki's are properly merged, not some hoped-to-be-temporary handshake files. What happens to those "exported dumps"? When are they discarded? How to know what's new and when conflicts are finally merged? Do 3 of 4 group members set a flag...file ...that says "ok, I merged your dump, now what"? When does the dump get deleted? Can I cancel that last commit? How is that feedback loop working? A single file won't do, obviously, as A might want to push to B might push to A.

- if you don't want to think about the implications by working with both wikis on sides A & B on one and the same syncStatus file, then you have to choose a different name for the import and the export file of one wiki - or respectively, choose a different name of the syncStatus import/export file on both sides A & B - then you have a handshake, don't you?!

If you want a single syncStatus file then that needs to be locked by the emanating wiki up until all others have merged the changes. I see a lot of problems already in that workflow. For TWc, there was TiddlyLock, which essentially put a lock-file next to TiddlyWiki saying "I am working on it, so you can't"... already this simple thing had so many known unknowns that essentially you ended up overwriting each other's edits... or not being able to work in TiddlyWiki at all!

- naming the import & export file different would be meaningful to make the sync robust for the unknown conditions of the cloud service (sometimes there's automatic file versioning, etc.)

It would also make it much more difficult to figure out what (still) needs merging and what doesn't... let alone where to get it from, filewise.

- so finally, the names of the files could be syncStatusA and syncStatusB and they would be synced by the cloud service and carry the information and data what has changed and will be imported automatically

Who / what gets to decide when to delete the file?

 

- and with minimal manual interaction of an handshake for a session you could have a synced tw5

You will need manual interaction, there is a lot of potential for merge conflicts. What if I fail to merge?

- a session would mean here that you start with a synced wiki (copies) and do handshakes by determining the name (convention) of the syncStatus file considering the current cloud conditions

At what point am I allowed to close my wiki? When is it saved?

- after the sync the syncfile of A has to be deleted by B and A is not allowed to write a new syncfile while the old syncfile exists
-- that would work for a little number (10?) of team members, at least for 2, wouldn't it?

Won't I be waiting for who knows how long until someone decides to close their browser session and allows me to push my changes? How do I even know they're still having that session open or not in their notebook (or mobile or desktop) on stand-by? Do I need to know? When I start my browser, am I irectly being greeted with my own yet unmerged sync-file, that of someone else?

- if you are more than 2 - say 10 - and you need for technical tw reasons not the naming convention but the exact name of the syncStatus file, then you would need the possibility to tell your tw not only one import/export naming convention of the syncfile, but 10

I guess a sync folder should do. Did you notice how we already have more than one file? So, I'm wondering, if there is no single file paradigm with multi-user, why bother struggling with such bricolage?

 

- in this case after updating a tiddler on side A nine (9) syncfiles will be created for the team members

Big ouch, don't you think? Again, when am I finally allowed to save my wiki? Do I, at some point, need to sync back my own file, because I couldn't save?

- so sync should be possible this way for small teams and I think you could put that functionality into a plugin

Here's one thing I am entirely positive about: Please don't make me the admin to that process.

 

- this would keep things quite handy for a 3 person team and would be a large benefit in workflow

In an ideal world with all things going smoothly, there's a window this can work, but in so many ways it's nothing I would spend my energy on. I would only recommend this process in the case where there is usually one maintainer of the wiki and only rarely contributions by others.

Here's a proposal: Make a flowchart / bpmn chart, whatever ... that dares to define this process in a water-tight manner. Once we get this design in a loophole-free manner, something could perhaps be implemented. Right now, I don't see an even remotely water-tight process, I see some strokes on a canvas that don't quite form a picture just yet that reads: Let's TiddlyCollaborate... "singlefile"-style.

So, here's my bottom line for now:

multi-user means

• single tiddler store
• with folders / bags / db tables (/ and also filters / recipes / permissions)
• proper server
• serving TW
• serving tiddler collections
• providing a CRUD interface per tiddler
• proper interfaces on both ends
• so a standalone wiki knows to communicate with that server
• and the server knows to handle those requests

So, this begs, above all, for a specification of a server-and-browser-agnostic-interface/-protocol that any server or browser can implement so as to talk with each other in their desire to exchange tiddlers in the context of a TiddlyWiki wiki or otherwise.

Best wishes, Tobias.

[Tobias Beer]

I'm sure there are quite some users, that would be happy with this mechanism, because it has no dependencies. It's extremely simple. An if you only did eg: 10 small changes in the text. Resolving will be fast too. 

Yes, an improved manual importer makes total sense.

The problem here is, though, that we're not having or wanting multiple wikis.

At the end of the day, you want one wiki all share.

So, perhaps, for a minimalist multi-user environment,

there need to be two masters...

• a master wiki
• this is where all edits end are merged into
• a master editor
• the one who merges all commits into it

Best wishes, Tobias.

[Danielo Rodríguez]

If somebody would ask me for a blueprint of file level syncing of TW5 in a efficient but also simple way I would suggest the following:
- keep things simple with the single file approach - over disadvantages it has many advantages

- on sync side A:
-- use the TW5 autosave capabilities to do not a save but an export of a file (call it syncStatusA) that has the following contents
--- the logic to say what has changed (tiddler updated)
--- the data that changed (tiddler data)
-- the exported file will be automatically synced (if thats possible) by the cloud service client you are using (it's possible in mine)
- on sync side B:
-- use the TW5 autosave logic to do an import (read) instead of an export (if that's possible???)
-- look for a exported file that has also been synced in the current file space (the syncStatusA file) and import it
-- maybe a file naming convention for the syncStatus file is neccessary & sufficient here, but a property inside tiddlywiki describing the exact name of the import file for the current session would do it

When I started tiddlydrive that was my goal. Currently the project is frozen, but you can check it out at  http://danielo515.github.io/TiddlyDrive4Community Basically is a single file HTML that syncs from and to google drive.

Regards.

[Tobias Beer]

When I started tiddlydrive that was my goal. Currently the project is frozen, but you can check it out at  http://danielo515.github.io/TiddlyDrive4Community Basically is a single file HTML that syncs from and to google drive.

With all the little unresolved (and unresolvable) bits and pieces ...this is much more tenable for multiuser than anything truly single-file can be. I would not call it single-file, though ...because that's not what it is.

Best wishes, Tobias.

[NT]

Am Dienstag, 27. Januar 2015 12:05:09 UTC+1 schrieb PMario:

- a session would mean here that you start with a synced wiki (copies) and do handshakes by determining the name (convention) of the syncStatus file considering the current cloud conditions

If you use your file based handshake, you need to sync the handshake = "lock file". ... But this is a "chicken / egg" problem.
To trigger the "3rd party syncing action" you need to delete the lock file on side A. -> syncing is triggered, but doesn't work -> So you are out of sync.

Big sorry, i didn't got your point. I think my word "handshake" is not luckily chosen here. What I mean with handshake here is just an agreement between (2) parties how to communicate. The agreement just consists of the names of the syncfiles.
!! And the important point I may have forgotten to mention explicitely is that the information about the syncfile of A is told to B in a meta way. Which means that the name of the syncfile of A is told B per comment / chat / social functionality of the sync carrier (the cloud service). In a case where you have a single wiki in a directory in sync over a cloud service you have to exchange this meta information only once! In theory the meta thing might be not elegant. In reality it would be highly practical, because you only have to exchange the meta information when your team changes or the cloud service does - that would be really convenient.
 

 

A thinks I'm ok
B thinks A is still locked ... The problem here is the 3rd party transport. Because you can't trust it. So the locking mechanism needs to be more sophisticated. ...

- after the sync the syncfile of A has to be deleted by B and A is not allowed to write a new syncfile while the old syncfile exists

This is a typical condition for a "dead lock", where both sides are locked, because the "delete" action didn't work, for some unknown reason.

Sorry, I did not understood that as well, perhaps my explanation above about the meta information brings some clarification?

 

[NT]

Am Dienstag, 27. Januar 2015 12:05:09 UTC+1 schrieb PMario:

- after the sync the syncfile of A has to be deleted by B and A is not allowed to write a new syncfile while the old syncfile exists

This is a typical condition for a "dead lock", where both sides are locked, because the "delete" action didn't work, for some unknown reason.

-m

 
I don't see the deadlock with the information provided regarding the meta information exchange.
If there is a syncfile from tw A for tw B, then tw B should delete it after:
- the syncfile was recognized (tw B could check every 10 seconds) and
-- tw B updated itself according to the syncfile from A
-- tw B does nothing, because it's already in sync

[Tobias Beer]

The agreement just consists of the names of the syncfiles. The important point I may have forgotten to mention explicitely is that the information about the syncfile of A is told to B in a meta way.

We are talking of ONE wiki, are we not? So, the "agreement" you refer to simply is that there are different modifier names being used throughout the wiki... and possible "syncfiles" for each. Making sure that each modifier actually does use its own username seems a bit of an issue. What if two (unknowingly) use the same? Anyhow, you want each user to use their own username when editing that wiki in the browser. This seems to ask for either local storage or a good old cookie, not a tiddler to store who everyone is.

Which means that the name of the syncfile of A is told B per comment / chat / social functionality of the sync carrier (the cloud service).

Considering the above, it does not mean that. Those users are hardcoded or merged into the wiki like any other content.

In a case where you have a single wiki in a directory in sync over a cloud service you have to exchange this meta information only once!

So, the challenge is not one of exchanging some "me, too" by means of communication, but of successfully adding a new modifier to the group of modifiers of a wiki.

In theory, the meta thing might be not elegant.

If not into the wiki, I wonder where you want to put it.

 

In reality it would be highly practical, because you only have to exchange the meta information when your team changes or the cloud service does - that would be really convenient.

Exchanging information is not the issue, telling TiddlyWiki about it is. :-)

Best wishes, Tobias.

[Tobias Beer]

I don't see the deadlock with the information provided regarding the meta information exchange.
If there is a syncfile from tw A for tw B, then tw B should delete it after:
- the syncfile was recognized (tw B could check every 10 seconds) and
-- tw B updated itself according to the syncfile from A
-- tw B does nothing, because it's already in sync

What if you have three users, or four, or five? What if their browser is down for lunch / over night / during vacation / during the weekend? ...with pending changes? Try not to think of TiddlyWikis as running without a user and a browser... they don't, unless they come with a server. They could do some background talky-talky in an open browser session, but doing background merging and also pushing syncfiles without user interaction?!? I don't think so.

Best wishes, Tobias.

[NT]

I think we're speaking about different thinks in this post. And I think it's a bit my fault because I have no clue about nodejs with tw.
The case I wanted to describe :
- a synced wiki in the cloud (single or multiple files not so important here)
- A begins a change
- B beginns a change
- A saves a change
- entire tw A or only one tiddler of it is synced
- B saves a changes
- entire tw B or only one tiddler of it is synced and overwrites the changes of A
-> if we are lucky and have the right cloud service the version of A is saved as a conflict file by the cloud service and also synced to B

Thats what I understood from your hangout what seafile does. Do I got it wrong??
And then my question:
Does seafile notice me about the conflict or does it only deliver the 'lost' version to me (I am B)?