Pub Sub multi-user tree application

[gordon.a...@gmail.com]

I have an ambition to create a multi user interactive application that presents network information to end users in a tree structure, and allows them to amend it, notifying other users of the changes using a publish/subscribe infrastructure.

I plan to use pouchdb/couchdb for persistence and pub/sub.

I want to use an off the shelf tree presentation tool for presenting and editing the nodes and links data.

I then plan to implement the glue between the two, which will hopefully be relatively small.

I have looked at loads of tree/table presentation tools, and superficially I think FancyTree is by far the best fit for my purposes. I do have some questions and suggestions.

Clones - This extension seems to handle the network <-> hierarchy mapping. My glue will need to chose one clone at a time as the "master". The key difference between the master and its clones is that the master can have children, whereas the other clones cannot. The end user can dynamically chose which of the clones can become the master, at which point the children will move. My pouch/couch server will allocate unique guids to nodes, and will know nothing of clones. My understanding is that if I use the refKey with the same value as the guid of the original master from the server, that will tie the clones together. I am not sure how the clones get unique keys. When a user creates a new node, then the guid cannot be set until the round trip publish / subscribe update has returned.

Spreadsheet - This is possibly a new extension. I want to be able to perform very simple calculations between cells in the same leaf row (eg charge_column = rate_column * days_column), and I want to provide aggregation up through the columns (eg summing the charge column). I think I may be able to do this by storing 2 optional functions in the column definitions, without requiring an extension? (Note this spreadsheet functionality is one of the main reasons why one of the clones is the master. If you are aggregating a sum for a project, you don't want the leaf adding in more than once).

Table vs UL/LI - Most of the examples use UL/LI for the markup, but the grid examples use Tables. I would like the ability to have row heights be the maximum height of any of the column cells. There is an example of this for UL/LI markup. Is it possible to achieve the same with tables? Also I have not stumbled across an example that allows columns to be resized / hidden. Is this possible?

AJAX vs PUB/SUB. Partly because of the spreadsheet aggregation functionality, I don't think my application will have any lazy data. Pouch/couch doesn't use AJAX, but rather maintains continuous connection between the client and server.

Existing tools.
Omni Outliner https://www.omnigroup.com/omnioutliner
My daughter uses a tool called OmniOutliner to achieve some of the above. But it isn't multi user, and it doesnt handle network links. She has to go through an error prone process cutting and pasting trees into different orders.
ag-grid https://www.ag-grid.com
I think ag-grid provides some spreadsheet and aggregation functionality, but the tree version is only available as an expensive expertise license, which I can't justify.

I retired some years ago, and having a little bit extra time, I thought I would try and make something better. Its 20 years since I did any serious programming, and I am a bit punch drunk with getting to grips with Git, Atom, npm, the mess that is the current state of javascript, React, JQuery... but I'm getting there, and its good to have such a brilliant tool as fancyTree as an example.

Comments welcome on the above, and on whether what I plan is achievable and or desirable.

[mar10]

An interesting project.

Some thoughts from my side:

'clones':

They indeed were developed to make make projecting a graph onto a tree structure more easy.

All clones share a refKey. There is no concept of 'master' however: all clones are made equal. That does not mean that you can't design the UI that way, of course.

The key on the other hand must be unique within the tree. If you don't set it explicitly, Fancytree calculates one from the refKey and the parent key: this makes sure that it is 1. unique and 2. reproducible, which is important if keys are used to store expansion state for example. (While Clones would always have the same title, they can have different expansion states.) Generally unlike refKeys, `key` typically have no meaning to the backend, but the 'persist' extension stores them in cookies or sessionStorage.

'spreadsheet':

I would agree that this can be build on top of the 'table' and maybe 'gridnav' extension, by adding  some keyboard and change event handlers.

'Table vs. UL':

Clones work with both.

Modifying special row attributes is not handled by Fancytree implicitly, but can probably be done in the renderColumn handler, render events, or simply using CSS.

'Ajax, Pub/Sub'

I don't know much about Couch, but I once made a PoC of a multi-user reactive tree based on Firebase, so that should be doable.

Are you planning to start an OpenSource (GitHub?) project for this, a spec, or mockup? That would probably help others to contribute.

[gordon.a...@gmail.com]

Thanks for you comments

I am planning to put this up onto GitHub. I have a repository ready for it, but the only content so far is a readme giving much the same info as in my first post.

I have been experimenting with clones a bit since you wrote, and I have an issue /comment about the semantics of clones after they have been created.

What is a clone - Is it a copy that having been created then takes on its own independent existence, or are all the clone nodes with the same refKey essentially the same object, with the same properties. My feeling is that it should be the latter. Whatever happens to one clone happens to all those with the same refKey. So for example if the end user edits a title on one clone, then all those with the same refKey get the same title. similarly for the column values.

How then to keep the clones in step.

With no change to the core of fancyTree, I think the only way is for me to intercept the edit events, and propagate to all the matching clones.

I don't particularly like this approach. I would rather that the matching clones behaved like wrappers sharing a common "pseudo node" that they dig into to for getting and setting the column values. It might be possible to share the "data" attribute, but the "title" attribute might be more problematic.

I would welcome your views on this conundrum.

Gordon

[mar10]

I would think of clones as s.th similar to 'hard links', so I'd agree that it is rather the latter of your examples.

However the 'clones' extension should make it easy to implement that thinking model, but not necessarily force it.

Also, the 'edit' and 'clones' extensions are both optional and independent of each other, so it might need some additional hooks to implement it inside the extensions.

On the other hand, I hope it is pretty straight forward to implement the behavior using edit-extension callback.

So I'd prefer to implement it that way in a first step, and learn from it.

If it turns out to be cumbersome, we can think about improving the API.

And then, if the spec has settled and it is running, we can think about adding this as option to a standard extension or create a new specialized one...

[gordon.a...@gmail.com]

Hi,
I have been experimenting with clones a bit more, and have a work in progress to show you at:

https://github.com/GordonJones/ambition/blob/master/index.html

Its essentially a copy of your clones index.html with my additions to date:

1. Since I am trying to represent a network, I declare the source not as a tree, but as an array of nodes, and an array of links, and then dynamically generate the tree (including creating clones) at start up.

2. The concept of a master clone is essential to my requirements, so I added that (as an attribute and a css class. The master clone holds the children (and eventually values). The other clones (lets call them slaves?) just hold the title. I feel this master concept might be desirable to others trying to use the clones extension.

3. There is a context menu with just one working function "branch". This recursively descends the tree from the clicked node turning slaves that it meets into masters (and the old masters back into slaves). I use your moveTo method to achieve the move of the children from the old master to the new one.

4. I have added type classes, so that I can style by type. It would be good to see that incorporated into the types extension? - Unless it is already but as far as I can see the types are just for icon selection?

5. I have also incorporated the node counter extension, but what I really want to do is adapt it to count clones (with the same refKey) not children.

Please excuse my coding. My method is largely copy paste. I find something that fits and steal it. So I think I have a mixture of ES5, ES6, and old-fashioned non OO code. Also, for the moment, all my code is in the html file, which is not good. I'll factor it out eventually I promise.

You will see from my example that the problem I am trying to solve is from the TV industry. My daughter and her team take scripts and break them down into the assets required (props, vehicles, furniture, sets to build...) The breakdown starts out in script order. But when the shooting schedule is decided, the breakdown must be shifted into filming order. This is a tedious task, and when the writers change the scripts errors creep in. Some teams use spreadsheets for this task, and sort into different orders. But spreadsheets don't handle the arbitrary level of breakdown very well. Some have said this is a job for a planning tool, but my feeling is planning tools are too inflexible and complex for this task.

This need to view data from different perspectives seems to me quite a general requirement.

I hope this gives you a better flavour for what I am trying to achieve. I'll keep enhancing it to get closer to my objectives.

[gordon.a...@gmail.com]

Hello Martin et al,

I have realised that eventually my app should become an electron desktop app. (Like Atom, WhatsApp, Slack, ....)

I am not going to attempt that yet, but I will add it to my set of ambitions.

I don't suppose you have any experience of running fancyTree within electron?

Regards Gordon

[mar10]

I once made up a simple example with electron and a tree, but not much beyond it (It is only JS so it should work).

2./3.

What is the reason behind the Master-Concept, i.e. why do you move nodes, instead of simply displaying them multiple times?

E.g. why is 'Annie's car' only a child of 'episode2/Scene 2.1' but not 'day1/Scene 2.1'. 

That car seems to be part of both categories (the node is reachable by different paths) and you could view the data from different perspectives by simply expanding either top node?

4.

'Types' is pretty new. Currently it does not even set icons automatically, but allows a simple pattern to do it in the 'icon' callback.

Adding common classes may be a good candidate as well.

5.

The clones extensions does not much more than appending a <span> tag in the nodeRenderTitle callback, so this may be a solution.

[gordon.a...@gmail.com]

Master/Slave and/or Collapse/Expand

Spreadsheet aggregation. We currently use OmniOutliner's ability to sum up a tree to produce cost estimates as we break things down. If all the clones with the same refKey in my network are equal, then the leaf costs would be added into the root total many times. I need some way of saying this is the currently important tree for summing purposes.

Cycles in the graph. My particular use case is probably a directed acyclic graph - ie no cycles. But 1. I am not sure in all the copy/paste/drag/drop/master/clone scenarios that I would be able to prevent cycles being generated, and 2. Why not just allow them anyway. In my master/slave scheme, if a clone appears beneath itself, then in any chosen tree, the first encountered clone will be the master (with children), and the lower ones slaves. Without master/slave I would need some other indicator that a clone was unsafe to descend.

The master/slave distinction seems to satisfy both these scenarios, but I can see that it is an additional concept for the end user to grasp, and that displaying children everywhere might more familiar. I would still need an indicator for the spreadsheet preference, and an analysis of where its unsafe to display children to prevent cycles.

I still lean towards a single mechanism Master/Slave to solve both these issues, but I can see what you suggest is achievable and may be easier to grasp. I'll keep thinking about which solution feels best for the end user. Neither solution will affect what I store persistently - its just nodes and links.

Gordon Jones

[gordon.a...@gmail.com]

> 2./3.
> What is the reason behind the Master-Concept, i.e. why do you move nodes, instead of simply displaying them multiple times?

And another thing....

If I did as you suggest and display the children under each of the clone nodes, I assume I would also have to recursively clone the children. (I am guessing a fancyTreeNode cannot have >1 DOM representation?)

If thats true, I don't like it - I should only have clones when the underlying network results in a node with multiple parents. To artificially clone the children just because they are children of clones seems wrong, and could result in a difficult to untangle mess of real and artificial clones.

Of course I may be wrong, and a fancyTreeNode can have >1 DOM elements... Let me know

[mar10]

No, 'clones' in fancytree are only nodes that happen to have the same `refKey`. The extension adds some convenience methods to find them.

In the DOM they must have distinct <li> tags of course.