[TW5] Another short video of my plugin. showing the filter mechanism

[Felix Küppers]

Hi guys,

just did another video to complete the previous.

http://youtu.be/0QFE54HAzH0

Don't worry, I'm not spamming the forum with videos, I just wanted to give a first impression.

Hope you like the new features.

Regards
Felix

[Danielo Rodríguez]

Absolutely awesome. I cant want to try IT out !!
The menú is with left or right click ?

[Felix Küppers]

Hi Danielo,

thanks :) At the moment the menu is always on top, above the graph. but you can hide it. Also if you save the canvas as png it will be hidden.

I still have to work on the static-graph mode (which was your idea). it has a little bug.

I try my best to publish everything next week at some point.

Regards
Felix

[Danielo Rodríguez]

Thank you for your efforts.

One thing I would love to have is to use your "tool" as a widget. Specifying the source data and so on. I will use it as a mind mapping tool so I need the hability to work with it inside a tiddler in full screen for example. The searching function is cool but I want to go much further.

[Felix Küppers]

You mean that you can create many different mindmap-instances and store them in separate tiddlers as adjustable-images?

This is truly a great idea for a feature Danielo! I want it too :)

I hope you understand that at the moment, I cannot implement everything. It took me already two weeks to build this plugin and I am short of time.

But I will put it on my todo list, ok? - might take a while though.

Regards
Felix

[Danielo Rodríguez]

Yes, that is exactly what I mean. Seriously I think your plugin deserves more than just a search results. It's an awesome way to show information. I totally understand your position, don't worry. I just want to play with it as soon as possible and maybe I can adapt it to my likings.

Regards

[Felix Küppers]

Ok Danielo, you will love this: I just implemented your idea and its so cool man.

http://youtu.be/8kk2FRIyf3o

Until later :)
Felix

[Felix Küppers]

ok the video didn't work. I will upload it again when I am back from sports.

[Danielo Rodríguez]

This is an ugly way of create HYPE :P

[Ed Dixon]

LOL I know right what he has going on here is so cool! We are all waiting like kids on an xbox game release day out side the shop! :)

[Felix Küppers]

Sorry, I am back from sports now.

The hype is justified :) It is cool, big promise:

http://youtu.be/Z-pQ59f7zCc

Regards felix

[Felix Küppers]

By the way. The philosophy here is that a snapshot will be a clone of the current filter used in the editor.

So say you want to create a mindmap or graph exclusively for a group of tiddlers, then you would actually

1. create a group and call it "graph xy" and
2. then begin to tag tiddlers with "graph xy" so they are displayed in the graph
3. and then create relationships

Note: the relationships are also displayed in other snapshots which share the same tiddlers (see minute 2:50 the edge is displayed in to snapshots embedded in the tiddlers "you will like this" and "New Tiddler" because both share the same tiddlers). if you don't want that, create tiddlers that are exclusive to a tag group.

Also note: all manipulation of snapshots is done in the main window.

I tried to cover all aspects with my plugin while keeping things easy :)

Felix

[Felix Küppers]

an addition to the workflow described in my previous post (https://groups.google.com/d/msg/tiddlywiki/Nj885Ia8kGA/RKRlL9id9JQJ)

because in my video (http://youtu.be/Z-pQ59f7zCc), i didn't really show a real workflow, this is how you would do it

[Ed Dixon]

This is so cool! 

[Felix Küppers]

Thanks Ed!

Maybe I can put a demo-wiki online tomorrow after I fixed the remaining bugs. However because I haven't bundled my plugin yet, you will not be able to copy it. It will be finally available after TW 5.1.5 is released.

Regards Felix

[Tobias Beer]

Hey Felix,

It's a really neat plugin for tiddler representation with lots of potential,

There are some things I would perhaps reconsider architecturally, one being the dependency of views on tags.

To me, a graph displays relations between tiddlers and the overhead of having to use dedicated tags to relate tiddlers to a given view seems redundant, to some extend bloating a tiddlers metadata.

So, my train of thought is... Why not keep those "graph views" as a self-contained entities that are independent from the tiddlers they represent, in terms of constraint requirements?

While being able to filter a view based on tags sure is a great feature, I would say that to actually model a view should not (necessarily) depend on a given tag-filter constraint. Why? Because, at some point you eventually may want a whole range of constraints, e.g. field constraints with tiddlers having field x of value y or some value <= z, etc... or a modified date < dadada.

Makes sense?

Perhaps it's a question of how the workflow of modeling a graph is envisioned.

For me it's this:

• define view (name ...may at some point even require an additional category(tag) )
  
• add / create nodes (easily select existing tids)
• add / define edges

So, these questions may be interesting to perhaps (re)consider...

• What defines the set of nodes that are represented in a graph?
  
• Are edges part of a dedicated view?
• In other words, do I want all edges in a view, just because some (tag)filter expression says the corresponding tiddlers match?
• Or: How could I make differentent kinds of edges appear in different views with the same tiddlers?

Right now, I guess, it is all set up such that the nodes and actions are global, i.e. all views share the same underlying node-tree... a paradigm that might be challenged at some point and begging for a solution as to: How can we model independent node-trees on the same tiddler metadata?

- tb

[Felix Küppers]

Hi Tobias,

thanks for the input. Let me answer some points because I think there is a misunderstanding here.

In general: You have to think of nodes like tiddlers and views like tw-filters. It is almost exactly the same.

 

To me, a graph displays relations between tiddlers and the overhead of having to use dedicated tags to relate tiddlers to a given view seems redundant, to some extend bloating a tiddlers metadata.

No metadata at all is stored in any tiddler. Edges are stored in dedicated json-stores for each edge-type and views are merely filters that select all tiddlers that match them. A view here is the exact same thing as a view in SQL or a TW-Filter: it is a precalculated select statement. At the moment, only tags are possible as filter, but I will add more ways to filter the graph. Everytime you create a clone of a view it is a clone of the current filter used.
 

So, my train of thought is... Why not keep those "graph views" as a self-contained entities that are independent from the tiddlers they represent, in terms of constraint requirements?

Yes. This is already the way it works. Imagine a view as a tw-filter which will select the nodes it will contain. No tiddlers are actually linked to a view. They just match a pattern that is specified by a view.
 

While being able to filter a view based on tags sure is a great feature, I would say that to actually model a view should not (necessarily) depend on a given tag-filter constraint. Why? Because, at some point you eventually may want a whole range of constraints, e.g. field constraints with tiddlers having field x of value y or some value <= z, etc... or a modified date < dadada.

Makes sense?

Yes. I totally agree and you will be able to do that. If you open the debug console of your browser and search the through the debug output you will see a line that says "new filter string to accept or reject graph-nodes: [!is[system]!tag[tiddler-relation]tag[todo]!has[draft.of]". It is only at the moment, that the GUI is not done and every view filters by tags. But theoretically I could also provide a textfield where you can manually enter a tw-filter.
 

Perhaps it's a question of how the workflow of modeling a graph is envisioned.

For me it's this:

• define view (name ...may at some point even require an additional category(tag) )
  
• add / create nodes (easily select existing tids)
• add / define edges

 

So, these questions may be interesting to perhaps (re)consider...

• What defines the set of nodes that are represented in a graph?
  

In its core: a tw-filter, as complex as you want it to be.
 

• Are edges part of a dedicated view?

no edges exist between nodes and are only shown if the view permits it. that means: when nodes match the filter or when not filtered by a special "edge-filter".

• In other words, do I want all edges in a view, just because some (tag)filter expression says the corresponding tiddlers match?

You can create use an edge filter that is also stored by each view. Its visible in the gui but not working yet.
 

• Or: How could I make differentent kinds of edges appear in different views with the same tiddlers?

edge filter will be implemented next time :)
 

Right now, I guess, it is all set up such that the nodes and actions are global, i.e. all views share the same underlying node-tree.

No they don't. They just pick the nodes (tiddlers) they want to based on their filter.


Hope that helped.

Please wait with in depth discussions when I published the code.

Regards Felix

[Tobias Beer]

Hi Felix,

Thanks a lot for all the details.

Correct me if I'm wrong, it appears I am indeed questioning the approach where a graph (rather than a view) was to be defined by a filter, which may for the time being be restricted to tags and, as you say, may be extended to allow entering a plaintext filter expression (yay!).

If one wanted to manually edit a graph, this is helpful to get to an initial set of tiddlers that one wants represented in a graph (rather than a view), but one may want to add additional tiddlers that don't correspond to any such filter expression.

In other words, while it is entirely neat to be able to generate dynamic graphs based on filters, perhaps some kind of "free graph" may be an approach to be able to select and add individual tiddlers without any filter constraints to a given graph, in fact, the filter being a means to only temporarily constrain a given graph to a certain subset of it and when removed, to revert to one particular graph, rather than an overall graph of all tiddlers and all edges.

So, the question seems: (How) might it be possible to have different graphs in one and the same wiki? When I say "graph", I am really not talking about a visual representation (view) but about a nodetree, the underlying information structure whereas two independent nodes of two separate graphs may indeed relate to one and the same tiddler.

For example, let's say I have the tids...

• Mum
  
• Dad
  
• Kiddo

I may want one graph for general relationships showing...

Mum => parent of => Kiddo
Dad => parent of => Kiddo
Mum => married to => Dad (btw. can an edge be bidirectional?)

...and another more task oriented graph that shows...

Dad  => brings to school => Kiddo
Mum => brings to bed => Kiddo
Dad => brings flowers for => Mum

I guess there's quite a difference between the filter-based, dynamically generated graph you produce now and a static, manually edited graph, whereas the former feels impressively powerful and dynamic, but the latter is what people may desire just as much, and in fact perhaps more corresponds to a basic capability one would hope to find in representing tiddlers in a graph.

For now, let's say I'm quite curious as to how you might envision / implement node-filtering :-)

- tb

[James]

Hi Felix,
This is really cool!  How could I get it?

[Danielo Rodríguez]

El miércoles, 12 de noviembre de 2014 03:00:04 UTC+1, James escribió:

Hi Felix,
This is really cool!  How could I get it?

This still on beta, so the only way to use it is downloading a full wiki containing it:

http://wkpr.de/hosting/tmp/tw5/ 

[Felix Küppers]

Hi Tobias

I appreciate your thoughts. but it is not possible to change the underlying paradigm i am afraid.

I just want to explain how it can easily be done with the functions I will provide. Consider your example:

For example, let's say I have the tids...

• Mum
  
• Dad
  
• Kiddo

I may want one graph for general relationships showing...

Mum => parent of => Kiddo
Dad => parent of => Kiddo
Mum => married to => Dad (btw. can an edge be bidirectional?)

you create a view (a node view) which filters all tiddlers tagged with "family" and other filter components may be possible.
additionally you will be able to select all the edgetypes via checkboxes in the menubar you want to be displayed: "parent of" and "married to" (this is an edge view)

 

...and another more task oriented graph that shows...

Dad  => brings to school => Kiddo
Mum => brings to bed => Kiddo
Dad => brings flowers for => Mum

you create a view (a node view) which filters all tiddlers tagged with "family" and other filter components may be possible. Additionally you will be able to select all the edgetypes via checkboxes in the menubar you want to be displayed: "brings to school" and "brings flowers for" (this is an edge view)

So it can be done, this is most important. Also this is the most convenient way for me to handle it in the code because it is all based on filters. In a DB you also operate on the same dataset only with different views. Because also in a SQL-DB you only get results via select statements which are views and the underlying pool of records is the same for all views (analog: no nodes belong exclusively to a graph). Otherwise it would be a whole different paradigm that doesn't go well with the view idea. The paradigm I follow is: if you want different results, you need different selects (=filters, views). However there will be at one point the possibility to add single nodes to a view,  which works in the backend like this: filter:title[nodename] and may be represented on a gui level via checkboxes.

Regards Felix

[Felix Küppers]

On Wednesday, November 12, 2014 3:00:04 AM UTC+1, James wrote:

Hi Felix,
This is really cool!  How could I get it?

Hi James. Thanks. Not yet, please wait till the end of the week.

[PMario]

On Tuesday, November 11, 2014 4:00:49 PM UTC+1, Tobias Beer wrote:

To me, a graph displays relations between tiddlers and the overhead of having to use dedicated tags to relate tiddlers to a given view seems redundant, to some extend bloating a tiddlers metadata.

At first, I had the same feeling. But not anymore. ...
As our TWs tend to grow, imo we need to add more tags anyway to group our tiddlers. Since TW filters can be very complex, I think it's possible to have good control.

I actually like the additional (curated) edge layer. It acts like an index and gives as very fast access to tiddlers with a special attribute, without the need, to modify the tiddler.
On the other hand, it introduces some syncing problems for export / import and in an multi user, multi wiki environment. (But we can deal with this later :)

If the TW is new, and has just some tiddlers, actually _no_ tag/filter is needed at all. We can work with the whole set. ... As discussed in the last hangout #68 [2] the vis library even has a clustering function [1]. IMO it's not used at the moment. ... but it's there. So it should be possible to handle a large ammount of tiddlers in the same view, if needed.

There is one more thing, I'm interested in. ... It's the TWs "hidden structure".

By "hidden structure" I mean ... references, tags, tagging. This structure is created automatically, when tiddlers are added and refactored. References may change, if you rewrite tiddler content. Tiddlers may be splitted .... I'd like to make this structure visible. We also did discuss this a little bit at hangout #68. ...

So I'm looking forward to mess with the code ;)


@Felix,
Don't be scared :) by the number of requests that pop up from all sides. That happens if you create something interesting :) As I said in the hangout. Make your core thing work. Extensions can be made later.

IMO it's good that you expose a lot of configuration data as json tiddlers. We can mess around with them.
As Jeremy said, the core may provide easy UI access for those parameters, because the core also needs this feature.
So you can focus on the vis.js stuff at the moment :)

have fun!
mario

[1] http://visjs.org/docs/graph.html#Clustering
[2] https://www.youtube.com/watch?v=H8G6BEQpvW0

[Felix Küppers]

Hi Mario,

Make your core thing work. Extensions can be made later.

yes, I will do that! :)

I hope it's all done by the weekend. I already locked my room and deliberately isolated myself from all my friends and family to finalize the plugin code ;)

Regards
Felix

[Tobias Beer]

Hi Felix,

Thanks for taking the time to explain your concept in detail, much appreciated!

It all sounds very reasonable, especially that...

• it's entirely possible to (later) add individual tiddlers to a filter expression in order to define a view
  
• one will be able to classify / select edge-types and thus model different representations on the same set of tiddlers, thus different graphs

Kudos, Tobias.

[Felix Küppers]

Hi Tobias,
 

Thanks for taking the time to explain your concept in detail, much appreciated!
 

No Problem, I even had similar thoughts when I designed the plugin two few weeks ago. But I had to stick to a design choice eventually. Everything has pros and cons as Mario already put it.
 

It all sounds very reasonable, especially that...

• it's entirely possible to (later) add individual tiddlers to a filter expression in order to define a view
  

Yes, I will make sure you will get the chance.
But at the moment I am foremost struggling with getting the first version published :)

• one will be able to classify / select edge-types and thus model different representations on the same set of tiddlers, thus different graphs

Yes, correct. eventually it is the same thing you are trying to achieve just with different means.

Regards
Felix

[Felix Küppers]

Hi Tobias,

just wanted to tell you that I thought about your remarks and I finally made a choice to allow binding edges exclusively to views. In contrast to nodes, which are regarded as universally unique subjects, which cannot exist twice, a new edge can now be decided to be a "non-exclusive edge" (displayed in every graph/view) or to be an "exlusive-edge", displayed only in a certain view.

That gives you all the freedom to create graphs/views with exclusive/private edges or to use global edges. Additionally, each view allows you with its filter (edge filter) to further filter edges you want to display.

See screenshot for the "creating-edge-dialog".

Felix

[Tobias Beer]

Hi Felix,

Both elegantly done and very well thought out. Kudos! ^_^

(I assume one can, of course, define a number of views actually using the same filter.)

Not sure how much this great plugin is related to tasks, but you could just call it graph, kraph, graf, or kraf (k=küppers, f=felix ;-)

One more thing, it's probably a small thing for you but a big step for humanity, being able to create a tiddler for a view in which one can put some info about what we're seeing in the graph, which perhaps also provides a button to open the graph-viewer with the corresponding view.

Best wishes, Tobias.

[Felix Küppers]

graph, kraph, graf, or kraf (k=küppers, f=felix ;-)

kraf :))))) great metamorphosis of the word graph! ...but doesn't it sound kinda stereotypic, I mean a german (we are often called "krauts") created a tool called "kraf" :)
 

One more thing, it's probably a small thing for you but a big step for humanity, being able to create a tiddler for a view in which one can put some info about what we're seeing in the graph, which perhaps also provides a button to open the graph-viewer with the corresponding view.

I put it on my list.

Maybe not inside a node but I think I can allow a view to have a description or title that is displayed above and also a small button that contains a reference to edit the view.

By the way I created a remember choice button (screenshot) so when you are working on a view/graph with only isolated edges you do not have to select your view all the time.

Regards
Felix

[Danielo Rodríguez]

Can I try this new awesome features¿?

[PMario]

On Thursday, November 20, 2014 3:23:52 PM UTC+1, Felix Küppers wrote:

By the way I created a remember choice button (screenshot) so when you are working on a view/graph with only isolated edges you do not have to select your view all the time.

It would be nice, to have a default setting for the edge type, so it would remove the dialog, needed to specify the edge, and make the workflow faster.
-m

[Felix Küppers]

Hi Danielo,

I have problems with the graph being properly displayed at the moment (the nodes overlap at the beginning in the center) but you can see a demo here
http://wkpr.de/hosting/tmp/tw5/

Regards Felix

[Felix Küppers]

Hi Mario,

I can create a default for the edgetype but I can never remove the dialog as you then have no other possibility to change the edgetype..

Regards Felix

[Tobias Beer]

http://wkpr.de/hosting/tmp/tw5/
Regards Felix

Felix, this stuff is just amazing! It's going to rock this house. ^_^ 

Best wishes, Tobias.