Lets talk about tables.

[HC Haase]

Lets talk about tables.

Lately there has been a focus on tables  of tiddlers and there fields (not static wikitext tables) and some great plugins has been developed. Lets talk about the what, the why and the future of tables in tiddlywiki

What tables

when we look at tiddler-tabels, i.e. tables that list tiddlers, not wikitext tables, I find it a bit confusing with the many options. I see three great plug-ins at the moment.

• fields2table by Mat: simple and quick to use, in-table edits. I like this because it is simple an fast to use and the inline editing is realy useful. The way you specify what fields to use as columns without editing the tiddler in edit mode is also nice.
  
• Tiddlytables by Alan Aldrich : very extensive, lots of options and nice GUI, beautiful tables. However for me this is overkill as I only have 5 tables.
  
• Dynamic tables as part of Shiraz by Mohammad: this has all the best from tiddlytable (that it is derived from), but keeps it more simple. beautiful tables, many cosmetic customisations, sort by column click (very useful) and it also has inline editing.

There is a trade of here between easy of use and features. Starting from fields2tabel it is simple fast and easy to edit, bam here is your table. Dynamic table a bit more to understand a bit of a barrier to get going but much more beautiful and realy handy with column sorting. Tiddlytable has, for my usecase way to much going on and to much learning, but looks realy powerful so if I had a greater need for tables this might be good.

Why tables

For some tiddlers like my index of books to read, I use a handful of fields in them. For those kind of tiddlers, tables are very handy to sort by different criteria (year, genre etc.) and it gives a good overview.

However I am not sure if it breaks with the TW-philosophy of the tiddler as the smallest unit and tags as the main organiser. Like TW has changed the way I think/interact with knowledge, I think these tables have done it too. when using tables I have a mindset of spreadsheets/DB. I tend to seek to find and overview, to fill missing fields, where the "wiki mindset" gives me a mindset of seeking connections and mental maps. I am not saying tables a bad but I think they change the room for doing an situating knowledge. Or is it just reinventing spreadsheets? that's worth pondering on.

Future use and development

The main organising mechanism in TW is filtered list of different kinds (list-link, toc, tags). Tables is in essence also just a filtered list that is showing more data. You can (and I have in the past) use the list macro, to make a list showing many fields of its entries. As an example I have a list of users in my company that shows there name, what special software they have and version no. What these tables plugin do different, is making it easier to make "list with fields showing", fast to sort ( and shift) by different criteria, and fast to edit field values in bulk. This is very useful.

So this is my proposal.

Imaging that you make a standard TW list, and at the top left corner (or elsewhere) there is a small toggle button. table view on/of. with this you get a table view of your list with all fields present for the filtered tiddlers. You can do some edits and turn list view back on, or you can leave the table view on if it is more appropriate for what you are doing. Or you could use it to get an overview, make a quick sort by the relevant field and turn back to list again.

I think this could be a good way to hop between wiki mindset and database mindset (as commented above) and give the flexibility to use the best of two worlds.

what do you think. would it be something worth perusing?

unfortunately I would not be the right person to implement his idea.

[TiddlyTweeter]

Ciao HC,

Very interesting post!

I think this could be a good way to hop between wiki mindset and database mindset

I'm thinking about it.

I would say that TW is NOT only a "wiki" in the normal sense of that term.

This is a case that kinda brings that into the foreground more? Its potential flex is VERY unusual.

My 2 cents

TT

HC Haase wrote:

Lets talk about tables.

Lately there has been a focus on tables  of tiddlers and there fields (not static wikitext tables) and some great plugins has been developed. Lets talk about the what, the why and the future of tables in tiddlywiki

<snip> 

[TonyM]

HC,

To me a table is as fundamental as a list, I would like to see table handling to move into the standard distribution. In TWC the ForEachTiddler plugin was my most used plugin which was effectively the List Widget and Table tool.

Perhaps a macro or variation on the list widget, I wonder if this can be achieved with the template parameter in the list? Then you could introduce a toggle that presents the list OR Table view that you suggest.

Regards

Tony

[Alan Aldrich]

HC,

As someone who's daily job is overwhelmed with tabular data in the form of spreadsheets and relational databases, I believe I know where you are coming from. TiddlyWiki's non-linear philosophy is part of what makes it so great. TiddlyTables was supposed to be more of a "query" tool than a "table builder" but TiddlyWiki's unique data structure lends itself to the best of both worlds. The advantages of non-linear data come in the form of efficient memory utilization and the ability for an element to exist at multiple levels of a hierarchical structure, but at the expense of a more complex structure that is generally more difficult to traverse. Regardless of how data is stored, humans love simple and "readable" information, and further, most seem to be especially comfortable in a spreadsheet format - hence the overuse of excel. There will always be a need to visualize linear "chunks" of non-linear data though. Think of it this way. you could create a TiddlyMap of your entire wiki with every field represented and edges representing all of their links and, if it doesn't crash your browser, you will likely see a useless spiderweb of unreadable nodes and lines. This is a purely non-linear representation of a non-linear database. On the other hand, you can export an entire wiki to a csv file, a spreadsheet. and likewise the data is practically useless in its raw form, mostly because it is 90% empty cells. Those empty cells are the wasted efficiencies of a non-linear database converted to a linear one. Luckily, TiddlyWiki is unique in that everything is represented in tiddlers which are represented by name:value pairs. So getting back to the "best of both worlds", creating a simple table in TiddlyWiki is a linear "chunk" of a non-linear database, which is an inevitable part of visualizing data. In the plugins you mention, editing a table only creates data in one field at a time therefore the non-linear philosophy is maintained. In my plugin, the expand feature was intended to make the tables less linear to hopefully avoid "spreadsheet overload". The idea is you can set up multiple tables to drill-down or traverse through rather than a bunch of separate tables or tables with vague links between them. 

That said, I think you make a good point, and we should always remind ourselves to stay true to the spirit of TiddlyWIki. Certainly TiddlyTables is too complex in its current form to be implemented as a standard feature, but it was a general lack of basic functionality that led me to build it. That is, the need for a "query builder". I found myself using the advanced search filter a lot and became familiar with the filter syntax, but I felt the filter search was missing one thing, columns. I would type in a filter that returned 15 titles and then wanted to know their values for "tags" or "modified". I would often create a quick unstructured table using the list widget but soon started working on a plugin. To answer your proposal, let me share my idea for a simple table/query builder that could/should become part of the core:

In advanced search under the filter tab, instead of just one text box, two. One to query a list of tiddler titles (filter format) and another to query fields/indexes (also filter format). If the second box is blank it defaults to a simple list of tiddler titles, if not it returns a table. You could even add a button to turn that query into a tiddler with the table rendered, but that isn't necessarily required. I think it would be a very useful feature and would expand the practicality of the advanced search.

I told myself I would keep this short. Sorry,

Alan

[Mohammad]

In advanced search under the filter tab, instead of just one text box, two. One to query a list of tiddler titles (filter format) and another to query fields/indexes (also filter format). If the second box is blank it defaults to a simple list of tiddler titles, if not it returns a table. You could even add a button to turn that query into a tiddler with the table rendered, but that isn't necessarily required. I think it would be a very useful feature and would expand the practicality of the advanced search.

This is great idea! Many times you search for a word and it is in different fields of tiddler and when you see a bare list of title, it just confusing and uncomfortable!

As you and Tony said, tables are a forgotten part of list and it is beset to have some simple form in the standard distribution.

Also when you come to bunch of data visualization tool, query tool are of high importance!

By the way, what you built in TiddlyTables shows how to create a solid and powerful app using modular extensible approach.

I hope we could see more contribution from your side!

--Mohammad

[TonyM]

Tables may appear to be structured, but are they if they have one or diverse members, or exist to discover relationships.

It is important to recognise the interplay of non linear and linear information. I can collect information of any nature or detail in a non linear datastore. However as we start to learn more we create informal structures. We identify and record relationships and we see opportunities to fill gaps discover new sets. We can Actualy create linear views of non linear data often to identify missing data or ensure comprehensive data sets. The value of non linearity can remain even as we start to uncover similarities, differences and build consistency.

Tiddlywiki is open to almost any structure but its strength starts with its ability to be open before adding, not imposing structure in parallelle.

There is a danger to imposing structure too early durring discovery but finding structure in the unstructured is often the ultimate value that can be found in the unstructured real world.

Just some thoughts
Tony

[PMario]

On Wednesday, December 11, 2019 at 10:23:03 AM UTC+1, HC Haase wrote:

..

However I am not sure if it breaks with the TW-philosophy of the tiddler as the smallest unit and tags as the main organiser.

I think "tags as the main organiser" is only true, because it has the best UX (user experience).

With TW5 Jeremy tried to make fields "first class citizens". And if you have a look at the filter syntax, the goal has been reached but the existing UI elements to work with fields are cumbersome.

 

Like TW has changed the way I think/interact with knowledge, I think these tables have done it too. when using tables I have a mindset of spreadsheets/DB. I tend to seek to find and overview, to fill missing fields,

right

 

where the "wiki mindset" gives me a mindset of seeking connections and mental maps.

:) I'd love to have a "stripped down version of TiddlyMaps" for brainstorming.

 

I am not saying tables a bad but I think they change the room for doing an situating knowledge. Or is it just reinventing spreadsheets? that's worth pondering on.

It's a "different" type of view, which can be better, depending on the usecase.

just some thoughts

mario

[PMario]

On Wednesday, December 11, 2019 at 10:23:03 AM UTC+1, HC Haase wrote:

...

So this is my proposal.

Imaging that you make a standard TW list, and at the top left corner (or elsewhere) there is a small toggle button. table view on/of. with this you get a table view of your list with all fields present for the filtered tiddlers. You can do some edits and turn list view back on, or you can leave the table view on if it is more appropriate for what you are doing. Or you could use it to get an overview, make a quick sort by the relevant field and turn back to list again.

I think this could be a good way to hop between wiki mindset and database mindset (as commented above) and give the flexibility to use the best of two worlds.

what do you think. would it be something worth perusing?

I think it's possible, because at the moment, most "list representations" only use 30% of the list-widget possibilities.

Even the existing TW UI only uses 80% because some core elements where only developed far enough to reach those 80%. ... Because that is good enough and it served us well.

But from time to time, there seems to be something missing. ... And I think your post names exactly that.

There is some brainstorming about navigation and improving the story list handling going on at GitHub.

I think, improving the "story list handling" and "navigation per list" will bring us to the point where it should be easy to implement a behaviour that you described.

I think every list is a "little story" and it should be possible to interact with it in a way similar as we do now with the sidebar and the main story river.

just some thoughts

mario

[Mohammad]

Tables give comfort and readability when you work on bunch of tiddlers.

Tiddlywiki core uses itself tables in several cases like

Tag Manager

https://tiddlywiki.com/prerelease/#%24%3A%2FTagManager

Tiddler Manager

https://tiddlywiki.com/prerelease/#%24%3A%2FManager

I think dynamic tables are just another representation for list output! $list has template parameter and customizing it can produce tables.

--Mohammad

[TonyM]

Folks,

In the same vein as lists, tables we can add toc's. If we wanted to be complete the ability to generate multi-level sort lists could first be introduced to tables but with clever design we can strip away the table "Chrome (headers, rows and lines)" and provide indented lists, or heading level lists all with select filters, sortable columns/rows sublists.  Basically with repeated values down the table only show the first.

Other features would be to 

• provide table paging, set the number of rows or columns and slidebars or page numbers for outside that.
• And printable tables with repeating headers and page breaks (HTML features already support this)

Ideally such filter/row column or Filter/tiddler/field lists could also be represented by other graphical views as well. We just need a kind of reports engine which can have alternative view templates.

I built a substantial columns tool (unpublished), which is mostly superseded with the tiddlytables solution, which made this approach obvious sooner or later.  It is Quite suited to a plugin that provides the architecture.

To keep this somewhat simple focusing primarily on tiddlers as rows and fields as attributes/columns would be a reasonable compromise initially.

Regards

Tony

[bimlas]

HC Haase,

Imaging that you make a standard TW list, and at the top left corner (or elsewhere) there is a small toggle button. table view on/of. with this you get a table view of your list with all fields present for the filtered tiddlers. You can do some edits and turn list view back on, or you can leave the table view on if it is more appropriate for what you are doing. Or you could use it to get an overview, make a quick sort by the relevant field and turn back to list again. 

I think this could be a good way to hop between wiki mindset and database mindset (as commented above) and give the flexibility to use the best of two worlds.

You may be interested in this: https://groups.google.com/forum/#!topic/tiddlywiki/Y6tbO_vnKmw