To structure or not to structure? Depends, eh?

[Charlie Veniot]

In my latest "brain-age" game (Coding Fun: My take on recipe ingredients), I've gone all-in with structured data.

(Aside: I tend to prefer using data tiddlers over fields, but that's the kind of conversation that deserves its own thread.)

Although structured data is very cool, I usually much prefer the loosey-goosey unstructured data.

Like just about all things, which is better (structured or unstructured)

• it depends

Structured data involves big effort up front, but with substantial benefits later.

• However, structure done wrong (big analysis up front did not consider some things until elucidation happened while knee-deep in the thick of it) can involve big effort re-jigging things if "quickly adjustable re-design" wasn't built it.  (Maintaining documentation, even if just bread-crumbs, makes a re-jigging effort so much easier, but even maintaining bread-crumbs can be some effort.)
• Building structure for possible future needs that never happen, that makes big effort up-front not so pretty re the cost-benefit ratio

Unstructured data involves little effort up-front (immediate benefit), but could require big effort later: i.e. having to move all of that unstructured data into fields when structure is needed.

Way too many thoughts about it all to write here.  I'd need a dedicated TiddlyWiki.

All of that to say that my "brain-age" game of structured recipe ingredients may turn into an expanded game that pits structured recipe ingredients head-to-head with unstructured ingredients.

Proof in the pudding, advantages and disadvantages to both, maybe some trickery.

Maybe via a shared TiddlyWiki running on nodejs, on a virtual machine, if anybody is interested.  I do have, I think, enough credit in my Google Compute Engine to setup a virtual machine for some collaborative "brain-age" structured vs unstructured recipe tomfoolery for a couple of months...

[TW Tones]

Charlie,

There is in fact a middle way between structured and unstructured. An example would be if you were building a contact database and when it came to meeting extended family at holiday times and asked them for their phone number, you also noted down their parents names. You could even record there children's names and more but if you only recorded their parents names this would be fine. What then happens is over time as you speak to each member of the family and get their parents name the family tree hierarchy simply "emerges" from the details.

You can see here that in the above example we have established that a hierarchy exists in the real world and ensure we simply collect enough information each time we talk to someone "Their parents" that the hierarchy builds over time. Such hierarchies need to tolerate missing information, but they can actually help us discover what information is missing, Which we can then seek.

There are plenty of hierarchies that exist in the real world that almost need not be stated like family trees and

earth > Country > state > county > town > street > number 

If one assumes these exist in the first place, it informs us of what it takes to get a full address, but a fuzzy hierarchy and tolerance for missing information. for example you may only record a state/town for where a cousin lives, you can assume the planet, country and county and perhaps for now live without knowing street and number.

The thing is by being aware of hierarchies that exist or you discover, and accounting for there existence, but not "slavishly" trying to build them, these hierarchies' just emerge from the shadows over time. In many ways this helps the unstructured data trend towards more complete information over time.

To me this is where an unstructured database can exist, in such a way that overtime, the obvious, but even hidden structures start to emerge. And you see here there is not problem having both at once. In fact within our unstructured database there will be other emerging structures like lists, tables, networks and common attributes or values. For example, if someone has the "same home phone number" (land line) as another person, perhaps they live at the same address? We may learn they live together, even although we don't have their address (however we have the phone number which we can and ask for the address).

This ability for tiddlywiki to accommodate the unstructured through to multiple and incomplete structures is, I believe, one of tiddlywiki's key attributes that can empower its application universally.

Regards

Tones

[Charlie Veniot]

Well, yeah.  I live in the gray.

Everything you're talking about, that was the whole point of this little project I had in mind:   pit the two extremes against each other, and then drop in everything in between.

It is what I hope my (subsequently posted) "Collaborative Recipes TiddlyWiki (Case Study)" winds up showing for reals.  A large variety of ways to enter/manage recipes (ingredients etc.) with various degrees (a whole spectrum?) of structured data and organization, with the added fun of finding ways of intertwingling information structured very differently together.

And then see whether or not various ways start converging.

Even if they don't an exercise of combining the structured and the unstructured might involve some interesting filtering, and I do love filtering.

Along with a practical showcase of tiddlywiki ways of thinking.

[Charlie Veniot]

Crap.  Forgot to say: your post is a damned fine contribution to the knowledge base.

On Monday, October 4, 2021 at 9:13:29 PM UTC-3 TW Tones wrote:

[Charlie Veniot]

Anybody interested in this kind of stuff, I am influenced by (re software development, database development, and "intertwingularity mapping"):

• Scott Ambler.
  • Scott Ambler's Home Page
  • Agile Modeling
  • Agile/Lean Documentation

• Ted Nelson.
  • Ted Nelson's YouTube Channel
  • Wikipedia's Ted Nelson Article
  • Intertwingled: The Work and Influence of Ted Nelson (Internet Archive)

• Open Knowledge Foundation
  • Home Page
  • The Four Principles of (Open) Knowledge Development
  • What Do We Mean by Componentization (for Knowledge)?

• Software Development.
  • Agile Software Development
  • Lean Software Development
  • Object-oriented programming
  • Rapid Application Development

(I won't get into the various Zen Buddhism and Tribal Wisdom stories/sayings/etc. I rather like and funny jokes/stories/analogies I've read over the years.)

Aside:  Intertwingularity Mapping (a work in progress that has been gathering dust.)

[TW Tones]

Thanks Charlie,

But thanks to your inspiration for raising the the "conceptual issue", in a way it allowed me to state my thinking on the subject. 

Ideas, I feel I have failed to express so far.

Tones

[Charlie Veniot]

Pff, it is stuff that has been swirling and expanding in my head since the early 90's and I still can't coherently spell it out.

You've got it down to an art-form from my perspective.  I am frigging envious.