Fields as First class citizens?

[TonyM]

Folks,

Perhaps I am off track, but I may as well voice observations during my learning process, that may have a baring on future enhancements. As I learn about filters and fields I was using the advanced search to investigate the filter [fields[]] which lists the names of all fields in the tiddlywiki. Of note is the list of links generated lend themselves to creating tiddlers using each fields name.

The parallel you would all be familiar with is tags and how you can create tiddlers for each tag. Usefully then you can see references, tagged, list and listed for that tag. In fact tag tiddlers are a great place to build logic or views of your tiddlywiki by that tag, for example a References tag, open the references tiddler and see a list of all references...

So, I thought, what if fields are treated the same?, any field-name can become a tiddler (already a fact), But what if we take this "feature" further and allow default or possible values to be placed in a fields corresponding tiddler, on that tiddlers fields and supplement the Info button with field references? This could betaken further to define a fields qualities even further such as allowable formats eg: number only, 2 decimal places etc...

As I discovered earlier it is actually practical to store in existing tiddler fields, macro names, variable names and more so such an approach to make fields a first class citizen could open up a universe of possibilities further strengthening tiddly-wikis universal and Quine nature.

Just food for thought
Tony

[codacoder...@outlook.com]

If "everything is a tiddler", this makes a lot of sense.

+1

[Mat]

I would also like for this (actually, I've even made this suggestion myself, but I fail to find where). 

IMO tiddlers, as they're currently manifested, are not general enough to be "the smallest semantically meaningful unit". IMO, what is "the smallest" depends on context - but at the very atomic level it must be the field.

But this does present a (solvable) problem; A field is a <name>:<value> ...but that would mean we can only use one instance of each fieldname. Or we'd need some other identification to differentiate between foo:bar and foo:frotz.... so then the minimum entity would not be <name>:<value> but instead:

<id>:[<name>:<value>]

I would very much like to see this as a first class citizen. Of course, the typical 'end-user use' may then be to deal with a complex unit of these that is identical to the current tiddler manifestation. But not always! Especially not for when you just want a single value container.

...which is related to a recent thread here where the concept of tiddlers as variables was discussed. 

This should make for very streamlined wikis.

<:-)

[codacoder...@outlook.com]

On Saturday, September 9, 2017 at 5:14:29 AM UTC-5, Mat wrote:

I would also like for this (actually, I've even made this suggestion myself, but I fail to find where). 

IMO tiddlers, as they're currently manifested, are not general enough to be "the smallest semantically meaningful unit". IMO, what is "the smallest" depends on context - but at the very atomic level it must be the field.

Right.  Tiddlers are objects.

[TonyM]

Mat,

Hopefully this is not classed as a rant, some thoughts.

I am inclined to keep the fieldname and value separate. Like in a standard database, a field is really just a column title, and the instances of it are the values associated with a key. Tiddlers are addressable by there unique name (a key), even if a field-name is used once, to access its its instance we needs to identify the tiddler title to which it belongs. To me this makes the tiddler remain the "atom" of tiddlywiki, fields are the sub-atomic particles.

If there is a field anywhere (and everywhere) in the wiki called "fieldname" then it is fine to have a single tiddler that describes it. If someone chooses to use fieldname in one set of tiddlers to mean X and in another to mean Y then they are free to do so. They could even note in the fieldname tiddler that fieldname has two or more uses.  Extending the Info page to tell you in which tiddlers the field is used and or has a value will help.

In keeping with the field concept I think data tiddlers can be considered external fields, they define a field, its instances(index name is the key not the tiddler name) and values all in a single tiddler, they are more like a global field not to be confused with others such as Text/Title/Tags so by thinking of them as external fields (for want of a better name) could help. Actually it could be helpful if we (users) could define a global field like text and Tags in a tiddlywiki that exists in every tiddler like changecount (from TWC).

However there is a good argument that maintaining a SINGLE tiddler for each fieldname, and each field name for a documented purpose, is a way of supporting a good naming standard, avoid unexpected results, support self documentation practices and more.

Never the less a Tiddler can be a tiddler with content, including containing text/code (macro) or references to other tiddlers, be a tag and a Field definition all at once. And as in the discussion you linked to, also a variable or container for a variable. It is with this possibility in mind that perhaps a state tiddler should not populate the "text" field but a "value" field in the state tiddler. Although using text should remain valid.

Of course if we extend the "every field is also a tiddler" idea and for example provide in a field tiddlers a "list field" the "possible values" for that field or some input value tests eg a number between 1 and 100 then users would be more inclined to use a field name more consistently (but are not forced to). The same can be said for variables, if a tiddler exists with the name of a variable we can choose to look in its "value" field, then its "text" field, and look in a "list" field for possible values and the "allowed" field for valid values or a validation script, even if the currentValue exists within the widget tree.

Thanks for the reference to the variables discussion. What you and Jeremy says has great value as well. Personally I do not yet grasp the whole widget tree model, but understand its importance.

It seems to me that there are things that vary such as field values and state tiddlers and variables that are contextual within multiple transclusions and the widget tree.

If we are to simplify the syntax I think we may need to start differentiating here because there are constants that can change, things that vary and things that vary within the tree. In someways "currentTiddler" causes a problem because there is a current tiddler "Displayed" and a changing "currentTiddler" as a result of a filter/list. Sometimes the current tiddler is not even a tiddler, or only a potential tiddler. Because I am keen for tiddlywiki to undergo reform not revolution if we keep currenttiddler and introduce a value "displayTiddler" for the parent that is static in the tree below, a templateTiddler when specified for the displayTiddler it would have helped me learn and do things sooner.

Perhaps the whole widget tree idea needs to be made clearer, perhaps by providing tools to navigate it, not through understanding the code but tools provided that help people visualise and tap into it, perhaps even a debug mode so we can see its value changing, Inheritance is is a good example. It is all fine having a many to many quine but a lot in tiddlywiki and the real world uses hierarchies and these should be more available, visible, understandable and reusable.

My 2cents (or more) again.

Tony

[Mat]

Tony,

I'm unsure if what your last post describe is as things currently are or as you'd want them to be.

But it seems your vision is not to have fields be separate entities like I envision them. But if you're happy to always see them integrated in tiddlers, then what do you mean by first class citizens? You can access a field value from anywhere with {{tiddler!!field}} and you can set fields and remove them.

Warning:

In the following I hope it is OK that I also rant a bit and clarify my thoughts/opinions, not as an attempt to hijack your thread but rather to bounce ideas:

Like in a standard database, a field is really just a column title, and the instances of it are the values associated with a key. Tiddlers are addressable by there unique name (a key), even if a field-name is used once, to access its its instance we needs to identify the tiddler title to which it belongs. To me this makes the tiddler remain the "atom" of tiddlywiki, fields are the sub-atomic particles.

I did not know this. Very interesting view of it, or maybe it is not even a view of it but how it actually is implemented. Either way, very interesting. 

My vision is then to rather make the cells be the atomic unit, with the row+column ID equivalent to tiddlertitle+fieldname. I.e an instance of such a unit could be something like "MyCar.Color:blue", where "MyCar.Color" is the unique ID. This unit should be independent - for example it should be exportable to other TWs. 

I'll here refer to these cells as fids ("field tiddlers") from lack of a better name. Tiddlers would still have an important role but they are complex entities consisting of fids, but they would probably differ in implementation depending on context/need. (Tiddlers in a notebook TW are probably different from tiddlers in a spreadsheet type TW.) 

..but really, the tiddler.fieldname as ID is just a special case. It is two dimensional. The ID could consist of any number of parameters that would also serve as fid kinship (categorization/belongness). For example MyCar.Color.MyWifesCar:blue  I intentionally did not use a hierarchical example here, like Vehicle.MyCar.Color:blue because I do not mean a hierarcy. The ID would be more comparable to tags! (OK, probably throw in a unique hash number also.)

So another fid might be Foobar.Color.MyWifesCar:whatever  and the ID shows clearly shows the partial kinship with the previous fid. 

The point is that, just like tiddlers, these would be independent units that are still linked to one another. They would be as targetable as cells in a speadsheet but in no way locked in like they are in spreadsheets. IMO this is pure tiddler philosophy. Not that we should be dogmatic about "tiddler philosophy" - I think just think "minimal chunks of meaningful content to create bigger narratives" is a brilliant strongly practical ideal.)

Likely, the most popular use of such fids would be to group tids into tiddlers as they are currently expressed. But not always. :-)

If there is a field anywhere (and everywhere) in the wiki called "fieldname" then it is fine to have a single tiddler that describes it. If someone chooses to use fieldname in one set of tiddlers to mean X and in another to mean Y then they are free to do so. They could even note in the fieldname tiddler that fieldname has two or more uses.  Extending the Info page to tell you in which tiddlers the field is used and or has a value will help.

In keeping with the field concept I think data tiddlers can be considered external fields,

Very interesting!!!

IMO, while data tiddlers are pretty concise they are still complex in that they list several more or less independent entries. IMO it would be truer to tiddler philosophy if data tiddlers really were lists of fids, comparable to lists of tiddlers.

Side note: it would be pretty cool to create fids by manually typing a data tiddler type of list, e.g like so:

tiddlertitle:Foo

text:

foo1:bar1

foo2:bar2

foo3:bar3

...and, on tiddler save, these fids are created (because they're fully defined). E.g number 2 fid would be Foo.foo2:bar2 ...or whatever the syntax would be - the main point is that this mini-thing defines the entire fid.

Recreating the list would be even simpler with something like {{{ [fid[Foo]] }}} ...or with focus on any of the other.identifying.kinship.attributes.that.it.may.have.

...but that was a digress.

Back to your thoughts.

 

they define a field, its instances(index name is the key not the tiddler name) and values all in a single tiddler, they are more like a global field not to be confused with others such as Text/Title/Tags so by thinking of them as external fields (for want of a better name) could help. Actually it could be helpful if we (users) could define a global field like text and Tags in a tiddlywiki that exists in every tiddler like changecount (from TWC).

What role/purpose would this serve in your vision? (I can, however, see it as part of 'fids' to serve as a part of the ID)

However there is a good argument that maintaining a SINGLE tiddler for each fieldname, and each field name for a documented purpose, is a way of supporting a good naming standard, avoid unexpected results, support self documentation practices and more.

Never the less a Tiddler can be a tiddler with content, including containing text/code (macro) or references to other tiddlers, be a tag and a Field definition all at once. And as in the discussion you linked to, also a variable or container for a variable. It is with this possibility in mind that perhaps a state tiddler should not populate the "text" field but a "value" field in the state tiddler. Although using text should remain valid.

Not quite sure what the problem is here. I must assume both the text fild and other fields are "just fields" even if with different properties. It may be aesthetically unpleasing to use the text field for states... but other than that, I don't know.

...

 

If we are to simplify the syntax I think we may need to start differentiating here because there are constants that can change, things that vary and things that vary within the tree. In someways "currentTiddler" causes a problem because there is a current tiddler "Displayed" and a changing "currentTiddler" as a result of a filter/list. Sometimes the current tiddler is not even a tiddler, or only a potential tiddler. Because I am keen for tiddlywiki to undergo reform not revolution if we keep currenttiddler and introduce a value "displayTiddler" for the parent that is static in the tree below, a templateTiddler when specified for the displayTiddler it would have helped me learn and do things sooner.

There is much consideration for backward compatibility. Pros/cons with that approach. I'd spontaneously guess that it is not possible to introduce a "displayTiddler" other than perhaps as some alias to currentTiddler. I also think that revolutions (to use your term...by which I assume you mean a total rework, or perhaps a fork) comes when there is enough tension between current status and identified/solvable improvement that it feels justified. The aforementioned backward compatibility approach, probably hastens this process. (Obviously it also takes super skills to be able to pull it off...)

 

Perhaps the whole widget tree idea needs to be made clearer, perhaps by providing tools to navigate it, not through understanding the code but tools provided that help people visualise and tap into it, perhaps even a debug mode so we can see its value changing, Inheritance is is a good example. It is all fine having a many to many quine but a lot in tiddlywiki and the real world uses hierarchies and these should be more available, visible, understandable and reusable.

Considering that I barely know anything about how the widget tree works, I'd like this... but maybe it is more a js matter?

<:-)

[TonyM]

Mat

On Tuesday, September 12, 2017 at 5:33:53 AM UTC+10, Mat wrote:

Tony,

I'm unsure if what your last post describe is as things currently are or as you'd want them to be.

Its a bit of both because it is a combination of what is in place, the way we describe, promote and explain fields, and also suggests some "minor" changes to support a more holistic or structured meaning for fields. The major part is providing a way to think about and use fields.

 

But it seems your vision is not to have fields be separate entities like I envision them. But if you're happy to always see them integrated in tiddlers, then what do you mean by first class citizens? You can access a field value from anywhere with {{tiddler!!field}} and you can set fields and remove them.

Let me me define 1st class by comparison. Tags are first class citizens. They have tools to select, tools to colour, filters, tags can be tiddlers, they can be nested. Fields have slightly different qualities and need a subset of features to make them also 1st Class citizens. Sure field shave many of these but they are largely left to the user.

 

Warning:

In the following I hope it is OK that I also rant a bit and clarify my thoughts/opinions, not as an attempt to hijack your thread but rather to bounce ideas:

Like in a standard database, a field is really just a column title, and the instances of it are the values associated with a key. Tiddlers are addressable by there unique name (a key), even if a field-name is used once, to access its its instance we needs to identify the tiddler title to which it belongs. To me this makes the tiddler remain the "atom" of tiddlywiki, fields are the sub-atomic particles.

I did not know this. Very interesting view of it, or maybe it is not even a view of it but how it actually is implemented. Either way, very interesting. 

You may benefit delving into a little bit on database design principals. Although I am happy to discuss them. 

 

My vision is then to rather make the cells be the atomic unit, with the row+column ID equivalent to tiddlertitle+fieldname. I.e an instance of such a unit could be something like "MyCar.Color:blue", where "MyCar.Color" is the unique ID. This unit should be independent - for example it should be exportable to other TWs. 

In database terms it seems what you are asking for here is a "Compound key", whilst they serve as a useful way to specify a reference database design principals suggest this should not be the underlying way to store the information. I am not sure what you are trying to achieve here, please explain further, not how but why!

 

I'll here refer to these cells as fids ("field tiddlers") from lack of a better name. Tiddlers would still have an important role but they are complex entities consisting of fids, but they would probably differ in implementation depending on context/need. (Tiddlers in a notebook TW are probably different from tiddlers in a spreadsheet type TW.) 

True - this is why we must try and keep the underlying structure "Pure", maybe your Fids should be a composite element, that lays on top of tiddlywiki when needed. Not that I am yet sure why.

..but really, the tiddler.fieldname as ID is just a special case. It is two dimensional. The ID could consist of any number of parameters that would also serve as fid kinship (categorization/belongness). For example MyCar.Color.MyWifesCar:blue  I intentionally did not use a hierarchical example here, like Vehicle.MyCar.Color:blue because I do not mean a hierarcy. The ID would be more comparable to tags! (OK, probably throw in a unique hash number also.)

So another fid might be Foobar.Color.MyWifesCar:whatever  and the ID shows clearly shows the partial kinship with the previous fid. 

The point is that, just like tiddlers, these would be independent units that are still linked to one another. They would be as targetable as cells in a speadsheet but in no way locked in like they are in spreadsheets. IMO this is pure tiddler philosophy. Not that we should be dogmatic about "tiddler philosophy" - I think just think "minimal chunks of meaningful content to create bigger narratives" is a brilliant strongly practical ideal.)

I think I understand what you are saying, and you are imagining some powerful use cases. I think we can extend your ideas and map them to others that are well known in specialist disciplines where many of the issues are already ironed out. 

Likely, the most popular use of such fids would be to group tids into tiddlers as they are currently expressed. But not always. :-)

If there is a field anywhere (and everywhere) in the wiki called "fieldname" then it is fine to have a single tiddler that describes it. If someone chooses to use fieldname in one set of tiddlers to mean X and in another to mean Y then they are free to do so. They could even note in the fieldname tiddler that fieldname has two or more uses.  Extending the Info page to tell you in which tiddlers the field is used and or has a value will help.

In keeping with the field concept I think data tiddlers can be considered external fields,

Very interesting!!!

IMO, while data tiddlers are pretty concise they are still complex in that they list several more or less independent entries. IMO it would be truer to tiddler philosophy if data tiddlers really were lists of fids, comparable to lists of tiddlers.

In someways they already are, but let me repeat my first answer and lets apply that to the data tiddlers / fids

Its a bit of both because it is a combination of what is in place, the way we describe, promote and explain fields, and also suggests some "minor" changes to support a more holistic or structured meaning for fields. The major part is providing a way to think about and use fields.

Side note: it would be pretty cool to create fids by manually typing a data tiddler type of list, e.g like so:

tiddlertitle:Foo

text:

foo1:bar1

foo2:bar2

foo3:bar3

...and, on tiddler save, these fids are created (because they're fully defined). E.g number 2 fid would be Foo.foo2:bar2 ...or whatever the syntax would be - the main point is that this mini-thing defines the entire fid.

Recreating the list would be even simpler with something like {{{ [fid[Foo]] }}} ...or with focus on any of the other.identifying.kinship.attributes.that.it.may.have.

Something like this may already be possible with the Import Process.

...but that was a digress.

Back to your thoughts.

 

they define a field, its instances(index name is the key not the tiddler name) and values all in a single tiddler, they are more like a global field not to be confused with others such as Text/Title/Tags so by thinking of them as external fields (for want of a better name) could help. Actually it could be helpful if we (users) could define a global field like text and Tags in a tiddlywiki that exists in every tiddler like changecount (from TWC).

What role/purpose would this serve in your vision? (I can, however, see it as part of 'fids' to serve as a part of the ID)

The wonderful designers of TiddlyWiki saw a key set of fields as essential to the operation of tiddlers, every tiddler has these field whether or not they contain a value. I would like us to provide a mechanism for an advanced user to introduce a new field to every tiddler (I know it can already be done, but I mean with more ease). If you are building a tiddlywiki to meet a design requirement this could be an essential add on.

 

However there is a good argument that maintaining a SINGLE tiddler for each fieldname, and each field name for a documented purpose, is a way of supporting a good naming standard, avoid unexpected results, support self documentation practices and more.

Never the less a Tiddler can be a tiddler with content, including containing text/code (macro) or references to other tiddlers, be a tag and a Field definition all at once. And as in the discussion you linked to, also a variable or container for a variable. It is with this possibility in mind that perhaps a state tiddler should not populate the "text" field but a "value" field in the state tiddler. Although using text should remain valid.

Not quite sure what the problem is here. I must assume both the text fild and other fields are "just fields" even if with different properties. It may be aesthetically unpleasing to use the text field for states... but other than that, I don't know.

To me the text field is a free-form detailed "Description" or text except that belongs only to that specific tiddler. It would be nice to document the meaning of a state tiddler either in the text field, or in a description field so that the text field can continue to be transcluded where it is needed. I would just like to see some conventions around this.

 

...

 

If we are to simplify the syntax I think we may need to start differentiating here because there are constants that can change, things that vary and things that vary within the tree. In someways "currentTiddler" causes a problem because there is a current tiddler "Displayed" and a changing "currentTiddler" as a result of a filter/list. Sometimes the current tiddler is not even a tiddler, or only a potential tiddler. Because I am keen for tiddlywiki to undergo reform not revolution if we keep currenttiddler and introduce a value "displayTiddler" for the parent that is static in the tree below, a templateTiddler when specified for the displayTiddler it would have helped me learn and do things sooner.

There is much consideration for backward compatibility. Pros/cons with that approach. I'd spontaneously guess that it is not possible to introduce a "displayTiddler" other than perhaps as some alias to currentTiddler. I also think that revolutions (to use your term...by which I assume you mean a total rework, or perhaps a fork) comes when there is enough tension between current status and identified/solvable improvement that it feels justified. The aforementioned backward compatibility approach, probably hastens this process. (Obviously it also takes super skills to be able to pull it off...)

I can already "foresee" methods by which this could be implemented and retain backward compatibility (which I see as essential). Like a gang of superheros I believe we do have plenty of superheros here, we just need match our different powers correctly. I expect as my skills develop I will make a plugin that does this.

 

Perhaps the whole widget tree idea needs to be made clearer, perhaps by providing tools to navigate it, not through understanding the code but tools provided that help people visualise and tap into it, perhaps even a debug mode so we can see its value changing, Inheritance is is a good example. It is all fine having a many to many quine but a lot in tiddlywiki and the real world uses hierarchies and these should be more available, visible, understandable and reusable.

Considering that I barely know anything about how the widget tree works, I'd like this... but maybe it is more a js matter?

It is about JS but it is also part of the fundamental model. My idea is to make some tools and features that make full use of it in a more accessible way, thus helping people come to understand the widget tree idea.

An example would be the idea of inheritance which drives many a security or data model. Inheritance could allow for example the idea that one tiddler inherits the value of a field of the displayTiddler.

 

<:-)

I appreciate discussing these issues, but I hope some of the more experience coders can jump in.

We do not always introduce ourselves here, and it is hard to keep track of who's who and their skills in the tiddlywiki community, and I do not only mean tiddlywiki skills I also could include, Data modeling, script Writing, document writing, Database design, interest in model railways...

Perhaps a voluntary register could be developed?

I for one have skills and experience in Collaboration, Remote working, Solutions design, Information and Knowledge Management, Advanced Windows OS, Office 365 Cloud services, Internet Hosting WebDesign and now developing my BigData skills. I also have an interest in startups and business development as I am doing this myself now. Before this I was a LAN/WAN network specialist, Helpless desk :)  team manager, Office Automation specialist, Contingency planner (Disaster Recovery), Business analyst, Programmer analyst...

[@TiddlyTweeter]

Whoah.

I haven't read this as the topic is way above my head.

I just want to remark that the incremental width of recent posts in this thread are in near Tolstoyan dimensions & aesthetically novel in a GG group. Its visually very exciting! xx

Buonanotte
Josiah

Mat wrote:

I'm unsure if what your last post describe is as things currently are or as you'd want them to be.

ENORMOUS stuff followed ...

[Joshua Fontany]

Many of the ideas here in this thread are why I coded my Json Mangler plugin:
https://groups.google.com/forum/#!topic/tiddlywiki/OMaRZ1mOZ2w

It allows you to write out a nested Json in the editor window that describes a tiddler's fields, then "flatten" it to a 1-object deep (i.e. tiddlywiki-valid) set of Name:Value pairs in Json format. There are then buttons to "push" or "pull" these to or from a parent tiddler (same title without ".json" at the end).

For example:

{ "rpg" : {
  "character" : {
        "name" : "Conan",

        "class" : "Barbarian",

        "shtick" : "Craftiest horse-thief alive."

    }

  }

}

Becomes - 

{
"rpg.character.name": "Conan",

"rpg.character.class": "Barbarian",

"rpg.character.shtick": "Craftiest horse-thief alive."

}

And then you push those as fields (& values optional) from the "Conan the Barbarian.json" tiddler's text field to be individual fields on the "Conan the Barbarian" tiddler.

Then manipulate or display the Conan tiddler as you wish: {{Conan the Barbarian||$:/view/oD&D/charactersheet}}

Best,
Joshua Fontany

[TonyM]

Thanks Joshua,

I looked at your mangler and thought it was well made and professional. I am only learning about Jason files and did not understand its use case so I will review again.

I am looking to feed a Json file of my large twc tiddlers now in tw5 into a bigdata analysis.

Tony

[Diego Mesa]

Ive been thinking a lot about this recently Tony. Just wanted to +1 this and bring this back into the communal consciousness! 

Diego

[TonyM]

Diego,

My ideas are maturing. I have been building dozens of solutions but none yet good enough yet to publish. 

A Simple inroad I made at the start was in the Edit view fields

The Link allows you to create a tiddler by the name of the field. Only one tiddler will be created for any field name.

I call these field tiddlers

This tiddler can be given a caption, an Icon, Description, tooltip, color tags etc... 

The idea is now to start using these fields on the fieldtiddler to bring forward to the user more useful tools for field manipulation

I have done this in a not totally generic way for date fields for another solution I am building. 

As soon as you define a date field there is a whole range of tools available to increment, defer, clear, set to now, determine days since and until.

Perhaps that describe my vision a little better.

Tony