IDEA Multi-line Fields ... Possible implementation

[PMario]

Hi Folks,

Just reading an other thread in the group: Change initial field/textarea setup.    I had an idea about multi line fields, which are requested here quite a bit.

The default UI for inserting fields is a bit "heavy weight", if you have many of them. AND because of the concept, how fields are stored in the html file and the .tid file, it's not possible to use line breaks atm.

The idea now is, to use the text area for "multi-line" fields. This wouldn't change the .tid file spec, because those fields are part of the text area.

We already know macro definition pragma in tiddlers eg:

\define test()
Paragraph 1

Paragraph 2
\end


<<test>>

A similar "pragma" could be used to define fields. eg:

\field my-field
Paragraph 1

Paragraph 2
\end

{{!!my-field}}

The macro parser does exist already. So the field parser should be very similar. ...

PROS

 - This idea, wouldn't need to change the .tid file format, like other ideas discussed here.

 - Database recorde-like tiddlers that contain many different fields, wouldn't need much UI witchery

   - They could be easily copy / pasted in text edit mode

 - If new "field-text" is added to old TWs, it will be recognized, as "plain text"

 - NO new edit-multiline-field widget needed, since the existing edit-widget should do it.

   - We need to test this.

 - If users want a form-like UI this is the same work as now.

CONS

 - There is some overhead for \field start end \end

 - Since existing versions of TW don't ignore unknown pragmas, the new text format wouldn't be backwards compatible.

   - but it wouldn't hurt too much (see pros above)

 - New .tid templates will be needed.

   - may be fields-widget will need some extensions

-------- A little bit more "specs" ----------

1) field names must be lower-case (same a now)

2) Fields, that contain 1 line of text stay in the text area eg:

\field my-field

1 line of text

\end

3) "1 liners" will be converted into fields when the tiddler is saved eg:

    - they go to the field input area of the edit-template

    (probably worth more discussion)

\field test-field 1 line of text

4) multi-line fields will always be transcluded in "block-mode" like

<$transclude tiddler=x field="my-field" mode=block/> even if the shortcut is used: {{!!my-field}}

5) ... ???

just some thoughts

@Jeremy and others

What do you think?

have fun!

mario

[PMario]

Proposed Spec. (this post will be updated.)

1) field names must be lower-case (same a now)

2) Fields, that contain 1 line of text stay in the text area eg:

\field my-field

1 line of text

\end

3) "1 liners" will be converted into fields when the tiddler is saved eg:

    - they go to the field input area of the edit-template

    (probably worth more discussion)

\field test-field 1 line of text

4) multi-line fields will always be transcluded in "block-mode" like

<$transclude tiddler=x field="my-field" mode=block/> even if the shortcut is used: {{!!my-field}}

5) action-setfield widget will need a new parameter

 - multi-line: yes (default is: no)

6) action-listops ... gets NO new functions in the first run

7)

With multi-line fields, it should also be possible to define field-types. eg:

• \field my-text-field(type:text) -> default if not type is selected
  
• \field my-date-field(type:date)
  
• \field my-currency-field(type:currency)
  
• \field my-tag-field(type:tag)
  
• \field my-list-field(type:list)
  
• \field my-binary-field(type:binary) -> true / false or 1 / 0
  
• ... you name it.

8)

currency may be (type:currency format:de-DE) or (type:currency format:en-EN)

format: language specific settings in a different config tiddler

9)

 - The field content should not be duplicated in core internal memory!

 - So the JSON like internal structure will contain a field: start-reference and a end-reference

10)

 - It should be possible to preserve field order. Similar to uni-fields.js

11)

-m

[TonyM]

Mario,

This sounds interesting can you please clarify. Are you saying, Such "field" definitions would make the content in the field pragma become a value of this "Virtual Field". If I use edit-text I can name such fields and edit them as if they were independent fields but I would only be editing part of the text field where this definition resides?

• If my above understanding is correct perhaps it would be misleading to call them fields, although they will behave as such. Perhaps they could be considered "sections" that are addressable like fields.
• Will they be able to be made global, or defined in a tiddler tagged view template?
• Can we then display them conditionally?

If this interpretation that they are like sections is correct then it could allow code modularisation within a Tiddler, such that sections of code are defined at the top and included below in a structured form.

Thanks for your innovative thinking

Tony

[PMario]

On Thursday, May 9, 2019 at 3:04:16 AM UTC+2, TonyM wrote:

...

This sounds interesting can you please clarify. Are you saying, Such "field" definitions would make the content in the field pragma become a value of this "Virtual Field".

Not a "Virtual Field" a real field. The same as existing fields, but with the possibility to have line-breaks in the text. It would be the exact same behaviour as line-breaks in the text field. ... With some CSS magic, it would even display single line breaks. Which is already part of the core for plain text .. see: TW-Docs about hard linebreaks.

 

If I use edit-text I can name such fields and edit them as if they were independent fields but I would only be editing part of the text field where this definition resides?

That's right. If tiddler content is rendered, the \filed pragma will be "ignored". The tiddler-save action and several other widgets actions can modify the new fields. The core internal API may be the same, except of a new "multi-line" parameter for several widgets.

With multi-line fields, it should also be possible to define field-types. eg:

• \field my-text-field(type:text) -> default if not type is selected
  

• \field my-date-field(type:date) or
• \field my-currency-field(type:currency) or
• ... you name it.
  

This behaviour has been discussed some time ago. It would give as a lot of new possibilities for "input-validation"

 

• If my above understanding is correct perhaps it would be misleading to call them fields, although they will behave as such.

I don't think so. They should exactly behave like fields, with some new functionality.

 

• Perhaps they could be considered "sections" that are addressable like fields.

For me a "section" is a term, that was used in TWclassic for "heading-sections". Sadly ... This function didn't make it to TW5 :/

 

• Will they be able to be made global, or defined in a tiddler tagged view template?

They will be displaied with:

 - {{tiddlerName!!field-name}} .. Exactly the way you do it now. or

 - {{!!fieldName}} if you want to work with currentTiddler

 

• Can we then display them conditionally?

Yes with the reveal-widget. Same as you do with fields now.

 

If this interpretation that they are like sections is correct then it could allow code modularisation within a Tiddler, such that sections of code are defined at the top and included below in a structured form.

As shown in the OP

\field my-field       <-- Definition

Paragraph 1

Paragraph 2
\end

{{!!my-field}}        <-- Usage




This would be also easy to use with tables. eg:

\field a1()
Multi Line text

Some more text
\end

| Cell | 1        | 2        |
| a    | {{!!a1}} | {{!!a2}} |

The table will be simple copy paste action and the fields are convenient to be edited. ....

 

Thanks for your innovative thinking

You are welcome!

In the future, it should be also possible to create a specialised EditTemplate, that works with a GUI for GUI type of users. ... But the first run could be text only mode, which should be much faster for experienced users. 

have fun!

mario

[Diego Mesa]

Hey Mario,

This is a nice idea. It also seems related to an "Advanced Edit Mode" I described here: https://github.com/Jermolene/TiddlyWiki5/issues/3308

[TonyM]

Mario

Could you share a few sentences of the workflow we would use with the field pragma?

A possibly wrong example
Either manualy, as a result of a view template tag or in the text field of a template tiddler, place the field pragma in the text to name the field and provide its default content. On saving such a tiddler the field definitions will create actual tiddler fields in the current tiddler and those fields will now be available using standard text references to the fields.

Thanks
Tony

[TonyM]

Mario,

There is a discussion of a use case for multi-line fields in Discord General Chat with RK Shaw

This issue raises my ongoing frustration with editing fields in the current tiddler. I know and have implemented the work around to make it "appear to work".

We do have a work around however the cognitive load it places on the designer to make it work is immense because it involves "self referential code at a distance".  

It seems to me we should be able to build a mechanism to hide the complexity from the designer and provide a way to appear to edit the current tiddlers fields, thus empower the use of forms to be simplified.

Regards

Tony

On Thursday, May 9, 2019 at 12:56:58 AM UTC+10, PMario wrote:

[Reece Shaw]

Hey y'all, RK from the discord.

I think there is a lot to be said for making (better) use of multi line fields. Text area with $edit-text has always seemed like a bandaid to a bigger problem with fields.

Say I'm a user trying to make use of tiddlywiki fields. Ostensibly the difference between the text field and the other fields is very blurred. If I want to display the text field of Tiddler1 as it's displayed when not in edit mode, I'm simply using the $view widget and pointing at the desired tiddler (Tiddler1). 

If I want to display the text of a user field (say, user-defined-field) and format it I have to then workaround by somehow editing Tiddler1 's user-defined-field while also making sure to avoid using $edit-text in a manner that would cause the re-rendering issue. For a user that wants to build a UI that's a complete headache. There's a lot to be said for the ability to have a multi-line field that is directly attached to the tiddler itself and not as a sub-tiddler

1. User defined fields, when made multi-line would not impact core
2. We already have the capability to embed multi-line fields into a tiddler, we just have to use a workaround
3. list filters would become less complex when displaying multiple pieces of data linked to a single tiddler provided the user wishes to have multiple formatted fields

I am also of the opinion that organizationally multi-line fields are easier to manage. I know it's a personal preference or opinion but sometimes I have trouble navigating particularly long list fields because the data I want to manipulate, change or reorganize might be over 100 characters in. Some list fields when manually reorganizing can become cumbersome, and this would help alleviate some of that as well.

In an ideal world I would love if a multi-line-field indicator was just a checkbox next to the indicated field in edit mode. Then, we can default all field editing to single line fields unless the user specifies otherwise. There's a ton of untapped design space therein that can prevent the need for "hacky" workarounds like users trying to over-utilize subtiddlers or creating custom UIs.

Just some thoughts...

-Reece

[Jeremy Ruston]

Hi Mario

If I understand the proposal correctly, the idea is that one could type multiline fields into the main textarea of a tiddler with the following syntax:

\field my-field

Paragraph 1

Paragraph 2

\end

Then, when the user saves the tiddler, the content of the textarea is parsed, the field value(s) extracted and saved into the tiddler in the usual way. I’m assuming that the field pragma wouldn’t be replicated in the text field as well as the specified field?

Conversely, when initiating editing a tiddler, multiline fields are prepended to the text of the tiddler and placed in the textarea.

If I’ve got that right, my first thought is that the idea of a process to convert a tiddler back and forth to a single textarea representation is interesting, and doesn’t present any particular issues if it’s just a UI thing. Re-using the pragma syntax is confusing because this “parsing” is nothing to do with the main wikitext parser; perhaps it would be clearer to use an extended .tid file syntax.

However, your comments about re-using the wikitext parser suggest that my understanding is way off the mark, and you are thinking of a system where we do have to parse every tiddler before we can access its fields. That would have profound implications for almost every aspect of how TW5 works.

Best wishes

Jeremy.

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/a184c225-0dc6-46b9-9d7e-45d35b32943e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[PMario]

On Saturday, May 11, 2019 at 5:18:20 PM UTC+2, Jeremy Ruston wrote:

...

If I understand the proposal correctly, the idea is that one could type multiline fields into the main textarea of a tiddler with the following syntax:

\field my-field

Paragraph 1

Paragraph 2

\end

Exactly.

 

Then, when the user saves the tiddler, the content of the textarea is parsed, the field value(s) extracted and saved into the tiddler in the usual way. I’m assuming that the field pragma wouldn’t be replicated in the text field as well as the specified field?

I don't want to store the content 2 times. So the permanent storage would be the text area. ... The wiki-store json format may only contain a reference, where to find the content. ....

.tid file may look something like this

title: test-tiddler
tags: a b c d
my-field: @@@my-field

\field my-field(type:binary)
0
\end

or something similar. .. Hoping that nobody starts field content with 3 @ :)

If a tiddler is saved, the fields are parsed and stored into the internal tiddler store. ...

The textarea as a permanent store is needed, since the field content needs to survive a store-to-html / reload round-trip, which is the main problem atm.

 

Conversely, when initiating editing a tiddler, multiline fields are prepended to the text of the tiddler and placed in the textarea.

right

 

If I’ve got that right, my first thought is that the idea of a process to convert a tiddler back and forth to a single textarea representation is interesting, and doesn’t present any particular issues if it’s just a UI thing.

I think, it's not an UI thing only, since multi line fields need to survive the save/reload roundtrip with HTML and the .tid file format.

 

Re-using the pragma syntax is confusing because this “parsing” is nothing to do with the main wikitext parser; perhaps it would be clearer to use an extended .tid file syntax.

I'm sorry about this. ... I just wanted to point out. that there is some code, that can read the existing \define ... pragma. ... So it will be easy to create _new_ code the can parse the multi line definitions. ... No reusing of the existing mechanism involved.

have fun!

mario

[PMario]

Hi Diego,

On Thursday, May 9, 2019 at 2:55:44 PM UTC+2, Diego Mesa wrote:

This is a nice idea. It also seems related to an "Advanced Edit Mode" I described here: https://github.com/Jermolene/TiddlyWiki5/issues/3308

Good catch :)

-m

[PMario]

On Friday, May 10, 2019 at 6:23:37 AM UTC+2, Reece Shaw wrote:

Hey y'all, RK from the discord.

Hi

 

If I want to display the text of a user field (say, user-defined-field) and format it I have to then workaround by somehow editing Tiddler1 's user-defined-field while also making sure to avoid using $edit-text in a manner that would cause the re-rendering issue.

That's a problem of browser UI focus handling. There are 3 github PRs that try to address this issue. There is an EXPERIMENTAL test version at tiddlyspot too: http://refocusing-inputs.tiddlyspot.com/ ... It allows you to edit the "currentTiddler" ...

 

For a user that wants to build a UI that's a complete headache. There's a lot to be said for the ability to have a multi-line field that is directly attached to the tiddler itself and not as a sub-tiddler

1. User defined fields, when made multi-line would not impact core

That's not true. They would have been implemented already, if it wouldn't be a problem.

 

1. We already have the capability to embed multi-line fields into a tiddler, we just have to use a workaround

The problem is, that it also needs to work with the .tid file format. Which probably isn't the case for the workarounds. I don't know the workarounds, so I'm not sure here.

1. list filters would become less complex when displaying multiple pieces of data linked to a single tiddler provided the user wishes to have multiple formatted fields

I'm not sure, what this means.

 

I am also of the opinion that organizationally multi-line fields are easier to manage. I know it's a personal preference or opinion but sometimes I have trouble navigating particularly long list fields because the data I want to manipulate, change or reorganize might be over 100 characters in. Some list fields when manually reorganizing can become cumbersome, and this would help alleviate some of that as well.

There are a lot of different views, about, how to work with TW. ... Most of the time, it is highly depending on the user preferences and the actual use-case. ... We don't have a solution atm. ...

That's the reason for this thread. I thought the format explained in the OP should be OK for users and it should be possible to implement it, with an acceptable amount of work.

 

In an ideal world I would love if a multi-line-field indicator was just a checkbox next to the indicated field in edit mode. Then, we can default all field editing to single line fields unless the user specifies otherwise. There's a ton of untapped design space therein that can prevent the need for "hacky" workarounds like users trying to over-utilize subtiddlers or creating custom UIs.

That's an implementation detail, that should be doable.

have fun!

mario

[TonyM]

Mario,

Perhaps a related hack

In an experiment in the past I noted I could build a slightly modified .tid exporter such that the tiddler text to export contained field value pairs as in the raw tiddler syntax.

Fieldname: field value (from memory)

once this exported tiddlier was reimported these fields became standard tiddler fields. It provided a mechanisium for user edits of text to be turned into field value pairs.

My thought is this method could be used along with your suggestion without the export/import step. Such field value pairs could be used in template tiddlers to create and populate fields and even restore default values if the fields are deleted. Of course also introducing multiline fields.

\fields
Fieldname: field value
Fieldname2: field value
Multilinetextfieldname:
text on next line indicates multi-
line field handled as you suggest
\end

The above would populate fields with default values if they do not exist, including the multiline text fields defined in the tiddlers text body.

Just a hack idea

Regards
Tony

[@TiddlyTweeter]

Ciao PMario & all

Regarding the AIM: to provide ability to have line breaks in fields.

I don't understand the underlying architecture. But I understand the need.

This is a basic example where this utility would add to TidlyWiki and ease usage. 

Take the case of MOVIES. Each movie is SINGULAR (one & unique).

-- At the moment if you want to maintain a dataset organised by Movie Title that includes rich content, including line-breaks, its over complicated. You have to have several Tiddlers per Movie as soon as you include lines breaks. A simple example is ACTORS. Say you have 35 actors in a movie. Is that 35 fields? 35 Actor Tiddlers? Or better 1 field with 35 lines? 

-- A "Movie" is a good example of a referenced object that is a whole. It does not need to be spread over multiple Tiddlers. Currently for richer data you have to (over)break it down.

-- One might argue a solution here is to emulate a relational database via JSON structures. But actually, I think complex (tree-nested) JSON is unnecessarily complex for the specific USE CASE. 

-- The Use Case Supposition (with good reason) is one-movie is one-thing. A movie is one object... Even IF George Clooney appears in 13 other movies, when I'm looking at the record for one movie I could still link to Clooney but I think over-spawning "actor-tiddlers" would pretty soon ruin a TW's utility in over-complexity.

---

Crucial point. In my usage I want to send students ONE Tiddler per movie so they can easily import it. Its needs to have a way to include line breaks in fields so that I can include Plot, Logline, Actor Field & Crew, all as a unit. Other routes get far too complicated.

Just thoughts

Josiah

[@TiddlyTweeter]

Ciao PMario & Jeremy

I sat with what this thread is about.

I'm not sure. I mean, I DO like the aim. 

But I'm slightly uncomfortable with using the text field as a surrogate field-holder. It seems counter-intuitive?

Just a misgiving. But maybe I misunderstood how its intended to work?

Best wishes

Josiah

[@TiddlyTweeter]

Ciao Jeremy & all

 

Jeremy: Re-using the pragma syntax is confusing because this “parsing” is nothing to do with the main wikitext parser...

 

Right. So a footnote. 

Please don't, if this goes ahead, conflate the proposed solution with the base pragma. It could be very confusing for users like me interested in (manipulating) the process through which TW assembles itself in the "pragma phase"!

I'm not so good on the architecture process in TW but from what I see the proposed solution is running after the core pragma have?

I'm not clear!!! Excuse me!

Best wishes

Josiah

[PMario]

On Sunday, May 12, 2019 at 11:53:43 AM UTC+2, @TiddlyTweeter wrote:

..

But I'm slightly uncomfortable with using the text field as a surrogate field-holder. It seems counter-intuitive?

It's very similar to the existing macro definition. If the macro definition seems counter-intuitve, then yes. From a "text" viewpoint it is the same.  

-mario

[@TiddlyTweeter]

Ciao PMario

Okay. Its simply then that I perceive a difference between data-holding that are "fields" and "pragma" "running fields" as surrogates.

The confusion for me, I think, is Tiddlers Have Fields Already so why are we adding others a different way rather than changing field behaviour? 

I do think there is a difference between field structure and field manipulation and in that sense I, perhaps naively, find the approach slightly confusing (BUT do like the ultimate aim :-)

Best wishes

Josiah

[PMario]

On Sunday, May 12, 2019 at 12:45:45 PM UTC+2, @TiddlyTweeter wrote:

...

The confusion for me, I think, is Tiddlers Have Fields Already so why are we adding others a different way rather than changing field behaviour? 

The problem is, that we don't have a structure, that allows us to use multi-line fields with the .tid file format.

eg:

title: tiddler-title
tags: a b c [[d e]]
my-field: 1 line of text is possible

text field starts here. Fields can't handle muliti-lines.

As you can see, there is no way at the moment to add multi-line fields in a sensible way.

There are 2 main points which are important:

1) It must be easy for humans to read and modify

2) It should be relatively easy for machines to parse the content.

No. 1) is more important than No. 2!

I personally think, that the existing field editing mechanism (UI) is cumbersome. Especially since fields are alphabetically sorted ...

ATM there are differences between core-fields eg: created, modified, ... or tags. ... They have built in behaviors, which are hard for ordinary users to duplicate. Eg: It's hard if I want to create a field eg: my-tags, that behave like the tag field. Especially I want to have an automatic UI, that let's me handle my-tags the same way as the existing tags-dropdown.

My proposal makes this possible with eg:

\field my-tags(type:tag)
a b [[c d]]
\end

Since fields can have a user defined "type" now, it will be possible to automatically create the tag-like UI for my-tags field ... and so on ...

-mario

[PMario]

On Sunday, May 12, 2019 at 12:45:45 PM UTC+2, @TiddlyTweeter wrote:

 

I do think there is a difference between field structure and field manipulation and in that sense I, perhaps naively, find the approach slightly confusing (BUT do like the ultimate aim :-)

Hi,

It's good to get feedback like this. It tells developers, where users may have problems.

-m

[PMario]

Hi,

With TWclassic we had a transclusion mechanism, that worked with "section headings". eg:

<<tiddler [[tiddlerTitle##sectionHeading]]>> .... this is TWc syntax.

In the example belwo <<tiddler [[my-movie##Title]]>> ... would return: Movie Title - bla bla ..

With this mechanism, you could transclude parts of 1 tiddler into an other. With this mechanism, users could create a "database-like" structure, using different heading levels. eg my-movie:

! Title

Movie Title - bla bla ..

! Description

Short  info about the movie.

!! Actors

* Actore A
* Actore B
* ...

! Details

Other stuff, that may be interesting.

Using headings, the user could create a structure. If several tiddlers did contain the same heading-structure, this structure was an "intrinsic" database schema. ... Transcluding ##Description would transclude everything until ##Details section starts.

------------------------

The main problem is, that this mechanism is "heavy weight", in terms of performance, since everything has to be calculated during tiddler rendering phase.

That's why it was replaced with TW5 DataTiddlers. ...

The problem with DataTiddlers [1] is, that JsonTiddlers are NOT convenient to edit for humans AND DictionaryTiddlers can't handle multi-line values.

-m

[1] https://tiddlywiki.com/#DataTiddlers

[PMario]

Typo fixed in above post.

That's why it was replaced with TW5 DataTiddlers. ...

-m

[TonyM]

Mario, did you reply on the wrong thread or go off topic?

tony

[Mat]

PMario

The idea to be able to create fields using wikitext is brilliant! As Diego requests, TW needs a smoother way for making notes. (I've commented in teh past that TW is great for managing notes but not for taking notes. It is currently way too fidgety and attention demanding to use e.g in a lecture or during a meeting.)

Regarding multiple lines, it should be noted that it is possible to use <br> to achieve line breaks, i.e transcluding this does give two lines as output.

my-field: this has<br>two lines

If this is insufficient I would rather treat "we need multiple lines" as something to deal with when an eventual non-backwards compatible step is taken because TW should either feature "multi-line fields" via the regular field creation way -or- we should be able to create fields directly in wikitext but in a very direct and smooth way.... particularly the .tid syntax would make sense. Maybe multi-lines could be signified using empty colons (ahemm, that term) . Very efficient wikitext.

my-field: this has
: two lines

<:-)

[damscal]

Hi all,

the idea seems very good to me, especially if coupled with a system aimed towards frictionless editing such as saqimtiaz's streams plugin plus some good autocompletion plugin.

I cannot tell which architecture might be the best, but I'd be looking forward to this.

Are there working implementations yet?

Regards,

dam

[Flibbles]

This seems like it adds a lot of complexity to anyone parsing the JSON tiddler store. If anyone is manually looking through it, then they won't have access to all of the fields unless they also parse the text for "\field" pragmas.

Out of question, what is the workflow for having multiline fields where having a subtiddler doesn't work (e.g. "myTiddler/description" relates to "myTiddler" instead of having a description field)? They've always seemed so esoteric to me.

[PMario]

Hi dam,

This was only an idea, prior to TW 5.2.0 ... There are different possibilities now

Have a closer look at: https://talk.tiddlywiki.org/t/intro-field-editor-plugin/1580  Which talks about 2 experimental plugins that I published recently. It's a different approach ... but ... experimental. ... It seems the plugins cause a bit of an issue with the view- and editTemplates of the latest TW version.

So you have to be careful testing them. ...

-mario