Structured data in TW

[AlanBCohen]

I've been trying to fully understand the ways you can record, edit,
and retrieve structured data in a TW. So far, I see the following as
the options:
- tables within a tiddler
- slices, where a field name is followed by the field value, and by
using the ForEachTiddler plugin, you can retrieve the field values
into lists and output displays.
- Form and Data tiddler plugins, using HTML forms within tiddlers to
format data, which can also use the ForEachTiddler plugin.
- custom fields, which seem to require changes in the Edit and View
templates and affect all tiddler in the file. And again, the
ForEachTiddler plugin can be used to retrieve values from multiple
tiddlers
These seem to be the options, as I understand them. I am successfully
using the first three methods, as they meet my needs in various
applications I've built; the latest using slices. At work, I have not
been able to import a copy of the FormTiddler plugin (I have
DataTiddler plugin loaded and it should work when I get it's partner
in shop).
But I really am not sure about custom fields yet, even after reading:
http://tiddlywiki.org/wiki/Getting_started_with_custom_tiddler_fields
It looks as simple as modifying the edit and view template to add
lastname, firstname, and contactinfo and those fields would appear on
every tiddler in the file, making a phonebook (yes, I know about
twab). Am I understanding this correctly? And, are there other
structured data techniques I've missed?

[wolfgang]

Hi Alan,

> the options:
> - tables ..
> - slices ..
> - Form and Data tiddler plugins ..
> - custom fields..
> ...

> ForEachTiddler plugin can be used to retrieve values from multiple
> tiddlers

Not only ForEachTiddlerPlugin, but also InlineJavascriptPlugin
dependent scripts can retrieve the values in all 4 cases from chosen
tiddlers and create specific output, for example in tables.. However,
there isn't as much documentation as with ForEachTiddlersPlugin.

> But I really am not sure about custom fields yet, even after reading:
> http://tiddlywiki.org/wiki/Getting_started_with_custom_tiddler_fields
> It looks as simple as modifying the edit and view template to add
> lastname, firstname, and contactinfo and those fields would appear on
> every tiddler in the file, making a phonebook (yes, I know about
> twab). Am I understanding this correctly?

Yes. For quick creation and editing of custom fields
FieldsEditorPlugin comes also very handy:
http://visualtw.ouvaton.org/VisualTW.html#FieldsEditorPlugin

Costom Fields also have the advantage or disadvantage to be able to be
retrieved even if the tiddler text itself is encrypted.

Regards..

[Dave Gifford]

Not a full response, to your question, just a note that you be aware
that form fields need not require you to have them in ALL your
tiddlers. You can create a custom viewtemplate and/or edittemplate for
a set of tiddlers that share the same tag.

Dave

[wolfgang]

> Not only ForEachTiddlerPlugin, but also InlineJavascriptPlugin
> dependent scripts can retrieve the values in all 4 cases from chosen
> tiddlers and create specific output, for example in tables.. However,
> there isn't as much documentation as with ForEachTiddlersPlugin.
>

Latest there are also 'transclusions' possible to retrieve any values
of above methods with custom output but without dependency to any
plugins, however it seems only Eric is able to do so. See for example:

http://www.tiddlytools.com/#ShowPlugins
http://www.tiddlytools.com/#BookmarkList

[wolfgang]

> Yes. For quick creation and editing of custom fields
> FieldsEditorPlugin comes also very handy:http://visualtw.ouvaton.org/VisualTW.html#FieldsEditorPlugin
>

Even better still, since its not only able to edit fields, but slices
and whole sections of a custom selection of tiddlers:

http://www.tiddlytools.com/#GridPluginInfo

[Eric Shulman]

On Oct 10, 6:22 pm, AlanBCohen <alanbco...@gmail.com> wrote:
> I've been trying to fully understand the ways you can record, edit,
> and retrieve structured data in a TW.  So far, I see the following as
> the options:

"Slices" and "Sections"
----------------------------------

Tiddler slices and tiddler sections are TW native methods of storing
*named* data values by entering information within the normal text of
a tiddler, surrounded by specific patterns of wiki-syntax formatting.

Because they are part of the regular tiddler content, it is easy for
users to add or update information stored in slices or sections,
without having to create any special input or display handling.

A slice is a simple name/value pair, specified using a 2-column TW
table, like this:

|slicename1|slicevalue|
|slicename2|slicevalue|
(e.g. at the top of published plugins, scripts, etc.)

or, using a 'description' syntax, like this:
slicename1: slicevalue
slicename2: slicevalue
(e.g., the default shadow ColorPalette definition)

Regardless of which syntax you use, each slicename should be unique to
the tiddler in which it is defined. Slice values are intended to hold
relatively short bits of text, and cannot contain any newlines.
Fortunately, when you need to store larger, multi-line blocks of text,
the tiddler *section* syntax is available.

A section begins with the TW 'heading' syntax, ! through !!!!!!, where
the heading text is used as the section name, and the value of the
section is everything following the heading, up until the next heading
(of any level), or the end of the tiddler, whichever is reached
first. Thus:
!SectionName
This content is on
multiple lines
!NextSectionName
This is a different section
with more lines of content
!AnotherSectionName
etc.

Once they are stored within the tiddler text, you can easily embed the
value of any given slice or section into another tiddler, by using the
standard <<tiddler ...>> macro, with a special syntax for referring to
slices or sections, like this:
<<tiddler SomeTiddler::SliceName>>
<<tiddler SomeTiddler##SectionName>>
Note: As with any tiddlername used in a macro, if the slice or section
name contains spaces, you will need to enclose the reference in
brackets:
<<tiddler [[SomeTiddler::Slice Name with spaces]]>>
<<tiddler [[SomeTiddler##SectionName with spaces]]>>
As a general rule-of-thumb, when possible, avoid using long text with
spaces when defining slices and section names... it's just easier that
way.

If you enclose your slice tables or sections within TW comment markers
(/%...%/), then they will not be *rendered* when the tiddler is
displayed, but will still be accessible as named slice or section data
using the <<tiddler>> macro as described above. These are called
"hidden slices" and "hidden sections", and can be used to store
'variables' associated with the tiddler, without affecting the
rendered appearance of that tiddler.

When definining a hidden section, it can be helpful to ensure that it
has a specific beginning *and* ending by adding a 'junk' heading
following the section content, like this:
/%
!SectionName
This is content
!end

!OtherSection
More content
!end
%/

Note the "!end" heading is repeated multiple times in the same
tiddler. The "!end" simply serves to mark the ending of each hidden
section. This is OK, as long as we don't actually try to retrieve a
*value* from the "!end" section... and, since the entire block of
content is *hidden*, it doesn't matter how many times "!end" is used,
as it is never actually displayed.

To get the value of a slice or section *programatically* (i.e., for
use with <<forEachTiddler>>, InlineJavascriptPlugin, or when writing
actual plugin code), you can use the TW standard function:
var txt=store.getTiddlerText("TiddlerName::slicename")
var txt=store.getTiddlerText(""TiddlerName##sectionname)

Of course, not every tiddler in a document will have the same slice
and/or section definitions (if any). While it is possible to query
each tiddler in turn to determine if a given slice/section is present,
it is often more useful to determine the kind of information stored in
a tiddler's slices/sections by tagging the tiddler with some pre-
determined value that desigates it's "type".

Then, you can use that tag value to 'filter' the list of tiddlers to
be retrieved, so that only those with the desired slices and/or
sections are processed.

For example, here's an inline script that gathers up information about
'bookmark' tiddlers, and outputs a summary, formatted as an "HR-
separated list"

<script>
var out=[];
var tids=store.getTaggedTiddlers("bookmark");
for (var i=0; i<tids.length; i++) {
var t=tids[i].title;
var d=store.getTiddlerText(t+"::Description");
var u=store.getTiddlerText(t+"::URL")
out.push("[[%0]] - %1\n%2".format([t,d,u]));
}
return out.join("\n----\n");
</script>

Hopefully, this should be enough to get you started with slices and
sections...

enjoy,
-e
Eric Shulman
TiddlyTools / ELS Design Studios

[AlanBCohen]

Thank you.
Alan

[Markk S.]

Hi Eric,

Great write-up!

When I first started TW, I wrote an application or two using the Data
plugin, mainly because the information about programming slices/
sections was less accessible (vi tiddlywiki.org) than information from
the author's site about the Data plugin. If I re-wrote them now, I
would probably tend to use slices/sections.

Your example demonstrates how to pull and used slice/section data
programmatically. But what if I want to push data back into a tiddler
(e.g., change "Name: Rowling" to "Name: Pratchett" ? Or change
something like:

!Latest Book
Nation

to

!Latest Book
Unseen Academicals

Is there some simple code to accomplish this, or would I have to
essentially replace the entire original Tiddler contents?

Thanks!
Mark

[wolfgang]

> Great write-up!
>

Indeed!

> But what if I want to push data back into a tiddler
> (e.g., change "Name: Rowling" to "Name: Pratchett" ? Or change
> something like:
>
> !Latest Book
> Nation
>
> to
>
> !Latest Book
> Unseen Academicals
>
> Is there some simple code to accomplish this, or would I have to
> essentially replace the entire original Tiddler contents?
>

> Even better still, since its not only able to edit fields, but slices
> and whole sections of a custom selection of tiddlers:
>
> http://www.tiddlytools.com/#GridPluginInfo

Regards..