Table building (noob problems)

[PWL]

Hello all,

I have recently started using Tiddly wiki to build a knowledge database and I am having difficulty figuring a few things out.  I am hoping somebody can help.

Like I said, I have run into several things that I can no figure out so I will start with what I figure will probably be the easiest problem.

This wiki is going to be huge when it is done (4500+) entries.  Each entry has a list of 14 stats that need to be displayed as a table.  I found how to build tables, the work perfectly, and I like the look.

But I have found that it is rather labour intensive to do this every time I make a new entry.  Is there a macro, or some code, or anything that can make this easier for me?

[@TiddlyTweeter]

Is this data intensive? Is the data structured, I mean with a consistent pattern?

[PWL]

Every entry has the same 14 stats, a gif and a significant amount of text.  The Pattern of the stats is consistent.  There are some exceptions but for the majority of the entries it will be the same pattern

STAT1 | value (alpha numeric)                                                                            Floating 100kb Gif

                                                                                                                      approx 400px X 400px

.

.                                                  

.

.

STAT14| Value

Average 50 to 100 lines of formatted text.

[Mark S.]

How are you storing the stats? As fields? As data dictionary entrees?  In separate tiddlers?

In general, it's not a good idea to directly put the numbers into tables in TW5 tiddlers unless you only want to display them as static tables. Forever. In TW5 there are no sections or slices for extracting the data later.

Whatever the data storage approach, as long as the data is consistent, it should be possible to create a view template that will display the data.

It's always good at the start of a project like this to consider what outputs you are expecting. Then you can write routines that work with a handful of tiddlers to verify that things work before scaling up the project.

-- Mark

[PWL]

The stats are static, they will never change.  Like I said it is a knowledge data base.  Almost like a specialized encyclopedia.  I have attached a capture of one of the entries so you can see what I am doing.  The only thing I really want to do with the stats is make the three that are highlighted searchable.  But I thought I would deal with that once I got the appearance the way I wanted.  I have that, but like I said building the table for each entry as I make it is somewhat time consuming.

[Mark S.]

Your image is too small for me to make out the fields, but I can guess some of them.

This is the whirlwind overview of using templates.

Let's say you have a tiddler with some fields:

image_url: simian.jpg

terrain: any

intelligence: 11

matrix: 3x3

Then you might set up a template tiddler, TEMPLATE that uses these fields:

<image src={{!!image_url}} float="left"/>

<table>

<tr>

<td>Terrain</td><td>{{!!terrain}}</td>

</tr><tr>

<td>Intelligence</td><td>{{!!intelligence}}</td>

</tr><tr>

<td>Matrix</td><td>{{!!matrix}}</td>

</tr>

</table>

Then let's say that all these (avatar?) tiddlers have something to identify them, like tag "Avatar".

Then you can make a tiddler, tag it with $:/tags/ViewTemplate, with contents:

<$list filter="[all[current]]">

{{||TEMPLATE}}

</$list>

And each tiddler tagged with Avatar will present your formatted data.

A general tip about database fields is that they should always only represent one thing. For instance, an intelligence field of 11-15 will be hard to parse and compare with other avatars because it has a range. So if you ever expect to want to compare intelligence values you should have intelligence_low and intelligence_high, or "intelligence" and "intelligence_range".

There's a lot more to say about templates and their use in TW5, but as I said, that's the whirlwind version.

-- Mark

[barro...@gmail.com]

PWL,

I have recently started using Tiddly wiki to build a knowledge database and I am having difficulty figuring a few things out.  I am hoping somebody can help.

I see your project looks RPG/MMORPG related.  With patience you'll see that TW is a great tool for such purposes-- a few individuals here have applied TW to RPG related purposes.  There's a learning curve but its definitely a worthwhile curve cause everything that TW can do.

 

This wiki is going to be huge when it is done (4500+) entries.  Each entry has a list of 14 stats that need to be displayed as a table.  I found how to build tables, the work perfectly, and I like the look.

But I have found that it is rather labour intensive to do this every time I make a new entry.  Is there a macro, or some code, or anything that can make this easier for me?

The templates Mark S mention is the best way to cut out the manual table coding (typing)  but templates work best with fields not raw text, which I'm not sure if you have gotten to learning fields yet.  Field value entry can also be labour intensive but templates again can be used to create tiddlers with all fields added.

How far have you gotten with making tiddlers, would be a major time loss going back to start and starting with templates?  Would you already have this data already on a spreadsheet?  there's ways to mass import spreadsheets into TW, creating tiddlers preformatted.

Once you get to the search function, I may have something that may help.

[PWL]

You are correct.  I am attempting to make the Monstrous Compendium from 2nd ed AD&D into a wiki.  Here is a link to the site I am using for my source data.   http://lomion.de/cmm/_index.php 

The site is an almost complete list of all entries from that version of the game.  However, it is a horrible resource to search, poorly organized, the links are all over the place and it cannot be searched beyond the index entries.  I spend far too much time sifting through the entries and my books looking for the stuff I want.   What I have been doing is copying the text from the page to a tiddler then formatting it, copying the image to my hdd and importing it to the tiddler.  Not exactly the most efficient method.  But it keeps it all local on my hdd which is a requirement.

I started out building an actual data base with LO.  But I found it started becoming cumbersome, slow and unstable; probably due to the sheer amount of text.  So I turned to personal wiki's.  Tiddly seems to be the best option I have found for this.  To this point I have learned how to make tabs, tables of contents, tags, tables, and how to format the text and lay out on my tiddlers.  Now I need to figure out how to make the information I use most often searchable; Climate / Terrain, Intelligence, and Hit Dice.  I was thinking fields would be the best way to go, but I am finding the information available confusing.  Reducing the amount typing, and coding I have to do would be a great bonus but learning how to make my data searchable should probably be the priority.  I have not done any serious programming since the 90's.  And I hated it then.  I would love to find a wysiwyg interface for this.

Any help you can give, or point me to a tiddly for dummies type book or site would be greatly appreciated. 

[Jed Carty]

The easiest way to do this is to make a template to display each entry. I had considered doing this a while ago but I haven't had time. For some things, like creature types, I think that the GenTags plugin may help too.

Conditional view templates are probably also helpful.

These may be useful links:

http://inmysocks.tiddlyspot.com/#

http://tobibeer.github.io/tb5/#Welcome

[Mark S.]

Although the best long-term solution is probably fields, you might want to start with companion data dictionaries. In this approach, each main tiddler has a companion dictionary tiddler. The advantage to this approach is that the data format on the website is almost formatted as a data dictionary. For instance, if you had an "Allin" tiddler then you might have a companion Allin-dict tiddler with contents like:

climate-terrain: Temperate or tropical/wilderness or subterranean
frequency: Uncommon
organization: Solitary
activity_cycle: Any
diet: Omnivore
intelligence: Average (8-10)
treasure: 1
alignment: Neutral
num_appearing: 1d4
armor_class: 4
movement: 6, Sw 15
hit_dice: 3
thac0: 17
num_of_attacks: 1
damage_attack: See below
special_attacks: Drowning
special_defenses: See below
magic_resistance: Nil
size: L (10’ tall)
morale: Elite (13-14)
xp_value: 270

These are the values from the website, with tabs converted to spaces, spaces in the fields (the part before the colons) converted to underscores. "no." Converted to "num", and all fields made lower-case because it's so easy to make mixed case mistakes.

To reference fields in the dictionary, you use a notation like:

{{Allin-dict##num_of_attacks}}

However, I think this can be adapted for use in the template previously described so that "-dict" is automatically applied and a table is automatically constructed by the template. It should be easy to convert dictionaries like this into fields somewhere down the road.

-- Mark

[PWL]

So how exactly would this work?  I build a template then dump all the stats into a 'data tiddler' and call it?

Also I found this...   https://groups.google.com/forum/#!msg/tiddlywiki/FC2KOtlyaYo/KqRIUjNbEbcJ

Kind of looks like it might work for a search function.  Thoughts?

[PWL]

I found this....   https://groups.google.com/forum/#!msg/tiddlywiki/FC2KOtlyaYo/KqRIUjNbEbcJ 

Kind of looks like it might work.  Thoughts?

[TonyM]

PWL,

By definition your requirement has a degree of complexity by virtue of the data, but I would keep it as simple as possible. Here are the steps I would take, and they are easy for me because I identified them some time ago and already have solutions for each (unfortunately that are not yet of publishing quality).

• Build a tool to create new tiddlers from a template tiddler, add what ever fields you want to this template, Idealy all possible fields for each "type" of object you decide to manage.
  
• Define each field you are using in their own tiddler which will show how to display or edit that field
• Create a toggle in view template that shows an edit-fields tiddler, this tiddler needs to use a "template" that allows editing the current tiddler fields (a little tricky) but it then transcludes the field definition tiddler for each field in the current tiddler allowing you view / edit its content. 

• Typically I fold every tiddler and toggle the edit-fields tiddler, on all "objects". I do not even need to select "edit" on each tiddler, just go strait to editing the fields. Thus I rarely add info to the text of a tiddler because it is all contained in other fields.

This is only high level but if gives you some idea of what can be done.

I have also created a list support tool that allows you to generate lists, choose the filter and columns in that list and define the columns to allow inline editing or time stamping of fields in the column cells when listing.

If I could raise some funds to pay my way I would publish these tools sooner.

Regards

Tony

[PWL]

How about this?  I will start with the absolute minimum.... I want to make 3 items in my list searchable....  Climate / Terrain: [alpha only], Intelligence:  [alpha numeric]  and Hit Dice:  [alpha numeric] I think it has been determined that fields are the best way to go with this.  How do I make a field?

[Mark S.]

This brings us to the problems of using a dictionary for each Avatar versus having fields.

There are few good tools for using data dictionaries in TW5. So a search list that searches all dictionaries for intelligence=Average and armor_class=4 would like something like:

<$list filter="[tag[Dictionary]]" variable=dict>
<$list filter="[<dict>getindex[intelligence]prefix[Average]]" >
<$list filter="[<dict>getindex[armor_class]prefix[4]]" >
<<dict>>
</$list>
</$list>
</$list>

I'm actually cheating here, just looking for "prefix". If your data is more complicated (like you have prefix 34) then more would have to be done.

If you use fields, then the search is easier:

<$list filter="[tag[Avatar]search:intelligence[Average]search:armor_class[4]]"/>

You can use the search operator to search individual fields. However, TW5 doesn't have good comparison operators (they've been hung up in committee for awhile). So you might need regular expression operators to check for armor_class 21. But I digress.

Once you have a data dictionary set up, it's easy to write code that will transfer the data dictionary into a tiddler as fields. For instance, if you have a  tiddler "Allin-Dict" tagged with "Dictionary" and are inside tiddler "Allin", and then paste this code into Allin and run it (tiddler in view mode):

<$button>Apply dictionary
<$list filter="[all[current]addsuffix[-dict]]" variable=dict>
<$list filter="[title<dict>indexes[]]" variable=field>
<$list filter="[title<dict>getindex<field>]" variable=value>
<$action-setfield $field=<<field>> $value=<<value>>/>
</$list>
</$list>
</$list>
</$button>

This will create a button that will transfer your fields over when you click it. (be sure to have a backup before trying any of this -- only tested once!)

When you open "Allin" and scroll to the bottom, you'll see all the new fields and values.

Looks like you're going to get lots of advice here, especially from people who understand RPG better than I do, which is good.

-- Mark

[Mark S.]

Edit the tiddler and scroll to the bottom of the tiddler (e.g. "Allin"). At the bottom you'll see:

"Add a new field"

Fill the box next to that with the name of the field you want (e.g. "intelligence") and then the field next to that with the value (e.g. "10") and then click on the "Add" button. Repeat for each field.

Don't use spaces in field names. For future sanity, use all lower case because field operators are case sensitive. I tend to eliminate all punctuation except _ and - .

Once you have made a field, the next time you add the same field to a new tiddler you will see a list of existing fields from which to select. This will help prevent spelling inconsistencies as you work (plus save a little typing time.)

-- Mark

[PWL]

Great! Thank you.  Okay next step... I figured out how to use the filters in the advance search ( I know, no big feat)  But they worked the way I wanted!  So now I just have to figure out how to put a search function in that doesn't require me to type [field:intelligence[average]]  into the advanced search every time I want to search something.  I was thinking a search button at the top of the index page I have or another tab on the side bar that has search options, something simple.  I haven't been able to find anything like that.  I even tried copying the code from the site and tweaking it.  No luck.

[Thomas Elmiger]

Hi PWL,

I think you are on the right track with fields and filters for your special searchable attributes.

Just a hint for copying the tables, in case you have to copy them manually. Your source offers nice clean HTML tables and you could copy them directly.

(Maybe you will have to activate Developer Tools in your browser if you can't find the commands mentioned.) Steps:

• right click on the first cell of the table, choose Inspect Element (or similar)
• in the panel opened by the browser look for the <table ...> tag two lines up from the marked line in HTML
• right click on the table tag, choose Copy Outer HTML
• insert in your tiddler

Way faster than disassembling and reassembling tables :)

Good luck!

Thomas

[PWL]

That is great! Thank you so much.  Dont suppose you can tell me how to insert a functional search box into my wiki could you?

[Mark S.]

Here's a quick and dirty way to set up a search by one attribute. It leverages the existing search field (which is in $:/temp/search) and the list-links macro.

Make a tiddler, say "Search-Intelligence". Create a "caption" field with a short title (say "S-Intel") and tag the tiddler with $:/tags/SideBar .

In the contents put:

<$list filter="[[$:/temp/search]] +[minlength[3]]">
<<list-links "[has[intelligence]search:intelligence{$:/temp/search}]">>
</$list>

The outer list just makes sure that no searching starts until at least 3 characters have been typed (you'll want to change that for numerical searches, of course). The list-links macro searches in the "intelligence" field, using the contents of the main search box. I used "search" and not find because I'm pretty sure that "find" wants exact matches, whereas you might want partial matches in this instance.

That's it! Save the tiddler and now a new tab will appear in the sidebar, S-Intel. Open it. Type "aver" (or Average) and assuming some of your entries have "Average" in the field they should be listed.

Always make backups before you try any of this, in case you accidentally create an infinite recursive loop or something.

-- Mark

[PWL]

Thanks Mark.  You have been a great help.  Don't be surprised if tomorrow I am looking to expand on this. 

[Eric Shuman]

This reply is AMAZING!

I'm doing a similar kind of project, except that I have a few different classes of entries. So I tried out your code, and it works great. Here's my variant, though:

Currently I can get the template to put the same three-category table (i.e., Terrain, Intelligence, Matrix) in every tiddler. But my classes don't have the same fields. Mine is more like this (for example)--

Astronauts -- Age, Rank, Ship

Ships -- Age, Home Planet, Speed

Planets -- Type, Satellites

Is there a way to call a particular template for a particular category? So that for ex. Planets pages won't try to display a Rank?

I tried changing "Avatar"'s code --

<$list filter="[all[current]]">

to --> ..."[tag[current]]", or "[tag[Planets]]" etc, with my actual data, but I seemed to alternate between getting it to print nothing at all, and somehow getting it to print the data of everything tagged with those fields.

Any tips?

Thanks again!

Eric

[Thomas Elmiger]

Hi Eric,

if I get your task right, you would have to add your own criteria to the filter, e.g.

<$list filter="[all[current]tag[planet]]">

This filter checks first if it is applyed to the current tiddler and then, if this also has the desired tag.

Hope that helps,
Thomas

[PWL]

Hello again Mark,

So I have made some progress building my search function.  This is what I have so far...

<$list filter="[[$:/temp/search]] +[minlength[3]]">

<<list-links "[has[Intelligence]search:intelligence{$:/temp/search}]">>
</$list>

<$button>
<$action-createtiddler $basetitle="Homemade Button" tags="$:/tags/PageControls" text={{$:/state/new-button-caption}}/>
Search
</$button>  <$edit-text tiddler="$:/state/new-button-caption" tag="input" default=""/>



<div class="tc-table-of-contents">
<<toc-selective-expandable 'INDEX' sort[title]>>

And this is how it renders.  Its pretty much exactly what I want.  Problem is I can't figure out how to make it actually search. 

The search field is on the index page, which is what I want.  But it doesn't seem to do any actual searching.  just prints the word I type in on the side of the screen.

On Tuesday, 2 October 2018 21:27:20 UTC-3, Mark S. wrote:

[Mark S.]

The list filter is using the standard search box. ($:/temp/search). If you have created another search field, then it needs to use the output tiddler for that search box instead. You probably need to show the contents of your INDEX tiddler.

-- Mark

[Eric Shuman]

Perfect! That hits the target right on. Thanks Thomas!

Eric

[PWL]

Okay so I figured it out. the search tab now works with the sidebar search function.  The only problem I am having is with the numbers.  If I am searching for an entry with a single number say 2 it returns every entry with the number two in the field.  12 2 22 21 etc.  Can I fix this?

[PWL]

Disregard my last post, I found a work around.  I have another problem now though.  I think I am going to start a new string for it though, this one is getting a little convoluted.  Again thank you for all your help. :)

On Thursday, 4 October 2018 11:10:46 UTC-3, Mark S. wrote:

[barro...@gmail.com]

My apologies  for the delay,

You are correct.  I am attempting to make the Monstrous Compendium 

cool

  However, it is a horrible resource to search, poorly organized, the links are all over the place and it cannot be searched beyond the index entries.  I spend far too much time sifting through the entries and my books looking for the stuff I want.   What I have been doing is copying the text from the page to a tiddler then formatting it, copying the image to my hdd and importing it to the tiddler.  Not exactly the most efficient method.  But it keeps it all local on my hdd which is a requirement.

TW will be good, it has tags, fields and data dictionary that will allow you to do all kinda things.  I made 2 custom TW's for MMORPG-- (which is basically tabletop RPG taken to a virtual level)-- development and all the ways it can organize and present information makes it a perfect management tool as well as database-- I'm reading Mark S's responses to you to see how I can use them as well-- been at this for just a few months so I hadn't gotten to data dictionaries yet..  The members here will be great assistance-- both Mark S and Tony helped me when I first joined.

 

I started out building an actual data base with LO.  But I found it started becoming cumbersome, slow and unstable; probably due to the sheer amount of text.  So I turned to personal wiki's.  Tiddly seems to be the best option I have found for this.  To this point I have learned how to make tabs, tables of contents, tags, tables, and how to format the text and lay out on my tiddlers.  Now I need to figure out how to make the information I use most often searchable; Climate / Terrain, Intelligence, and Hit Dice.  I was thinking fields would be the best way to go, but I am finding the information available confusing.  Reducing the amount typing, and coding I have to do would be a great bonus but learning how to make my data searchable should probably be the priority.  I have not done any serious programming since the 90's.  And I hated it then.  I would love to find a wysiwyg interface for this.

Fields will be good, not only for searching but if you want to beuild tables, reading from fields is easy to build your tables with.

Other users have made some nifty search enhancements-- the official plug in library or Community section on the main TW site will help you find them.

Cool thing about TW, is its preview panel, let you see your wikitext markup and html/css in real time as you edit-- love how I can see the table form as I edit it.

Be careful of embedding images-- too many can overload your wiki.  If possible to have the images in a directory with your images or a set directory there is certain link coding that can display your images without embedding.  I use that trick to link to images in several different project directories. 

 

Also backup's, especially as your time invested and data gets considerable-- I have lost my TW completely (as in disappeared completely )do to Firefox save crashes.  So every time _before I close the browser_ for the day or session, Isave 2 seperate copies in 2 separeate directories and open up the saved copies in another browser window.  Once I'm sure my saved copies are readable, then I close the original browser window for the day.  Also you may want to have your raw data saved separately somewhere-- an exported json file or on a spreadsheet-- in case you have to restore your starting data.

Any help you can give, or point me to a tiddly for dummies type book or site would be greatly appreciated. 

 

I have several search function trick I'llgladly share-- a tiddler that works with the main search box under the TW title that will search fields (which th search box normally doesn't do) and displays the results in a tiddler on the left (something I learned from one of the group members here), As well as a visual step by step configurable 3 stage filter search/list tiddlers.  Not only will it display the end result but also show the pools of tiddlers of each selection step you are selecting from. Using tags it will sort through them and produce lists such as item(s) that is also this, located in this particulararea.  Once you learn the filters in it,it would be easy to change the steps and the logic functions such what area has this item(s) but not these second item(s) or what area had this item(s) and this second item(s).

Also table templates that show field values and their headers are clickable to sort the values ascending or descending (something I learned from JD McCarty iirc-- I just showed a slight mod to it recently)