Shiraz plugin: dynamic table gets search ui

[Mohammad]

I am EXPERIMENTING how to implement a searchable dynamic table in Shiraz!

My top priorities are

• modular design (extensible)
  
• minimalist design
• simple to use
• simple to understand

Today we had a fruitful thread with Eric Shulman who created a simple powerful UI to search and destroy!

https://groups.google.com/d/msg/tiddlywiki/OeVzRs9FeYo/BNNRK5hEBgAJ

I used his design and approach with some minor changes and used it with table-fd the dynamic table builder from fields in Shiraz plugin.

I have pushed an EXPERIMENTAL update and setup a wiki on tiddlyspot to get feedback!

http://tw-tables.tiddlyspot.com/

Now we have searchable dynamic table builder!

I am afraid it deviates the minimalist design principle and gets complex to use!

What do you think?

NOTE: Some UI element like the number of tiddlers (records) in table and constructed filter ARE displayed intentionally and will be removed later!

--Mohammad

[TonyM]

Mohammad,

It looks good. Perhaps placing this, or a variation of it, as an advanced search tab would be appropriate, because that is what it is.

I wonder if the field displayed in the preview slider which is currently text, could be set as well?

Just for fun I extracted your data tiddlers and created both a json and (minimalist) plugin to demonstrate sharing test data

Regards

Tony

[A Gloom]

Mohammad

VERY NICE!!  Will be watching with interest to see if you come up with something better than my mad coding : D

[Mohammad]

Hi Tony!

On Friday, December 13, 2019 at 6:53:20 AM UTC+3:30, TonyM wrote:

Mohammad,

It looks good. Perhaps placing this, or a variation of it, as an advanced search tab would be appropriate, because that is what it is.

I thought to use this as en example (use case) but keep the Shiraz distribution itself minimal!

This table can be used for task management also! with tags and fields derived from standard task template 

I wonder if the field displayed in the preview slider which is currently text, could be set as well?

Yes, all the input can be set as variable and taken from UI.

 

Just for fun I extracted your data tiddlers and created both a json and (minimalist) plugin to demonstrate sharing test data

Nice, how do you pack those data as plugin? :-) 

[Mohammad]

On Friday, December 13, 2019 at 7:39:08 AM UTC+3:30, A Gloom wrote:

Mohammad

VERY NICE!!  Will be watching with interest to see if you come up with something better than my mad coding : D

LOL! 

Where is that? Is it the same you use in your documentation resource project?

--Mohammad

 

[TiddlyTweeter]

Ciao Mohammad

Very good! I have one suggestion ...

• Put this in a wiki of its own. 

Even though it is largely an inspired sub-set of TiddlyTables I think it would still be better in a dedicated wiki.

Best wishes

TT 

Mohammad wrote ...

 <snip>

[A Gloom]

Where is that? Is it the same you use in your documentation resource project?

It was too volatile for the Ref Collective, so I left it out along with the Tiddler Explorer and Whatis! the Wiki Assistant

my way to do the table search is much simpler-- it changes the table filter so it re-renders the table just with content meeting the search and excluding all other previous table rows (previous table content would be the initial total pool of tiddlers being searched)

I showed some parts of it in these topics (its actually constructed from several tiddlers like the ref wikis sidebar search in  the Ref Collective and the Tiddler Explorer)--

the doubling up of column header sorting values (2 fields display and independent ascending/decending sort of each) (Jed Carty's original table coding iirc, I just modified it)

https://groups.google.com/d/msg/tiddlywiki/Y7O0V4Xy_CM/MIwVgiPuAwAJ

the search bar with regexp (applied to search data tiddlers, this made it into one of the Ref Collective sidebar searches)

https://groups.google.com/d/msg/tiddlywiki/_Nukt4YgUPc/8npz6WWOAgAJ

experimenting with using tiddlers as modules to build complex utilitites as I'll show with the Game UI (User interface)

the chart hers shows some of the modular construction

https://groups.google.com/d/msg/tiddlywiki/gsKttLKxG_Y/9--O_YWzAgAJ

screenshot of Tiddler Explorer-- the selections and entry fields are modules, you'll recognize the regezp search bar/radio buttons in there

https://groups.google.com/d/msg/tiddlywiki/lNoMFwnZ3iw/wPdLN-eIAQAJ

[Mohammad]

Hi A Gloom,

 Many thanks for all these great pieces of work you made!

 I am experimenting to see what can I learn and use as improvement in Shiraz!

Best

Mohammad

[Mohammad]

Hello TT,

 I will add an install instruction on Tiddlyspot site for who likes to have only  dynamic tables!

 I also keep it as a dedicated wiki for searchable dynamic table! 

By the way, my main focus is on Shiraz and will maintain it! So, tiddlyspot only may get bug fixes!

--Mohammad

[A Gloom]

Many thanks for the compliments-- they mean a lot!  Sorry I wasn't more help but isolating and preparing the code will take more time I have at the moment-- its holiday-- 19 years and I decided to rededicate Rags & Shadows to my beloved and less of a memorial piece-- need to scour folklore for a new edition name-- something along the lines of renewal...

 

 I will add an install instruction on Tiddlyspot site for who likes to have only  dynamic tables!

 I also keep it as a dedicated wiki for searchable dynamic table!

What you can do with the dynamic tables and search function--

Have the search function its own tiddler/"module" that its transclusion is hidden in a reveal in the dynamic tables tiddler & triggered by a fireld value/setting set by the separate search plugin installation.

[TonyM]

Mohammad

How to pack data as a plugin is surprisingly simple.

In advanced search filter list the tiddlers you want to include and export them as json.

Use the import button to import the json you just created but when the $:/import tiddler appears edit it, rename it, and keep only the plugin-type field with the value plugin.

Ideally I would like a method to bypass the export to file step for both plugin and JSON tiddler creation from a filter.

Regards
Tony

[A Gloom]

Mohammad,

Sorry for the lateness-- was on holiday then out of action...

Here is what I created for myself-- I took out the tags and fields displays 2nd, 3rd & 4th rows of each table entry in the first screenshot which I believe will make it more stable.  Its 12 tiddlers (attached), the second screenshot shows how the top/selection section is made of modules --dotted borders around every other tiddler/module-- which I intend on re-using in other utilities.

There still some unfinished parts and I need to streamline the whole thing with more use of macros and templates, but its operational-- more operational feature than I can briefly describe at the moment-- maybe after I get more rest.  I never made any documentation for it unfortunately, I decided against offering it for use.  Hopefully, it has something you can use-- I believe it has some unusual features.

All tiddlers should be tagged "It's alive" so you can track all the components

[Mohammad]

Many thanks!

I will come back to you after evaluating them.

Cheers

Mohammad

[A Gloom]

An explanation of the code

Other than as a Windows type explorer of tiddlers, it serves other purposes-- a filter builder for count widget, lists for use in other tiddlers, Advanced Search> Filter tab for exporting selection of tiddlers

---

Selection section

Table section

Table header

Table display (in scrollable frame)

---

Selection section (4 transclusions)

-- each section can be disabled to remove them from the filter

Tag selection
Field selection
Type selection
Reg Expression search

-- reverse search (!), regexp (need to add literal, case sensitive, whitespace, words, can be easily done), search field, (single, title-tag-text or all)(need to all multiple field selection) or exclude field (-), start of word/line/exact word
-- Some selectors have tooltips describing their use, a help popup is above it to the right

Table section

Table header

-- filter resulting from selections is shown
--- selection count display with copy to clipboard for ready to use count widget code
--- table list filter display, copy to clipboard for use in other tiddlers (pasting) & copy to Advanced Search> Filter tab for exporting selection of tiddlers
--- Only selections made will show in filter, if a selection section is disabled by checking none, it disappears from filter.  The regexp section has a circled X that clears the search and removes it (the search operator) completely from the list filter code

Table display (in scrollable frame)

Table column collapse control bar for 6 columns (not for the title column) (transclusion)

Table (7 columns)

5 column having field display selection (5 transclusions)
-- when created/modified is selected-- column automarically applies custom date format template
all columns - ascending/desending sortable *(missing directional icons)

removed from demo version: revealable tags and fields/values rows display for each table list result entry

To the right of the tags selection is the starting filter pool selection (content, shadows, system)-- haven't gotten around to applying it

[Eric Shulman]

On Thursday, December 12, 2019 at 1:02:19 PM UTC-8, Mohammad wrote:

I am EXPERIMENTING how to implement a searchable dynamic table in Shiraz!

My top priorities are

• modular design (extensible)
  
• minimalist design
• simple to use
• simple to understand

Today we had a fruitful thread with Eric Shulman who created a simple powerful UI to search and destroy!

https://groups.google.com/d/msg/tiddlywiki/OeVzRs9FeYo/BNNRK5hEBgAJ

I used his design and approach with some minor changes and used it with table-fd the dynamic table builder from fields in Shiraz plugin.

I have pushed an EXPERIMENTAL update and setup a wiki on tiddlyspot to get feedback!

http://tw-tables.tiddlyspot.com/

Now we have searchable dynamic table builder!

I did a little experiment myself, to combine my latest version of "PowerSearch" (renamed from SearchAndDestroy) with the dynamic table output from Shiraz.

Here's what I did:

1) From http://tiddlytools.com/filtergenerators.html, import these tiddlers (drag-and-drop from the sidebar list)

* TiddlyTools/FilterGenerators/PowerSearch

* TiddlyTools/Macros/edit-list

since these are just macro definitions, there's no plugins involved and they can be used right away, without needing to save-and-reload.

2) Create a new tiddler containing:

\import TiddlyTools/FilterGenerators/PowerSearch

\define doSearch_showresults()
<$macrocall $name=table-fd
 filter=<<filter>> tblClass=""
 fields="tbl-expand title fa fb fc caption tags"
 stateTiddler=n
 tblCaption="''Table 2. Dynamic table example''"/>
\end

\define getOptions() <!-- NO DISPLAY OPTIONS -->

<<setStyles>>
<<getInputs>>
<$vars
  tids={{{ [<tidinput>get[text]]   ~[[tiddlers]]        }}}
   pre={{{ [<preinput>get[text]]                        }}}
   tag={{{ [<taginput>get[text]]                        }}}
 field={{{ [<fieldinput>get[text]] ~[[title,text,tags]] }}}
  flag={{{ [<flaginput>get[text]]  ~[[words]]           }}}
  term={{{ [<terminput>get[text]]                       }}}
  sort={{{ [<sortinput>get[text]]  ~[[title]]           }}}>
   <<doSearch>>
</$vars>

What it does:

1) import all the macros from PowerSearch

2) override doSearch_showresults() to invoke the "table-fd" macro call

    (note: use filter=<<filter>> instead of filter=<<uifilter>>)

3) override getOptions() button to remove the button and popup

    (note: they only apply to the default PowerSearch "showresults()" output)

4) get the inputs and do the search

   (same as the PowerSearch code)

note that I don't apply your styles to the inputs, since the edit-list display

relies upon overlapping of edit-text and select controls, which wouldn't

look right if you change the input control styles.

The result: my latest search input interface, with the Shiraz table output!

enjoy,

-e

[HansWobbe]

@Eric: @Mohammad:

This is a really nice collaboration result! 

Many thanks to both of you.

And, of course, Season's Greetings to one and all. Best wishes for the New Year and indeed throughout the next New Decade.

Hans

 

The result: my latest search input interface, with the Shiraz table output!

enjoy,

-e

----

appended Edit (as another part of my gGroups formatting tests associated with a_tag)

[Mohammad]

Hi Eric,

 many thanks! this is a really powerful search tools!

 I will make some experiments and will return with the result.

[Mohammad]

Thanks Hans!

 I wish you all the best in new coming year!

--Mohammad

On Tuesday, December 24, 2019 at 4:13:42 PM UTC+3:30, HansWobbe wrote:

@Eric: @Mohammad:

This is a really nice collaboration result! 

:-)

[Mohammad]

Hi Eric,

 I am wondering how we can use a kind of power search you developed with Tiddler Commander.

As I described in separate email, there we use two step search

1. filter search
2. select among the search by step 1

--Mohammad

[Eric Shulman]

As I illustrated in a previous post, you can override the doSearch_showresults() macro to produce whatever kind of output you like by using the generated <<filter>> variable as input to further code.

Thus, step 2 could be something like:

<$list filter=<<filter>>>
   <$checkbox tiddler="$:/temp/selectedTiddlers" index=<<currentTiddler>> checked=" "
        checkactions="""<$action-setfield $tiddler="$:/temp/selectedTiddlers" $index=<<currentTiddler>> $value=" " />"""
      uncheckactions="""<$action-setfield $tiddler="$:/temp/selectedTiddlers" $index=<<currentTiddler>> />""">
         <$text text=<<currentTiddler>>/>
   </$checkbox><br>
</$list>
<$button> DO STUFF!
   <$list filter="[title[$:/temp/selectedTiddlers]indexes[]>
      <$action... tiddler=<<currentTiddler>> ... />
   </$list>
</$button>

The first part uses the generated <<filter>> to render checkboxes for each tiddler that matches the search.

Selecting a checkbox adds the name of the corresponding tiddler as an index in $:/temp/selectedTiddlers.

Then, the second part provides a button that invokes some action widget(s) for each item listed in $:/temp/selectedTiddlers

Of course, the actions performed can be based on whatever is specified by the TiddlerCommander interface.

-e

[Mohammad]

Hi Eric,

 Many thanks!

As you note, I have used a similar approach for TiddlerCommander! 

https://kookma.github.io/TW-Commander/#%24%3A%2FCommander

The problem arises when I do some actions like title manipulation (add remove prefix, ...)

a tiddler selected already will be remained in $:/temp/selectedTiddlers

The same is true for other operation! You may search and filter some tiddlers by tags or field value and then you do operation on such fields

The selected tiddlers still are in $:/temp/selectedTiddlers while the do not meet the search criteria any more!

Right now after each bulk operation I remove all selections and user has to reselect them.

--Mohammad

[passingby]

Hello Eric,

I have both of your tiddlers copied. And as expected the PowerSearch works on its own perfectly. I have also installed the Shiraz plugin and working on its own as well. Then I created another tiddler containing the overriding definitions which you've given here( I had to change the macro call name to table-dynamic which I assume changed at a later date). I can see that PowerSearch is being called because the result table is shown but I don't have PowerSearch UI. What might I be doing wrong?

[Eric Shulman]

On Thursday, April 14, 2022 at 11:04:08 PM UTC-7 passingby wrote:

I have both of your tiddlers copied. And as expected the PowerSearch works on its own perfectly. I have also installed the Shiraz plugin and working on its own as well. Then I created another tiddler containing the overriding definitions which you've given here ( I had to change the macro call name to table-dynamic which I assume changed at a later date). I can see that PowerSearch is being called because the result table is shown but I don't have PowerSearch UI. What might I be doing wrong?

It's been more than 2 years since I wrote those instructions for "overriding definitions".

As you noted, in that intervening time, the Shiraz macro name changed from "table-fd" to "table-dynamic".

In addition, the underlying PowerSearch and edit-list code have also been extensively re-written and

re-factored several times, so some of those macro names and surrounding syntax have changed as well.

Instead of writing:

 

\import TiddlyTools/FilterGenerators/PowerSearch

\define doSearch_showresults()
<$macrocall $name=table-fd
 filter=<<filter>> tblClass=""
 fields="tbl-expand title fa fb fc caption tags"
 stateTiddler=n
 tblCaption="''Table 2. Dynamic table example''"/>
\end

\define getOptions() <!-- NO DISPLAY OPTIONS -->

<<setStyles>>
<<getInputs>>
<$vars
  tids={{{ [<tidinput>get[text]]   ~[[tiddlers]]        }}}
   pre={{{ [<preinput>get[text]]                        }}}
   tag={{{ [<taginput>get[text]]                        }}}
 field={{{ [<fieldinput>get[text]] ~[[title,text,tags]] }}}
  flag={{{ [<flaginput>get[text]]  ~[[words]]           }}}
  term={{{ [<terminput>get[text]]                       }}}
  sort={{{ [<sortinput>get[text]]  ~[[title]]           }}}>
   <<doSearch>>
</$vars>


 

Try this:

```

\import TiddlyTools/FilterGenerators/PowerSearch
\define doSearch_showresults()
<$macrocall $name=table-dynamic filter=<<filter>> tblClass=""

 fields="tbl-expand title fa fb fc caption tags"
 stateTiddler=n tblCaption="''Table 2. Dynamic table example''"/>

\end

<<styles>> <<inputs>>
<div style="clear:both;padding-top:0.5em;"/>
<<doSearch>>

```

Note: if you also omit the `<div style="clear:both;padding-top:0.5em;"/>`, then the PowerSearch inputs and the Shiraz output table will appear side by side (as long as there's enough room for the table)

enjoy,

-e

[passingby]

🙏 As always, par excellence.
Thank You!