Plugin / Pull Request: Comined Search: Find system + shadow tiddlers, use filters in simple search

[bimlas]

What do you think? Is it usefull enough to let it in the core?

Regular search (nothing new, just want to show the current behaviour): https://i.imgur.com/ovBjJeV.png

Find system + shadow tiddlers: https://i.imgur.com/yMTWt5d.png

Search by filters: https://i.imgur.com/xIXtmn6.png

Demo: https://bimlas.gitlab.io/demo/tw5/combined-search.html

Note: The "Filter error: Missing closing bracket in filter expression" is not shown at all.

To achieve this behaviour, rewrite the contents of $:/core/ui/DefaultSearchResultList with this:

\define filterExpression() {{$(searchTiddler)$}} +[limit[$(resultsLimit)$]]
 
\define filterSearch()
 
<$wikify name="transcludedFilterExpression" text=<<filterExpression>> >
<$list filter=<<transcludedFilterExpression>> template="$:/core/ui/ListItemTemplate"/>
</$wikify>
 
\end
\define regularSearch(scope)
 
//<small>{{$:/language/Search/Matches/Title}}</small>//
 
<$list filter="[$scope$search:title{$(searchTiddler)$}sort[title]limit[$(resultsLimit)$]]" template="$:/core/ui/ListItemTemplate"/>
 
//<small>{{$:/language/Search/Matches/All}}</small>//
 
<$list filter="[$scope$search{$(searchTiddler)$}sort[title]limit[$(resultsLimit)$]]" template="$:/core/ui/ListItemTemplate"/>
 
\end
\define searchResultList()
 
<$list filter="[{$(searchTiddler)$}prefix[[]]">
 
//''<small>{{$:/language/Search/Filter/Caption}}</small>''//
 
<<filterSearch>>
 
</$list>
 
<$list filter="[{$(searchTiddler)$}prefix[$:/]]">
 
//''<small>{{$:/language/Search/System/Caption}} + {{$:/language/Search/Shadows/Caption}}</small>''//
 
<<regularSearch "all[system+shadows]">>
 
</$list>
 
<$list filter="[{$(searchTiddler)$}!prefix[[]!prefix[$:/]]">
 
<<regularSearch "!is[system]">>
 
</$list>
 
\end
<$vars resultsLimit="250">
<<searchResultList>>
</$vars>

[Mark S.]

That would be a handy improvement on the standard search.  The one thing is that it might require a different character limit. The standard default is 3 characters, but if you type in $:/ you've already met that limit and are exposed to a massive result list.

Somewhere down the line, it would be great if TW could save your searches (like in other search engines) so you could quickly go back to a prior search.

-- Mark

On Wednesday, November 14, 2018 at 10:47:55 AM UTC-8, bimlas wrote:

What do you think? Is it usefull enough to let it in the core?

Regular search (nothing new, just want to show the current behaviour): https://i.imgur.com/ovBjJeV.png

Find system + shadow tiddlers: https://i.imgur.com/yMTWt5d.png

Search by filters: https://i.imgur.com/xIXtmn6.png

[Thomas Elmiger]

Hi Bimlas,

Honestly, I did not try* your solution but looked at your screenshots. Thanks for your suggestion, we are in the same boat, trying to improve searching TW.

My feedback on first sight:

• Counting results: Your filter search shows a counter with zero results (instead of 7), system & shadows counts 6 instead of 7.
• Filter search: How would I search for (e.g. standard) tiddlers containing a certain filter (as text)? Which leads to the next point:
  
• Search mode: Is it possible to switch search mode somehow like in Advanced Search?
  
• Hints on filter errors: I appreciate them and would miss them ...

Advanced Search in TW is just one or two clicks/taps or key strokes away from standard TW search. So I am sorry to say, this would not be an improvement for me. Just advanced features merged into a simple search, but with less control and less meta info.

* I am happy with my own plugin Simple Search :)

Cheers & keep on improving!

Thomas

[bimlas]

The problem with the minimum character is also apparent to me, the solution would be to ignore the length of the prefix, but the length of the search term is managed at another level - it would require more serious intervention.

I do not really understand the purpose of the search history. Could you give a practical example of where it would be useful? Is the $:/tags/Filter tag not enough to solve this problem?

[Thomas Elmiger]

Mark,

This is a good point, partially covered in Advanced Search:

Somewhere down the line, it would be great if TW could save your searches (like in other search engines) so you could quickly go back to a prior search.

You can store filter searches in the dropdown menu using $:/tags/Filter – but only for filters. And you have to do it yourself. It would be really great if TW would

• offer such dropdowns or suggestions in all modes
• register recent searches automagically

-- Thomas

[bimlas]

Hi Thomas,

 

Honestly, I did not try* your solution but looked at your screenshots. ... I am happy with my own plugin Simple Search :

Here's a demo: https://bimlas.gitlab.io/demo/tw5/combined-search.html

Just tried out to install in to your wiki - it works well with your plugin. :) 

 

• Counting results: Your filter search shows a counter with zero results (instead of 7), system & shadows counts 6 instead of 7.

 

The counter is defined elsewhere - if I actually make a pull request or a plugin, I have to modify it too. Thanks for the remark!

• Filter search: How would I search for (e.g. standard) tiddlers containing a certain filter (as text)? 
  

 Just add a leading space to the filter (`[tag[$:/tags/Macro]]` behaves filter, while ` [tag[$:/tags/Macro]]` is a regular search).

• Search mode: Is it possible to switch search mode somehow like in Advanced Search?
  

 I do not quite understand what you mean: the prefix determines the mode ($: / - system + shadows; [... - filter]. If you need more precise search, you can still use Advanced Search.

• Hints on filter errors: I appreciate them and would miss them ...

I will fix it when it comes to plugin or PR.

 

Advanced Search in TW is just one or two clicks/taps or key strokes away from standard TW search. So I am sorry to say, this would not be an improvement for me. Just advanced features merged into a simple search, but with less control and less meta info.

 Maybe you're right, but I'm just bothering to click: if I need to search for a shadow, I have to go into Advanced Search and switch to the Shadows tab. Why is this necessary if we already have a search bar in the sidebar?

[PMario]

Hi,

Nice stuff! --- but ---

I think, this should be a plugin. We do have the Advanced search, which has an interface, where it is clear, what you get.

Testing your code with $:/core returns an unusable list of results. IMO it would be confusing for "standard" users.

just my thoughts.

mario

[bimlas]

Hi PMario,

 

I think, this should be a plugin. We do have the Advanced search, which has an interface, where it is clear, what you get. 

That's why I put it for the first time in the forum: I'm not sure that this would be "standard".

I originally wanted to embed Advanced Search into the sidebar (instead of the standard search), which can be done with prerelease, but the results are not a dropdown list, which is bad. Then came the idea that the search term determines the subject of the search.

Anyway, this is probably just a plugin. Thanks for the comments!

[Diego Mesa]

Hello all,

As usual Bimlas beautiful work. I disagree with Mario - I dont think this would confuse new users, and should be a part of the core. There seems to be a big wave at NOT changing the way TW works, but in my experience trying to onboard many new people onto TW there is A LOT of confusion as to how things work in TW. And search is definitely one of those things. 

[bimlas]

Hi Diego,

There seems to be a big wave at NOT changing the way TW works

I agree that everything should be in the core, which is why TiddlyWiki did not break even in 20 years.

I do not know, for example, that Jeremy likes the idea, but if it does not get into the core, the documentation can still contain a link to it. :)

[bimlas]

Sorry, Google Translate: I agree that NOT everything should be in the core...

[Diego Mesa]

I dont want to hijack this thread - I think your plugin is great and I think it should be part of the core

Ill end by saying this: There is a tension between having TW work well for 20 years for 20,000 people over the world, and having TW work well for 2 years for 2 million people all over the world. 

Especially when you consider that TW is just a file you download - so if the TW core changes more and more frequently, its very easy for your TW to remain feature-frozen (like many linux OS with Long Term Support)

[TonyM]

On the subject of adding to the core, 

Remember such things could be part of the core plugins, In Get more plugins, as codemirror currently is. Or even in the empty distribution, but removable.

No need to bloat the minimum which could be used for anything and not need such features.

On ANY search result, it would lovely to be able to one click save it in a list, perhaps even better tag them all, then it would be easy to step through each found tiddler and add or remove items from the list. Being able to use multiple searches to create one tagged list would be fantastic. And even better if this would be an add or remove filter.

Lists, search and tagging are fundamental features so why not provide the facility to combine them?

Personalty I want the following things I do not yet have in a search box in the sidebar

• A search that will also list System tiddlers
  
• A Search which is like TWC go to that only searches on title
• The ability to copy the tiddler title wrapped [[as required]] from the search results list
• The possibility to open for edit a tiddler from the search list without viewing it first (in case its causing a fault)
• Saving search results as previously mentioned.

Regards

Tony

[bimlas]

Search history seems to be a more lacking feature than a combined search.

I was thinking how to do it. The browser saves the input in the search history when you press Enter, but Tiddly does not need it because it keeps updating the list of results continously.

A solution could be a timer such as $:/config/Drafts/TypingTimeout: after the last typed character after X time adds the current input to the search history, the first 3 history could appear at the beginning of the list of results. I'm not sure whether the history should be in advanced search, for example, exporting tiddlers would want to export search history as well, but it's just a list of strings, not tiddlers.

Concept art: https://i.imgur.com/6SoY5P8.png

What do you think of this idea?

[bimlas]

I've found the already existing way to easily use Advanced Search in standard search: it displays various lists in tabs.

See the demo at https://bimlas.gitlab.io/demo/tw5/combined-search-by-tabs.html

[bimlas]

Just fixed some bugs:

• Search by title and content was the same (?!), beacuse I uses `is[]` everywhere, while `shadows` needs `all` (weird, see https://gitlab.com/bimlas/demo/blob/9794dbe2a6856d484a4bf4478f75b0261d36d256/public/tw5/combined-search-by-tabs.html for previous, buggy version)
• Reordered the tabs to look like advanced search

[HC Haase]

This is cool. I can see this being very useful or maybe not at all - I will have to wait and see if this solves a problem I didn't knew I had.

However In my advanced search I frequently use Tobais´ in fields search. this is very useful as it search and list all fields.

I have made a crude search and replace version of tobias´ field search, to include it in your combined search. I don't think it mess up any thing, but as an amateur I don't know.

But now I am not sure how to name this.

$:/.tb/ or $:/bimlas/

Anyway If anyone else is interested here it is.

[bimlas]

Probably your name suits the best for prefix of your own "plugins": name it $:/.hchaase for example. :)

As I see, you just using the $:/tags/SearchResults from my "plugin": it's not my invention, it's a builtin feature, see https://tiddlywiki.com/#Customising%20search%20results

[TonyM]

Bimlas,

Very nice, once again.

Thanks

[bimlas]

I've added a new tab that allows you to search for tags: https://bimlas.gitlab.io/demo/tw5/combined-search-by-tabs.html

I think this one could be in the core, because the tags are densely used and currently there is no built-in tag search, just the MoreSidebar tags tab (try to import $:/bimlas/combined-search/tags into https://tiddlywiki.com/ to see what I mean).

[bimlas]

I hope you like it :)

https://bimlas.gitlab.io/demo/tw5/combined-search-for-tonym.html

Feel free to ask for modifications if something is different from your "requirements".

[TonyM]

Bimlas,

Thanks so much, its beautiful. Fast

I cant wait for the 5.1.18 release to be come General availability, I am reluctant to upgrade some of my key wikis until this is so.

Regards

Tony

[bimlas]

I added a stylesheet to show the buttons with the list item in the same row ($:/bimlas/combined-search/stylesheet-for-tonym).

I forgot to mention that it works with 5.1.17, not just prerelease.

[bimlas]

Another improvement: moved the "regular" tiddlers to the top of the list, system and shadows are on the bottom (if you don't need to search for shadows, you can remove them from the search query of course).

[HC Haase]

Hi bimlas

I find your "stylsheed for tony" with the buttons next to the search result, quite nice (I only use the buttons part in combination with the tabbed combined search). However if you use the "insert link" button in edit mode of a tiddler you get these big grey buttons stacked on top of the search result - Not looking nice. (screenshot from your demo page).

[HC Haase]

would you consider to make this a plugin? It gets quite confusing what function/bug comes from where, when you have many plugin and custom modifications (like this). If packaged as a plugin it is much easier to turn things off when searching for bugs.

[bimlas]

@HC Haase: Sorry for late reply, I corrected the mistake, grab $:/bimlas/combined-search/stylesheet-for-tonym from https://bimlas.gitlab.io/demo/tw5/combined-search-for-tonym.html.

The problem is that the buttons are not designed to appear in dropdown lists, therefore, it is not certain that this is the best solution for placing the "action buttons".

I plan to implement a similar behaviour, but not by puting buttons on every list item, so I do not release this (combined-search-for-tonym.html) as a plugin.

[HC Haase]

thanks

I plan to implement a similar behaviour, but not by puting buttons on every list item, so I do not release this (combined-search-for-tonym.html) as a plugin.

Looking forward to this