Searching Tiddlers with transcluded content

[Emily Dies]

Hi, 

I'm creating a tiddlywiki to be used as a reference tool for non-technical users. 

One issue that I am running into is that if I transclude content "ABC" into tiddler X, the search function does not find X when I search for ABC. 

Is there a workaround for this? I've made some changes to the tiddler $:/core/ui/DefaultSearchResultList to capture additional fields, but I'm not sure if it's possible to search within transcluded content. 

Thanks in advance for your help. 

[S. S.]

Emily, there was a discussion some months ago which may shed some (very dim) light : Backlinking' transclusions ?

---

The rest of this is not an answer - just some thoughts relating to the question.

As I create more and more "written" material, I keep running against this "smallest semantic unit" - and "small tiddlers re-used by transclusion."

My constant concern is, when I want to find that "written" material, will I be able to find it in its proper context (with the other main material of which it is a part) - quickly, easily, and without fail.

For example, I take an excerpt of a few paragraphs from a book by one author and put it in a tiddler. Within that excerpt is a quote from another book with another author. If I put the quote in a second tiddler and transclude it back into the first tiddler, I would need a way to know later, which tiddler has transcluded this second tiddler, as a search will find the quote in the second tiddler, but in itself, without its context, this quote is useless.

I may be able to find its place by searching for the literal " {{second tiddler}} ". But what if the transclusion is done by a macro, or <$transclude .../>, or <$list filter=" ... "/>

One solution would be to have a "related" field in the second tiddler (or maybe first tiddler?) that is manually updated each time it is transcluded, but this is more cumbersome to do than first imagined.***See the note below

To avoid this happening, whenever I have material that I expect may need to be searched for and found within its proper context - and I cannot find a way to "safely" transclude it - I do not separate it into a smaller unit. I keep it within its context no matter how big the tiddler becomes.

I have a feeling this philosophy I am using is inherently wrong. Perhaps one day I will understand a better way of doing it.



*** Note:
Automagically keeping track of related tiddlers
Handling Changing Relationships Between Tiddlers

[TonyM]

Emily and S S

Especially when it comes to intrinsically related data, I tend to start with a body of text and excise it, into our partial-tiddler(s) not only will the excise replace it but It also tags the new tiddler with the title of where is was excised from.

Thus if I was to search for something and it is found in a particular tiddler, then what that tiddler is tagged with is likely where I will find it transcluded as well.

Since the excise uses the transclusion of 

{{partial-tiddler}}

Your search could also be extended to the above string just on finding your partial-tiddler name.

When you use list and other methods to transclude content, this is presumably done using tags or values in fields, as long as you set yourself a standard it will be possible to find the information you need in the partial-tiddler to find the places in which it is referenced. For example you could use the new subfilter operator and a define to create a filter containing [subfilter<mysubfilter>] and then you could search for the use of "<mysubfilter>". Set up some subfilters you use exclusively for the types of transclusions you are doing and searching from them will reliably return what you are after. 

In my consolidating tiddler, or a version of it, I place links to the transcluded tiddlers so so I can come back the other way. Ie Open the consolidated tiddler, search in the browser window, locate the link to the partial tiddler and open it.

You could have a consolidated tiddler, designed just to identify the partial-tiddlers(s) from which something comes.

<$list filter="[tag[Essentials and Desirables]sort[created]]">
<h2>{{!!title}}</h2> {{||$:/core/ui/Buttons/edit}}
<$transclude mode="block"/>
{{!!print-footer}}
</$list>

The above transcludes all tag[Essentials and Desirables] and provides an edit link to each

Regards

Tony

[FrD]

Hi,

I'm wondering whether an option could be added to the search operator allowing to search a "wikified" version of the text (or the field) of a tiddler.

The change in the code could be in the search function that lives in wiki.js. The field that is to be searched could be wikified before the search according to an option or a field value for this particular tiddler.

I believe there should be more tools to manage transclusions as transclusions are an ordinary way to organize knowledge in TW. I feel sometimes that :

- on one hand we don't have "sections" anymore (I was a fan of sections in TWC) and so we have to cut information into small pieces and use transclusions. And I'm fine with that.

- but on the other hand we don't have many tools to deal with transclusions.

A search with wikified content could be such a tool.

Regards

FrD

[Jeremy Ruston]

I'm wondering whether an option could be added to the search operator allowing to search a "wikified" version of the text (or the field) of a tiddler.

It's an interesting idea, and I'm sure worth some experimentation but there is a substantial performance issue in having to wikify all tiddlers before every search operation.

The closest thing we've got at the moment is the orphan and missing filters which have to parse every tiddler in order to gather all the explicit links. Accessing the Missing and Orphans tabs is very slow on big wikis because of that parsing overhead.

Adding the overhead of rendering on top of parsing would be a lot slower, in some cases several orders of magnitude slower. Some tiddlers are inherently slow to render, and we'd probably need a way to exclude them from the search, or restrict the search to tiddlers of interest.

Best wishes

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/3062125d-37ee-4d19-a601-b306db345141%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mark S.]

Running it all the time would be resource-intensive, for sure.

But what about a tool that renders all the tiddlers ON DEMAND with a title like $:/rendered/<title> with a tag "rendered" and the original title in the caption field. Before the tools runs, it would delete existing items tagged as "rendered".  The search engine would display the original title and link of any item marked as "rendered". So to a user it would feel like they're finding everything, including transcluded text. And of course, the rendered tiddlers could be deleted whenever needed to improve performance.

-- Mark

[Jeremy Ruston]

Hi Mark

Running it all the time would be resource-intensive, for sure.

But what about a tool that renders all the tiddlers ON DEMAND with a title like $:/rendered/<title> with a tag "rendered" and the original title in the caption field. Before the tools runs, it would delete existing items tagged as "rendered".  The search engine would display the original title and link of any item marked as "rendered". So to a user it would feel like they're finding everything, including transcluded text. And of course, the rendered tiddlers could be deleted whenever needed to improve performance. 

I think that’s equivalent to my suggestion of restricting the search to a subset of tiddlers combined with something equivalent to our existing caching mechanism. We already cache the results of parsing tiddlers, and we could reuse the same mechanism to cache the wikified tiddlers. The trouble is that caching is of limited value cache invalidation would be very hard. The current refresh mechanism doesn’t work at the level of a whole tiddler, it works at the level of a node in the render tree. In order to find out whether a cached rendered tiddler needed re-rendering we’d have to run the refresh process over its render tree. In fact, right now things would actually work more efficiently by rendering all the tiddlers direct to the DOM, and then searching via the DOM (ie using the browser search).

Best wishes

Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/88cc5ba0-23dc-4229-8189-8581bdc1a03e%40googlegroups.com.

[TonyM]

Folks,

Speculation

Would it be posible to export a fully wikified tiddlywiki to a file, a bit like an index file, then use search in that file to identify the target tiddlers with rendered content? Is the static html version like this?

Now somehow intergrate this with the original wiki so that the above process is somewhat hidden from users? Perhaps even in an iframe, in which you can see the rendered content and search results with a way to open tiddlers in the main wiki.

A solution like this could also assist search engin optimisation for rendered content, perhaps with links to the main wiki.

Regards
Tony

[Mark S.]

Idea #665

Modify excision so that when the new tiddler is made, it has a "backlinks" field pointing to the tiddler from which it was excised. Modify the search routine to add the backlinks field to the output list. Possibly each time an excision occurs, add to the backlinks field, so all tiddlers leading to the original text end up in the final list.

-- Mark 

[TonyM]

Mark,

Love it.

If as I hope Bimlas's Kin operator becomes part of the core, you could use the kin operator to find the parents of this "backlinks" field, or for that matter any other kin relationship. The field should be a title list allowing multiple references such that if transcluded in more than one place the second and subsequent references will be appended to the "backlinks" field.

You could also use tiddler-citations as a toc structure to picture the relationships

My Black hat, What happens if the transcludion, reference is removed after the citation link is created?

PS I do nit recommend the field-name backlinks as it has other meanings that it will not live up to or confuse. Perhaps tiddler-citation(s) and we could build other manual interlinks or tools around this.

Regards

Tony