'Backlinking' transclusions ?

[Mark S.]

Hello everyone,

We can find backlinks for links, but what about for transclusions?

Use case:

I want to archive off tiddlers tagged, let's say, 2015. But I also want to find and archive off any other tiddlers that are transcluded in those same tiddlers. How can I find them, since they might be transcluded with {{tiddler}} or with <$transclude .../>. Am I stuck doing multiple regexp passes?

Ideas?

Thanks!
Mark

[PMario]

On Sunday, December 10, 2017 at 11:50:47 PM UTC+1, Mark S. wrote:

...  since they might be transcluded with {{tiddler}} or with <$transclude .../>. Am I stuck doing multiple regexp passes?

There are a lot more of dependencies. ... eg: <$list filter="[tag[test]]"><$view>.....</$list>  
Just to name one.

To avoid the headache, save the whole TW, name it "Includes 2015.html". In the working copy delete the stuff, that you don't want anymore,  and you are good.

have fun!
mario

[Mark S.]

I'm more worried about transclusions, since they would be used for insertion of _canonical_uri images. It's the images and the supporting tiddlers that would be hived off.

-- Mark

[PMario]

On Monday, December 11, 2017 at 4:31:01 AM UTC+1, Mark S. wrote:

I'm more worried about transclusions, since they would be used for insertion of _canonical_uri images. It's the images and the supporting tiddlers that would be hived off.

The code would be similar to the link-widget detection, which has to check the parsetree for every tiddler. Since this calculation is "expensive", the results are cached internally, to speed up further searches.

So detecting the transclusion-widget would result in similar code, which is relatively low level. But the code is nicely separated, so it should be doable. 

Link detection can be found in wiki.js

-m

[TonyM]

Mario,

Same technology different idea. Sorry for of topic.

If every tiddler (title) is cached internally for the link-widget detection could this also be used to highlight all text in a tiddler, on rendering a tiddler, that matches any existing tiddler title (or tag)?, this would highlight somewhat relevant information in text and provide opportunities to make links to tiddlers where none currently exist. If at the same time the rendering process would also highlight text that matches the last search string, it would allow people searching for a word or phrase to open tiddlers and see all occurrences of the search string highlighted.

It seems to me if we already spent money on an "expensive" calculation lets get more in return.

Such an approach will support a number of requirements raised in the forum where we are struggling to find reasonable and inexpensive solutions.

Regards

Tony

[PMario]

On Tuesday, December 12, 2017 at 12:40:53 AM UTC+1, TonyM wrote:

If every tiddler (title) is cached internally for the link-widget detection could this also be used to highlight all text in a tiddler, on rendering a tiddler, that matches any existing tiddler title (or tag)?,

Not really possible. There isn't enough info in the cache atm. It just holds a links[] array. And the parse tree has no positional info. ...

 

this would highlight somewhat relevant information in text and provide opportunities to make links to tiddlers where none currently exist.

I don't understand this. [[doesn't exist]] is already possible now. ?!?

 

If at the same time the rendering process would also highlight text that matches the last search string,

not possible. ... You can't style text-nodes. You first need to find the text -> convert it to eg: a span-node -> give the span a styling. ...

 

it would allow people searching for a word or phrase to open tiddlers and see all occurrences of the search string highlighted.

If I need this behaviour, I use the browser built in search.

 

It seems to me if we already spent money on an "expensive" calculation lets get more in return.

Good idea, but time consuming.

 

Such an approach will support a number of requirements raised in the forum where we are struggling to find reasonable and inexpensive solutions.

"inexpensive" is the key word here.

have fun!

mario

[TonyM]

Perhaps I don't get it then, No need to respond if what I say is clearly barking up the wrong tree.

On Tuesday, 12 December 2017 22:10:49 UTC+11, PMario wrote:

On Tuesday, December 12, 2017 at 12:40:53 AM UTC+1, TonyM wrote:

If every tiddler (title) is cached internally for the link-widget detection could this also be used to highlight all text in a tiddler, on rendering a tiddler, that matches any existing tiddler title (or tag)?,

Not really possible. There isn't enough info in the cache atm. It just holds a links[] array. And the parse tree has no positional info. ...

What I thought was there was a List of All Tiddlers - a set of titles we could use to search a tiddler to be rendered with, for such titles that match the text create Virtual or Temporary links on any text in the tiddler that matches an Existing Tiddler name. The Links will be valid because they link to existing tiddlers, they just do not have the [[this is a tiddler name]] around them in the tiddler., ie we do not update the source tiddlers text field, only add the "virtual"  links at render time. One could then scan the tiddler for relevant matches.

 

this would highlight somewhat relevant information in text and provide opportunities to make links to tiddlers where none currently exist.

At least manually 

I don't understand this. [[doesn't exist]] is already possible now. ?!?

What I mean is lets say a tiddlers text contained the following "using Alphanumeric values in" - No links present, but the existence of a Tiddler named "Alphanumeric values" we would  alter the rendering process to treat "Alphanumeric values" as [[Alphanumeric values]]

 

 

If at the same time the rendering process would also highlight text that matches the last search string,

not possible. ... You can't style text-nodes. You first need to find the text -> convert it to eg: a span-node -> give the span a styling. ...

Perhaps you are right, I am out of my depth, noting ventured, nothing gained. 

 

it would allow people searching for a word or phrase to open tiddlers and see all occurrences of the search string highlighted.

If I need this behaviour, I use the browser built in search.

That is  all I can do now, Search with Tiddlywiki, open all tiddlers that contain the text then copy the same search string into the Browser find

 

 

It seems to me if we already spent money on an "expensive" calculation lets get more in return.

Good idea, but time consuming.

If there were value in the idea, lets put it on the shelf, perhaps one day it will become easier with another development.

I was looking for quick wins, if not lets not bother for now.

 

 

Such an approach will support a number of requirements raised in the forum where we are struggling to find reasonable and inexpensive solutions.

"inexpensive" is the key word here.

Yup

 

have fun!

mario

\

Thanks for your feedback

Tony 

[PMario]

================== slightly OT ==================

On Tuesday, December 12, 2017 at 1:30:25 PM UTC+1, TonyM wrote:

I don't understand this. [[doesn't exist]] is already possible now. ?!?

What I mean is lets say a tiddlers text contained the following "using Alphanumeric values in" - No links present, but the existence of a Tiddler named "Alphanumeric values" we would  alter the rendering process to treat "Alphanumeric values" as [[Alphanumeric values]]

Hmmm,

But how should a program know, that it should search for a title "Alphanumeric values" and not "using" or "Alphanumeric" or "values" or "in" or "using Alphanumeric" or "Alphanumeric values" or "values in" or "using Alphanumeric values" or "Alphanumeric values in" or "using Alphanumeric values in" ... (I hope I got them all ;)

You get the problem?

What we can do with plugins already is eg: <<aka "Alphanumeric values">> wich may be defined in a tiddler "aliases" field. The link is created to open the tiddler title.

At the moment I'm finishing the "uni-link" plugin. It will allow us to use something like this:

"using [[Alphanumeric values]] in"  ... Where the uni-link renders the "subtitle", "caption" or "title" of the tiddler. ... "backlinks" are defined.

"using [[Alphanumeric values|?]] in" ... Now "Alphanumeric values" is an alias, and links to the tiddler, that contains the alias. .... Backlinks "work in progress" .. much more work as it seemed to be :/

...

=========== OT end ==========

I think, that a similar mechanism can be used to detect {{}} or <$transclusion> ...

have fun!

mario

[TonyM]

Mario,

Sounds like we are most of the way there, Love your work, If mark can find his target text because they are highlighted as links he may be able to do what he wants via a different avenue, So it may still be on topic. However I would think all he needs to do is tag the tiddlers he plans to archive off, then run a search for the tiddlers titles in all tiddlers, open each of these tiddlers, and review the "virtual links" which will highlight text that matches his search strings (his tagged Tiddler Titles), if he then wants to archive the viewed tiddler he could tag that tiddler and run the search again and discover any additional tiddler to archive.

Let me clarify by answering one of your questions, and simplify my suggestion, I think your existing solutions means you could implement this easily

But how should a program know, that it should search for a title "Alphanumeric values" and not "using" or "Alphanumeric" or "values" or "in" or "using Alphanumeric" or "Alphanumeric values" or "values in" or "using Alphanumeric values" or "Alphanumeric values in" or "using Alphanumeric values in" ... (I hope I got them all ;)

Only search for strings that exactly match (all) existing tiddlers titles and or captions. In this case "Alphanumeric values" ONLY, perhaps case insensitive.

It makes sense to only do so on tiddlers flagged/tagged as requiring automatic links to be generated. Also if you want "virtual links created, then it makes sense to create a tiddler of that name, such that it becomes one of the search strings. 

Back to marks original Question, if he transcludes {{tiddler name}} and "tiddler name" exists in text, then it will be found in text content, perhaps the render process could place a link icon to "tiddler name" and stlll allow the transclusion.

To put it simply why not generate links to tiddler titles when that title is found in text? It is a different story turning these into valid back-links.

Anyway just thinking.

Regards

Tony

[PMario]

On Tuesday, December 12, 2017 at 11:09:33 PM UTC+1, TonyM wrote:

Only search for strings that exactly match (all) existing tiddlers titles and or captions. In this case "Alphanumeric values" ONLY, perhaps case insensitive.

:) ... seems simple but isn't. That's a performance nightmare, for big TWs. ... Without a propper indexing mechanism, for only that one usecase, we can forget it.

------

I also doubt, that it makes much sense at all. ... At tiddlywiki.com we use the term TiddlyWiki very often and most of the time we need to undo the CamelCase linking with ~ ... like: ~TiddlyWiki. So it isn't rendered. ..

The same problem will happen here. If you have a tiddler "Alphanumeric values" and every time you use the term it will be auto-linked. So if you use it 3 times in a paragraph, that's simply wrong.

For me as a user, I would not like the texts very much. They create a lot of redundant links, which only waste time, if you click them.

----------

The other problem is eg: If I do have a tiddler named "done" ... Everytime I now write the word "done" it will be linked. ... Other tag tiddlers that are common: high, low, medium, todo, ...

Conclusion: ...

 - We have a lot of trouble with CamelCase autolinking. ...

 - Creating "non user defined" links, based on tiddler names, will create the same sort of problems, but worse.

 - Managing a "do not link" list, in the end will be more work than using the proper link syntax from the beginning:

    - like [[Alphanumeric values]]

just some thoughs

-mario