How to generate list of missing tiddlers, sorted by most references

[Beckstrom]

Is there a way to generate a list of missing tiddlers sorted by the number of times they have been referenced?

Right now I have a hundred or so "missing tiddlers," and I'd guess most of them are only referenced from other tiddlers once or twice - but there are probably a handful that are referenced many times. I like to periodically clean up and add things to my wiki, and knowing which tiddlers are most needed would be cool.

thanks in advance!

[TW Tones]

Yes is the answer.

First identify the titles of missing tiddlers then for each search for that title in all tiddlers, list or count.

This may also detect matching text that does not form a tiddler title but that can be useful as well.

The Freelinks plugin could also help combined with search.

[TW Tones]

Backstrom,

This macro does what I said in my last reply, but without sorting it.

<$list filter="[all[missing]] -[[New Tiddler]]" variable=missing-title>

<h2><$link to=<<missing-title>>/> <$count filter="[all[]search:text<missing-title>]"/></h2>

   <$list filter="[all[]search:text<missing-title>]">

   </$list>

</$list>

Note how the majority found on tiddlywiki.com are the result of using camelCase

Also click to create the missing tiddler

The next step is to sort the list if you need to but since the sort order is a value we determine from our search its not so strait forward.

For a one off or occasional review perhaps the above is sufficient since  

[TW Tones]

Sorry,

I accidentally posted before signing off, 

<$list filter="[all[missing]sort[]] -[[New Tiddler]]" variable=missing-title>

<h2><$link to=<<missing-title>>/> <$count filter="[all[]search:text<missing-title>]"/></h2>
   <$list filter="[all[]search:text<missing-title>]">

   </$list>
</$list>

This sorts by title and excludes "New Tiddler" 

As I said

The next step is to sort the list if you need to but since the sort order is a value we determine from our search its not so strait forward

Just ask if you still want the original sorted list, this requires either a trick, possibly the new subsort operator or a temporary separate list.

You can see also that perhaps !is[system]] may make sense in your wiki.

Regards

Tony

[Felicia Crow]

Hi,

so after playing around with filter operators and trying to find a way over a data tiddler I wrote a javascript macro I attatched below.

The only caveat with this is that you have to close and reopen the tiddler calling the macro due to how javascript macros are called. Maybe someone has an idea for how to reload a single tiddler.

For now the macro - called simply with <<missing-tiddlers>> - renders as a list of all missing tiddlers sorted by how often they are referenced:

It can be changed to render as a table by changing this line of code in the macro:

return renderList(tiddlerData);

To this:

return renderTable(tiddlerData);

Which then looks like this:

Please be aware that after importing the macro/changing it you have to reload your wiki for it to take effect.

Also as an aside unlike Tony's solution this uses the backlink filter operator so it only counts hard links and excludes system tiddlers completely. A little more about what hard and soft links are here.

The reason I did this is mostly to ensure that these are actual references and not just the tiddler title being in the text even if it isn't meant as an actual reference.

Should you want to use Tony's solution with search instead of backlinks it is easy change the filter either by yourself or I can update the macro.

Regards,

Felicia

[Felicia Crow]

Sorry forgot to actually attach the file.

[Eric Shulman]

On Friday, July 24, 2020 at 3:34:52 AM UTC-7, Felicia Crow wrote:

so after playing around with filter operators and trying to find a way over a data tiddler I wrote a javascript macro I attatched below.

The only caveat with this is that you have to close and reopen the tiddler calling the macro due to how javascript macros are called. Maybe someone has an idea for how to reload a single tiddler.

Here's a version that does the same results using just wikitext instead of javascript:

\define getData()

<$list filter="[all[missing]!has[draft.of]sort[title]]">

   <$text text="[["/>{{{ [<currentTiddler>backlinks[]count[]divide[1000]removeprefix[0.]] }}};<<currentTiddler>><$text text="]]"/><br>

</$list>

\end

\define renderTable()

<table>

<tr><th>Missing</th><th>Times Referenced</th></tr>

<$list filter="[enlist<missing>!sort[]]">

   <tr>

   <td>{{{ [<currentTiddler>split[;]rest[]join[;]] }}}</td>

   <td style="text-align:right;"><$text text={{{ [<currentTiddler>split[;]first[]divide[1]] }}}/></td>

   </tr>

</$list>

</table>

\end

\define renderList()

<ul>

<$list filter="[enlist<missing>!sort[]]">

   <li> {{{ [<currentTiddler>split[;]rest[]join[;]] }}} (<$text text={{{ [<currentTiddler>split[;]first[]divide[1]] }}}/>)</li>

</$list>

</ul>

\end

<$list filter="[all[missing]limit[1]]" variable="has_missing_tiddlers"

   emptyMessage="<p>No Missing Tiddlers</p>">

   <$wikify name="missing" text=<<getData>>>

   <<renderList>>

   </$wikify>

</$list>

I patterned the code structure similar to your javascript code.

* the outermost $list checks that there are missing tiddlers and displays either "No Missing Tiddlers", or proceeds to get the list of missing tiddlers and show the output

* getData() generates a list all missing tiddlers

* each item in the list is a combination of the backlinks[]count[] (zero-padded) and the missing tiddler title, separated by a semi-colon and enclosed in doubled square brackets (to handle spaces in titles)

* the results of getData() are passed through $wikify to convert the macro output into an actual list of items

* renderList() and renderTable() enlist[] all items, sorted in descending order using the zero-padded count, and then splits each item back into its title and count and then shows the title followed by the count

enjoy,

-e

[Felicia Crow]

Hi Eric,

I am fascinated by what is possible with tiddlywiki alone. Thanks for sharing your solution and letting me learn more about it.

Regards,

Felicia

* the outermost $list checks that there are missing tiddlesr and displays either "No Missing Tiddlers", or proceed to get the list of missing tiddlers and show the output

[Hans Wobbe]

Eric:

"Very nice" code that is Functional, and Educational.

Thank you very much for yet another in a long list of contributions!

Cheers,

Hans

* the outermost $list checks that there are missing tiddlesr and displays either "No Missing Tiddlers", or proceed to get the list of missing tiddlers and show the output

[Pit.W.]

Thank you Hans for asking and Eric for answering.

This solves a problem I had so far regarded as unsolvable.

Pit.W

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/601789af-b470-4a5b-a3fb-10b8afeecdc2n%40googlegroups.com.

_________________________________________________________________
________________________________________________________
Ihre E-Mail-Postfächer sicher & zentral an einem Ort. Jetzt wechseln und alte E-Mail-Adresse mitnehmen! https://www.eclipso.de

[Eric Shulman]

On Sunday, July 26, 2020 at 5:27:25 AM UTC-7, Pit.W. wrote:

Thank you Hans for asking and Eric for answering.

This solves a problem I had so far regarded as unsolvable.

Just to be accurate:

"Beckstrom" asked the question.

"Felicia Crow" responded with a solution using javascript code

I re-wrote her solution to use macros and wikitext

"Hans Wobbe" said thank you.

-e

[bimlas]

Not a perfect solution, but you can relatively easily create tables in the Javascript console that you can sort by columns by clicking on the header.

SAVE YOUR WIKI BEFORE TRYING OUT: Although it is just a query and does not change anything, it is better to prevent trouble.

Open the DeveloperTools console in the wiki window (right click anywhere on the page -> inspect element -> console tab) and issue this command:

let results = [];

$tw.utils.each($tw.wiki.getMissingTitles(), (tiddler) => {

  const backlinks = $tw.wiki.getTiddlerBacklinks(tiddler).length;

  results.push([

    tiddler,

    backlinks

  ]);

});

console.table(results);

In another example, we can print a list of tiddlers, which can be sorted by the number of links and backlinks:

let results = [];

$tw.wiki.forEachTiddler((tiddler) => {

  const links = $tw.wiki.getTiddlerLinks(tiddler).length;

  const backlinks = $tw.wiki.getTiddlerBacklinks(tiddler).length;

  // Filter unnecessary results

  if(links === 0 && backlinks === 0) return;

  results.push([

    tiddler,

    links,

    backlinks

  ]);

});

console.table(results);

Because the table can display up to 1000 rows, it's a good idea to filter out irrelevant results (such as those with no links and backlinks).

[Beckstrom]

this is all fabulously useful and super cool, thank you, everybody!!!

[TW Tones]

Felicia,

Since you a quite able to build Javascript solutions into Tiddlywiki now, I would like to ask a quick question, if it makes sense to you.

Would you know how to define a global variable(s) that we can update (repeatedly) via a macro call? As a result it will need to have a step to initialise it.

Imagine that inside a list widget someone could conditionally change the variable, ie add 1 to a number or set a flag = yes, such that the value can be accessed outside of the list widget?

This is a missing mechanism as far as I can see.

Regared

TW Tones

[Felicia Crow]

Hi Tony,

after some testing I may have a first idea, but there are two things:

• Calls: Javascript macros are called when the tiddler is parsed. So you would need something that rerenders the tiddler - for instance the list widget you mentioned upon changes in the filter result - to call the macro multiple times without closing and reopening the tiddler. As an example if you change your variable in one tiddler and in another already open tiddler get the variable's value it won't be updated until the tiddler is reloaded. I currently don't have an idea for that.
  
• Permanent Save: Due to javascript macros not being able to directly write to a field by themselves currently the variables only last as long as the wiki is not closed or reloaded. For this I might have a solution, but will need to play around a little more with it, before being able to say if it actually works or not.

Regards,

Felicia

[TW Tones]

Felicia

Thanks for considering this issue. Perhaps we should start a new thread, since this is related but off topic.

Some more thoughts

• Felicia to be a true "local" variable it would more often be used in a single tiddler so you need not worry about its display elsewhere, however there is I believe mechanisms to refresh the tiddlers and their transclusions in the story.
• Since this would be quite different behaviour we should perhaps use an alternate defacto naming standard eg _varname or 
  
  <<localvar "varname" "filter">>
  
  
• Where filter can compute the next result?
• [<vaname>add[1]] to increment the value
• 33 to set the value...
• There is no need to save the value permanently, a separate step can be taken to do this if needed. 
• For example add an additional action to the save button to save named variables into a field and restore them on load with start up actions.
• A simple way to reload a tiddler is to navigate to itself, however this needs a trigger. 
• However my thought was the initial load of the tiddler was where the rendering takes place and the variable "increments" during the render leaving its transitory values in the rendered output (if displayed) or affecting the conditional flow.
• Then once this is finished we can display the final value if needed.

In closing

• This "problem" may face some difficulties in the way tiddlywiki operates
• From a users designer view point it is however in my view an important gap to fill
• I can name dozens of cases where this would help.
• The key difference with such a variable is its use need not be inside an open and close, just to set its value.
• Perhaps it has to be a widget.

Regards

TW Tones  

[Felicia Crow]

Tony,

Last post here as an answer, will open a new topic with the solution after rewriting it.

Sorry, when you first wrote about global variables I assumed you meant it as in a global scope - so usable within the whole wiki - not local to the tiddler, this of course changes things somewhat.

Also with your answer I now have a better understanding of what you actually wish to achieve, which will take some time to rewrite what I have to allow filters to set variables, especially since I am not sure if you mean tiddlywiki filters or filters especially written for the macro, with the first maybe being very easy or not depending on how getTiddler is setup to work on non existent tiddlers.

What I thought you meant from your first question was more along the lines of a mixture between set and actionSetField widgets so having a more global place to set/get variables without having to use a button to asign the value to a field if that makes sense.

My current solution uses a simplet getter/setter setup plus the ability to save variables to a tiddler and load them back in on startup. So the logic would be still in tiddlywiki itself and only the saving to a global space is done via the respective macos.

Regards

Felicia