Bug? each[] and each[title] only work for existing tiddlers

[Stephan Hradek]

Create a "New Tiddler" on tiddlywiki.com

Paste this content

<$list filter="[[test]] [[New Tiddler]]">

</$list>

---

<$list filter="[[test]] [[New Tiddler]] +[each[]]">

</$list>

---

<$list filter="[[test]] [[New Tiddler]] +[each[title]]">

</$list>


You will get

test

New Tiddler

---

New Tiddler

---

New Tiddler

that's because "test" does not exist as a tiddler and thus has no fields.

While this matches the definition of "each" (If a tiddler does not contain field F, it is treated as if the value of the field were empty.), I think this confusing and a bug.

1. A non existing tiddler - to my understanding - has exactly one field. This is the "title" field.
2. It's not possible (with "each") to filter a list for duplicates.

I came across this when testin my new "pick" filter. I wanted to remove duplicates from the list of picked texts. The result was nil - no hit at all because none of the hits is an existing tiddler.

So either we require a unique (or uniq) filter which just cares for the names in the list and not for existance of fields, or we should fix the bug.

What do you think?

Maybe treat "each[]" as just caring for the tiddlers listed and ignoring fields and "each[SOMETHING]" handle like it does now.

[Stephan Hradek]

I just created a pull request (#3116) which will fix the issue.

Please use at your own discretion…

with my example above the output will be:

test

New Tiddler

---

test

New Tiddler

---

New Tiddler

So if each[] is used, the "title" field is not accessed and thus non-existing tiddlers can be used. With each[title], the title field has to exist.

I did this by not letting the parameter default to "title". Additionally the condition "if (tiddler)", which discards every non-existing tiddler, is changed to "if (tiddler || field === "")".

I hope this won't break anything in the core.

As I said: My understanding is that non-existing tiddlers have an "inherited" title-field. It seems this understanding isn't shared by everyone…

[Stephan Hradek]

Please check #3117 / PR #3122 as I do not think this proposal is matching how other operators work.

 Discussion in this thread should stop (or better: not start) but go to https://groups.google.com/forum/#!topic/tiddlywikidev/-GYFcugEoHQ

[Jeremy Ruston]

Hi Stephan

I'll respond to the GitHub PRs, but wanted to give a bit of context here.

The first thing is that when I started TW5 it wasn't at all clear to me that I would be able to achieve my primary objective of creating an algebra of tiddlers that was rich enough to support constructing TiddlyWiki's own user interface. The components that you see today were built up gradually over the two or three years from 2011.

Part of the process of TW5's development has been a sometimes jumpy process of rethinking prior assumptions - the situation with filters is fairly typical of the development of a number of features: I knew from TiddlyWiki Classic that I wanted a concise notation for structured searching of tiddlers, and so had a basic filter language early on. But initially I was thinking of the filter language as a way to select tiddlers: the results of evaluating a filter would be a list of real, actual tiddlers.

Accordingly, the implementation of the early filter operators tended to look up titles to retrieve the underlying tiddler, and would silently return nothing if the target tiddler didn't exist. (The each.js and field.js operators are a good example).

Gradually (and this is really the point of these remarks), I realised that it was better to think of the filter language as a generic way to manipulate lists of values, and only have a small set of distinct operators that treat the values in those lists as tiddler titles.

The trouble we have now is, as ever, backwards compatibility. We've got nearly a decade of wikis in the wild that depend on the existing behaviour.

We can and do change the behaviour of filters but it's generally by extending them with special new modes and the like. We certainly can't change existing behaviour just because it is inconsistent or confusing.

Best wishes

Jeremy

--

Jeremy Ruston

jer...@jermolene.com

https://jermolene.com

On 15 Feb 2018, at 07:26, Stephan Hradek <stephan...@gmail.com> wrote:

Please check #3117 / PR #3122 as I do not think this proposal is matching how other operators work.

 Discussion in this thread should stop (or better: not start) but go to https://groups.google.com/forum/#!topic/tiddlywikidev/-GYFcugEoHQ

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/79804c40-ce72-4237-a2b4-cfe1503e7e7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Stephan Hradek]

Am Donnerstag, 15. Februar 2018 09:03:39 UTC+1 schrieb Jeremy Ruston:

Hi Stephan

I'll respond to the GitHub PRs, but wanted to give a bit of context here.

The trouble we have now is, as ever, backwards compatibility. We've got nearly a decade of wikis in the wild that depend on the existing behaviour.

We can and do change the behaviour of filters but it's generally by extending them with special new modes and the like. We certainly can't change existing behaviour just because it is inconsistent or confusing.

I feared this. But I'm wondering if anyone is really relying on the fact that "each" filters out nonexisting tiddlers.

Anyhow: To resolve this I propose:
a) Amend the documentation for each (and maybe others) so that people know that non-existing tiddlers will be dropped
b) Create a new filter "uniq" (named after the UNIX command) which will do the same as each, except that it will handle non existing tiddlers as well
c) I'm happy to do b) ;)

[PMario]

On Thursday, February 15, 2018 at 10:32:23 AM UTC+1, Stephan Hradek wrote:

Anyhow: To resolve this I propose:
a) Amend the documentation for each (and maybe others) so that people know that non-existing tiddlers will be dropped

IMO you could create a PR for this one!
 

b) Create a new filter "uniq" (named after the UNIX command) which will do the same as each, except that it will handle non existing tiddlers as well

I'm not really happy with that name: uniq. IMO uniq has a completely different meaning. It removes existing duplicates. ... I think tw each operator already does this!

For me the each-operator is similar to the js-function forEach. eg: ["a", "b", "c"].forEach() ...

So my proposal would be a new operator named: foreach

It's similar enough, to be found by users and it's different enough, to ask:    What's the difference between "each" and "foreach", to read the docs

foreach takes every input given, without the requirement to be an existing tiddler.

c) I'm happy to do b) ;)

I'd be happy to test it.

just my 2 cents

have fun!
mario
 

[PMario]

On Thursday, February 15, 2018 at 10:32:23 AM UTC+1, Stephan Hradek wrote:

... But I'm wondering if anyone is really relying on the fact that "each" filters out nonexisting tiddlers.

:) As a developer I can tell you, that you can't rely on assumptions. Users will prove you always wrong.

-m

[Stephan Hradek]

Am Donnerstag, 15. Februar 2018 11:54:12 UTC+1 schrieb PMario:

I'm not really happy with that name: uniq. IMO uniq has a completely different meaning. It removes existing duplicates.

Okay… uniq is for adjacent identical lines.

I'm happy to use the name "unique"
 

... I think tw each operator already does this!

Please follow the discussion. "each" does this plus a bit more. This "a bit more" is the main reason for "uniq(ue)"
 

For me the each-operator is similar to the js-function forEach. eg: ["a", "b", "c"].forEach() ...

For me - not ;)
 

So my proposal would be a new operator named: foreach

No. Absolutely no. Because "foreach" implies that you want to do some common operation with each input element. That's not the case here.
 

c) I'm happy to do b) ;)

I'd be happy to test it.

Already did, But it's still called "uniq".

[Jeremy Ruston]

In fact, rather than a brand new filter, I'd recommend adding an optional suffix to modify the existing behaviour of the ‘each' filter — I did something similar when fixing a similar problem with the "tag” filter.

Perhaps here we could say that the existing behaviour is equivalent to "each:tiddler", and the new behaviour is "each:value"

Best wishes

Jeremy

--

Jeremy Ruston

jer...@jermolene.com

https://jermolene.com

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/610a5466-610f-4938-8b19-c257d9ce5ef1%40googlegroups.com.

[PMario]

On Thursday, February 15, 2018 at 2:45:31 PM UTC+1, Jeremy Ruston wrote:

Perhaps here we could say that the existing behaviour is equivalent to "each:tiddler", and the new behaviour is "each:value"

I would be OK with this suggestion.

-m

[PMario]

It may give us the possibility to "fix" other inconsistencies too, in a consistent way.

-m

[Stephan Hradek]

Me as well. Please check https://github.com/Jermolene/TiddlyWiki5/pull/3124

Please check carefully as I included a bit of code vs. speed improvement (I hope)

Instead of checking for each tiddler which field we check, I do this once upfront.

so instead of (pseudocode)

foreach tiddler {
  if tiddler exists {
    is the field "title" use title as value else get the field as value
    push the value if it has not been seen before
}

I do

is the field "title" {
  foreach tiddler {
    if tiddler exists
      push the title if it has not been seen before
  }
} else {
  foreach tiddler
    if tiddler exists and the field value has not been seen before
      push the field value
  }
}

we save at least 5 [length of "title"] * (n [number of tiddlers] -1) compares in each use of "each".