[Comment] Data Dictionaries are GOOD for some things ...

[TiddlyTweeter]

In basic mode data dictionaries are just an INDEX and a VALUE, but a LOT of things that we play with in life are that simple binary ...

In TW I think we could use them a bit more easily?

They are NOT difficult to edit once you know how to reference the index/value.

Just a comment

TT

[TW Tones]

I agree TT

It is a way to pump a list into a tiddler and reference it by key going forward and much more. Same can be done with a JSON but editing directly is easier.

Tones

[Mohammad Rahmani]

TT,

For me dataTiddlers (both dictionary and JSON) are an essential part of my daily use in Tiddlywiki!

Even with this one level (flat) you can do many things using them!

Best wishes

Mohammad

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/e5c63c57-f6e8-41f5-a404-a5f6a5f28e0cn%40googlegroups.com.

[TiddlyTweeter]

Ciao Mohammad and TW Tones

Right!

I also think it worth noting a couple of things that maybe not used that much yet ...  

1. TW Data Tiddlers will take WikiText. That is quite UNlike most regular databases! Basically, you can use one dictionary to present anything! And pre-formatted!
2. You can DUPLICATE INDICES in different dictionaries (Normally in TW every title is ONCE only. In DIFFERENT dictionaries you can INDEX with the same pseudo-title). That gives the possibility to do a crude "relational database".

Just thoughts!

TT

[Stobot]

I agree with all of the above and add that these are crucial (in my experience) for performance reasons. For example I've built company lookups of customer statuses for people to easily find information. We have something like 100 locations, and 1000 customers per location - so 100,000 customers in total. Comparing the two ways:

1. 100,000 customers doing a search is *very* slow

2. Making 100 data tiddlers of 1000 indexes each runs quite well

Every once in a while I see either posts on "wiki is too big" when it's filled with simple lists and I know in at least *some* of the cases they could benefit from this structure. 

While this thread is here, I'd just just throw out there again that while there's a indexes[] filter operator, there's not a corresponding values[] one. This comes up every now and then and it would be very useful, but I don't think any of the people that have the skills to add an operator must use them like I do. To illustrate the use-case, I can iterate through the 100,000 customers and manually look for the ones with say an "active" value, but I can't just return a list of customers with the "active" value in the filter. 

[TiddlyTweeter]

Stobot wrote

1. 100,000 customers doing a search is *very* slow

2. Making 100 data tiddlers of 1000 indexes each runs quite well

VERY useful to know! 

TT, x

[TiddlyTweeter]

Stobot wrote:

... while there's a indexes[] filter operator, there's not a corresponding values[] one... I can iterate through the 100,000 customers and manually look for the ones with say an "active" value, but I can't just return a list of customers with the "active" value in the filter. 

Great observation!

IMO being able to easily filter VALUES as well as INDICES would open a big, big door.

IMO data dictionaries (binary animals) are really excellent for a LOT of ordinary data presentation issues which are very often ONLY needing an index/value. 

TT 

[PMario]

On Thursday, May 13, 2021 at 12:48:08 PM UTC+2 Stobot wrote:

While this thread is here, I'd just just throw out there again that while there's a indexes[] filter operator, there's not a corresponding values[] one. This comes up every now and then and it would be very useful, but I don't think any of the people that have the skills to add an operator must use them like I do. To illustrate the use-case, I can iterate through the 100,000 customers and manually look for the ones with say an "active" value, but I can't just return a list of customers with the "active" value in the filter. 

Hi,

I did create the keyvalues plugin: https://wikilabs.github.io/editions/keyvalues/

That allows you to directly show the key and the value plus a separator. The operand is used to allow filtering. It's easy to use it as a values[] operand, with the right parameters.

Jeremy considers data tiddlers as a mistake, so he doesn't want to add more complexity to the core, to give it better support. If we have better support in the core, more users will use it, which makes it harder to get rid of it.

Due to backwards compatibility promises, we will always have data-tiddlers in TW5 .. But it may be gone in TWx

-mario

[si]

>>> Jeremy considers data tiddlers as a mistake, so he doesn't want to add more complexity to the core, to give it better support. If we have better support in the core, more users will use it, which makes it harder to get rid of it.

@PMario Why does he consider data tiddlers a mistake? This seems related to my recent post here: What's the point of JSON tiddlers?

[PMario]

On Thursday, May 13, 2021 at 1:31:22 PM UTC+2 si wrote:

>>> Jeremy considers data tiddlers as a mistake, so he doesn't want to add more complexity to the core, to give it better support. If we have better support in the core, more users will use it, which makes it harder to get rid of it.

@PMario Why does he consider data tiddlers a mistake? This seems related to my recent post here: What's the point of JSON tiddlers?

That's not so easy to explain, but I'll try. It's part of the "selective refresh" mechanism of TW. ...

"Selective refresh" means, that widgets or parts of a widget are only redrawn, if their parameters have been changed.

Eg: You have 2 checkboxes, that are connected to the same data-tiddler, which may be seen as an advantage.

cb1: <$checkbox tiddler="test" index=1 checked="selected" unchecked="" default=""> Selected?</$checkbox>
cb2: <$checkbox tiddler="test" index=2 checked="selected" unchecked="" default=""> Selected?</$checkbox>

So with this setup, whenever you change the "selected / index" status of cb1, the core also has to recalculate the "selected" status of cb2, because the tiddler "test" has been changed. ... This wouldn't be the case if there would be 2 different tiddlers.

You may say, that's not a problem, because it's convenient. ... Yes it is convenient and for this example it isn't a problem. ... BUT

If you have several 100 indexes, that are "connected" to completely different UI elements, it starts to be a problem: You will have to redraw a lot of elements, that don't really need it.

--------

I also think, that using data-tiddlers as "configuration" and / or "lookup-tiddlers" that don't control visual elements, is convenient. And it won't go away anyway.

hope that helps

mario

[Mohammad Rahmani]

On Thu, May 13, 2021 at 1:29 PM TiddlyTweeter <Tiddly...@assays.tv> wrote:

Ciao Mohammad and TW Tones

TT, 

Right!

I also think it worth noting a couple of things that maybe not used that much yet ...  

1. TW Data Tiddlers will take WikiText. That is quite UNlike most regular databases! Basically, you can use one dictionary to present anything! And pre-formatted!

In the Todolist plugin which is highly based on dataTiddler you can enter any kind of wikitext! 

It even uses macro calls like <<fixme>> to show an icon for todo item!

So, yes this part needs to be documented much better, many do not know!

 

1. You can DUPLICATE INDICES in different dictionaries (Normally in TW every title is ONCE only. In DIFFERENT dictionaries you can INDEX with the same pseudo-title). That gives the possibility to do a crude "relational database".

Just thoughts!

TT

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/32b8d336-f637-4b1a-9fb3-82be1f804928n%40googlegroups.com.

[si]

Thanks for the explanation @PMario. I don't know why, but I imagined it would be more of a philosophical/conceptual reason rather than a technical one. Good to know.

[Mohammad Rahmani]

On Thu, May 13, 2021 at 5:33 PM PMario <pmar...@gmail.com> wrote:

On Thursday, May 13, 2021 at 1:31:22 PM UTC+2 si wrote:

>>> Jeremy considers data tiddlers as a mistake, so he doesn't want to add more complexity to the core, to give it better support. If we have better support in the core, more users will use it, which makes it harder to get rid of it.

@PMario Why does he consider data tiddlers a mistake? This seems related to my recent post here: What's the point of JSON tiddlers?

That's not so easy to explain, but I'll try. It's part of the "selective refresh" mechanism of TW. ...

"Selective refresh" means, that widgets or parts of a widget are only redrawn, if their parameters have been changed.

Eg: You have 2 checkboxes, that are connected to the same data-tiddler, which may be seen as an advantage.

cb1: <$checkbox tiddler="test" index=1 checked="selected" unchecked="" default=""> Selected?</$checkbox>
cb2: <$checkbox tiddler="test" index=2 checked="selected" unchecked="" default=""> Selected?</$checkbox>

This is exactly the case with the Todolist plugin! and it is quite common to have several 10s of todo items connected to three four dataTiddler!

So, I should be aware to not use one Todolist with many items!

So with this setup, whenever you change the "selected / index" status of cb1, the core also has to recalculate the "selected" status of cb2, because the tiddler "test" has been changed. ... This wouldn't be the case if there would be 2 different tiddlers.

You may say, that's not a problem, because it's convenient. ... Yes it is convenient and for this example it isn't a problem. ... BUT

If you have several 100 indexes, that are "connected" to completely different UI elements, it starts to be a problem: You will have to redraw a lot of elements, that don't really need it.

--------

I also think, that using data-tiddlers as "configuration" and / or "lookup-tiddlers" that don't control visual elements, is convenient. And it won't go away anyway.

hope that helps

mario

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1de0941f-227f-4624-8ebb-ba139c9ad710n%40googlegroups.com.

[Mohammad Rahmani]

Best wishes

Mohammad

On Thu, May 13, 2021 at 3:18 PM Stobot <sto...@gmail.com> wrote:

I agree with all of the above and add that these are crucial (in my experience) for performance reasons. For example I've built company lookups of customer statuses for people to easily find information. We have something like 100 locations, and 1000 customers per location - so 100,000 customers in total. Comparing the two ways:

1. 100,000 customers doing a search is *very* slow

2. Making 100 data tiddlers of 1000 indexes each runs quite well

Every once in a while I see either posts on "wiki is too big" when it's filled with simple lists and I know in at least *some* of the cases they could benefit from this structure. 

Very interesting results!

While this thread is here, I'd just just throw out there again that while there's a indexes[] filter operator, there's not a corresponding values[] one. This comes up every now and then and it would be very useful, but I don't think any of the people that have the skills to add an operator must use them like I do. To illustrate the use-case, I can iterate through the 100,000 customers and manually look for the ones with say an "active" value, but I can't just return a list of customers with the "active" value in the filter. 

On Thursday, May 13, 2021 at 4:59:06 AM UTC-4 TiddlyTweeter wrote:

Ciao Mohammad and TW Tones

Right!

I also think it worth noting a couple of things that maybe not used that much yet ...  

1. TW Data Tiddlers will take WikiText. That is quite UNlike most regular databases! Basically, you can use one dictionary to present anything! And pre-formatted!
2. You can DUPLICATE INDICES in different dictionaries (Normally in TW every title is ONCE only. In DIFFERENT dictionaries you can INDEX with the same pseudo-title). That gives the possibility to do a crude "relational database".

Just thoughts!

TT

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/e3cdfd59-92fc-4d36-8a9f-e0e36a59cea0n%40googlegroups.com.

[Stobot]

Wow @PMario - I can't believe I missed your plugin! It's extremely impressive how much flexibility you've put in there. Thanks!!

[PMario]

On Thursday, May 13, 2021 at 6:38:42 PM UTC+2 Mohammad wrote:

Eg: You have 2 checkboxes, that are connected to the same data-tiddler, which may be seen as an advantage.

cb1: <$checkbox tiddler="test" index=1 checked="selected" unchecked="" default=""> Selected?</$checkbox>
cb2: <$checkbox tiddler="test" index=2 checked="selected" unchecked="" default=""> Selected?</$checkbox>

This is exactly the case with the Todolist plugin! and it is quite common to have several 10s of todo items connected to three four dataTiddler!

So, I should be aware to not use one Todolist with many items!

As I wrote, if you have a countable number of items in the data-tiddler, the impact won't be critical. ... But there is one.

Eg: If 1 checkbox = 1 tiddler, the refresh will execute about 10-15 lines of code 1 times. ..

If it uses a data-tiddler it will execute about 30-50 lines of code for every checkbox, that is visible at the moment. For the first checkbox it will be more lines of code, since the data-tiddler cache needs to be rebuilt. The others will be faster, since they can use the cache. ..

So it probably won't kill the performance, but there is more code that needs to be executed.

--------------------

If 2 checkboxes use 1 tiddler but 2 different fields, there will be the same refresh problem, but with 10-15 lines of code per checkbox. ... Accessing tiddler fields is faster as accessing data-tiddler indexes. ...

-mario

[PMario]

On Thursday, May 13, 2021 at 6:42:12 PM UTC+2 Stobot wrote:

Wow @PMario - I can't believe I missed your plugin! It's extremely impressive how much flexibility you've put in there. Thanks!!

Thanks. I basically made it for me ;) ... I'm using it for 1 of my "bigger" plugins. ... I found out, that there is still some room for improvements.

So if you have problems using it, let my know at github: https://github.com/wikilabs/plugins/issues

That's the best place for issues that shouldn't be forgotten ;)

-mario

[TiddlyTweeter]

Ciao PMario

Looks very interesting, and relevant to what I'm working on at the moment! 

IF I need advice on its use could the open a Discussion on your GitHub?

Best wishes

TT  

PMario wrote:

I did create the keyvalues plugin: https://wikilabs.github.io/editions/keyvalues/

That allows you to directly show the key and the value plus a separator. The operand is used to allow filtering. It's easy to use it as a values[] operand, with the right parameters.

 

So if you have problems using it, let my know at github: https://github.com/wikilabs/plugins/issues

[PMario]

On Friday, May 14, 2021 at 9:04:04 AM UTC+2 TiddlyTweeter wrote:

IF I need advice on its use could the open a Discussion on your GitHub?

Sure

-m

[Mohammad Rahmani]

Mario,

 One more question:

If you add a new key:value to a dataTiddler in Tiddlywiki, does it recreate the whole tiddler again? Seems yes ( I am talking about this with your recent discussion on immutability of tiddlers)

If s, then a big dataTiddler like the one has few hundred keys, then it will affect the performance 

Best wishes

Mohammad

On Thu, May 13, 2021 at 5:33 PM PMario <pmar...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1de0941f-227f-4624-8ebb-ba139c9ad710n%40googlegroups.com.

[PMario]

On Friday, May 14, 2021 at 6:37:16 PM UTC+2 Mohammad wrote:

 One more question:

If you add a new key:value to a dataTiddler in Tiddlywiki, does it recreate the whole tiddler again? Seems yes ( I am talking about this with your recent discussion on immutability of tiddlers)

Yes. It's just like any other tiddler with the same size. .. But writing to internal storage is fast.

 

If s, then a big dataTiddler like the one has few hundred keys, then it will affect the performance 

As I wrote. If you add a new entry to a data tiddler, the per tiddler "cache" will be cleared. The next time you access an element the cache will be created again. Every other "read" action will directly use the cached indexes. So it should be reasonably fast.

The performance penalty for data tiddlers is in the refresh cycle, if many elements of the data tiddler are shown _and_ the tiddler is changed. ... It may cause a "refresh cascade" that is less performant than using single tiddlers.

I'm currently working on a "filter performance" widget, that will allow users to see, how filters behave. ... It may be part of the Advanced Search "Filter tab", if "performance instrumentation" option is active. ... The discussion can be found at: https://github.com/Jermolene/TiddlyWiki5/issues/5674

-mario

[PMario]

There is one thing, that is important for program usability. .. It's the "UI responsiveness".

If the user triggers an action by clicking a button, or inserting a "search term" and the result is shown within 400ms a program feels responsive.

If it's between 400ms and 1 second, users can work with the program, without being stressed.

If the UI response is constantly >1 second, "waiting stress" starts and frustration starts to build up.

That's the time, when optimizations are needed and will really help users.

just a thought.

mario

[Mohammad Rahmani]

Thank you Mario!

Very useful!

Best wishes

Mohammad

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/2e0bf9f9-5aa3-415c-b918-3b5bffe42798n%40googlegroups.com.

[TiddlyTweeter]

Mohammad wrote ...

In the Todolist plugin which is highly based on dataTiddler you can enter any kind of wikitext! 

It even uses macro calls like <<fixme>> to show an icon for todo item!

So, yes this part needs to be documented much better, many do not know!

Right!

It is also worth noticing ...

3.  That TW natively has great INTELLIGENCE on links. For instance, put "https://tiddlywiki.com"  in a data dictionary and automatically, when you list for render, it will generate the link: <a class="tc-tiddlylink-external" href=" https://tiddlywiki.com" rel="noopener noreferrer" target="_blank">

This kind of clever behavior is vast in TW, though barely consciously known.

Best wishes

TT