Data tiddlers replacement

[Hubert]

Hello,

With data tiddlers being deprecated, would anyone suggest a best practice approach to storing large data sets in the flat format key:value? I know that Jeremy suggested plugins but I could not find anything remotely related.

Among other things, I'm using TW to log my cardio workouts. I now have thousands of dictionary entries relating to cardio workouts as well as their types and durations that I dynamically retrieve to generate stats, summaries, forecasts and SVG charts. And so I was wondering if there's now a better option to have the underlying data stored within TiddlyWiki other than in a single data tiddler. I would obviously wish to avoid a scenario where each data point is a single tiddler. I would also like to avoid plugins because there's no warranty they will be maintained forever. However, if there's a "clean" solution (perhaps with the use of system tiddlers) or anything else generally considered "better" than data tiddlers, while still being as future proof as possible, then I'm open to suggestions.

Thank you (and apologies for the wall of text).

Regards,

Hubert

[Jeremy Ruston]

Hi Hubert

Sorry for the confusion, data tiddlers are not deprecated yet, and won’t be until there are enough better ways to achieve the same goals. I’m on my phone and so can’t easily reference the thread, but the discussion on GitHub was prompted by a request to extend the support for data tiddlers which I want to avoid if they are to be eventually deprecated.

Best wishes

Jeremy

--

Jeremy Ruston

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/612d3717-b18b-4bbc-952b-bd95f505846f%40googlegroups.com.

[TonyM]

Jeremy, Hubert,

I assumed data tiddlers tends to refer to DictionaryTiddlers rather than JSONTiddlers

Perhaps JSON tiddlers is the way to go. 

And you can extend JSON handling using Joshuas JSON Tools for Tiddlywiki 5.1.20! if you want to get really serious.

Regards

Tony

[Jeremy Ruston]

Hi Tony

“Data tiddlers” refers both to tiddlers formatted as JSON and those formatted as a dictionary tiddler.

Best wishes

Jeremy

--

Jeremy Ruston

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/38d413bb-2f93-4803-af11-0373df29eecd%40googlegroups.com.

[Jeremy Ruston]

To emphasise that the discussion about deprecating data tiddlers is part of the long term conversation about the evolution of the core of TiddlyWiki. As a user, there's nothing to worry about thanks to our commitment to backwards compatibility.

And to clarify further: "deprecated" in this context doesn't mean that something goes away, just that we recommend against using it for new work. 

Best wishes

Jeremy

--

Jeremy Ruston

jer...@jermolene.com

https://jermolene.com

On 19 Aug 2019, at 12:31, Jeremy Ruston <jeremy...@gmail.com> wrote:

Hi Tony

[Hubert]

Thank you for the valuable info, Jeremy, and Tony -- for the suggestion.

As a user, there's nothing to worry about thanks to our commitment to backwards compatibility

This is exactly what I wanted to hear :)

Thanks again, I really appreciate the hard work you put into TiddlyWiki and I'm excited to see it evolve.

Best regards,

Hubert

--

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

[PMario]

Hi Folks,

The whole discussion was probably started by a PullRequest, that I made some time ago.

It is about a new filter operator named: keyvalues. We do have indexes and getindex which allows a "list-widget" to list all indexes or to read a defined value. Using those operators you need a nested list-widget, if you want to list both: the "key and the value".

That's why I did create the operator. Usage examples can be seen at an example wiki.

The suggestion at the moment is, to implement it as a plugin. ... Which I am considering. ...

In the PR Jeremy pointed out, that handling many small tiddlers is the same as handling 1 bigger one. .. From a program point of view this is right. ... From a users point of view I think it isn't.

As Hubert pointed out in the OP, he doesn't want to use plugins. But if I do create a lot of related tiddlers, where some of them are system tiddlers, users need a specialized UI to export / import different states. ... If there is no UI users can't reliably handle a given state.

If the "given state" is saved in 1 data-tiddler that's a simple exercise.

So IMO there is a usecase for data-tiddlers. An other example:

Let's say I have a sudoku game UI, which consists of 9 x 9 fields, which results in 81 cells. That's OK for most users, for 1 sudoku.

BUT every cell has 9 helper-cells, which gives us max 729 tiddlers.

In my special case the game field is 11 x 11 (sandwich sudoku), which gives us: 11 x 11 x 9 = 1089 tiddlers. ... That's OK for me as a programmer, but if I need to transfer or save a special state, its close to impossible for a user to save without a specialized UI.

At the moment I do create 121 tiddlers per game, and I think that's already too many. ... Especially, since 1 TW will / can contain many different games. 

Game initial settings and game-states should be easy to save and reload. Unlimited "Undo" and "Redo" is essential for a sudoku. So data-tiddlers allow me to bundle information, because it belongs together.

Just some thoughts!

have fun!

mario

[@TiddlyTweeter]

PMario wrote:

Let's say I have a sudoku game UI, which consists of 9 x 9 fields, which results in 81 cells. That's OK for most users, for 1 sudoku. 

BUT every cell has 9 helper-cells, which gives us max 729 tiddlers.

In my special case the game field is 11 x 11 (sandwich sudoku), which gives us: 11 x 11 x 9 = 1089 tiddlers. ... That's OK for me as a programmer, but if I need to transfer or save a special state, its close to impossible for a user to save without a specialized UI.

At the moment I do create 121 tiddlers per game, and I think that's already too many. ... Especially, since 1 TW will / can contain many different games. 

Game initial settings and game-states should be easy to save and reload. Unlimited "Undo" and "Redo" is essential for a sudoku.

 

>> So data-tiddlers allow me to bundle information, because it belongs together <<

I find it a very interesting issue. And your last sentence  good.

But IS the point more about UI than numbers?

I am struck by Jeremy's continuing strong love for the "raw unit fragment".

Best wishes

TT

[Mohammad]

I support your idea!

Related information is better to be kept in one single tiddler specially when these information are small! Having a lot of tiddlers, tags, fields makes confusion, increase entropy of system

and normally difficult to maintain and are more error prone.

In few of plugins I used data tiddlers and it results to much compact, better to follow and understand structure.

--Mohammad