Best practices for data structure

[Dave Parker]

I'm going to be keeping track of progress of patients across multiple visits, and I have a couple ideas of how to do that.

[[cc1]] (short for chief complaint 1) would be the first complaint tiddler, lets say with an alias of "headaches"

In that tiddler would be several fields filled in on the first visit, e.g. location, onset, quality, severity, frequency, etc.

On each subsequent visits (visit tiddlers would be dates, e.g [[2020-04-27]]) there would have to be updates about that cc1 condition, say "VAS-high" and VAS-low" for the patient's pain levels between visits, and there would potentially be several more data points like that to record and keep track of over time (things in the general headings of Subjective, Objective, Assessment, and Plan Of Management)

My initial thought (Visit-centric) would be to have in the visit tiddler ("2020-04-23") a field called cc1.vashi and cc1.vaslo and keep the info there: 

[[2020-04-23]]

field "cc1.vashi"=9 

field "cc1.vaslo"=4

The alternative (Complaint-centric) would be to have tiddlers called cc1.VAS-high etc (probably tagged with "cc1") and have them as data tiddlers with keyvalue pairs like:

[[cc1.VAS-high]] 

2020-04-23:9 

2020-04-27:7

Or maybe it would be better to put info into the visit tiddler as keyvalue pairs: 

[[2020-04-27]] 

cc1.vashi:7 

cc1.vaslo:3

Having never done a really big TW project before I'm not sure if any of these makes more sense than the other, but I envision making notes on anywhere from 10-30 variables for each visit and I plan on being able to attempt to correlate changes in one variable to changes in others as a sort of in-office research project (do changes in leg length correlate to headaches or a certain treatment procedure, e.g.)

    • Question: Is there actually a "best practice" for a project like this, or is one data structure as good as the next?

I guess I'm wanting the best way to A) track progress over time and B) find previously unseen data relationships and manipulate it the way TW nicely does.

P.S. Does anyone ever put keyvalue pairs in fields (e.g. field="vashi" text="2020-04-23:9,2020-04-27:7", or would that be a nightmare to use in macros and filters later on?

thanks,

- Dave

[TonyM]

David,

I think tiddlywiki could be a great tool for this, lean on us if you need help.

This is a complex subject you ask questions about, here are a few tips.

My Gut feeling is to make the key object the patient not the complaint, but if the complaint is selected from or arrives on a list then you should be able to look at if from a complaint perspective. Its really good to design your solution so that you can look at it from every possible perspective.

I am concerned your using tiddlers with a date in it. Why not use visit 1, or patient - visit 1, patient - visit 2, date is only a detail I would put in a date field.

Fields and their values can be represented as key value pairs, I think you are making too much work for yourself adding key value pairs inside fields because then you can use simple tiddlywiki field operators in filters. If Creating a new tiddler you can provide key=value pairs to create fields.

There is no best practice for this in tiddlywiki for this to my knowledge, there are best practice guides for designing any database, or practice management, but that is a broad subject.

If you are wanting the best way to A) track progress over time and B) find previously unseen data relationships and manipulate it the way TW nicely does. then consider the following;

• Use tiddlers as objects and tiddlers as transactions, link the two by tag or field eg; every transaction has a field or tag containing the patient it belongs to.
• Let all tiddlers that represent real things, be known only by their name, perhaps adding an object-type field that classifies them like object-type = person. 
• Don't get hung up on tiddler names for recurring items, use a simple increment made unique to that which it applies eg "patient - visit N", or have a separate visit number for each visit no matter who it is and assign the person to it.
• Do not use compound values in any title, they are the "key" so don't make life difficult.

In the database world they say

The data should be related to the key, the whole Key and nothing but the key.

The data should be related to the key (person contains person info), the whole Key (no compound keys) and nothing but the key (don't put visit details, which can happen multiple times in a person tiddler).

Perhaps that can get you started.

Ask here for more detailed guidance in separate focused posts.

Regards

Tony

[Mat]

Dave Parker wrote:

    • Question: Is there actually a "best practice" for a project like this, or is one data structure as good as the next?

I'd say there are some general best practices using TW, but not for your particular context because this obviously depends on your needs - not least future needs like how you will want to extract the data and how to sort it. (You might not even know this at this moment.) To name one specific example, recently discussed: the most basic way to structure your data is by using tags. This is much better than to e.g try to use special prefixes direclty on the tiddler titles.

You talk about key:value pairs (a.k.a name:value pairs). As Tony notes, this is what fields are in TW. So it may be a good idea to use your keys as field names (and the value as value). But, again, it depends what you want to do. It is fairly easy to e.g extract all tiddlers that have a specific field or a specific field value. 

<:-)

[Mat]

Overall, the less flexible a system is the easier it is to pronounce best practices. This is yet another example of how TiddlyWikis excellence is causing problems for itself.

<:-)

[TonyM]

Mat,

Good Point "Best Practice" is often in a context, It is easy to design and document a "Best  practice" in an oversimplified context. I believe tiddlywiki can take on almost any context, so it is much harder to decide on a best practice. What we do know is there are many "really Good Practices", but its hard to choose one, so people don't publish them (so often), they just try and do them.

I think there are quite a few good elemental best practices in tiddlywiki, like the no compound tiddler title etc.. but they too can change with the context.

TiddlyWikis excellence, is its unbound possibilities as well.

Regards

Tony

[PMario]

Hi Dave,

As Tony and Mat wrote, it always depends on the usecase. That's why I only want to explain my ideas, using the description you provided.

I personally would go with 1 TW file per patient. ...

TW is a very open system. I think it is very likely that you need to discuss decision making with your patient. So I think it would be fatal, if they see other data then their own. Both from privacy perspective and confusion, if they see content that is not theirs.

For the "visits" I'd use your 3rd approach. 

[[2020-04-27]] 

cc1.vashi:7 

cc1.vaslo:3

Where cc1.vashi: 7 is a "key: value" pair. .. The "key" is the field name (has to be _all_ lowercase) and the "value" can be any text you want.

Using this structue should make filtering relatively easy, since "field - filters" are very powerfull in TW.

It should be possible, to list all existing values from other tiddlers as a dropdown, to speed up entering those values.

With 1 tiddler per visit you have all patients data in 1 tiddler. So no filtering needed to get an overview about the last visit.

There can be a field eg: my-thoughts, which links to a tiddler with your thoughts, that a patient may or may not be able to see. This could also be done with a backlink.

----------------

I think there are 2 more important things, how to create your "visit data". ... You wrote: " ... I envision making notes on anywhere from 10-30 variables for each visit ..."

a) About Questions

Since the number of your "questions" will probably grow over time, it is important, that every visit-tiddler also knows, which "questions" you did ask, for later statistics. ...

eg: You can't compare new data-points (questions), that didn't exist in older visits, where this question wasn't asked. So every tiddler needs to know, what you asked.

b) Separation of "your data" and "patient data"

Everything that is in a "visit" tiddler is data that the patient gave you. ... Your thoughts are kept separated, but linked.

just some thoughts

have fun!

mario

[PMario]

On Tuesday, April 28, 2020 at 6:46:54 AM UTC+2, Dave Parker wrote:

I'm going to be keeping track of progress of patients across multiple visits, and I have a couple ideas of how to do that.

What are you thoughts about privacy?

Do you intend to work with encrypted wikis?

What about backups and storing those backups?

Did you think about those topics?

-mario

[Dave.2]

Thank you all for your feedback (this is the same Dave at another computer now)

You've convinced me to stick with the fields method of key value pairs.

Yes, I forgot to mention that the entire file will be for a single patient.  I have visions of having the entire set of patient TWs in a Bob instance which apparently can handle multiple wikis, but single file wikis are fine to start with (as I'm sure there is currently a way to import a single file wiki into that system)

The concept of separating personal notes from patient notes is something I hadn't thought of.  I'll have to check with my association if that's even allowed.  In most cases where I might not want a patient to necessarily know what I'm talking about I usually use code language (e.g. when a patient has bad body odor and the room needs to be sprayed with air freshener I'l send a message to the front desk that we need a "red pen" in the room, ha ha

As for privacy, I'd thought that the best way would be to possibly encrypt the hard drive its on and use syncthing (also encrypted) to keep current copies sync'd between the treatment rooms and my office computers.  I guess I'd have to see if there are Tiddlywiki ways to encrypt a single TW file in case a patient wanted a copy.

** the other thing I'm concerned about is what if I have TW patient files that are a couple years old, and over time I end up modifying say the layout of how I view or modify information?  In TW Classic there was a way to import a tiddler automatically on startup which would ensure old files are updated with the newest changes (would have to be only tiddlers tagged "sometag" or whatever).  I'm not sure if TW5 has that capability yet.

* and what about updating TW version?  Currently I drag my TW file onto a thing at tiddlywiki.com/updates, but that can't be a secure enough method as far as privacy legislation, can it?

thanks again

- Dave

[Mat]

Dave.2 wrote:

Yes, I forgot to mention that the entire file will be for a single patient. 

(I was surprised at this. If you have 100 patients you'll need to update 100 wikis, install each new pluign on 100 wikis, modify on 100 wikis etc etc... Or maybe I misunderstand. Or maybe multi-updates etc is possible via TiddlyDesktop or sth..)

 

The concept of separating [...]

With the node.js version, and Bob, you'd have separate tiddlerfiles but for the same wiki.

 

As for privacy, I'd thought that the best way would be to possibly encrypt the hard drive its on and use syncthing (also encrypted) to keep current copies sync'd between the treatment rooms and my office computers.  I guess I'd have to see if there are Tiddlywiki ways to encrypt a single TW file in case a patient wanted a copy.

There is the native full wiki encryption and Danielo just revamped the single-tiddler encryption.

 

* and what about updating TW version?  Currently I drag my TW file onto a thing at tiddlywiki.com/updates, but that can't be a secure enough method as far as privacy legislation, can it?

As is stated on the updates site, the data doesn't leave your computer. It is not uploaded. Instead, what you see (just as for any webpage) is a local copy of the updates file. So you drag'n drop your old wiki onto it and the core replaced with the new one and you save this locally. So nothing leaves your browser so, yeah, that should be secure.

<:-)

[Dave Parker]

With the node.js version, and Bob, you'd have separate tiddlerfiles but for the same wiki.

 

 How would that actually work? would you have to have the patient tiddler files tagged a certain way to keep them separate (I assume in separate folders), but the main system tiddlers would remain the same and kept in the parent folder?  This would be ideal.

Also, would it be possible to run filters/reports etc on the entire patient database? Or would this be something I'd have to use bash or some other external thing to look at?

[Mat]

Dave Parker wrote:

With the node.js version, and Bob, you'd have separate tiddlerfiles but for the same wiki. 

 How would that actually work? would you have to have the patient tiddler files tagged a certain way to keep them separate (I assume in separate folders), but the main system tiddlers would remain the same and kept in the parent folder?  This would be ideal.

Naw, each tiddler is a separate file in a big pile (@node users, am I right?). It wouldn't make sense with "folders based on tags" since a tiddler can have multiple tags... (unless you then wanted it in multiple folders I guess... but then you'd have multiple instances of the same tiddler... or maybe they'd only behave as one if both folders/tags were active... anyway, no, that's not how it works.)

Also, would it be possible to run filters/reports etc on the entire patient database? Or would this be something I'd have to use bash or some other external thing to look at?

If your patient database is in one wiki -- regardless if a single file or node.js version with one file per tiddler -- then, sure, filters and aggregations etc is what we do with TW all the time. 

<:-)

[PMario]

On Tuesday, April 28, 2020 at 5:52:18 PM UTC+2, Dave.2 wrote:

Yes, I forgot to mention that the entire file will be for a single patient.  I have visions of having the entire set of patient TWs in a Bob instance which apparently can handle multiple wikis, but single file wikis are fine to start with (as I'm sure there is currently a way to import a single file wiki into that system)

I think, that's a good way to start.

 

The concept of separating personal notes from patient notes is something I hadn't thought of.

:)

 

As for privacy, I'd thought that the best way would be to possibly encrypt the hard drive its on and use syncthing (also encrypted) to keep current copies sync'd between the treatment rooms and my office computers.  I guess I'd have to see if there are Tiddlywiki ways to encrypt a single TW file in case a patient wanted a copy.

Using an encrypted HD imo is a good way to go. .. Encrypted syncing too.

TW contains an option to encrypt all tiddlers at save and decrypt them, when the file is loaded. ... It's only important to remember the password. Otherwise the info is gone.

 

** the other thing I'm concerned about is what if I have TW patient files that are a couple years old, and over time I end up modifying say the layout of how I view or modify information?  In TW Classic there was a way to import a tiddler automatically on startup which would ensure old files are updated with the newest changes (would have to be only tiddlers tagged "sometag" or whatever).  I'm not sure if TW5 has that capability yet.

I did think about this too. The easiest way to update to a new master version imo is:

 - BACKUP your old wiki

 - Open the "old" wiki

 - Drag and Drop import an empty "master wiki" onto it

 - Import everything

 - Test

Since the master only contains the changes to the UI there shouldn't be a problem with the data tiddlers. ... Sure you have to be careful, but that's the same with a TW update

* and what about updating TW version?  Currently I drag my TW file onto a thing at tiddlywiki.com/updates, but that can't be a secure enough method as far as privacy legislation, can it?

tiddlywiki updates is loaded to your browser memory. No information is sent to the server. The whole update thing is done in your system only.

You can check this, if you open the developer console with: F12

Select the Network tab

Open tiddlywiki.com/upgrade ... There will be a GET, OPTIONS, HEAD ... which are all reads

Import your wiki

You'll see there is no network communication. ... You can even unplug the network cable if you want.

 

thanks again

- Dave

Have fun!

mario

[Dave Parker]

When I get close to completion I'll ask Jed about this as I'm pretty sure Bob can access different wikis in different subfolders.

Re all in one wiki, I currently have my patient data in the form of a few thousand individual ods (spreadsheet) files totalling 1.3 Gigs so far, so I think one big wiki might be a little unwieldy ;)

If there's no "include" function available by the time I need it, I'll just have to live with individual updating I guess.  I wonder what would happen if you drag a tiddler to an unopen set of TW files...

[Dave Parker]

PMario: I'd never thought of just dragging an empty new wiki onto an old existing wiki - Good Idea!! 

[PMario]

On Tuesday, April 28, 2020 at 11:03:16 PM UTC+2, PMario wrote:

* and what about updating TW version?  Currently I drag my TW file onto a thing at tiddlywiki.com/updates, but that can't be a secure enough method as far as privacy legislation, can it?

tiddlywiki updates is loaded to your browser memory. No information is sent to the server. The whole update thing is done in your system only.

You can check this, if you open the developer console with: F12

Select the Network tab

Open tiddlywiki.com/upgrade ... There will be a GET, OPTIONS, HEAD ... which are all reads

Import your wiki

You'll see there is no network communication. ... You can even unplug the network cable if you want.

Just forgot to mention the obvious.

 - You can download the upgrader and use it offline.

Top right is the << chevron, which will open the right sidebar. If you click the "Save" button it will download a file named "upgrade". You can rename it to eg: x.html and open it. ... The rest is the same as with the "online" version.

BUT do _not_ use the upgrader as a starting wiki. It contains a huge (16 MByte) upgrader plugin, that contains the new core and every existing "core plugin". The upgrader will only update those plugins, which your "old" wiki uses.

3rd party plugins will be not touched.

-mario

[PMario]

On Tuesday, April 28, 2020 at 11:10:59 PM UTC+2, Dave Parker wrote:

PMario: I'd never thought of just dragging an empty new wiki onto an old existing wiki - Good Idea!! 

If the core version is the same, there should be no problems.

If you need a new core, you should use the upgrader first. 

A little bit of experimentation with some "throw away wikis" will give you enough experience.

-mario

[Dave Parker]

What do you mean by "core"?  if the version starts with a "5"?

[Saq Imtiaz]

On node.js you can configure how tiddlers are saved to file, and in what directory structure, by providing filters. So you could separate tiddlers for patients into separate folders off a tag, a field value or anything really. You could have one folder for all patient tiddlers, one folder per patient, however you like.

https://tiddlywiki.com/#Customising%20Tiddler%20File%20Naming

[Dave Parker]

That's good to know for saving, but does that mean when you load the TW file it'll load all tiddlers from all patients, even if they're in separate subfolders?  That would slow load time considerably.  Or maybe there's a way to selectively load from different folders already?

[TonyM]

Dave,

An alternative from a Single File Wiki enthusiast.

Since you want to share a custom wiki with each patient, I would maintain a single master wiki (not OK) and use a custom template to generate their wiki as needed. Set the $:/status/UserName field so if they do make edits you can actually extract them if necessary.

An alternative to a custom template, is using the innerwiki plugin. You get to set what tiddlers are including in addition to the core, in a new wiki which appears in an iframe. If in the inner wiki you can save the whole new wiki to a new wiki file, with only the tiddlers that the innerwiki plugin included, or any subsequent edits. This would include the wiki filename, based on the patient, the custom wiki tiddlers look and feel and a single patients records. Also consider adding a version number and published date to the Patient wiki, so you can ask over the phone if they were possibly not on the most recent.

Sending updates to someone is sometimes best handled by you holding the source of truth and just sending a totally updated result.

Regards

Tony

On Tuesday, April 28, 2020 at 2:46:54 PM UTC+10, Dave Parker wrote:

[Dave Parker]

Thanks Tony,

I don't actually envision having patients directly make changes that stick to the patient record, but I do have to be able to provide a copy of the record if asked, or to a lawyer if the patient was involved in a car accident, e.g.. (on the other hand I have thought of making personalized health coach TWs to give to patients, but that will have to wait for the next pandemic, ha ha.)

This is the first I've heard of the innerwiki plugin - I'll check that out!

There's so much available to do in making your own TW solution I'm almost suffering from Choice Paralysis!  But now that I can't see patients right now I guess I have no excuse but to work on it :)

- Dave

[TonyM]

Dave,

If you make a tiddler that displays well, you can open it in a new window and print it, ctrl-p, to PDF or save as PDF and send to patient, that would most likely be the best method, and keep the copy as a record. Print to a printer for some clients who don't use the "new fangled interweb thingy"

regards

Tony

[Eric N.]

Hi,

Reacting to Mat's message below, stating :

good elemental best practices in tiddlywiki, like the no compound tiddler title 

In my use case, I couldn't do without compound tiddler title, i.e. titles prefixed by their types.

Any recommendations about how to improve my data model and data exploration solutions ?

= = = = = =

My use case

My use case is about multidisciplinary knowledge acquisition, exploration and consolidation.

I am dealing with the following Tiddlers categories:

1. Source, information source - ex: "so An Analysis of Kant's Critique of Pure Reason -- by author X"
   
1. tagged with Domain(s), Thinker(s), Topic(s)
2. Domain, Domain of Knowledge - ex: "do Philosophy"
1. recursively tagging sub (sub (sub...)) domains, to form a tree of domains, 
2. the head of domains being called "do Domain"
3. Thinker, the producer of the views in question - ex: "th Kant"
4. Topic, the subjects discussed - ex: "to Reasoning", "to Enlightenment", ...
5. and Notes, summaries of the views exposed in the sources
1. main note tagged with the source
2. other notes tagged with it's father note (classical tagging way to deal with large information)
3. thus all these notes about this source form one / several tree(s) of notes all going back to the source
4. if a specific note is about a specific Thinker, Field or Topic, that is not already mentioned in the source, then it is tagged with these specific tags.

At first: no compound titles, just tagging

At first, all topics were tagged by Topic, all sources by Source, etc. and titles didn't contain a prefix.

But when I wanted to explore categories (i.e. main tags) all the data was mangled in the simplest data exploration tool I knew : the table of content, as in

<<toc-selective-expandable "Domain">>

• includes all the domains and subdomains and subsubdomains in a nice tree view
• but also all the sources (indeed tagged by domains)
• and all the notes (indeed tagged by sources, and by domains)
• ...

and I did't want that. I just wanted to explore Domain names, and get more information only when I click on a specific domain name.

Then: compound titles + tagging for visual separation

To solve the above issue I thought : OK if data get mangled, but at least I want elements to be clearly grouped in the TOC display.

Prefix + alphabetical sort made it work. With the same <<toc-selective-expandable "Domain">>, I was better. I had all the sub domains listed, then thinkers, then topics, sources. And notes in between these groups, because they had no prefix. At least I could easily find my subfields.

Then: compound titles, no tagging for just what I need

No more sources tagged with Source, thinkers tagged with thinkers, etc. Only using prefix to indicated the category.

One exception: Domains, because they are a tree. So each domain was tagged by its parent domain, and the top Domain being "do Domain"

With <<toc-selective-expandable "do Domain" "prefix[do ]">>, I could have the table of content display I was looking for : domains and subdomains.

So....

Are there any TW benefits I am using with these prefixes and lack of main tags categories ?

And could I have dealt with my issue in a better way ?

Best,

Eric N.

 

On Tuesday, April 28, 2020 at 5:10:39 PM UTC+4, TonyM wrote:

Mat,

Good Point "Best Practice" is often in a context, It is easy to design and document a "Best  practice" in an oversimplified context. I believe tiddlywiki can take on almost any context, so it is much harder to decide on a best practice. What we do know is there are many "really Good Practices", but its hard to choose one, so people don't publish them (so often), they just try and do them.

I think there are quite a few good elemental best practices in tiddlywiki, like the no compound tiddler title etc.. but they too can change with the context.

[...]

[PMario]

Hi Eric,

Please open a new thread with your post. ... Otherwise you'd probably hijack this thread. ... Which imo isn't best practice ;)

kind regards

mario

[TonyM]

Eric

I understand how your post was somewhat related. But I am looking for the new thread so I can respond.

The main trick is creating a new tiddler getting it to respond to context. I believe your move to compound titles was in response over use of tags.

Lets continue the conversation.

Regards
Tony