Local-First Storage, with backup to cloud (Github)

449 views
Skip to first unread message

ludwa6

unread,
Apr 25, 2020, 9:59:35 AM4/25/20
to TiddlyWiki
Having been away from TiddlyWiki for many moons (i.e. pre-HTML5 days), i'm very happy with how the UX has improved, except for this one aspect of managing data storage. Have been playing with this for some hours, but cannot find any good way to get what i want on this score, which is
  1. In keeping with the Local-First principle, the file on my machine should be updated immediately on save of a newly-edited tiddler; and then..
  2. That updated file should sync with its online-accessible version (Github.io, in my case) with as little friction as possible.
Initially, i was pleased to discover how easy it is to save my incremental changes to Github, using the "GitHub Saver" in Control Panel... Until i discovered that the file in my local Github directory was not being updated, which violates the Local-First principle that is one of my non-negotiable requirements (i spend a fair bit of time offline, so cloud storage cannot be the default, but is instead the backup).

Then -finding no affordances in the default UI for local storage- i explored the plugins, and found this  "Browser Storage: Local storage in the browser" plugin, which i have installed, but this still doesn't solve the problem, AFAICT;  nothing gets saved until i click the download button, which results in a new file with an incremented filename in my local download directory -which i then have to rename, so i can then overwrite the previous version in my local Github directory, so i can then push it live to Github.io.  This is not a sustainable workflow.

So:  i'm thinking: there has *got* to be an easier way... But then, browsing this forum for an answer, the most current & relevant thread on the topic, i.e. "When do my wiki edits get saved to the local file system?" would seem to indicate that this requirement of Local-First storage is not such an easy one to solve.  

Now, as i have reliable internet access at the moment, i can do it the wrong way round -i.e. enable the GitHub Saver for now, and then just pull to my local GitHub directory for backup- but meanwhile, if some more experienced user here can tell me a way to get Local-First storage working in a relatively frictionless way, i'd be much obliged!

Jeremy Ruston

unread,
Apr 25, 2020, 10:29:09 AM4/25/20
to TiddlyWiki
Hi Ludwa

Your local first principle is roughly the setup I use for my personal wikis. I use TiddlyDesktop on my Mac and Quine on my iPad/iPhoen. Both are a dedicated browsers for TiddlyWiki that save changes automatically to the local file system, which can then be synced as desired (I use both Dropbox and Git).

Best wishes

Jeremy


--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/c0f52f20-b910-468d-bebf-9dd8536c9af5%40googlegroups.com.

Jed Carty

unread,
Apr 25, 2020, 10:38:35 AM4/25/20
to TiddlyWiki
If you don't mind my asking, what features are missing from the savers available? I haven't used anything but bob in a while but my understanding is that the browser extensions save to the local file system without any problems.

I am looking at improving the saver bob uses for single file wikis so I am looking for feature ideas.

Mike Manchester

unread,
Apr 25, 2020, 11:09:31 AM4/25/20
to TiddlyWiki
Check out Nextcloud you can signup for a free account or setup your own Cloud. I found this solution to work for editing offline on my computer and then syncing when I was connected to the internet. I'm using Tiddlydesktop on my lap top's and Quine on my IOS devices I only wish I had a solution for Android.

ludwa6

unread,
Apr 25, 2020, 11:48:42 AM4/25/20
to TiddlyWiki
Hey - Thanks, Jeremy for the good & quick response.  Have installed Tiddly Desktop version 0.0.13 (13), seems to be working and auto-saving fine on my Mac.

But (never content to leave well enough alone ;-) i now wonder: can i get the GitSaver affordance in the desktop app working?  Turned it on, but now when i save a tiddler, it pulls a 404 error msg.  Is this supposed to work somehow?

Meanwhile: i'll deactivate that option, and pushTo/ pullFrom Github from my console as i usually do.  Good enough for now!
To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

ludwa6

unread,
Apr 25, 2020, 12:16:18 PM4/25/20
to TiddlyWiki
IMHO Jed: The default option -active on installation, and working with the very first edit/save on a tiddler- should be to update (not duplicate) the file wherever it sits on the user's machine... And the "Saving" control panel should have a line of contextual guidance to say in effect: "this is the default, and these other options -if activated- will result in the following change to that default" -if any (i am still not clear if these saving options are all mutually compatible, or exclusive, or what).  

I just know that i have clicked the two relevant-sounding saver options - Saving:General: AutoSave, and Download Saver: Enable AutoSave for Download Saver - both on and off, to no apparent effect, i.e. i still wind up having to save a new file with a different name (version digit incremented by one) on my desktop.  For me, this is both non-intuitive and unduly frictional, even if you know what you have to do. 

Anyway: that's all in the web browser UI (Chrome / Firefox / Safari -have tried all)... But since i have as Jeremy suggested moved my workflow to Tiddly Desktop, this is no longer an issue: local save seems to work fine -albeit *without* the GitHub Saver turned on.

Jed Carty

unread,
Apr 25, 2020, 12:30:17 PM4/25/20
to TiddlyWiki
ludwa6,

In a browser it isn't possible to save things the way you are talking about without some external help, like the browser plugins discussed on tiddlywiki.com. These are restrictions built into browsers and we can't change them. The way around them is to make a new browser, which is essentially what tiddlydesktop is, or to use one of the non-git or tiddlyspot savers that saves locally. Unfortunately the only thing that works out of the box without any other installation is saving on remote servers. I could say a lot about this but it would take a long time and be about 90% profanity.

As far as the documentation goes, I am probably the wrong person to write that, but a GitHub issue would probably get some discussion and results. I have learned that there is almost no overlap in the documentation that I find useful and what most other people seem to find useful, if the reactions to Bobs documentation are any indication.

ludwa6

unread,
Apr 25, 2020, 12:48:36 PM4/25/20
to TiddlyWiki
Thanks for the good explainer, Jed; clarifies some things i've been wondering about.  

As to documentation, i'm still pretty fuzzy (still finding my way around the many granular pieces at tiddlywiki.com) but would like to know: what is this "Bobs" of which you speak? 

Mark S.

unread,
Apr 25, 2020, 2:15:48 PM4/25/20
to TiddlyWiki
Bob is a node-based server for TW that can run multiple wikis at a time and allow mult-user access. It can also save single-file wikis


Here's another approach to saving.

When your copy of TW saves numbered copies of your file into your browsers download directory, it's using the default save mechanism. This mechanism works pretty much with any browser, making it the most universal save solution. The problem is that the name and location of the saved file is not convenient. The Polly project runs a PS* script in the background that can find and copy your version in the download directory back to it's original location. One idea is that the script could be enhanced at some point to allow sub-scripts to be run. These scripts could do something like a git pull on your TW's directory. That way your local and cloud (github) versions would be kept in synch. Unfortunately, Polly hasn't grabbed a lot of attention, so I just work on it when there seems to be a need:


* PS is powershell, which runs on Windows, Linux, and Mac

---------------

There are some 20+ solutions to saving listed at TiddlyWiki.com. Another possibility for GH and TW is to put your GH folder below the downloads directory. Then use PMarios' file-backups extension to save your file. From time to time you would need to commit and push your saved file.

Another possibility is to use the local storage saver in conjunction with the GH saver. In this configuration, TW is constantly saving itself to the browser's storage without having to press the save button. When you press the save button, the file gets pushed to GH. You can then periodically do a pull to upate your local copy. In the event that you have a net disconnection, turn off the GH saver and save a local copy (in this case, setting the browser to ask where you want to save might be a good thing).

Local browser can be confusing, at least it was for me. Jeremy warns us that the browser could cleanse it's internal cache. So far no one has reported that event. Then again, I'm not sure whether many are using it. But used in conjunction with GH the loss of an internal cache might not be a total disaster. I'm relating this as best as I can from memory, since I only played with local storage/GH for a short time a couple weeks ago.

Birthe C

unread,
Apr 25, 2020, 2:20:06 PM4/25/20
to TiddlyWiki
You can find the latest thread here:Bob


TonyM

unread,
Apr 28, 2020, 4:32:44 AM4/28/20
to TiddlyWiki
Mark,

I know the browser supposed to be able to reclaim the space. But a recent plugin of mine asked for this permission so I wonder if there is a way around his?

Snag_6351d6a.png

Regrds
Tony

Mark S.

unread,
Apr 28, 2020, 10:34:53 AM4/28/20
to TiddlyWiki
Interesting. Note that it is browser extensions that can request this ability. That would suggest that there would need to be a special TW extension for local storage. That would make TW act almost like the old days, where we only needed one extension. Of course, you would need to take additional steps to save the data to your local hard drive (or perhaps use github for that).

Jed Carty

unread,
Apr 28, 2020, 10:53:34 AM4/28/20
to TiddlyWiki
Without installing additional things using localStorange in the browser is unreliable. There isn't any real nuance there, the browser can clear it out without informing you. The browser developers have been pretty clear that this isn't going to be the default state for the foreseeable future and isn't likely to ever change.

If you install something additional, like a browser extension, you can get around this limitation. But we know that because we have browser extensions that save wikis (tiddlyfox, etc). Using a browser extension that saves the wiki in the browser is a step backwards from those other solutions because you have to install something anyway and, while the saved wiki isn't necessarily in danger of being lost without warning, the wiki is only available in that browser on that computer, it isn't sharable unless you use some other saver, like the built-in download saver, to get a copy that you can then move around or share.

Riz

unread,
Apr 28, 2020, 11:20:00 AM4/28/20
to TiddlyWiki
You can use timimi to save standalone tiddlywikis anywhere in your file system. So that is local first. Now if you have a Dropbox/Google Drive/OneDrive folder, you can place your files in those folders- which will give you cloud backup

https://ibnishak.github.io/Timimi/

Sincerely,
Riz

Mark S.

unread,
Apr 28, 2020, 11:30:42 AM4/28/20
to TiddlyWiki


On Tuesday, April 28, 2020 at 7:53:34 AM UTC-7, Jed Carty wrote:
being lost without warning, the wiki is only available in that browser on that computer, it isn't sharable unless you use some other saver, like the built-in download saver, to get a copy that you can then move around or share.

But in terms of an initial user experience, there's only one easy extension to install, and they're good to go. Yes, down the road they'll have to learn how to save or export the file, but it would give an easy first step. And for people who never move around, it may be the only step they need.

TonyM

unread,
Apr 28, 2020, 8:33:31 PM4/28/20
to TiddlyWiki
Riz,

Agreed,

And I refer to my desire to enable checkin/out or lock to make this workable on a shared drive.

Tony

TonyM

unread,
Apr 28, 2020, 8:42:51 PM4/28/20
to TiddlyWiki
Mark/Jed,

On Tuesday, April 28, 2020 at 7:53:34 AM UTC-7, Jed Carty wrote:
being lost without warning, the wiki is only available in that browser on that computer, it isn't sharable unless you use some other saver, like the built-in download saver, to get a copy that you can then move around or share.

But in terms of an initial user experience, there's only one easy extension to install, and they're good to go. Yes, down the road they'll have to learn how to save or export the file, but it would give an easy first step. And for people who never move around, it may be the only step they need.

Of course when it comes to single file wikis once loaded even without Local storage they continue to operate and accept changes even if the the save mechanism fails. The browser download option remains and changes could be exported as JSON files. To me this is already a degree of local first, perhaps which we have not fully made use of.

For example;
Load a wiki from a URL on your mobile and make changes, disconnect from the network and keep using the wiki indefinitely (don't close the window and don't let the phone restart). Now return to a stable internet connection and save the wiki at its original address. If we had a invisible save changes only on our mobile we could restore only if necessary we are a long way there.

Jed,

I mentioned you here, because I wonder if there is a way that a bob/node served wiki could wholey move into browser memory pending reconnection, as is the case for single file wikis, not withstanding this could render the wiki single user.

Regards
Tony

Jed Carty

unread,
Apr 29, 2020, 5:06:55 AM4/29/20
to TiddlyWiki
Tony,

You are going to have to explain what part of that isn't the case now. Once it is in the browser Bob can act just like a single file wiki does in the browser, and it has had features to let it reconnect to the server after being disconnected almost from the beginning, I made it to handle my inconsistent internet connection so that was a very important feature.

ludwa6

unread,
Apr 29, 2020, 7:52:53 AM4/29/20
to TiddlyWiki
Such a wealth of options for Local-First storage have emerged in this thread (must say: i'm super impressed with the strength of this community i have just joined), i've been able to quickly implement the simple approach suggested by Jeremy (TW Desktop, + cloud sync to Github), which is serving me well enough for now. 

Yet, as i'm fast creating quite a mountain of data in TW, i'm also bothered by the growing sense that what i really want at the back end of this beautiful thing is a database -ideally SQLite, which has all that local-first/ single-file/ portability goodness of TW, while bringing the full set of functionalities needed to manage a large database with integrity.  With a single SQLite file stored locally and replicated to the cloud, that gives us a solution that ticks all the boxes in that seminal Local-First paper: Fast, Multi-Device, Offline, Collaboration, Longevity, Privacy, User-Control.  Combining this with the power of TW5 interface (really an "interface builder for the rest of us," i would call it), minus mess of .html backup files i've got to keep cleaning off my machine (SQLite has a full transactional history, so no need of any backups but that one file, replicated to cloud), and that in broad-strokes would be the shape of my dream machine. 

So i must ask: has anyone tried using SQLite as storage for a TiddlyWiki?  If this is indeed technically feasible, can anyone point me to prior art on which i might be able to build such a solution?  Though i am no programmer, i am reasonably comfortable with SQL as a data manipulation language, if someone can just show me how to get the TW data in there!

TonyM

unread,
Apr 29, 2020, 8:53:19 AM4/29/20
to TiddlyWiki
Jed

Thanks for that. I expected it was so but never got to confirm it, including with node and tiddlyserver. Thats great and I think we can make use of this a little more.

I kind of imagin one of the mobile apps acting like a dedicated browser containing such a bob wiki ensuring the session is never lost, just waiting to reconnect to the bob server when you get home to your trusted network.

It would be a nice way for single user multiple device wikis.

The bob server would be like the home server. I imagin it works with the single file bob saver as well.

Regards
Tony

TonyM

unread,
Apr 29, 2020, 9:01:24 AM4/29/20
to TiddlyWiki
Some quick points about sql.

The closest example I know is noteself which use pouch and couchdb. I would be keen to have wiki tiddlers use my WordPress db or cpanel sql allowing integrations.

I will help you go this direction.

I think the term is build a saver or sync module. Tw-reciever does this for php, bob to the node server, noteself to couchdb etc... Regular wikis can use the default download saver or timimi and other savers.

Regards
Tony

Regards
Tony

Jed Carty

unread,
Apr 29, 2020, 9:03:29 AM4/29/20
to TiddlyWiki
ludwa6,

There have been a few database-backends made for tiddlywiki, I don't remember if anyone made a sqlite one or not, but it wouldn't be too hard and there are some examples around for it. I considered it for bob but I don't like sqlite, so I didn't use it.

Tony,

There is some misunderstanding here somewhere. I am completely lost. What is 'such a bob wiki' in this context? It acts like any other html wiki in the browser, it just has a plugin. I don't know what is new here or how being treated the same by the browser as an html wiki is something special.

What works with the single file bob saver?

Arlen Beiler

unread,
Apr 29, 2020, 10:06:52 AM4/29/20
to TiddlyWiki
I have tried doing SQLite but there are some caveats that need to be accounted for. 

What I would recommend you do is use data folders instead of single file wikis. Data folders store tiddlers in individual text files using a custom format that Jeremy came up with. It works very well for text based storage solutions like GitHub. I'm actually really surprised no one mentioned this yet. 

To get started, install tiddlywiki globally by running npm install tiddlywiki -g

Next, to create a blank data folder, run tiddlywiki ./MyNewWikiFolder --init server

Then run tiddlywiki ./MyNewWikiFolder --listen

Now open your browser to localhost:8080 and start exploring. 

TiddlyServer is a good way to load multiple data folders on the same port. 

Hope that helps, 
Arlen

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/856ad777-dc63-4491-bd74-c0391c4bf413%40googlegroups.com.

Arlen Beiler

unread,
Apr 29, 2020, 10:10:56 AM4/29/20
to TiddlyWiki
I believe Tony is referring to the fact that a browser session is quickly lost, including the unsaved data, once the user switches to a different app on mobile devices. The only workaround that I know of would be to store unsaved changes in local storage until they get synced. Which is basically what noteself did.

So many different pieces of the puzzle. I often wish I could put them together. 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

ludwa6

unread,
Apr 29, 2020, 12:26:00 PM4/29/20
to TiddlyWiki
Hey Tony: Thanks for the quick & supportive response.  
Though i don't know anything about pouch & couchdb, nor cpanel SQL, i do know a bit about WordPress -am managing a few such sites myself- and tho the idea of WP integration appeals to me personally, this solution i'm trying to put together is a special-purpose thing (a user-friendly interface & documentation stack to a remote sensing & control gateway) that's to run on a Raspberry Pi, which has SQLite installed as a necessary part of the stack.  
I don't see WP as being a good fit for this application, and i'm quite fond of SQLite, but am open to other database technologies at this point, if they will fit. 
Thanks for the pointers about Saver or Sync modules; gotta get me up to speed on these things!
/walt

ludwa6

unread,
Apr 29, 2020, 12:37:36 PM4/29/20
to TiddlyWiki
Hey Arlen: Great to hear, thanks! 
So i ran those commands, installed TiddlyServer on my Raspberry Pi gateway... But hit a little snag setting up the listener, in that port :8080 is owned by another application server.  Can't change that for the moment, as it's a service i can't really mess with.  Could talk to that developer about changing the port he's using, i suppose...

But just to see this working, i tried to install on my Mac, but terminal tells me:

-bash: npm: command not found

Suppose i could use a different package manager, if i knew how (yes, i am a complete unix n00b  =8-(

Will try to bone up on the requisite skills, but any more crib notes would of course be appreciated!

/walt

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

Mark S.

unread,
Apr 29, 2020, 1:56:45 PM4/29/20
to TiddlyWiki

The listen command can take a port number. Like

--listen port=8090

Arlen Beiler

unread,
Apr 29, 2020, 1:58:23 PM4/29/20
to TiddlyWiki
I'm not at my computer right now, but you can specify the port in the config file. 

{ "bindInfo":{port:8090}}

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ae0debc5-de0e-4b2a-8e94-7c7915a868dc%40googlegroups.com.

Arlen Beiler

unread,
Apr 29, 2020, 2:00:25 PM4/29/20
to TiddlyWiki
Sorry, what I said was for tiddly server. What mark said is for tiddly wiki.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/9571530e-806f-4a61-ae2f-ef1498d49f7b%40googlegroups.com.

ludwa6

unread,
Apr 30, 2020, 6:12:00 AM4/30/20
to TiddlyWiki
YES!  that worked, got 'er working on my R-Pi server now.  Thanks Mark!    /w

ludwa6

unread,
Apr 30, 2020, 6:42:34 AM4/30/20
to TiddlyWiki
Arlen: Putting your earlier instructions together with Mark's declaration of listener port, i *think* i've got he server running on my Pi server... But in fact i'm still confused about this.  

The UI looks & seems to be performing pretty much like TiddlyWiki single-file version, except that default "Getting Started" tiddler asked me to make & save one, & confirm that it worked... So i did, and now i see it in the file system, inside 'MyNewWikiFolder/tiddlers/ , along with $_StoryList.tid .  I then created a 2nd tiddler, which appeared as a 2nd file in the same directory... So can i safely presume i've got tiddly server working as it should?

Now looking at the schema of these files, it seems pretty simple, i.e.: 
  • created: (date/time stamp in yyyymmddhhmm format, followed by a 3-digit numeric)
  • modified: (same format as above)
  • tags: (if any, a horizontal list)
  • title: (as declared)
  • type: text/vnd.tiddlywiki
  • (single line space, followed by...
  • (full text of tiddler, followed by...
  • (some 16 lines, blank except for '~' as initial character...
  • <ikiFolder/tiddlers/(TitleOfTiddler).tid
  • [noeol] 7L, 132C      7,23      ALL
That last line is most mysterious to me, as i don't know what those codes refer to, nor why there's so many spaces preceding the last 2 strings. 

Best candidate for Primary Key here, as far as i can see, would be the first attribute -the "created" date/time stamp- but i don't know how to turn these files into rows in the SQLite database where i want to store the data. From what Tony said, i gather it would involve building a sync module of some sort, but i have no idea how that might be done.  Any prior art, or ideas about this?

Arlen Beiler

unread,
Apr 30, 2020, 7:23:00 AM4/30/20
to TiddlyWiki
Those last lines are a mystery to me. I've never seen them before. I'll check make sure I'm not missing anything. 

The tiddler title is the primary key in Tiddly wiki. I don't really think you would need to store it anywhere else though.

Everything is in flux right now, but hopefully we'll have it mailed down soon and then I'll be able to better recommend how to use a database with this. But it certainly isn't required. 

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/e11cd690-1d1b-4ad4-a301-049640052549%40googlegroups.com.

Arlen Beiler

unread,
Apr 30, 2020, 7:25:30 AM4/30/20
to TiddlyWiki
Yes, if they're properly showing up in the folder then everything is working. Data folders are the second officially supported format in tiddlywiki, after single file wikis. So feel free to just use the data folder, in my opinion you don't need to store it anywhere else. 


ludwa6

unread,
Apr 30, 2020, 8:40:03 AM4/30/20
to TiddlyWiki
Arlen: storage in a RDBMS (requirement for this application i want to develop[1]) requires a Primary Key that is both guaranteed unique AND cannot be changed -but tiddler Title *can* be changed, so that is not a good candidate, seems to me. The one element of this schema that appears immutable to me is that "created" date/time stamp, which has 3 digits beyond minutes (thousandths of a minute, perhaps?), so that is fine-grained enough to guarantee uniqueness, i guess. 

[1] To explain: i need RDBMS storage for this particular application because i need to correlate tiddlers with records in other tables of the database that they are meant to document, or otherwise extend. This RDBMS lies at the heart of a gateway that facilitates data I/O with a set remote nodes for sensing & control of farm operations -initially a climate-controlled greenhouse.  The data flowing between nodes in this environment is structured as a rule, such that humans cannot mess with it. Yet it is precisely those "messy" human inputs -e.g. observations, photos, links, questions, etc.- that i aim to capture thru TiddlyWiki.  The real power of this application will lie in its ability to corrrelate machine data with user-generated data... And for that, i need both to be stored in RDBMS (just in case you were wondering why :-)

Mark S.

unread,
Apr 30, 2020, 9:23:57 AM4/30/20
to TiddlyWiki


On Thursday, April 30, 2020 at 3:42:34 AM UTC-7, ludwa6 wrote:
  • (some 16 lines, blank except for '~' as initial character...
  • <ikiFolder/tiddlers/(TitleOfTiddler).tid
  • [noeol] 7L, 132C      7,23      ALL


Are you sure these aren't visual artefacts of  the editor you're using? Some old editors display a tilde at the start of each blank line, and that line on the bottom looks like an info display. Maybe you could post one of the new tiddlers?


ludwa6

unread,
Apr 30, 2020, 12:22:05 PM4/30/20
to TiddlyWiki
Heh -looks like you are right, Mark: i was viewing text of tiddler in the terminal, but when i opened it in a text editor , it was simply this:

created: 20200430094059869
modified: 20200430094129604
tags: test
title: FirstPost
type: text/vnd.tiddlywiki
What hath God wrought?


- and nothing more.  All that afterbirth in my previous msg in this thread was some irrelevant terminal screed, i guess.... But if this is all there is to a tiddler, it should be simple to store in SQLite, i imagine, for one who knows how to write a "Sync module" (wish i knew how!).  /w

Arlen Beiler

unread,
Apr 30, 2020, 12:52:28 PM4/30/20
to TiddlyWiki
If I was doing that I would auto assign an integer primary key, but if that's more work you could use the created timestamp. 

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/41b844a9-ec92-48f3-82ba-577224128761%40googlegroups.com.

Mark S.

unread,
Apr 30, 2020, 1:08:24 PM4/30/20
to TiddlyWiki
I'm not sure the created field is never cleaned off. Like possibly during export/import procedures. Or when packed/unpacked from a plugin.
Maybe use a separate id field just to be sure.

On Thursday, April 30, 2020 at 9:52:28 AM UTC-7, Arlen Beiler wrote:
If I was doing that I would auto assign an integer primary key, but if that's more work you could use the created timestamp. 

On Thu, Apr 30, 2020 at 8:40 AM ludwa6
Arlen: storage in a RDBMS (requirement for this application i want to develop[1]) requires a Primary Key that is both guaranteed unique AND cannot be changed -but tiddler Title *can* be changed, so that is not a good candidate, seems to me. The one element of this schema that appears immutable to me is that "created" date/time stamp, which has 3 digits beyond minutes (thousandths of a minute, perhaps?), so that is fine-grained enough to guarantee uniqueness, i guess. 
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscribe@googlegroups.com.

ludwa6

unread,
Apr 30, 2020, 3:03:19 PM4/30/20
to TiddlyWiki
If there's any doubt about using the "Created" field, then the auto-assigned integer always works for PK (most RDBMs do it that way by default, in fact). 

Meanwhile, i am playing with tiddly server, trying to get Github Saver to work, with no joy as of yet.  Is this supposed to work, or does it work only for single-file instances, i wonder?

Mark S.

unread,
Apr 30, 2020, 3:38:51 PM4/30/20
to TiddlyWiki
AFAIK, the github saver is for single-file TW's.

TonyM

unread,
Apr 30, 2020, 8:17:59 PM4/30/20
to TiddlyWiki
Walt,

Using a automatically issues ID is wise in these circumstances, however I have done work of fixed serial numbers for tiddler so Personal Message me if you want to discuss.

I am not in favor of using such a large serial number as created date/time, however the date can be transferred to an independant field, then tested to ensure its unique, simply incrementing milliseconds until it is unique. Then you hide this field from edit. The problems come about when dragging and dropping tiddlers from other wikis, but a test for uniqueness can occurs again, unless we were to include a wikiname in the serial numbers.

I expect a database solution may assume an empty.html with the stored tiddlers anything added or overwritten. There is a real question however with tiddlers delivered via the database that are needed as the wiki loads. 

You may not know couchDB but Noteself implementing a database model and would inform you, perhaps even give you code examples. Two notable features are tiddler versions and the use of a local couchDB when PouchDB not available and before connected. A Design path is to build first to a browser internal Database, then look at connecting to an external Database. this could allow one Tiddlywiki to switch databases, or one database share multiple wikis.

Regards
Tony

Joshua Fontany

unread,
May 2, 2020, 11:34:54 AM5/2/20
to TiddlyWiki
You should investigate Danielo's "NoteSelf" adaptation if TW which I belive syncs to a PouchDB server.

Also - TW defines the Title field as primary key. When title "changes" TW is actually storing a new tiddler under the new Title and deletes the old copy.

Best,
-Joshua F

Reply all
Reply to author
Forward
0 new messages