Proposal: Tiddler Serial Numbers

[TonyM]

Folks,

I am working on a Tiddler Serial number (TSN) tool and would like some community input. Like always I try and design to avoid impacting on any other related solutions and with a view to sharing.

Features

• When you save a tiddler it will obtain the next available serial number and assign it to the tiddlers tsn field.
• Tiddlers you do not create (or some other process creates eg import) will provide you a button to add a serial number if desired, also for existing system tiddlers
• Any tiddler with a serial number will display it like the subtitle, in the right hand side under the tiddler buttons.
• An optionally displayed, copy to clipboard button next to the TSN will allow you to copy a macro command to include 
• A link to the tiddler eg `<<tsn n [link]>>`
• Plane text version of the current tiddler title `<<tsn n text>>`
• Transclude the content of the tiddler `<<tsn n transclude>>`
• Or the above for a named field in the tiddler with that serial number.
• Or just copy the serial number
• When the title changes for any reason the serial number in the above macros in your tiddlers will be used to lookup the correct tiddler title/content
• You can use a full macrocall to the tiddlers with given serial numbers for programmatic listing and actions to support the serial numbers.
• A new version of qualify will allow you to name state tiddlers based on the serial number so these state tiddlers will follow the tiddler whatever its name.

Futures;

I am thinking of taking this further to detect the occasions on which a tiddler name changes as a result of edit and rename. Then provide a method for you to trigger a rename process to update other content based on the old and new name such as your own list fields, further supporting the ability to rename tiddlers and list those containing the tiddler name or serial number references like `<<tsn n`. 

• Keep a log of all names a given tiddler serial number has over its lifetime
• Ability to restore the previous name

I also would like to generalise this so you can store serial number related tiddlers under another namespace eg; $:/mydata/n such that they will follow the current tiddler no matter its title.

Regards

Tony

[@TiddlyTweeter]

TonyM

I am working on a Tiddler Serial number (TSN) tool and would like some community input. Like always I try and design to avoid impacting on any other related solutions and with a view to sharing.

Features

• When you save a tiddler it will obtain the next available serial number and assign it to the tiddlers tsn field.

• When the title changes for any reason the serial number in the above macros in your tiddlers will be used to lookup the correct tiddler title/content

Erm ... Would not the "creation" date already be a unique number? 

TT

[Mohammad]

Hi TT,

That's true!

This is what I used in Trashbin plugin!

--Mohammad

[HC Haase]

Just some toughs.

1. why do you want this? is it to solve the problem with links?

2. what would this extra field mean for size of html wikis? maybe nothing?

3. performance? could this speed up filtering.. ?

regarding renaming of links etc (I assume this is your motivation). I think this is a important limitation of tw and elsewhere I have seen people ditching TW because of this. There already other solutions like pmarios uni-link and other alias plugins. How do this differ ( or am I missin the point of this)? The resonantly published re-limk plugin solves the renaming of links/filters etc really well I think. It just works!

I like the simplicity in that tiddler IDs are human readable. it is in the minimalistic spirit of TW, and it makes it a bit easier to debug stuff when you get a strange output from your experiments, however this ID philosophy give birth to the relinking problem.

The solutions like uni-link is for me too heavy and complicated, and more important require to much user action and geekery. That's why I realy like the re-link plugin. It just updates the links and the user dont have to do much about it.

If your work has nothing to do with renaming, I apologise for my rant

[@TiddlyTweeter]

HC Hasse

I think TonyM has a point about identifying Tiddlers uniquely beyond the title. In normal TW the Title IS the unique identifier. 

I think he is thinking about cases where the title changes.

BUT I find it hard to think of a case where the "creation date" isn't already a unique identifier. 

So I find his question, in a way, already answered. 

TT

[bimlas]

Josiah, TonyM,

I like the basic idea, because this way it is possible to use the same ID for tiddler, so you don't have to change it everywhere when you rename it + you can refer to it with the same link (URL).

Using the `created` field is a good idea, it works right now (create a new tiddler on TW.com with this content):

<$link to="HelloThere"/>

{{{ [created[20130822170200000]] }}}

[PMario]

Hi Tony,

;TLDR

Absolutely no offense intended. ... The internet has the same problem and didn't solve it ... yet.

-----------

This discussion has been going on for ages. A "serial number" has the exact same problems as the tiddler-title. It only uses a different name.

If you replace "serial number" in your OP with "tiddler title" there would be no difference, from a technical point of view.

tiddler-titles have to be unique within 1 TW file. -> This causes problems, if you import tiddlers with the same title.

serial number has to be unique within 1 TW file. -> This causes problems, if you import tiddlers with the same number.

... I think you can see the "differences" ... it's "serial number"

IMHO: Since technically the proposed change/addition suffers the same problems, it can _not_ be used as a solution.

---------------

To make a data-point unique, there are different approaches (partial solutions) at the moment.

 - We can use an URL link as the origin. eg:

    - https://tiddlywiki.com/#HelloThere ... which is different to

    - https://example.com/#HelloThere

Using this "origin" would be already better as using a "per tiddler serial number" ... BUT ... tiddlywiki.com  needs to be paid on a yearly basis. So as soon as nobody pais for it, it will be gone -> and what's much worse. ... It can be reused, after some time!!

IMO we need something "more unique" and STABLE ..

 - We could use an URI (Uniform Resource Identifier) as an origin field.

   - URL is an URI ... but it's not useful as we found out already

   - URN may be a better choice.

from wikipedia: 

A Uniform Resource Name (URN) is a URI that identifies a resource by name in a particular namespace. A URN may be used to talk about a resource without implying its location or how to access it. For example, in the International Standard Book Number (ISBN) system, ISBN 0-486-27557-4 identifies a specific edition of Shakespeare's play Romeo and Juliet. The URN for that edition would be urn:isbn:0-486-27557-4. However, it gives no information as to where to find a copy of that book.

So URN is a real solution used in the "isbn" field. ... plus whatever unique field exists in a TW ...

  - THE problem is: .. it costs something.

  - The ADVANTAGE is: it doesn't expire.

Since TW is similar to an e-book, I do think this a real consideration. Since the price goes down if I get a 100, or more, of them, I did consider this possibility. ..

There is an other advantage. ... You can dramatically increase the "reach" of your TWs if you list them, where it's allowed to list ISBN numbers. ...

In Austria, it would go down to € 0.60 per TW, that gets an isbn number. As a member of the "Hauptverband" it would be € 0.30

BUT it still costs something :/

That's why I'm so excited about the dat:// protocol. ... The DAT-address is a universal unique ID and a "public access key"

So if a tiddler contains a "dat" or "origin" field, that contains its origin, you can walk the chain, even if a tiddler is a clone. (This field would have "privacy related" challenges, that have to be considered!) ... see privacy NOT security!

IMO renaming isn't a problem anymore, since a dat-archive contains the full history. ... Full links like dat://<address>+<version>/#HelloThere  don't expire, since you can be sure, that address + version will know about this tiddler. ...

TWs won't die, as long as there is 1 party that "seeds" the content. So if you can't pay for your archive, there may be someone else, who has an incentive, that your content stays online.

Such a party may be someone that "links" to your content, from their own content. ... or the Internet Archive.

There is still the problem with renaming tiddler titles.

Yes but now it can be solved using per TW unique aliases or titles that never change and plugins like "uni-link" and "relink-plugin" which create human readable links.

-----------------------

Using dat:// we can recreate a TiddlySpace like structure using a P2P (peer 2 peer) architecture and mechanisms that TW already contains. Like: Shadow tiddlers for archives withOUT write access and normal tiddlers for archives with write access.

The only thing we need is a browser that knows how to deal with dat archives. Like beaker-browser or I think even better: Cliqz .. A FireFox clone, where the latest Developer-Edition can read and probably write to dat:// (I'm investigating it atm.)

just some thoughts

have fun!

mario

[@TiddlyTweeter]

PMario

I'd like to understand why "creation time" is not good enough?

TT

[PMario]

On Friday, August 30, 2019 at 1:44:22 PM UTC+2, @TiddlyTweeter wrote:

PMario

I'd like to understand why "creation time" is not good enough?

It's probably better than a 1,2,3,.. number, but it's not really human readable and most system tiddlers _don't_ have one.

-m

[@TiddlyTweeter]

@TiddlyTweeter wrote:

I'd like to understand why "creation time" is not good enough?

 

PMario wrote: 

It's probably better than a 1,2,3,.. number, but it's not really human readable

I doubt any solution to this would be :-)

 

and most system tiddlers _don't_ have one. 

Ah! How much of a real issue is that?

TT 

 

[PMario]

Hi,

For HelloThere in classic.tiddlywiki.com created is: 201102111106 ... So for the whole TWclassic system there isn't enough uniqueness to be useful.

-m

[PMario]

On Friday, August 30, 2019 at 1:57:06 PM UTC+2, @TiddlyTweeter wrote:

PMario wrote: 

It's probably better than a 1,2,3,.. number, but it's not really human readable

I doubt any solution to this would be :-)

As shown in my first post. I do think "tiddler title" or 1 of the existing fields like "aliases", maybe "caption" or "subtitle" are already good and unique enough to create unbreakable links. Especially in combination with dat://

I personally use tiddler titles like: hello-there, readme, something-i-want-to-show-you ... and never change them again.

The only thing I change is "heading 1" in hello-there which initially is something like eg: ! Hello There. I can change h1 as often as I want.

I do combine this convention with a theme like "slant". .. For me there is no need to change tiddler titles anymore.

The disadvantage is, that I need more time to find good titles.

An other advantage is: "deep links" hare human readable: https://wikilabs.github.io/editions/slant-01/#01-slant-01-theme

have fun!

mario

[TonyM]

All,

Thanks for your responses. I love to hear passionate technical arguments. Let me try and address them.

Don't read unless you are interested in algorithms, solutions and the discussion of technical merits there of.

Towards the end I will explain my key use case

• In someways simple serial numbers is similar to using created date (and forcing the milliseconds to be unique), I did that, I build a method to make unique (to this wiki ID's based on created date) but they were "big numbers", kind of blur when you look at them, and they are not uniquely guaranteed unique across wikis - but we will return to that.
• Trash bin works fine. We tend to add members in human interaction time, so created is ok.
• This is not simply a matter of solving problems with links, or renaming tiddlers, it is providing a simple mechanism for people to build data structures where this is possible. I believe it will be a path for designers and users to use as needed.
• The size overhead is limited, when what tiddlers have serial numbers that can be optional and they start at 1, but we do not need to do this to all tiddlers just those we want to do this for.
• Perhaps if could be used for performance, but this is in my mind for cognitive grasp of managing special cases where you want or expect a tiddler may be renamed, or as you will see compound tiddlers.
• The idea of a serial number is easy to grasp and code extensions for, even to type manually.
• I have a particular and powerful case where this proposal is more than sufficient.
• A transclude would be
  <<tsn NNN transclude>>
  , easy to read and even type, a link
  <<tsn NNN>>
  you could even remember a number for long enough to type them.
• How many tiddlers fit into this requirement in yourwiki? 1000 is a readable number.
  
• Mario, you never know what is written within that you do not read. I have being there with you with most of the conversations on this subject. I have never being content with the answers, although I love the plugins and tools that have being generated as a result eg uni-link and more. I think there is room for another solution.
• Like Mario I avoid the need to rename tiddlers very easily, and would not even concern myself with this except for one powerful use case
• Tiddlers have titles that are designed to change, I propose a serial number that does not. That is not replacing title with serial number.
• I have no desire to make tiddlers unique in the tiddly universe. Perhaps I need to filter out / reset the tsn field in imports so they do not interfere.
• Actually I think serial numbers like 1, 2 and 3 are cool, and simple.

So what is my use case?

• I am building a tabs tool that helps you make compound tiddlers. 
• Compound tiddlers can contain a number of subtiddlers who's content is always accessible from the main tiddler
• The "subTiddlers" can be defined by tags, list fields, delimiters "/" and any other useful method.
• Some subtiddlers can actually be shared with other tiddlers and others unique to its "parent".
• Almost all tiddlers will never need to be renamed, but if they do I want the rich set of subtiddlers to move with it.
• Subtiddlers will often have the same name as a subtiddler in another tiddler eg "Overview" "Notes", a serial number lets me make them unique as needed.
• With this compound tiddler method available the pressure goes off needing to come up with unique names because the subtiddlers are made unique by the serial number.
• If I want to move compound tiddlers across wikis I have to make a mechanism anyway and I can just acquire a new serial number in the destination.

I have written a sophisticated "Root cause analysis methodology" for information technology support, based on past experience it provides suggestions and useful links even guidelines . Tiddlywiki would is an ideal tool, except for this complexity of the rename.  My proposed solution is ideal for this use case. I would only bother setting a Tiddler Serial Number for such master tiddlers (as soon as I create a subtiddler of any type), it can contain a reference to the serial number of the parent and never be lost.

In my current use case I do not even need to use the macros I suggested, I just thought they would allow the entry of cross referencing as needed between master tiddlers and the subtiddlers.

Regards

Tony

[@TiddlyTweeter]

Ciao PMario

What is interesting here is you need to WORK on them a bit ;-)

I was most interested in full automation without anything else.

My impression is "date created" is pretty good for that.

You gave example of some system tiddlers lacking the creation date.

I'm still unclear how much of an issue that is?

TT

[@TiddlyTweeter]

I'm reading despite not grasping "algorithms, solutions and the discussion of technical merits". Excuse me.

My basic question is this ...

You want a unique field reference. RIGHT?

What I am struggling with understanding why "creation date" won't do for your purpose.

Excuse me if I am an idiot.

TT

[PMario]

On Friday, August 30, 2019 at 3:49:25 PM UTC+2, @TiddlyTweeter wrote:

What is interesting here is you need to WORK on them a bit ;-)

Yea, That's an ongoing process.

 

I was most interested in full automation without anything else.

hmm, I think, there will always be a little bit of manual work left. Especially with TW there are a lot of possibilities to do the same thing. Most of the time it's: use-case related conventions.

 

My impression is "date created" is pretty good for that.

With TW5 and manually created tiddlers yes. .. But if a program / plugin creates tiddlers, it's too easy to create several tiddlers with the absolute same timestamp.

 

You gave example of some system tiddlers lacking the creation date.

I'm still unclear how much of an issue that is?

You can't use them that way. That _is_ an issue. At least for me. --- I want it all - and I want it now ;)

-m

[PMario]

On Friday, August 30, 2019 at 3:37:05 PM UTC+2, TonyM wrote:

• I have no desire to make tiddlers unique in the tiddly universe. Perhaps I need to filter out / reset the tsn field in imports so they do not interfere.

OK ..  I do.

• Actually I think serial numbers like 1, 2 and 3 are cool, and simple.

hmm, 1, 11, 2, 3 ... I'm OK with that using +[nsort[]] with it. ..

I don't have an issue using 1, 2, 3 in the aliases field. It works pretty well.

 - [[1|?]] ... shows a link text: 1

 - [[1|?c]] ... shows the caption field as link text

 - [[1|?any-name]] ... shows the "any-name" field as link text

 - 1 2 +[is[alias]] ... gives you a nice list, that allows you to do everything you want.

and so on.

-m

[TonyM]

Mario

Uni-link is a nice companion for tiddlers with serial numbers. I will revise its capabilities for integration.

All,
If you managed to read my last post you will see that a very large set of use cases exist for a local wiki only, on demand serial numbers.

You do not need a serial number for every tiddler if the tiddlers that need to be addressable "by any other name" we only give them the serial number and write references with that serial number.

However in many wikis a serial number added to non system tiddlers at tiddler creation this will remain very low overhead.

For those wanting to develop multiwiki or universal serial numbers perhaps you could consider laying it on top of a locally issued serial number.

With my current use case, the subtiddlers so created for a tiddler with a serial number are easy to identify but perhaps an indicator on any tiddler referencing a serial number would prepare tiddlers for export.

Regards
Tony

[TonyM]

Mario,

Perhaps for universal serial numbers we could use the public url and compress it to give a unique wiki serial number and store it in the wiki. If the compression were reversible the wiki serial number could be decompressed to find its url. In this case it would be the wiki owners responsibility to deal with url changes etc.. But anyone with the wiki serial number could find the wiki and a locally managed tiddler serial number would be the compound key. This would allow the publisher to reassign the key locally. The local key could be found by search or mapped to tiddler name.

This would make serial number unique to a public url.

Regards
Tony

[TonyM]

Mario post script

Using the url to create the wiki serial number with a lossless compression could be used with the dat files in beaker browser as well.

Tony

[bimlas]

PMario,

I have not heard of the DAT protocol and I do not know how Peer-2-Peer works, but it is clear that it has to do with networks. One of TiddyWiki's basic ideas is to be free of external dependencies and work on its own. Does the DAT protocol break this principle? Can the ID be generated offline or do I have to register it on an online server?

I found a brief summary of what DAT is like, if it wasn't clear to anyone:

Beaker Browser is an experimental P2P web browser that natively integrates the Dat protocol. You can think of Dat as being an improved version of BitTorrent that’s optimized for sharing folders of data that are constantly changing. This makes Dat a good fit for hosting static websites and building hostless applications because the author of a website or application can publish updates that users can download from anyone on the network and still be sure that they are downloading the same updates as everyone else. (See this blog post for more info). With Beaker, creating a new website becomes as easy as entering a name for the site and clicking on a button to generate a new “key pair.” The “private” key gives the author the power to change the site while the “public” key gives other users the power to find and download the site.

https://medium.com/offline-camp/offline-first-peer-to-peer-networks-da2bc4bc92af

[bimlas]

This made it more clear: https://github.com/datproject/dat#getting-started

• Track your files with automatic version history.
• Share files with others over a secure peer to peer network.
• Automate live backups to external HDs or remote servers.
• Publish and share files with built in HTTP server.

 

 Can the ID be generated offline or do I have to register it on an online server?

Yes, just tried out the followings with disabled network:

$ cd dat-test
$ dat share
dat v13.13.1
Created new dat in /home/bimlas/Letöltések/dat-test/.dat
dat://5d5b02b61e7d883732cc5007743d4ef987157073824a489330bcbf791a8cae6e
Sharing dat: 1 files (3.0 KB)


0 connections | Download 0 B/s Upload 0 B/s


Watching for file updates

Exiting the Dat program...%     

[PMario]

On Saturday, August 31, 2019 at 12:25:15 PM UTC+2, bimlas wrote:

I have not heard of the DAT protocol and I do not know how Peer-2-Peer works, but it is clear that it has to do with networks.

Yes, it has to do with networks. It establishes direct communication between 2 peers. ... There are some 3rd party servers involved at the moment, because you need to find other peers. 

 

One of TiddyWiki's basic ideas is to be free of external dependencies and work on its own. Does the DAT protocol break this principle?

No - in the contrary. It is an attempt to avoid the dependency on large companies. As soon as your browser is running, it also can be a host if the user wants it. If someone else knows the ID = dat://<address> they can connect to your archive and load the content.

They can only read the info. ... The only person, that can write to a dat archive is the "private key" owner.

 

Can the ID be generated offline or do I have to register it on an online server?

The ID is generated offline. Either in the browser (that supports it) or with an external command line program.

There are some efforts going on atm, to allow browsers to read / write dat archives. TiddlyWiki already contains a default saver, that is able to save, if you use the beaker browser. ... others will follow in the future.

--------------------

The OP discusses the need to create a serial number. ... The problem is, that this number is only unique within 1 TW. ... That's the same as with tiddler titles. ...

The dat://<address> is unique by default. .. It belongs to 1 user. So in combination with a dat://<address> a tiddler title can be universal unique. ... Including it's history. ...

That's only 1 reason, why I'm so excited about the dat-project.

have fun!

mario

[@TiddlyTweeter]

bimlas wrote:

I have not heard of the DAT protocol and I do not know how Peer-2-Peer works, but it is clear that it has to do with networks.

PMario wrote: 

... There are some 3rd party servers involved at the moment, because you need to find other peers.  

Right.

It is interesting and works. A downside is, right now, is 3rd party servers in-between.

I think this is part of the problem with it. 

The protocol fosters true peer-to-peer, but, given how most of the net now works, true peer-to-peer seems not possible under DAT?

Maybe I'm an idiot?

TT

[PMario]

On Sunday, September 1, 2019 at 10:08:27 AM UTC+2, @TiddlyTweeter wrote:

bimlas wrote:

I have not heard of the DAT protocol and I do not know how Peer-2-Peer works, but it is clear that it has to do with networks.

PMario wrote: 

... There are some 3rd party servers involved at the moment, because you need to find other peers.  

Right.

It is interesting and works. A downside is, right now, is 3rd party servers in-between.

That's not necessarily a problem, as long as the software is open source and you can run it on your own, if you want to.

 

I think this is part of the problem with it. 

As above.

 

The protocol fosters true peer-to-peer, but, given how most of the net now works, true peer-to-peer seems not possible under DAT?

That's not true. ... You only need a "discovery service" to establish the communication. Then it is _true_ P2P.

-m

[HansWobbe]

After some preliminary testing of Containers, I think it might be possible to set up a proof-of-concept that would let a few folk interact as Peers that way.  I personally don't have the ability to set this up, but I do see enough potential in this apporach that I would participate and contribute as best I can.

Thoughts?

Cheers,

Hans

[Diego Mesa]

I think it would be useful to separate the discussion/use-cases of creating/using "serial numbers" that are unique inside a wiki, to creating/using "serial numbers" that are provably unique **on the entire internet for all time**.

Just because its relevant here, ill copy/paste what I said in this other thread:

In my mind, linking is one of the two major pillars of TW (searching/filtering being the other - why I think a lot and comment a lot about search). If I want to link to something there are three issues of concern:

1. what I want to type

• Aliases are magical here! A tiddler with "long title of method" can save me many keystrokes (and potential typos!) if I just type the name of the alias.

• Critically, this is decoupled from where I want the link to go, and what I want it to render as. I don't ever want to type "a very long title", or if the core also supports uniqueIDs, I don't want to type timestamps. 

• Something that I think should be incorporated into the core (!!!!) (or at least with uni-link) is the Edit-Comptext dropdown plugin. This plugin already lets you define custom dropdown templates, so its natural to incorporate the filters uni-link provides. 

1. where I want it to go

• If we're not just focusing on aliases for a second, this isn't as straightforward as I initially thought - frequently I want a link to go to a tiddler with a specific title. Sometimes I want a link to go to a specific tiddler, regardless of its title. For example, my TW is constantly evolving as my system of knowledge is evolving - I rename things, reorganize, etc frequently. If I rename a method, or person, etc. all of my links to this tiddler no longer work (there have been previous discussions of renaming tiddlers triggering a search/replace). I am more interested in linking by a UNIQUE ID (for example, created timestamp) - that way renaming a tiddler's title does not trigger massive changes in the rest of your tiddlers (I use version control, so its a bit annoying when I rename one tiddler, my commit object contains changes to 20 other tiddlers whose links to this one tiddler also had to change). 

• Uni-link address this issue, by introducing a (hopefully) unique field: aliases. So now I can link to [[coolMethod|?]] regardless of the actual title of that tiddler. This is a wonderful feature. 

1. what I want to render

• TW already supports some version of this, as sometimes its appropriate to render a tiddler's title, other times its caption. I just want to extend this so that the user has more fine-grained control over this. 

[TonyM]

Folks,

Please find attached a work in progress experiment bundle of tiddlers. Drop in an empty wiki and see how new tiddlers (Via done button) gain a serial number and existing tiddlers can have one added. This replaces the standard edit template done button (hides existing), and modifies $:/core/ui/ViewTemplate/subtitle to show the serial number under the view tiddler buttons.

See $:/PSaT/Tiddler-serial-numbers for more

Try the following where N is a valid serial number

:<<tsn N>>
:<<tsn N link>>
:<<tsn N text>>
:<<tsn N transclude>>

Or

:<<qualify-tsn>>
:<<tsn>

PS this is very initial and has inconsistent notes and pieces.