NoteTaking in Streams

1186 views
Skip to first unread message

ludwa6

unread,
Jun 24, 2021, 9:21:54 AMJun 24
to TiddlyWiki
Picking up where this earlier thread left off, i am following Saq's advice to start another conversation on the more specific question of HOW best to use -and perhaps adapt- the Streams plugin, to make TW into a great tool for not only "intertwingly" NoteTaking (which it certainly is), but also great for agile NoteTaking, as defined in aforementioned thread. 

Having just installed the plugin (Streams 0.2.18) in my TiddlyDesktop instance, i am already fairly drunk on the power of having what feels like a full-fledged outliner within a tiddler, where i can bang out a stream of bullet points, promote & demote, indent & outdent, etc... All as fast as i can type the keystrokes, or drag&drop on mobile. 

But then: the result is a slew of tiddlers with long numeric IDs that, tho nicely presented in the edit window as a clean hierarchal outline, cannot be either flattened into a single tiddler, nor exported or even copy/pasted into the tiddler body via any built-in affordance.  I've been going the copy/paste way so far, but it's a hard road to travel, given any significant length and/or complexity to your outline. 

Now: plugin author Saq said essentially that the only reason he hasn't built in any such affordance is that he doesn't know what users want: a simple concatenation of the stream? a hierarchial structure in some form? 

For me, the best (and simplest?) solution would be to convert the stream to Markdown: each node being just a line of text preceded by a number of asterisks (1->any) to reflect its level in the hierarchy.  This could be interpreted & displayed correctly in any tiddler that is formatted Markdown; TW has affordances for this -as does Github (.md is default format of ReadMe files there) and most web publishing & word processing apps you can find these days.  Indeed, Saq indicated that a  .md<->.tid converter would be no big deal (for him, at least :-) to create; this would enable not only "flattening" of a Stream in such manner, but also importing a .md file as presentable tiddler content.  Now THAT would be cool, IMHO... But that's just me, so far.

SO this is to ask all of you with any experience of Streams plugin: Can you relate to the need i am expressing?  Or have you found a good way "flatten" your streams, and integrate your Streams content properly into a TW instance?  If so, anything you might care to share in terms of workflow and/or code would be most appreciated.

NB: I did learn in the course of that earlier thread about this Streams-to-text solution (thanks @Si for the pointer), but Saq warned that the plugin (actually a microAddOn/Mod, according to author Jan) is not compatible with current version of Streams as it stands, though he believes that Jan is working on it.  So i'm holding off on installing that for now, but would love to hear any news or user reports about that.

/walt

Saq Imtiaz

unread,
Jun 24, 2021, 10:59:33 AMJun 24
to TiddlyWiki
Hi Walt, 

Hopefully you will get input from people actually using Streams as to what works for them.

One of the things I would dearly like to know is how many people are actually using Streams on a regular basis, and what their workflow looks like. My feeling is its a very small handful and I'll admit that probably influences how much time I devote to working on Streams.
  
But then: the result is a slew of tiddlers with long numeric IDs that, tho nicely presented in the edit window as a clean hierarchal outline, cannot be either flattened into a single tiddler, nor exported or even copy/pasted into the tiddler body via any built-in affordance.  I've been going the copy/paste way so far, but it's a hard road to travel, given any significant length and/or complexity to your outline. 

Have you considered any of the methods outlined here?

Now: plugin author Saq said essentially that the only reason he hasn't built in any such affordance is that he 
doesn't know what users want: a simple concatenation of the stream? a hierarchial structure in some form?

It's a bit more than that. Each node in a stream is a tiddler and can have any content. Streams was originally envisioned not just for rapid note taking but also for the ability to divide your text into smaller tiddlers as you write/edit. As such it is difficult to envisage a single export format that would work for everyone's content.  However using the approaches outlined in the link above you can easily set up your own markdown export.

 
For me, the best (and simplest?) solution would be to convert the stream to Markdown: each node being just a line of text preceded by a number of asterisks (1->any) to reflect its level in the hierarchy.  

The issue is that MD tiddlers will always be second class citizens in TiddlyWiki. I think in the long run this isn't going to be a feasible approach unless all you want to do is write your notes and export them outside of TiddlyWiki. Also note that while MD syntax does support multiline content in lists, TW markup does not ( at least not cleanly).

As an aside, I think something like this unfulfilled experiment would suit your use case from what I know of it:
Before you ask, no plans to take that any further, in part due to the reasons outlined above.

Cheers,
Saq

ludwa6

unread,
Jun 24, 2021, 12:12:38 PMJun 24
to TiddlyWiki
Hey @Saq: That demo of Streams-as-editor plugin (what i presume is performing the magic, from browsing the plugin list) is indeed interesting... But it freezes up when i push the "save" button (have downloaded and tried an instance in both TiddlyDesktop and current Chrome browser; no joy), so i don't know what sort of result it yields.  I'd install the plugin (the one labelled version 0.0.01, right?)  if i weren't afraid of what havoc it might wreak in my wiki, and how it plays alongside current Streams plugin (0.2.18 is what i have just installed).  Won't ask you to touch it, but if you have any sage advice to share, i'm all ears :-)

As to those other methods you outlined in the docs: I did consider them, but none yields the result i want.  The first one disregards node hierarchy, while the others yield html output.  What i need is a text format that preserves hierarchy, whether .MD or native wikitext and saves to a single tiddler; if that save process be a one-button save, then it should be wikitext, since that is default.  I hear you say that, given those approaches you laid out, i might "easily set up [my] own markdown export," but here my usual IABASF (I Am But A Simple Farmer) copout applies <8-)

Still: i'm not afraid to diddle with my Tiddly innards a bit, as time permits - which it does not just now, alas. 
Meanwhile: i too would very much like to know who else is using Streams regularly, and in what sort of workflow(s). 
So if you are one, dear reader, please let us know!

/walt

Saq Imtiaz

unread,
Jun 24, 2021, 1:01:16 PMJun 24
to TiddlyWiki
@Walt

On Thursday, June 24, 2021 at 6:12:38 PM UTC+2 ludwa6 wrote:
Hey @Saq: That demo of Streams-as-editor plugin (what i presume is performing the magic, from browsing the plugin list) is indeed interesting... But it freezes up when i push the "save" button (have downloaded and tried an instance in both TiddlyDesktop and current Chrome browser; no joy)

It's a demo of a concept that works on the demo site itself. I would not recommend it for anything else.
 

PMario

unread,
Jun 24, 2021, 5:01:12 PMJun 24
to TiddlyWiki
Hi Walt,

I don't use streams regularly but from time to time it's convenient for brainstorming and refactoring to get ideas sorted.

I did a heavy test with an early version in August 2020 for 2 days, where I did follow an online conference about DAT (now Hyper). It was possible to follow the live-stream and the chat to keep notes in real time. The wiki can be seen at: http://dat-conf-2020.tiddlyspot.com/ ... BUT it doesn't make sense on it's own. You'll need the Agenda and the Videos .

I did set streams up in way, that tiddler titles create an automatic "timeline" ... Which will allow my to connect the notes to the videos, if needed.

Every tiddler has this format DAT-CON/2020/07/30/16:08:49-582  DAT-CON/<year>/<month>/<day>/0hh:0mm:0ss-XXX

So the TW tree-macro can also be used to view the timeline in a different format. The tree-macro in the example only shows the tiddler titles. To show more info I needed to modify the macro according to my needs, but that's not public.

I basically use the notes with the default search function. .. I open the tiddlers .. have a look for my own keywords to find the tiddler with the exact "timestamp"... With the timestamp I can find the related discussion or presentation in the Videos. ---> That's the main purpose of this outline.

The published version doesn't contain any personal notes. ... So for me there is absolutely no desire to "flatten" the structure. ... I "widen" it with my notes and additional links to loosely related blog posts and github repos ...

My initial feedback can be found here: https://groups.google.com/g/tiddlywiki/c/1jTwdmq8cgI/m/Orm9JRVNBQAJ ... The main problems that I did mention there with streams V0.1.10 are fixed with the latest version (0.2.18) of streams. So I'm perfectly fine with the possibilities it offers.

For my personal convenience I need to adjust 1 little bit of CSS. I need a different colour for the "collapse indicator", if a parent node is collapsed. It would be cool, if this was configurable, but in normal cases I only copy paste the CSS tiddler.

I need to add the "keyboard shortcut" cheat-sheet to the right sidebar by adding the tag $:/tags/Sidebar. @Saq This could be part of the control-panel config options ;) I need it for the shortcuts that I use seldom.

For me "streams" is probably _the_ "easiest to use" "realtime" outliner I've ever used.

For exporting you should probably have a closer look at the filters that Saq mentioned. Especially the last one. ...

You are right, there is some room for improvement if "lists of stream tiddler titles" are shown. They are not very informative ... But imo the problem is, the needed list templates are highly use-case specific. .. So I actually don't know how they should look like. Not even form my own usecases.

Since stream uses a field named "parent" to link to parent tiddlers my tocP macro can also be used to show the structure. BUT in the current form this isn't too useful, since it only shows the title. ... I was experimenting to show the "subtitle" field, but adding that field manually is way too much work, to be done in realtime. (Mainly because finding good tiddler sub-titles is hard) ... I'm still not sure, how to solve this in a convenient and fast way.

I hope this helps.

have fun!
mario

Si Si

unread,
Jun 24, 2021, 7:01:09 PMJun 24
to tiddl...@googlegroups.com
Hi Walt,

SO this is to ask all of you with any experience of Streams plugin: Can you relate to the need i am expressing?  Or have you found a good way "flatten" your streams, and integrate your Streams content properly into a TW instance?  If so, anything you might care to share in terms of workflow and/or code would be most appreciated.

I don't personally have any need to convert a stream into a bulleted list within a single tiddler. As you note, outliners very useful for agile note taking, but in my opinion a hierarchical list is not the best way to present a single idea. So in general once I have moved my streams nodes to their evergreen home, I aspire to convert my notes into more readable prose. I do this by rearranging the nodes into a flat list, re-writing as required, and then flattening with a button described here: https://groups.google.com/g/tiddlywiki/c/rEtegfiGYJE/m/L0-euDfWAAAJ

In reality I rarely actually get to that stage, and am generally happy to leave my evergreen notes in "node-form". I don't personally see any benefit in converting them to a bulleted markdown list, but it depends on what your use case is?

Meanwhile: i too would very much like to know who else is using Streams regularly, and in what sort of workflow(s). 
So if you are one, dear reader, please let us know!  

 Hi Saq,

One of the things I would dearly like to know is how many people are actually using Streams on a regular basis, and what their workflow looks like. My feeling is its a very small handful and I'll admit that probably influences how much time I devote to working on Streams.

You may have seen it but I roughly described my workflow here: https://groups.google.com/g/tiddlywiki/c/gbEHUyX8dc0/m/r1yF5JdXCAAJ

For me Streams excels at quickly capturing information that I cannot yet fit into any main ideas within your knowledge. Being able to re-jig this captured information is helpful for making sense of it, eventually integrating them with the main ideas that I want to internalise. In particular this can help to break through some of the constraints of linear reading.

I aspire to one day build some incremental reading tools with TiddlyWiki, which may partially replace my use of streams, but for now I have found streams to be the best solution for non-linear reading.

On the topic of how many people use it, maybe you could consider creating a Google Form and linking to it from the download page, asking users if they would take a second to answer some brief questions?

I was also wondering if you had considered setting up a way for users to donate? Not necessarily to fund development, just to show appreciation for great free software! I am not currently in a position to contribute much, but in the future I would certainly be happy to subscribe to a patreon or whatever.

I would add that like Mario I am very happy with the current features of streams, though I have made the following tweak to the text display for the breadcrumbs. It's a very minor thing, but in my opinion looks nicer than just cutting off the title mid-word:

<$text text={{{ [<display-title>length[]compare:number:lt[40]then<display-title>] ~[<display-title>split[]first[40]join[]trim[]addsuffix[...]] }}}/>

 I wonder if you think it should be added when you next update?

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/2485302e-c5a4-4b3d-a9d8-46a367a2c65cn%40googlegroups.com.

Frédéric Demers

unread,
Jun 24, 2021, 11:01:37 PMJun 24
to TiddlyWiki
I use Streams daily (all my TW interactions are Streams-based). I my workflow, I treat them as sub-tiddlers really. You could say I replaced larger tiddlers with multiple headings with much smaller streams nodes (but not single line ones); main advantage is I can move nodes around, indent them, etc... and drag them from one tiddler to another (although I have recently noted this has an unfortunate side effect). And the double-click edit is very handy, focused to the one sub-tiddler I care to change.

I have customized the node titles to be  <$macrocall $name="unusedtitle" baseName=<<parent>> separator="/" /> (requires pre-release). In my workflow however, the return key does not create a new nodes; shift-return does (ctrl-return exits edit mode). This way I can enter several lines, bullets, etc.. and when I am ready, shift-return for the next node. I like that better. I use alt-right, alt-left to indent, so I can manage lists of bullets, number or checkboxes.

In all I am a big fan; I have on occasion wished for the ability to flatten and move content around to overcome the unfortunate side-effect, but have done so manually.

Cheers; and thanks Saq!

ludwa6

unread,
Jun 25, 2021, 5:57:56 AMJun 25
to TiddlyWiki
That's an interesting UseCase, @mario -along w/ some interesting content, including talk by Don Blair (NB: that farm in Portugal he talks about is mine :-).  

Wish i'd had access to such toolchain during all those COVID-year Zoom conferences i attended, proceedings of which are mostly lost in time "like tears in the rain"

In a standalone-scenario like this, where the wiki serves sole purpose of conference record, with just one tiddler for the conference, serving as "parent" to all those subtiddlers that it has spawned, then:  yes: no need to "flatten" the stream; that would actually defeat the purpose - purpose being to summarize and/or elaborate on the "live" content. 

The challenge i see with this is all the manual node-wrangling involved -greatly facilitated by "Streams" affordance for outlining via keyboard shortcuts (esp w/ shortcut "cheatsheet" in the sidebar, as you suggest), but to whatever extent that process could be further automated, i see that as a real opportunity, especially w/r/t the work of correlating these nodes with actual video timecodes.  Your system correlated the node tiddlers to your own clock-time, it appears... But then when video from Zoom (Skype, whatever) gets cut up and pushed out to separate YouTube videos, as it. has done in this case, then what is the optimal workflow? 

I guess what i might do to refine this to my prefs is have each talk in the conference with its own "parent" tiddler;  then if i could zero-out that timeline of which you speak at start of each talk, it should then track pretty close to the final video, so long as they don't edit-out any portions of it in post-production.  To achieve perfect alignment, presuming a continuous stream, you might need to tweak these codes after -which would involve a bulk renaming of all related tiddlers i guess. Am just thinking out loud here, w/ benefit of hindsight...

Anyway, as it stands: this looks to me like a very good solution to that problem in the majority of YouTube videos published, whereby most of them have either no "shownotes" at all, or else notes that serve some promotional purpose(s) that most viewers would MUCH rather see complemented (if not entirely replaced) with a helpful time-coded set of jump-links to whatever section(s) of content they consider to be of interest.
To that end, Mario, this represents a solid step forward. Thanks for sharing it here!

/walt

ludwa6

unread,
Jun 25, 2021, 6:16:31 AMJun 25
to TiddlyWiki
@Si : Dude!  I just used your clever little button script to flatten a caffeine-fueled Stream of a couple dozen nodes into a single tiddler to memorialise that epic train of thought :-). 
Think i'm gonna add this button feature to my Journal tiddler template, to see if this makes that sticky business of daily Journal-writing flow eaasierr, without making such a mess in my Zettelkasten. 
Thanks for this, mate...  and, 

PS, i'm likewise interested in the idea of using Streams as an incremental reading tool;  AND

PPS: Also +1 the idea of finding some mechanism whereby we could send some monetary tokens of appreciation Saq's way!

/walt

ludwa6

unread,
Jun 25, 2021, 6:23:40 AMJun 25
to TiddlyWiki
Hey @fred: Good to hear, thanks ; sounds like you too might make good use of the button script Si has provided, as i am now doing.

Also, i am curious to try your node-naming macrocall... But i wonder what you mean by "requires pre-release".  Does that mean a specific version of TW and/or Streams is needed? If so, can you say which it is, specifically?

/walt

Frédéric Demers

unread,
Jun 25, 2021, 6:32:43 AMJun 25
to tiddl...@googlegroups.com
@ludwa6
 the macrocall takes advantage of a new feature in 5.1.24 (or 5.2.0)
  • extended unusedtitle macro to use the prefix parameter
(up until then I had copied/modified the unusedtitle macro to do the same)

You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/-xTFWPwzq6g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/60ad6bb8-4472-44a8-ac50-4b5e5a891720n%40googlegroups.com.

Si

unread,
Jun 25, 2021, 2:51:52 PMJun 25
to TiddlyWiki
@walt Glad you found it useful, but you'll see from that thread that most of the credit goes to Saq!

PMario

unread,
Jun 25, 2021, 3:12:42 PMJun 25
to TiddlyWiki
On Friday, June 25, 2021 at 11:57:56 AM UTC+2 ludwa6 wrote:
That's an interesting UseCase, @mario -along w/ some interesting content, including talk by Don Blair (NB: that farm in Portugal he talks about is mine :-).  

That's funny. And we see again, how small the earth has become :)
 
... I guess what i might do to refine this to my prefs is have each talk in the conference with its own "parent" tiddler;  then if i could zero-out that timeline of which you speak at start of each talk, it should then track pretty close to the final video, so long as they don't edit-out any portions of it in post-production.  To achieve perfect alignment, presuming a continuous stream, you might need to tweak these codes after -which would involve a bulk renaming of all related tiddlers i guess. Am just thinking out loud here, w/ benefit of hindsight...

That's right. It's probably 1 thing we can learn from the wiki: We need a way to create "relative" timestamps. .. I think Eric Shulmans timer-tools may be an option here.
 
Anyway, as it stands: this looks to me like a very good solution to that problem in the majority of YouTube videos published, whereby most of them have either no "shownotes" at all, or else notes that serve some promotional purpose(s) that most viewers would MUCH rather see complemented (if not entirely replaced) with a helpful time-coded set of jump-links to whatever section(s) of content they consider to be of interest.
To that end, Mario, this represents a solid step forward. Thanks for sharing it here!

You are welcome!
mario

Saq Imtiaz

unread,
Jun 25, 2021, 4:47:39 PMJun 25
to TiddlyWiki
@Fred could you please elaborate on what this side effect is that you mention?

I my workflow, I treat them as sub-tiddlers really. You could say I replaced larger tiddlers with multiple headings with much smaller streams nodes (but not single line ones); main advantage is I can move nodes around, indent them, etc... and drag them from one tiddler to another (although I have recently noted this has an unfortunate side effect).

Thank you. 

Frédéric Demers

unread,
Jun 25, 2021, 11:13:07 PMJun 25
to tiddl...@googlegroups.com
@saq; certainly.
When dragging a node from one root tiddler to another, it retains the naming convention of previous tiddler.....
I believe this is working as intended. The parent and stream-list fields are adjusted correctly.

In my case, I sometimes wish to export and re-import a set of root tiddlers and their stream nodes.
At that point, I can't just bulk select (check) all tiddlers (including nodes) that start with the same title; a portion of those may actually now belong to different root tiddlers (thus I don't want to import them). Similarly, I can miss ones I should include because, whilst they are valid node tiddlers, their name is not associated with what I am looking for in the tiddler import dialog.

So I need to more cleverly bulk export one or more root tiddlers with their complete node hierarchy, regardless of title names, quite simply. I don't believe this to be particularly challenging to be honest, so whenever I need to do this, I should be able to do it smartly. 

Cheers,

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/-xTFWPwzq6g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/5407c9c6-b723-4d61-88c1-ad3a1bef24e5n%40googlegroups.com.

PMario

unread,
Jun 26, 2021, 12:32:20 AMJun 26
to TiddlyWiki

-m

Frédéric Demers

unread,
Jun 26, 2021, 7:25:12 AMJun 26
to tiddl...@googlegroups.com
exactly @PMario; thanks for the pointer; coupled with the TWCommander plugin, one can export based on a filter result if I am not mistaken....

+@saq: you will however note this pattern of moving nodes from one tiddler to another also disrupts the powerful relink tool when you rename the root. Only child nodes that match the title prefix are updated; meaning anything you dragged from a different origin retains its old name. The alternative would be the drag n' drop action also updates the child node title to the destination tiddler.



Saq Imtiaz

unread,
Jun 26, 2021, 8:48:28 AMJun 26
to TiddlyWiki

+@saq: you will however note this pattern of moving nodes from one tiddler to another also disrupts the powerful relink tool when you rename the root. Only child nodes that match the title prefix are updated; meaning anything you dragged from a different origin retains its old name. The alternative would be the drag n' drop action also updates the child node title to the destination tiddler.

I've added an issue so that the idea isn't lost: https://github.com/saqimtiaz/streams/issues/22

However, note my reservations about how such a behaviour would be highly dependent on the title configuration being used and therefore adding it may not be feasible. This is the downside of allowing such a high level of configuration and customization.

 

Saq Imtiaz

unread,
Jun 26, 2021, 9:21:52 AMJun 26
to TiddlyWiki
@Walt hopefully you are getting some helpful feedback here. Apologies for the slight digression caused by my replying to Streams specific comments and feedback.

@fred you can also export using a filter from the Filter tab in Advanced Search

@Si,
 
On the topic of how many people use it, maybe you could consider creating a Google Form and linking to it from the download page, asking users if they would take a second to answer some brief questions?

I've been considering it for some time actually. Apart from feedback on workflow, just knowing whether the number of people regularly using Streams is < 10, 10-20, or 20+ would be very helpful.
 
I was also wondering if you had considered setting up a way for users to donate? Not necessarily to fund development, just to show appreciation for great free software! I am not currently in a position to contribute much, but in the future I would certainly be happy to subscribe to a patreon or whatever.

I think the most value for myself would be in just getting regular, precise and on-topic feedback. Implementing new features in the absence of user feedback is just driving blind from my perspective. I also find it very frustrating and motivation sapping when I post ideas or demos and at times get feedback that is far off topic or just so poorly written that it is more frustrating than helpful.  Mind you, I try to be very understanding of users for whom English is not their first language. It is actually very pleasing how constructive and helpful this thread has been thus far in comparison.  

I've also been considering asking for volunteers for a small group of regular users of Streams for testing new features.. from my perspective however how, much effort I put into adding new features to Streams is highly dependent on how many people regularly use it and find that it imparts value. Call it a personal quirk but wanting to impart the most value possible is something that often drives my decision making in terms of what I dedicate my time to. Otherwise the motivation for work on Streams is just whatever seems interesting for me personally from a technical perspective.
 
I would add that like Mario I am very happy with the current features of streams, though I have made the following tweak to the text display for the breadcrumbs. It's a very minor thing, but in my opinion looks nicer than just cutting off the title mid-word:

<$text text={{{ [<display-title>length[]compare:number:lt[40]then<display-title>] ~[<display-title>split[]first[40]join[]trim[]addsuffix[...]] }}}/>

That is definitely a nice idea. I wonder if we might take it a step further and just drop the last word (which may be incomplete). This is untested:

 <$text text={{{ [<display-title>length[]compare:number:lt[40]then<display-title>] ~[<display-title>split[]first[40]join[]trim[]search-replace:g:regexp[(.*)\s(\w+)$],[$1]] }}}/>

What do you think?
Cheers,
Saq

TW Tones

unread,
Jun 26, 2021, 9:30:56 AMJun 26
to TiddlyWiki
Saq,

Well, I not one person that uses streams every day, Me. I am also continuing to extends its features with my own tools.

The big boost to my use was discovering how much I can make us of drag and drop in both Streams and Projectify. This has changed my everyday use of tiddlywiki.

I also see Streams as a source of ideas to develop even more general tools. 

I will volunteer for anything to do with streams, but yes I have to manage the rest of my life, but if you are happy if sometimes I make take time to get around to something count me in.

Love your work
Tones


ludwa6

unread,
Jun 26, 2021, 10:24:16 AMJun 26
to TiddlyWiki
Great feedback, all!  And to your comment, Saq:

On Saturday, June 26, 2021 at 2:21:52 PM UTC+1 saq.i...@gmail.com wrote:
@Walt hopefully you are getting some helpful feedback here. Apologies for the slight digression caused by my replying to Streams specific comments and feedback.

Not a digression at all!  this is about "NoteTaking in Streams" -which adds such new dimensions for expression in TW, it's great to hear about these different applications & adaptations. 

Saq, i just hope that the enthusiasm of your user community here is sufficient to retain your continuing interest in Streams.  As said in that earlier thread:  if the authoring world does indeed break more-or- less neatly into "Architect" types and "Gardener" types, then by that same token the intersection set between Outliners and Wiki-Weavers may be pretty small... But then, those of us who fall into that crossover ground REALLY need this TW+Streams solution.  
I for one have been on the hunt for a software like this some >25 years ;  there's nothing else that even comes close, AFAICT.

/walt 

Saq Imtiaz

unread,
Jun 26, 2021, 11:29:50 AMJun 26
to TiddlyWiki
@walt 
Saq, i just hope that the enthusiasm of your user community here is sufficient to retain your continuing interest in Streams. 

Unless something very unexpected happens, I am committed to at least maintaining Streams in its current form for the long term. So you need not worry that it will stop working or not be updated for new TW versions etc.

Cheers,
Saq

Si Si

unread,
Jun 27, 2021, 3:59:28 PMJun 27
to tiddl...@googlegroups.com
Saq,

I've also been considering asking for volunteers for a small group of regular users of Streams for testing new features..

I'm happy to test stuff out as required.  
 
from my perspective however how, much effort I put into adding new features to Streams is highly dependent on how many people regularly use it and find that it imparts value. Call it a personal quirk but wanting to impart the most value possible is something that often drives my decision making in terms of what I dedicate my time to.

It says a lot of positive things about your character if that is what motivates you!

That is definitely a nice idea. I wonder if we might take it a step further and just drop the last word (which may be incomplete). This is untested:  
 <$text text={{{ [<display-title>length[]compare:number:lt[40]then<display-title>] ~[<display-title>split[]first[40]join[]trim[]search-replace:g:regexp[(.*)\s(\w+)$],[$1]] }}}/>
What do you think?
--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/-xTFWPwzq6g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/0b65a408-28d6-4709-8e16-feb01355858en%40googlegroups.com.

Si Si

unread,
Jun 27, 2021, 4:04:38 PMJun 27
to tiddl...@googlegroups.com
Whoops, sent the previous message before finishing it:
 
That is definitely a nice idea. I wonder if we might take it a step further and just drop the last word (which may be incomplete). This is untested:  
 <$text text={{{ [<display-title>length[]compare:number:lt[40]then<display-title>] ~[<display-title>split[]first[40]join[]trim[]search-replace:g:regexp[(.*)\s(\w+)$],[$1]] }}}/>
What do you think?
 
I did consider this, but I'm actually not sure which way looks better. I don't think that cutting off an incomplete word half way through looks bad, and I might even prefer it, though I couldn't say why. Possibly just because that's what windows explorer does so I'm just used to it.

Saq Imtiaz

unread,
Jun 28, 2021, 1:44:26 PMJun 28
to TiddlyWiki
@si
 
I've also been considering asking for volunteers for a small group of regular users of Streams for testing new features..

I'm happy to test stuff out as required.  

I think I will start out with a questionnaire and also use that as an opportunity to ask for volunteers for testing.

I will give some more thought to the breadcrumbs.

Regards,
Saq

Steph Butera

unread,
Jun 28, 2021, 4:08:26 PMJun 28
to TiddlyWiki
Hi Saq, I'd also be happy to take part in the questionnaire. I use Streams daily and appreciate it very much.

All best,
Steph

stan...@gmail.com

unread,
Jun 28, 2021, 7:07:37 PMJun 28
to TiddlyWiki
You can add me to the list for the questionaire.  Streams is an excellent addition to TW.
Stan

On Monday, June 28, 2021 at 1:44:26 PM UTC-4 saq.i...@gmail.com wrote:

Saq Imtiaz

unread,
Jun 28, 2021, 7:42:47 PMJun 28
to TiddlyWiki
Thank you Stan and Steph. 
I'll post a link here and on reddit once the questionnaire is ready. 

TW Tones

unread,
Jun 28, 2021, 8:52:12 PMJun 28
to TiddlyWiki
Saq,

The breadcrumbs in Stream are nice. As I understand it the breadcrumbs find the parents to the given node. In this case using parent and streams-list fields.

I thought of the following because I expect streams includes 90% of the code to do as follows.

Mario's TopP plugin Allows new here on a nominated tiddler and creates the Parent field. 

It seems the breadcrumbs feature could be split out into a generic plugin itself the simple specifications would be;
  • Using the currentTiddler (or a supplied one) as the node in a breadcrumb trail
  • Provide the parent fieldname
  • Provide the children's list field.
  • If there is no parent or child list field (but the other is present) indicate top or Bottom
    • If neither do not display at all
  • Allow horizontal (default) or vertical breadcrumbs (in side bar)
Ways to display such breadcrumbs could be in the above and below story, a menu bar second row or floating bar. One trick I have used for a tool I call "focus tiddler" is to access {{$:/HistoryList!!current-tiddler}} which contains the last edited or navigated to tiddler, and a simple viewToolbar button can set this to the set tiddler. Thus the breadcrumbs will be displayed for the current tiddler with focus (on screen but optionally outside the tiddler). It may also look nice in the subtitle in small text.

Just some ideas to leverage your existing work.

Tonesza

Saq Imtiaz

unread,
Jun 29, 2021, 5:06:11 AMJun 29
to TiddlyWiki
It seems the breadcrumbs feature could be split out into a generic plugin itself...

@Tones please feel free to do so. It is only about ten lines of wikitext. As I've previously mentioned, the initial motivation for writing Streams and doing so almost entirely in wikitext was to motivate and inspire others in terms of what was possible in wikitext.

Cheers,
Saq

Jan Johannpeter

unread,
Jun 29, 2021, 7:42:47 AMJun 29
to tiddlyWiki
Hi Walt, I am happy to announce that the streams to text is finally updated: https://szen.io/stream/ . Thanks to Saq who has gently helped me and provided hooks within the plugin, that make it fun tinkering with the plugin, this version should be more future-proof than the last ones
.
As always: Backup before using it to recompress the ideas, you collected with streams.

Enjoy,
Jan



Am Do., 24. Juni 2021 um 15:22 Uhr schrieb ludwa6 <wlud...@gmail.com>:
Picking up where this earlier thread left off, i am following Saq's advice to start another conversation on the more specific question of HOW best to use -and perhaps adapt- the Streams plugin, to make TW into a great tool for not only "intertwingly" NoteTaking (which it certainly is), but also great for agile NoteTaking, as defined in aforementioned thread. 

Having just installed the plugin (Streams 0.2.18) in my TiddlyDesktop instance, i am already fairly drunk on the power of having what feels like a full-fledged outliner within a tiddler, where i can bang out a stream of bullet points, promote & demote, indent & outdent, etc... All as fast as i can type the keystrokes, or drag&drop on mobile. 

But then: the result is a slew of tiddlers with long numeric IDs that, tho nicely presented in the edit window as a clean hierarchal outline, cannot be either flattened into a single tiddler, nor exported or even copy/pasted into the tiddler body via any built-in affordance.  I've been going the copy/paste way so far, but it's a hard road to travel, given any significant length and/or complexity to your outline. 

Now: plugin author Saq said essentially that the only reason he hasn't built in any such affordance is that he doesn't know what users want: a simple concatenation of the stream? a hierarchial structure in some form? 

For me, the best (and simplest?) solution would be to convert the stream to Markdown: each node being just a line of text preceded by a number of asterisks (1->any) to reflect its level in the hierarchy.  This could be interpreted & displayed correctly in any tiddler that is formatted Markdown; TW has affordances for this -as does Github (.md is default format of ReadMe files there) and most web publishing & word processing apps you can find these days.  Indeed, Saq indicated that a  .md<->.tid converter would be no big deal (for him, at least :-) to create; this would enable not only "flattening" of a Stream in such manner, but also importing a .md file as presentable tiddler content.  Now THAT would be cool, IMHO... But that's just me, so far.

SO this is to ask all of you with any experience of Streams plugin: Can you relate to the need i am expressing?  Or have you found a good way "flatten" your streams, and integrate your Streams content properly into a TW instance?  If so, anything you might care to share in terms of workflow and/or code would be most appreciated.

NB: I did learn in the course of that earlier thread about this Streams-to-text solution (thanks @Si for the pointer), but Saq warned that the plugin (actually a microAddOn/Mod, according to author Jan) is not compatible with current version of Streams as it stands, though he believes that Jan is working on it.  So i'm holding off on installing that for now, but would love to hear any news or user reports about that.

/walt

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/2485302e-c5a4-4b3d-a9d8-46a367a2c65cn%40googlegroups.com.

TW Tones

unread,
Jun 29, 2021, 9:41:14 AMJun 29
to TiddlyWiki
Saq,

You certainly have "motivate[d] and inspire[d] others" - me!

Tones

arun babu

unread,
Jun 29, 2021, 12:45:39 PMJun 29
to tiddl...@googlegroups.com
Saq,

Can you help regarding the first request mentioned in this issue.  That is text clipped using tiddyclip from a webpage should be added as child nodes of a single parent tiddler 



On Thu, Jun 24, 2021 at 8:29 PM Saq Imtiaz <saq.i...@gmail.com> wrote:
Hi Walt, 

Hopefully you will get input from people actually using Streams as to what works for them.

One of the things I would dearly like to know is how many people are actually using Streams on a regular basis, and what their workflow looks like. My feeling is its a very small handful and I'll admit that probably influences how much time I devote to working on Streams.
  
But then: the result is a slew of tiddlers with long numeric IDs that, tho nicely presented in the edit window as a clean hierarchal outline, cannot be either flattened into a single tiddler, nor exported or even copy/pasted into the tiddler body via any built-in affordance.  I've been going the copy/paste way so far, but it's a hard road to travel, given any significant length and/or complexity to your outline. 

Have you considered any of the methods outlined here?

Now: plugin author Saq said essentially that the only reason he hasn't built in any such affordance is that he 
doesn't know what users want: a simple concatenation of the stream? a hierarchial structure in some form?

It's a bit more than that. Each node in a stream is a tiddler and can have any content. Streams was originally envisioned not just for rapid note taking but also for the ability to divide your text into smaller tiddlers as you write/edit. As such it is difficult to envisage a single export format that would work for everyone's content.  However using the approaches outlined in the link above you can easily set up your own markdown export.

 
For me, the best (and simplest?) solution would be to convert the stream to Markdown: each node being just a line of text preceded by a number of asterisks (1->any) to reflect its level in the hierarchy.  

The issue is that MD tiddlers will always be second class citizens in TiddlyWiki. I think in the long run this isn't going to be a feasible approach unless all you want to do is write your notes and export them outside of TiddlyWiki. Also note that while MD syntax does support multiline content in lists, TW markup does not ( at least not cleanly).

As an aside, I think something like this unfulfilled experiment would suit your use case from what I know of it:
Before you ask, no plans to take that any further, in part due to the reasons outlined above.

Cheers,
Saq

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

ludwa6

unread,
Jun 29, 2021, 1:16:58 PMJun 29
to TiddlyWiki
Hey @Jan: Great to hear!
So following these instructions, i have installed everything on that page below the Streams and Relink plugins (which i already have) -everything but the German language thingy.  That is to say: additional tiddlers numbered 1-5, plus stream-row-template, the two things under "A button to show the keyboard shortcuts," plus the MobileTweaks... But for all that, i cannot get any new button to reveal, nor does alt+f do anything useful ( on my mac, it enters the ƒ character).  Of course i've saved, refreshed, tried in different browsers, etc.
Any idea what i might have got wrong, or what else must i do, to get this add on working?

/walt


On Tuesday, June 29, 2021 at 12:42:47 PM UTC+1 Jan wrote:
Hi Walt, I am happy to announce that the streams to text is finally updated: https://szen.io/stream/ . Thanks to Saq who has gently helped me and provided hooks within the plugin, that make it fun tinkering with the plugin, this version should be more future-proof than the last ones
.
As always: Backup before using it to recompress the ideas, you collected with streams.

Enjoy,
Jan
... 

Saq Imtiaz

unread,
Jun 29, 2021, 1:39:28 PMJun 29
to TiddlyWiki
@walt

For the flatten capability you only need the tiddlers numbered 1-5
In particular I would advise caution against using the  stream-row-template override.

If you drag and drop and import those 5 tiddlers, the flatten button will turn up at the bottom of the stream as long as the stream has some nodes.

ludwa6

unread,
Jun 29, 2021, 2:45:43 PMJun 29
to TiddlyWiki
Thanks, Saq: I deleted the stream-row-template object, and now -tho i still don't see any new button at bottom of stream, i find that, by clicking on the bullet that appears at top left of a stream w/ nested nodes, it opens up the UI on this modal, which delivers what's promised -and more!

@Jan: Thanks a heap for this update; it adds real utility to the app for me!  A nice bonus (maybe it was also in earlier version/ demo, and i didn't notice) is that pulldown option in the modal UI  to select either bulleted or numbered list, as alternative to the paragraph-formatted option.  
What could add even more value to this feature would be to have the number of bullets or hashmarks correspond to the indent level;  that would preserve the essential function of "flattening" the many nodes into a single text block, while also yielding text in format that TW could interpret and render with treelike indentation, as an option.  (there's my €0.02 of user feedback, fwiw! :-)

/walt

Jan Johannpeter

unread,
Jun 29, 2021, 5:06:48 PMJun 29
to tiddlyWiki
Hi Walt,
I guess you have to install Saq's brandnew version 0.2.19 of  his plugin. He programmed a hook to make the bottom-buttons appear because they are tagged.

As for the requested sensor for the indent level I am sorry to admit that I have no idea how to build this.

Cheers Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

Keelan Cook

unread,
Jun 30, 2021, 6:33:00 AMJun 30
to TiddlyWiki
It's been a bit since I've had time to chime in on the group here, but I wanted to drop my two cents in as well. I use Streams significantly. In fact, it's the plugin that has made TiddlyWiki a superior tool for me in my research and writing. My work has me doing a great deal of historical research and then writing long form output. 

Walt notes the architect/gardner distinction. I actually think Streams in TW5 is one of the best crossover tools I've seen to bridge the space between those things. In my lane of knowledge work, I have to start in one and transition to the other. There is usually an ugly ditch between the two when it comes to tools that are good for one or the other. I've used a lot of different tools at this point, but nothing comes close (for me at least) to allowing the idea generation phase to seamlessly transition into long form writing. The friction for me right now is found in "last mile" portions of my workflow as I'm trying to get it out of TiddlyWiki into industry standard publication formats.

@Saq, I've said this before, but you've developed one of the most helpful tools I've used in years. It not only works seamlessly until the final steps, but it also does so in a clean, minimalist way that doesn't interrupt the creative process. It's not cluttered up with extra boxes or constantly requiring some form of context switching to get thoughts down in draft form. 

My use case: 
  • I collect notes from works I'm reading (books, journal articles, etc.) into source streams. These are titled with a bibtex key using the bibtex plugin.
  • I use Streams for drafting my writing. This is what is most important to me. I develop the outline for my long form writing (think academic articles or book length) by creating a root stream and building out the outline. The beauty of Streams is that it allows me to build that initial outline out into an entire draft of my article, from simple headings all the way to full body text. It grows easily. Nodes can be full paragraphs, so I begin writing my article inside the outline itself. This allows the flexibility to easily move paragraphs and whole sections around as I'm writing in draft mode. In addition, since I tie my notes to a tiddler titled with a bibtex key, I can link to my notes by citing the bibtex key for a source in my draft. This becomes helpful later, when I want to move this work to LaTeX or elsewhere for publication.
  • The need to flatten and export for publication. Of course, when the draft is complete, I need an easy way to export my written article into a format that I'm able to publish. This is currently the most fragile part of the process. Though, that is likely my lack of understanding when it comes to coding or writing my own solution.
On flattening & exporting: 
I've played with some of the approaches above, and they work. In specific Jan's mod is a helpful step in the right direction. However, some of Walt's initial points in this thread really resonate with me. Ideally, I think I would like to keep the initial draft of my work as an outline and create a copy that is flattened for export. To Saq's point above, a wikitext tiddler is superior to a markdown tiddler. I wouldn't want to lose my initial streams set by flattening in a way that replaces it. I'd rather keep it as a stream and export a flattened tiddler. If I'm not mistaken, the default behavior of most flattening techniques I've seen is to flatten the actual collection of tiddlers instead of outputting a copy. I'd love to have a setting to make a copy instead. Perhaps that is something that could be added to Jan's dialogue box of settings alongside the ability to choose bullets or paragraph format.

Furthermore, thinking toward export, Walt's suggestion of parsing into Markdown would be really helpful for me. Even more helpful would be LaTeX, but that's probably too niche for most people. In a perfect world, I would draft up a full article in Streams, then have an easy "one-button" export process that (1) flattens the text into a single tiddler as a copy of the original and (2) parses the results into either markdown or (preferably) LaTeX for easy copy and paste export into my publishing tool. Bonus points if I could choose to export as markdown or LaTeX!

Grateful for the work of this community, and the conversation that is always going on around here about how to make better tools for thought. 

ludwa6

unread,
Jun 30, 2021, 7:55:28 AMJun 30
to TiddlyWiki
Ah -nice surprise! Thanks @Jan for the heads-up about new Streams version (i musta missed the memo ;-)  Have just upgraded to 0.2.19, which is now working like a champ w/ your add-on.

@Saq: Obviously you meant what you said about staying committed to this product, given sufficient interest from users, so... You can count me among the tribe of heavy (i.e. daily) users, and if ever you are wanting someone(s) to try out something new, please do contact me directly.  

/walt


On Tuesday, June 29, 2021 at 10:06:48 PM UTC+1 Jan wrote:
Hi Walt,
I guess you have to install Saq's brandnew version 0.2.19 of  his plugin. He programmed a hook to make the bottom-buttons appear because they are tagged.

As for the requested sensor for the indent level I am sorry to admit that I have no idea how to build this.

Cheers Jan

Am Di., 29. Juni 2021 um 20:45 Uhr schrieb ludwa6 <wlud...@gmail.com>:
Thanks, Saq: I deleted the stream-row-template object, and now -tho i still don't see any new button at bottom of stream, i find that, by clicking on the bullet that appears at top left of a stream w/ nested nodes, it opens up the UI on this modal, which delivers what's promised -and more!

@Jan: Thanks a heap for this update; it adds real utility to the app for me!  A nice bonus (maybe it was also in earlier version/ demo, and i didn't notice) is that pulldown option in the modal UI  to select either bulleted or numbered list, as alternative to the paragraph-formatted option.  
What could add even more value to this feature would be to have the number of bullets or hashmarks correspond to the indent level;  that would preserve the essential function of "flattening" the many nodes into a single text block, while also yielding text in format that TW could interpret and render with treelike indentation, as an option.  (there's my €0.02 of user feedback, fwiw! :-)

/walt
...

ludwa6

unread,
Jun 30, 2021, 8:20:38 AMJun 30
to TiddlyWiki
Thanks-you, @keela, for chiming-in; so good to hear from another fellow-traveller on this path-less-travelled.  Were it not for that "ugly ditch" of which you speak, i suspect we'd have a lot more company -which will likely be the case, i suspect, if this Tiddly-Streams solution ever comes to the attention of some like-minded mainstream influencer(s).

In fact: having bridged that "ugly ditch," it is just the "last mile problem" that remains to be solved, for Tiddly-Streams to be a real category-killer, IMHO.  I don't know (being a civilian in this world of code-slingers) what it would involve, but it seems to me that if Tiddly-Streams had a solid .md<->tid converter, that would make for interop with so many other publishing tools (many of which can easily convert to LaTex), that would then make this the most versatile tool available for anyone who slings ideas for a living -bar none!

That being said: this is still more of a toolkit, not such a polished solution as some others, which have had the benefit of significant investment capital.  Still: in terms of usability & utility right out of the box, Tiddly-Streams beats the pants off Roam or Obsidian or Node, or so many of other solutions that are getting way more attention of late, for delivering just a subset of what we have here.  IMHO!

/walt

TW Tones

unread,
Jun 30, 2021, 9:33:32 AMJun 30
to TiddlyWiki
Walt,

Please let me restate your statement "if Tiddly-Streams had a solid .md<->tid converter", actually, I believe it should be "if TiddlyWiki had a solid .md<->tid converter".

Sure streams may be the best front end for what you discuss, but the last mile is one for tiddlywiki. There are other front ends other users prefer in TiddlyWiki (in no way diminishing Streams). Also it not about tid<->md more about tiddlywiki<->md.

My preference is to build a converter and add it to https://pandoc.org/ then the last mile to another format can become the last mile to many formats. See examples in Wiki markup formats.

The fact is a path most likely exists already, and example is generate compliant HTML code and you can convert to markdown. However it would be more robust if we could go in both directions with TiddlyWiki's own markup. The issue may be the more advanced tiddlers using widgets, however these can be used to generate "static" content not containing widgets for publishing.

Its all about structure and content. If you ensure you retain all the  structure and content you need you can typically convert anything to anything.

TiddlyWiki is already "a real category-killer", in many categories.

Regards
Tones

ludwa6

unread,
Jun 30, 2021, 10:21:47 AMJun 30
to TiddlyWiki
As always @Tones: thanks for helping to clarify my fuzzy thinking :-)  Quick comments inline below:

Please let me restate your statement "if Tiddly-Streams had a solid .md<->tid converter", actually, I believe it should be "if TiddlyWiki had a solid .md<->tid converter".

Of course, that makes sense; the conversion affordance should live at the lower TW level.  I've just been w/ tunnel vision the last few days about moving content (including hierarchal presentation) from Streams:nodes to Tiddler body, but this is indeed a platform issue.  /w

Sure streams may be the best front end for what you discuss, but the last mile is one for tiddlywiki. There are other front ends other users prefer in TiddlyWiki (in no way diminishing Streams).

Yes! And i'm still using a mitfull of those other tront ends myself; again, shame on me for the narrowminded thinking.  /w 

Also it not about tid<->md more about tiddlywiki<->md.

Indeed?  Must confess, this is a nuance of understanding that eludes me.  Can you shed additional light as to what this distinction means, in practical terms?  /w
 
My preference is to build a converter and add it to https://pandoc.org/ then the last mile to another format can become the last mile to many formats. See examples in Wiki markup formats.

Ah... You mean that TiddlyWiki format (whatever that means; i thought it was .tid) should join that list on Pandoc page of "Wiki markup formats" supported?  What would be involved in achieving that, i wonder... /w

The fact is a path most likely exists already, and example is generate compliant HTML code and you can convert to markdown. However it would be more robust if we could go in both directions with TiddlyWiki's own markup. The issue may be the more advanced tiddlers using widgets, however these can be used to generate "static" content not containing widgets for publishing.

My simpleton (i.e. text-wrangling, not code-slinging) perspective is: the simplest step for those us writing in TW to move content back-&-forth with the outside world would be: a mainstream text-based format that supports a minimal modicum of formatting, that is machine parsable to html and other formats, but still readable by humans.  The only format i know that meets these requirements is Markdown.  This is probably the simplest and highest-leverage step we could take, to make that breakthru into mainstream viability -dontcha think?   /w

Its all about structure and content. If you ensure you retain all the  structure and content you need you can typically convert anything to anything.

If you say so, mate -i'm happy to hear it!  /w 

TiddlyWiki is already "a real category-killer", in many categories.

Amen, @Tones!

/walt

TW Tones

unread,
Jun 30, 2021, 6:49:53 PMJun 30
to TiddlyWiki
Walt, comments in line.

On Thursday, 1 July 2021 at 00:21:47 UTC+10 ludwa6 wrote:
As always @Tones: thanks for helping to clarify my fuzzy thinking :-)  Quick comments inline below:

Please let me restate your statement "if Tiddly-Streams had a solid .md<->tid converter", actually, I believe it should be "if TiddlyWiki had a solid .md<->tid converter".

Of course, that makes sense; the conversion affordance should live at the lower TW level.  I've just been w/ tunnel vision the last few days about moving content (including hierarchal presentation) from Streams:nodes to Tiddler body, but this is indeed a platform issue.  /w

#1 Hey, that is why the community is so valuable, some times we point out subtle tips in others the blindingly obvious. I would run out of fingers counting the times the community has pointed out a silly mistake of mine (not that yours was :)
 

Sure streams may be the best front end for what you discuss, but the last mile is one for tiddlywiki. There are other front ends other users prefer in TiddlyWiki (in no way diminishing Streams).

Yes! And i'm still using a mitfull of those other tront ends myself; again, shame on me for the narrowminded thinking.  /w 
see #1 

Also it not about tid<->md more about tiddlywiki<->md.

Indeed?  Must confess, this is a nuance of understanding that eludes me.  Can you shed additional light as to what this distinction means, in practical terms?  /w

What ever the source one or many tiddlers, a macro or widget, output taking content stored within tiddlywiki deserves to be totally free,  confining it to a single tid file is unnecessary but you can if you want. Any tiddlywiki content to any pandoc supported format is my dream.
 
 
My preference is to build a converter and add it to https://pandoc.org/ then the last mile to another format can become the last mile to many formats. See examples in Wiki markup formats.

Ah... You mean that TiddlyWiki format (whatever that means; i thought it was .tid) should join that list on Pandoc page of "Wiki markup formats" supported?  What would be involved in achieving that, i wonder... /w

I do not know, technically, to be clear the main need is to provide a tiddlywiki markup <-> other markup/mark down formats. I could do it but I hope we may find someone with pre-knowledge about pandoc.

 

The fact is a path most likely exists already, and example is generate compliant HTML code and you can convert to markdown. However it would be more robust if we could go in both directions with TiddlyWiki's own markup. The issue may be the more advanced tiddlers using widgets, however these can be used to generate "static" content not containing widgets for publishing.

My simpleton (i.e. text-wrangling, not code-slinging) perspective is: the simplest step for those us writing in TW to move content back-&-forth with the outside world would be: a mainstream text-based format that supports a minimal modicum of formatting, that is machine parsable to html and other formats, but still readable by humans.  The only format i know that meets these requirements is Markdown.  This is probably the simplest and highest-leverage step we could take, to make that breakthru into mainstream viability -dontcha think?   /w

The other you know is Tiddlywiki's markdown. In the end all you should need is a plugin, then select content and click a button. Agreed!
 

Its all about structure and content. If you ensure you retain all the  structure and content you need you can typically convert anything to anything.

If you say so, mate -i'm happy to hear it!  /w 

I am basically saying to move between any two formats the obvious need is to convert the content of one format in the other, the less obvious is to ensure information about the way is is organised/structured survives the conversion. The example you are familiar with is Streams and its indents.

 

TiddlyWiki is already "a real category-killer", in many categories.

Amen, @Tones!

A secular Amen from me too walt :)
 
Regrds
Tones

Xiubo Zhang

unread,
Jul 1, 2021, 8:22:38 PMJul 1
to TiddlyWiki
@Saq
> One of the things I would dearly like to know is how many people are actually using Streams on a regular basis, and what their workflow looks like. My feeling is its a very small handful and I'll admit that probably influences how much time I devote to working on Streams.

I've been using streams regularly for a bit over a month, initially as a replacement for RoamResearch.  After awhile I have to say TiddlyWiki combined with Streams is so much more flexible and "sensible" than vanilla RoamResearch.  For example, I was able to implement RoamResearch-style backlinks with Streams' macros:
backlink.PNG
So Just joined the group to say thanks to you for your marvellous work!

TW Tones

unread,
Jul 1, 2021, 8:33:42 PMJul 1
to TiddlyWiki
Zhang.

Please share that solution if possible, it looks great.

Tones

Si

unread,
Jul 2, 2021, 12:25:35 PMJul 2
to TiddlyWiki
@Zhang. I second Tones' request! Would love to see the code for that linked references section.

Saq Imtiaz

unread,
Jul 10, 2021, 11:56:15 AMJul 10
to TiddlyWiki
Hi everyone. I've been away on an impromptu leave the last week, apologies for the late reply. 

I am very grateful to everyone that provided feedback or even just taken the time to acknowledge that they use Streams. Thank you Keelan, Zhang, Stan, Steph and everyone else.

@keelan It is very good to hear that you are still finding Streams to be of value. I thought of you a few weeks ago actually as yours was a workflow and use case that really stood out to me, and I wasn't sure if you had stuck with or turned to other tools. It is interesting how this works in terms of feedback and involvement. From a user's perspective, if you find a good tool you just want to use it and be productive rather than focus overmuch on talking about the tool. However, for a very small software project like this, occasional user feedback or even just holding your hand up to acknowledge that you are still using it goes a very long way. So thank you for taking the time to chime in here.

On flattening and export:
In terms of export formats, the issue is that everyone's content and use case is different which makes it difficult to come up with universal solutions. One of the approaches here that could be interesting is if everyone gave an actual real world example of what their content in Streams is like, and what an ideal exported version in a single tiddler in TiddlyWiki would look like. It might mean crafting that single consolidated tiddler by hand to serve as an example. This might help generalize and come up with a flexible solution.

Note also that for some users, such as @si, it is a valid use case to not want to consolidate all your nodes but keep them as separate tiddlers.

I think the focus on markdown in this discussion is misplaced as it isn't the primary barrier to exports from Streams. It really is two steps. a) consolidating your individual nodes into a single tiddler as wikitext, and b) then converting it to another format (HTML, markdown, etc.) Figuring out the first step is necessary before looking at what comes after.

Use cases:
We now have descriptions of a few different workflows with Streams. One of the things that would be very helpful if we gathered this information and added it to the documentation. This could be the first step towards facilitating sharing of tweaks and settings etc that are being used in those workflows. Any assistance with this would be very welcome.

Questionnaire, plugin updates and what comes next:
I will likely post a questionnaire in the next week or two. A rather preliminary one to get an idea of how many people use Streams and recruit volunteers for testing new ideas and features.

My policy moving forwards is going to be to make small incremental plugin updates whenever I get the chance. Rather than spamming the group with announcements, I will provide a simple opt-in plugin updates mechanism until we have something in the core that serves the same purpose. See https://groups.google.com/g/tiddlywiki/c/5eewU5jOUlc

Group announcements of new updates will only be made if there is a breaking change, or a critical bug discovered.

I am tracking ideas for further development at github: https://github.com/saqimtiaz/streams/issues
Constructive and relevant input is always welcome.

Apart from the plugin update notifications, there are two other features that I have prototyped and would like to find time to flesh out:

ii) a mechanism to easily see Streams keyboard shortcuts. In the video, the overlay is triggered by holding ctrl+space for 2s, and automatically closes when you let go of the keys: https://drive.google.com/file/d/1y-hLpDZAhVLKEr9QA_4_jMV2oqla-Kys/view?usp=sharing

Cheers,
Saq


ludwa6

unread,
Jul 11, 2021, 6:13:01 AMJul 11
to TiddlyWiki
Encouraging news, @Saq -especially your process of continuous updates (with active notification in core would be GREAT!) and end-user engagement; best practices to emulate, IMHO.

As to the node-flattening issue: OK, so if we forget about that 2nd step of conversion to .md (or other) format for now, and focus on "consolidating your individual nodes into a single tiddler as wikitext" -this makes sense. As to next step: from my perspective, using @Jan 's streams-to-text addon, i would like to see "wikitext" join the other 3 options (bullet list, numbered list and paragraphs) in that pulldown menu of "Structure:" options at top RH corner of modal window... And this for me should preserve hierarchial indentation scheme, instead of just adding a * or a # to a flat list, as the first two options in existing pulldown do. Could this be a good next step to develop from your perspective, @Saq , @Jan?

The other issue you've raised of particular interest to me is the mobile UseCase - which may be more properly considered a UserModel  that could serve as extension to various UseCases. This in my case involves using Quine2 on iOS devices: iPhone and iPad -2 different form factors, which lend themselves to somewhat different modes of interaction. What's needed here is the ability to manipulate position in the hierarchy via swipe L/R, drag Up/Down. Is this what you're talking about in that Reddit thread you shared?  Or is there some other mode of mobile interaction to consider?

As to documenting useCases: I'd be happy to pitch in on this, Saq, if you could advise as to which of the above two UseCases is most likely to be of constructive value in the near term.  
Q: Where to send such input? Is it to create "UseCaseMobileNoteTaking" as an issue in your Github repo? Or do you prefer to engage such discussion here, or in Reddit, or...?

/walt

Saq Imtiaz

unread,
Jul 11, 2021, 7:13:14 AMJul 11
to TiddlyWiki
@walt

As to the node-flattening issue.... i would like to see "wikitext" join the other 3 options (bullet list, numbered list and paragraphs)

So when I wrote "an actual real world example of what their content in Streams is like, and what an ideal exported version in a single tiddler in TiddlyWiki would look like" my expectation was for users to provide wikis with a representative example of their content in streams nodes, as well as a single tiddler version of it that they created by hand. For example one tiddler with some stream nodes, the content of which is representative of what their content is usually like. Plus one tiddler that is an amalgamation of those nodes into a single tiddler.

Having a few such examples might allow generalizing a few export options that fit most use cases. The problem with receiving proposals for solutions, rather than detailed description of the problem and related content, is that often the proposed solutions don't fulfil the actual needs or consider technical limitations they would impose. For example, your proposed solution would limit you to only ever having single line content in all your nodes.
 
Regarding use cases/workflows, I was referring to the manner in which people are using Streams for note taking (the original subject of this thread). We have a few examples now from Keelan, Si and others. It would be good to gather the descriptions of their workflows and add it to the documentation. This could later be fleshed out with details on other plugins, or wiki settings etc that they use to achieve their desired workflow. Since there is not a single prescribed workflow for using Streams, nor a vertical TiddlyWiki edition designed to accommodate it, having examples of how people use Streams would allow knowledge sharing amongst Streams user and be useful to the next person who has the same question as you regarding how to use Streams for note taking.

Help with collecting this to add it to the documentation would be greatly appreciated. The easiest thing to do would be to collect the information in a TiddlyWiki file. 

The other issue you've raised of particular interest to me is the mobile UseCase - which may be more properly considered a UserModel  that could serve as extension to various UseCases. This in my case involves using Quine2 on iOS devices: iPhone and iPad -2 different form factors, which lend themselves to somewhat different modes of interaction. What's needed here is the ability to manipulate position in the hierarchy via swipe L/R, drag Up/Down. Is this what you're talking about in that Reddit thread you shared?  Or is there some other mode of mobile interaction to consider?

Mobile usage is a tricky issue for TiddlyWiki. The design and UX really isn't optimized for it. In particular, the drag and drop mechanism doesn't work for all mobile browsers. This is a limitation of the browsers not implementing the relevant spec at all or properly. As such, implementing proper support for this is outside the scope of a single plugin.

As I've expressed during earlier conversations on the same topic, I do not intend to work on any mobile specific features or affordances for Streams beyond any low hanging fruit that can be achieved with minimal effort. Currently in Streams you can swipe on a node on mobile to trigger the context menu. Add your own commands to the context menu is the way to go for any features you absolutely need on mobile.

The interest in the swipe with visual feedback feature actually comes from a technical perspective. It is very impressive that such a thing can be achieved with just wikitext, so I would like to see it realized (time permitting) to serve as an example for others wanting to explore similar UX.

Cheers,
Saq

ludwa6

unread,
Jul 11, 2021, 2:14:48 PMJul 11
to TiddlyWiki
OK @Saq: I've taken a little run at creating a wiki to hold UseCase documentation for "TiddlyStreams" ; it's online at tiddlystreams.github.io , and if that looks to you like a move in the right direction, then i'll keep on going with it as time permits.  If you or anyone wants to assist, i can add as users to the "tiddlystreams" organization, or i guess anyone can send a PR (it's a public repo)... or else just click the Comment Link (i.e. mailto:) at bottom of each tiddler.

Funny thing: your feedback has caused me to question my understanding of the term UseCase, so i did a bit of digging; sounds to me like what you want (if this article can be considered definitive) is more like a UserStory set, while i- the naive (i.e. non-dev) user in this conversation- have been trying to communicate more in terms of a technical solution.  Awkward!  (like when i'm insisting on speaking Portuguese to a local who really wants to speak English with me :-)

/walt

Saq Imtiaz

unread,
Jul 11, 2021, 2:53:22 PMJul 11
to TiddlyWiki
Hi Walt,

It seems like we are getting our wires crossed and I think I am partly to blame for which I apologize. I am very accustomed to speaking to a technical audience, OR a completely non-technical one. However here the borders are muddled and it has led to me using terms more colloquially than technically accurately, which it seems has caused confusion.

What you have created is very impressive and interesting in its own right, but not what I was thinking of. So let me take a crack at explaining it once more using every day language. 

There are two separate topics here:

1) Understanding how other people use Streams and allowing Streams users to learn from each other.
We have in this thread descriptions of how several different users make use of Streams for their intended purposes. For example this is how Keelan uses Streams. It would be good if we collected and added these descriptions of how people are already using Streams to the Streams documentation. Assistance with this would be fantastic. 

Later on details could be added to this where relevant, like exactly which plugin and wiki settings were used and which other plugins. Some users may even be comfortable sharing empty copies of their wiki for others to learn from.

The key here is that we are talking about descriptions of how people are already using Streams. Not how they might hypothetically use it.

2) The issue of creating better options around export.
What would be helpful here is if users that have such a need, shared examples of their content. So let's say you used Streams to take notes on fertilization methods in a tiddler called "Fertilization". I want to see that tiddler and its streams nodes. Additionally, I want to see how you would consolidate those nodes into a single tiddler if you had to do it by hand.

If you are not comfortable sharing actual tiddlers from your wiki due to privacy concerns, then you could create a dummy version that has similar content in length etc to your actual tiddlers but does not contain anything private. The kind of things I need to understand are: how long are the nodes typically? Is it just plain text? Are their widgets? Are their images? Tables? etc..

Now if all of your content is similar to the nodes on the Fertilization tiddler, then that is enough. However if you are using Streams for creating different styles of content, such as say a To Do list, meeting notes and writing essays, then such an example for each would be needed.

Let us assume 6 users do this. That will hopefully give me enough examples of the content people are writing with Streams to have a better chance at coming up with a generalized solution that works for most people.

If any of this still isn't clear, I would suggest we find a time for a call to discuss before you put any more effort into it, so as to avoid any chance of further misunderstanding.

For the record what you did create is fantastic and if this were a properly funded development effort we would indeed be using User Stories representing hypothetical users and how they might want to use Streams as a starting point to understand what was needed. However, this is  one of many side projects for someone that has their fingers in far too many pies, so that approach isn't feasible.

Regards,
Saq

TW Tones

unread,
Jul 11, 2021, 8:08:23 PMJul 11
to TiddlyWiki
Saq,

I will endeavour to share examples of my extensive use of streams. For me streams may suffer from the same issue of TiddlyWiki as a whole and that is it is so flexible there may be as many uses as there are users, multiplied by the number of wikis they have. Some quick thoughts;
  • You may recall my input in the first phase of development, suggestions and ideas that urge a generalises solution, you took this on an have thus created an extraordinary tool. 
  • I think this can continue going forward because of this underlying strength. for example;
    • Streams uses a parent and list field to organise nodes, it uses an automatic naming of tiddlers to let the user ignore this, and focus on the content.
    • As a result of this ground work I suggest not developing a solution for "flattening streams", but flattening hierarchical lists because this is the same effort and will have even broader applications.
    • Also don't really flatten such lists, perhaps mostly visually, because we should never design solutions that loose information, perhaps just hide it (see hide not flatten below)
  • I suspect many want to flatten a stream to move it, or copy it, again this is not necessary and potential loss of information. A way to export/import or drag and drop the whole stream as a JSON is possible
    • I have build such a tool for "packages" of tiddlers driven by a filter, I can share this.
  • The reverse process to (apparent) flattening is another generic solution we could use that would allow any structure such as the table of contents etc... to be converted to a parent/list structure including streams.
  • As you have done so well, allowing the generation of sub tiddler nodes titles to be altered is also a great benefit, as well as the ability to rename node tiddlers. Renaming a tiddler can also change its use eg temp system to standard tiddler.
    • Personally I believe we as a community need to start to share the algorithms for generating new and unique tiddler titles using different methods for different purposes. Personally all my streams currently use the following so nodes remain hidden until I consciously rename a node to make it appear in searches. "$:/s/<<now "[UTC]YY-0MM-0DD 0hh:0mm:0ss XXX">>"
    • Recent developments in 5.1.23/24 and 5.2 coming will assist in this such as variable/macros in filters can now include parameter's, the "unusedtitle Macro", format title list operator and more.
Hide not flatten
  • I have being working on a range of hacks to support easy more and slider to display or hide content solutions in response to Mats discussion
    • I see parallels with flattening where we can use existing, new or arbitrary html tags to store content in a nested manner without it showing in the output. 
    • Streams can be the beginning of "a maturing of handling outlines and hierarchies in tiddlywiki", especially their encapsulation in a single tiddler or a stream (hierarchical list of tiddlers)
How best may we collaborate and share the load?

Regards
Tones

Keelan Cook

unread,
Jul 14, 2021, 9:20:55 PM (12 days ago) Jul 14
to TiddlyWiki
@Saq, I'm happy to contribute toward both of those ends you have mentioned. Do let me know what else I should add to my description of my workflow with Streams and TW, and I can fill in those blanks. I'd be happy to write up something more thorough or brief, if that was of benefit. 

I'm also happy to share some specific examples of how I'd like to see it flatten. My end game is getting a whole stream of tiddlers into one document in either LaTeX or some kind of word processor, formatted as an academic paper. My streams will consist of an series of tiddlers, some of which are bold and serve as paper headings, and others which are entire single paragraphs in the body of the paper draft. So imagine, a first tiddler being a simple heading in bold followed by five or six more tiddlers each one-paragraph long. Mash several of these sections together, and that's a paper draft stream for me. If there's a way to flatten to wikitext and somehow make the conversion to LaTeX or something that would copy easily (and maintain formatting) into Google Docs or Microsoft Word, then that wins the prize for me. I know you asked for specific tiddler examples. Let me pull some out and I'll add them here, unless you'd prefer they go to github instead. 

Again, I'm continually encouraged by your work on this and the direction this conversation continues to take.

TW Tones

unread,
Jul 15, 2021, 8:42:44 AM (11 days ago) Jul 15
to TiddlyWiki
keela,

The result of tiddlywiki rendering is html. With the internals plugin you even get a html preview. Jeremy just pointed out a html file saved as doc can be opened in Microsoft word as a document (Word can save as html as well).


keep in mind many before us have used TiddlyWiki for academic papers so search is your friend.

Tones

ludwa6

unread,
Jul 15, 2021, 1:22:33 PM (11 days ago) Jul 15
to TiddlyWiki
@Saq: As shared in another post i've been too busy farming lately to think about this -but when i do get a chanc, the focus Q is: how can i use this software to simplify the crazy-making challenge of keeping track of all my crops and animals and machines and markets and all the other moving targets there are swirling about the complex open dynamical ecosystem that is the farm -this one, or any other.

While much of that data is scattered about in spreadsheets and Google docs and Trello cards and image archives etc., my Mission right now (as mentioned in the above-linked post) is to get it all centralised in version 2.0 of our farmOS, but it is one Very Big Rock to move -as evidenced by this browsable (JSON) view of farmOS API , if you really want to know the bloody details... But here's a high-level view i've created in TiddlyStreams for popular consumption.

As to how Streams is facilitating my workflow: i use it for rapid outlining of ideas and documentation and links to all the online resources i am having to wrangle into my "One Wiki to Rule Them All," managed in TiddlyDesktop -from which i push information out to other wikis like TiddlyStreams.

That last step is still too frictional to be very productive; it's the archetypal "last mile" problem of impedance mismatch between input and output: the notational velocity of Streams vs the highly constrained aperture of channels for reception and distribution. Keelan has described that problem well, in context of academic publishing, and if you want to know about my context/ requirements in more detail, please do get in touch directly by whatever means.  Given such range of specific requirements, maybe Tones idea of wikitext<->pandoc integration makes good sense, if i understood it right (but i probably didn't :-)

Yours, Walt