TiddlyWiki5 Progress Report
Jeremy Ruston
development work. You can see the latest code at
http://tiddlywiki.com/tiddlywiki5
The heart of TiddlyWiki is a representation transformation engine that
converts tiddlers between different forms (for example, wikitext to
HTML) and can selectively update the representations if the underlying
tiddlers change. This engine has to satisfy many constraints in order
to be flexible enough to run on the server under node.js or within the
browser, and to be efficient enough to serve as the basis for the
interactive features of TiddlyWiki.
There are still quite a few features to be implemented, such as
slices, sections and filters, but the basic design of the engine is
now complete. Over the last few weeks I've been gradually adding more
experimental interactive features, notably the slider and chooser
macros, so that I can verify that the design is adequate to support
the interactivity I'd like.
Over the next few weeks I'll be filling out the available macros,
matching all of the core TiddlyWiki ones. The next big area is to
implement editing, which I'm thinking through now. I'm hoping to have
something usable by the early summer.
Some characteristics of the new engine compared to classic TiddlyWiki:
* Everything is a macro. In classic TiddlyWiki the main story column
and individual tiddler links behave somewhat like macros, but are
treated completely differently. Now, they are both implemented as
macros. This change makes the design easier to hack and change, and
makes TiddlyWiki easier to comprehend, because it reduces the number
of entity types one has to learn about.
* No cookies. The idea is to store user interface state data in
tiddlers instead. You can see this in the story and slider macros. The
story macro takes as a parameter a reference to a tiddler containing a
list of tiddlers. To display a new tiddler one just adds it to the
list, and the story macro will automatically refresh to display the
newly added tiddler. Similarly, the slider macro can optionally store
its state in a specified tiddler. One can then flip the slider open or
closed by setting the text of the tiddler to "open" or "closed". See
the tiddlers SliderTests and StoryTiddlers to see these features.
* Efficient sliders. The content of sliders isn't rendered until the
slider is opened for the first time
* Base64 embedded images and SVG tiddlers
* Syntax highlighting for JavaScript tiddlers and fragments. Tiddlers
of the MIME type application/javascript are parsed to an abstract
syntax tree which enables syntax elements to be colour coded. In
addition, comments are treated as wikitext. See the tiddlers
TypedBlockTests, SampleJavaScript and SampleJavaScriptWithErrors, and
the TiddlyWiki source code tiddlers whose titles start "js/"
* Syntax highlighting for JSON tiddlers and fragments. The CSS is
pretty terrible at the moment; see SampleData
* Safe inclusion of tiddlers containing untrusted JavaScript. There
would be a lot of work to fully implement and verify this feature, but
the idea is that by parsing and recompiling the JavaScript code we can
use a combination of static analysis and code injection to ensure that
it doesn't do anything bad.
Under the covers, the API for creating and saving tiddlers is much
simpler than before. For example, here is the code that updates the
ClockTiddler:
window.setInterval(function() {
me.store.addTiddler(new Tiddler({
title: "ClockTiddler",
text: "The time was recently " + (new Date()).toString()
}));
},3000);
Finally, my approach to the user interface of TiddlyWiki5 is to try to
design it for touch and mobile first (see the tiddler
UserInterfaceSketches). With that in mind, I've been experimenting
with a pervasive new 'chooser' that makes it quick and easy to locate
and open tiddlers. Right now it's just an alphabetical list of tiddler
titles, but it's going to evolve into a full hierarchical menu. I made
a short video to demonstrate it on the ipad here:
You can follow the development along at https://github.com/Jermolene/TiddlyWiki5
Best wishes
Jeremy
--
Jeremy Ruston
mailto:jeremy...@gmail.com
http://www.tiddlywiki.com
Martin Budden
standardised than TiddlyWiki and some of the unnecessary quirkiness is
being eliminated.
One minor area where TW5 still retains this quirkiness is in the
template syntax, see:
https://github.com/Jermolene/TiddlyWiki5/blob/master/tiddlywiki5/tiddlywiki5.template.html
in particular the use of HTML comments <<!--@@name@@-->> for the
template parameters. It would be good if TW5 could, rather than invent
its own template syntax, use an already established syntax, or a
subset of such a syntax. One possibility is to use the variable syntax
used in Jinga2 Templates
I'm not particularly advocating Jinga2. If someone thinks there is a
more widely used or more suitable template language, then I am fine
with that. One advantage of Jinga2 is that it is used by TiddlyWeb.
What I am advocating is that TiddlyWiki5 uses an existing standard,
rather than create its own.
I agree this is a minor point. But every time TiddlyWiki creates its
own standard, rather than uses an existing one, it makes it slightly
less usable/adoptable. And it means TiddlyWiki potentially looses out
from some unexpected benefits that result from the use of a
pre-existing standard.
Martin
> --
> You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
> To post to this group, send email to tiddly...@googlegroups.com.
> To unsubscribe from this group, send email to tiddlywikide...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/tiddlywikidev?hl=en.
>
Ben Gillies
> more widely used or more suitable template language, then I am fine
> with that. One advantage of Jinga2 is that it is used by TiddlyWeb.
>
> What I am advocating is that TiddlyWiki5 uses an existing standard,
> rather than create its own.
My vote would go with Mustache (http://mustache.github.com/) which has
the benefit of cleaner syntax, and more portable templates (it
supports 21 languages).
> I agree this is a minor point. But every time TiddlyWiki creates its
> own standard, rather than uses an existing one, it makes it slightly
> less usable/adoptable. And it means TiddlyWiki potentially looses out
> from some unexpected benefits that result from the use of a
> pre-existing standard.
I fully agree.
Jeremy Ruston
> standardised than TiddlyWiki and some of the unnecessary quirkiness is
> being eliminated.
Good, that's certainly the idea.
> One minor area where TW5 still retains this quirkiness is in the
> template syntax, see:
>
> https://github.com/Jermolene/TiddlyWiki5/blob/master/tiddlywiki5/tiddlywiki5.template.html
Right, yes, definitely, there's a number of things that carry over
from the old cook/ginsu world that are distinctly creaky.
The other one that concerns me more at the moment is the recipe file
format. I think it conflates two things: specifying tiddlers to load
into a store, and plugging those tiddlers into an output schema. My
thinking is to separate them out into two separate file formats.
> in particular the use of HTML comments <<!--@@name@@-->> for the
> template parameters. It would be good if TW5 could, rather than invent
> its own template syntax, use an already established syntax, or a
> subset of such a syntax. One possibility is to use the variable syntax
> used in Jinga2 Templates
> http://jinja.pocoo.org/docs/
>
> I'm not particularly advocating Jinga2. If someone thinks there is a
> more widely used or more suitable template language, then I am fine
> with that. One advantage of Jinga2 is that it is used by TiddlyWeb.
>
> What I am advocating is that TiddlyWiki5 uses an existing standard,
> rather than create its own.
>
> I agree this is a minor point. But every time TiddlyWiki creates its
> own standard, rather than uses an existing one, it makes it slightly
> less usable/adoptable. And it means TiddlyWiki potentially looses out
> from some unexpected benefits that result from the use of a
> pre-existing standard.
I suspect that you'll disagree violently, but here's how I see it:
The wikitext in TiddlyWiki is already its own templating language. I'm
talking about PageTemplate, ViewTemplate etc. In classic TW these
templates are written in HTML, but in TW5 they are written in
wikitext. The difference will be subtler when I add support for HTML
tags within wikitext. Even today, evaluated parameters give users a
tiny bit of the expressive power of templating engines like mustache
(and especially with things like the ForEachTiddlerPlugin).
The HTML template in the recipe file will thus become an ordinary
tiddler that when wikified renders the full HTML page.
So, given that perspective, my approach to mustache, Jinga2 etc. is to
use them as inspiration for extending the templating-like capabilities
of wikitext.
Going back to your original points about easing the comprehension of
TiddlyWiki by reducing the number of unique characteristics that need
to be learned, my primary goal is to do that by having a very small
set of features that work together in a uniform manner at many
different scales. So the mechanisms that people will use to arrange
their tiddlers into a sequenced story would be the same mechanisms
that you'd use to arrange menu items in a sequence. And of course the
primary mechanism within TiddlyWiki is this transformation of tiddlers
from one representation to another, and so you can see the
attractiveness of developing a single paradigm to accomplish that, and
then using it to provide multiple features within the application.
I also note that people who want to write Jinga2 or mustache templates
are already pretty well catered for; they should just use those
products, and should be able to do so happily with the TW5
representation transformation engine if they want. TiddlyWiki isn't
aimed at web developers, it's designed to give capabilities to
ordinary, reasonably intelligent users that would normally be the
domain of expert web developers.
chris...@gmail.com
> I agree this is a minor point. But every time TiddlyWiki creates its
> own standard, rather than uses an existing one, it makes it slightly
> less usable/adoptable. And it means TiddlyWiki potentially looses out
> from some unexpected benefits that result from the use of a
> pre-existing standard.
+many
One of the biggest missed benefits is cross-pollination (both in terms
of technology and community) with other projects.
--
Chris Dent http://burningchrome.com/
[...]
Martin Budden
> talking about PageTemplate, ViewTemplate etc. In classic TW these
> templates are written in HTML, but in TW5 they are written in
> wikitext. The difference will be subtler when I add support for HTML
> tags within wikitext. Even today, evaluated parameters give users a
> tiny bit of the expressive power of templating engines like mustache
> (and especially with things like the ForEachTiddlerPlugin).
>
> The HTML template in the recipe file will thus become an ordinary
> tiddler that when wikified renders the full HTML page.
So, if I understand you correctly, the existing template file will
become obsolete and so its syntax is moot. That's a good result.
> So, given that perspective, my approach to mustache, Jinga2 etc. is to
> use them as inspiration for extending the templating-like capabilities
> of wikitext.
I had presumed the templating mechanism for wikitext would be via the
use of macros. The above makes me think you have something else in
mind as well. Care to elaborate?
> my primary goal is to do that by having a very small
> set of features that work together in a uniform manner at many
> different scales.
Yep, I certainly agree with that goal.
> I also note that people who want to write Jinga2 or mustache templates
> are already pretty well catered for
This misses the point. The purpose of using Jinga2 is not to support
people who want to write Jinga2 templates. It's, as Chris puts it, to
support cross-pollination. So I'm imagining a situation where a Jinga
guy starts playing around with TW5 and we get a posting to one of the
TW groups, something like: "I've noticed TW5 uses Jinga. Did you know
that if you combine TW5 with this Jinga tool you can do this amazing
thing..." The amazing thing being some new use for TW5, that nobody
had thought of before. And, as Chris points out, the cross-pollination
also works with communities. So some Jinga guy posts in a Jinga forum
"Have you seen this TW5 thing that uses Jinga, check it out" and as a
result we get a whole load more TW5 users and developers.
So if you do extend wikitext templating, it should not just be
inspired by mustach or Jinga2. It should be conformant with one of
those standards. I don't care which one, but it should avoid ploughing
its own furrow.
Martin
Jeremy Ruston
> become obsolete and so its syntax is moot. That's a good result.
Yes, that's correct. We'll look back and laugh at it.
> I had presumed the templating mechanism for wikitext would be via the
> use of macros. The above makes me think you have something else in
> mind as well. Care to elaborate?
I'm still thinking a lot of this out. One option I'm considering is
recasting macros as a more conventional textual substitution and
expansion mechanism, and introducing a new plugin invocation model
that executes target tiddlers as JavaScript. Then we'd have plugins
called things like "http://tiddlywiki.com/plugins/slider", and end
users would start with a "slider" macro that invokes that plugin, but
could invent their own macros that shortcut the invocation of the
plugin with different parameters.
In any case, I'd like to pull the execution of JavaScript parameters
up out of the macro syntax and be a first class part of the wikitext
syntax.
> Yep, I certainly agree with that goal.
Good.
>> I also note that people who want to write Jinga2 or mustache templates
>> are already pretty well catered for
>
> This misses the point. The purpose of using Jinga2 is not to support
> people who want to write Jinga2 templates. It's, as Chris puts it, to
> support cross-pollination. So I'm imagining a situation where a Jinga
> guy starts playing around with TW5 and we get a posting to one of the
> TW groups, something like: "I've noticed TW5 uses Jinga. Did you know
> that if you combine TW5 with this Jinga tool you can do this amazing
> thing..." The amazing thing being some new use for TW5, that nobody
> had thought of before. And, as Chris points out, the cross-pollination
> also works with communities. So some Jinga guy posts in a Jinga forum
> "Have you seen this TW5 thing that uses Jinga, check it out" and as a
> result we get a whole load more TW5 users and developers.
Yes, I agree with all of that, I was making a different point: that
TW5 is much more usable as a component by ordinary web developers who
don't want to deal with the historical eccentricities.
TW5 already incorporates a bunch of external components where it makes
sense, and will continue to do so.
> So if you do extend wikitext templating, it should not just be
> inspired by mustach or Jinga2. It should be conformant with one of
> those standards. I don't care which one, but it should avoid ploughing
> its own furrow.
I'm not so confident that I can devise a syntax that is harmonious
with the existing wikitext principles while also being 100% compatible
with either of those existing syntaxes. And neither would I expect to
be able to do that: Mustach and Jinga2 are trying to solve a different
problem from TW5, with different constraints. They only look like the
same problem from a distance.
Cheers
UnclePhil
Just to give you some user feedback about your new chooser.
It work very nice on an Ipad/safari machine, but when I try it on
android with the Dolphin browser, it's a real pain .... because the
Dolphin creator had the same idea before you.
Menu, bookmark, history are accessible with a left slide, and tools
are accessible with a right slide.
So the idea itself is very interesting for mobile/tablet user, but i
think you need also to implement it through a basic menu
You are doing a fantastic rewriting of my favorite info container, and
like many others I'm waiting the first stable release to use it
intensively.
ph Koenig
UnclePhil tc.unclephil.net
On Mar 14, 4:04 pm, Jeremy Ruston <jeremy.rus...@gmail.com> wrote:
> I wanted to give the group a brief progress report on the TiddlyWiki5
> Best wishes
>
> Jeremy
>
> --
> Jeremy Ruston
Jeremy Ruston
> Just to give you some user feedback about your new chooser.
> It work very nice on an Ipad/safari machine, but when I try it on
> android with the Dolphin browser, it's a real pain .... because the
> Dolphin creator had the same idea before you.
> Menu, bookmark, history are accessible with a left slide, and tools
> are accessible with a right slide.
Ha! Does the right hand side of the screen have a similar function defined?
> So the idea itself is very interesting for mobile/tablet user, but i
> think you need also to implement it through a basic menu
Yes, the sideswipe thing wont be the only way to access the chooser;
I'm hoping to be able to swoosh out from any link to explore the link
in the context of related links.
The menus exposed by the chooser will in turn be generated by macros
that can be used outside of the chooser context. I'd like to make it
easy to explore other navigation approaches.
> You are doing a fantastic rewriting of my favorite info container, and
> like many others I'm waiting the first stable release to use it
> intensively.
Thank you. It's an honour to have an appreciative audience.
Building TiddlyWiki is a bit like making a bracelet; it's not until
the loop is complete that it becomes usable for its intended purpose.
I'm trying hard to strike a balance between racing to complete the
loop and taking the time to ensure that I've got a solid enough
foundation to withstand wear and tear. Thanks for your patience as it
unfolds.
Best wishes
Jeremy
FND
>> Menu, bookmark, history are accessible with a left slide, and tools
>> are accessible with a right slide.
>
> Ha! Does the right hand side of the screen have a similar function
> defined?
As I understood Phil, sliding on the right-hand side brings up a tools menu.
Either way, Android's browser has an experimental mode (on tablets
anyway; it's called Quick Controls IIRC) where you can slide in from
either side to get a menu. I assume that would conflict as well.
-- F.
Jeremy Ruston
All this touchy swipey stuff is highly experimental, and will evolve
over time. I wanted to do a little work on this to steep myself in the
capabilities of touch events, which are a new area for me, and so to
help me think of the UI from a touch perspective.
I should also emphasise that these features are currently somewhat
undiscoverable, and so I'm treating them rather as the touch
equivalent of keyboard shortcuts, features that needs to be shown or
explained. Hopefully I can make them easy to learn once experienced by
making them engaging and natural.
I remain very excited about touch interfaces. Talking to a computer
with a keyboard and mouse always felt a bit like poking at something
with a long stick through a narrow slot; touch puts you in direct
control with high bandwidth and high precision. We think of touch as
being a mismatched interface for text applications, but it seems that
we spend most of our time navigating and reading text. Perhaps even
the time we spend writing text is mostly about rearranging it.
Best wishes
Jeremy
--
Jeremy Ruston
jeremy...@gmail.com
Michael Mahemoff
My vote would go with Mustache (http://mustache.github.com/) which has
the benefit of cleaner syntax, and more portable templates (it
supports 21 languages).
I fully agree.
> I agree this is a minor point. But every time TiddlyWiki creates its
> own standard, rather than uses an existing one, it makes it slightly
> less usable/adoptable. And it means TiddlyWiki potentially looses out
> from some unexpected benefits that result from the use of a
> pre-existing standard.
+1
It might be worth considering something Markdown-inspired. The various flavours of MD have a lot of momentum (GitHub, StackOverflow, Google+, Jekyll/Octopress, many Dropbox editors) and familiarity is a worthy goal. It can't be the same as Markdown, but it could be one-way compatible with it (MD->TW).
Jeremy Ruston
+1
It might be worth considering something Markdown-inspired. The various flavours of MD have a lot of momentum (GitHub, StackOverflow, Google+, Jekyll/Octopress, many Dropbox editors) and familiarity is a worthy goal. It can't be the same as Markdown, but it could be one-way compatible with it (MD->TW).
Chris Dent
On Saturday, March 17, 2012 8:56:23 PM UTC, Jeremy Ruston wrote:
I remain very excited about touch interfaces. Talking to a computer
with a keyboard and mouse always felt a bit like poking at something
with a long stick through a narrow slot; touch puts you in direct
control with high bandwidth and high precision. We think of touch as
being a mismatched interface for text applications, but it seems that
we spend most of our time navigating and reading text. Perhaps even
the time we spend writing text is mostly about rearranging it.
Alex Hough
In the context of a online notebook there is a lot of cut and pasting.
I was struck by the comment "the time we spend writing text is mostly
about rearranging it" and started to bear it in mind as I constructed
a TW specifically to research a new topic. I consciously thought about
my process and was surprised that the tweaking of a fresh vanilla TW,
looking at the source and altering the way the TW prompted through its
design, influenced way the hypertext evolved. It was interesting to do
this with a self imposed contraint: no plugins.
I found that I wanted to compare things and comment on them as well as
classifying them and I found transclusion useful for comparing. I
would do it more if it were easier i think. On an Ipad, if there
there were a gesture (like in dolphin browser) to evoke "select
tiddler for transcultion" then I think it could be a powerful feature.
A the moment transclusion involves quite a bit of cutting and pasting
and bracket adding, and double brackets are a nightmare to find.
I found tended to refactor my notes with transclusion in mind. The "
boundary between narrative text and computer code" is useful as
inspiration for sense making in hypertext. I would not have come
across re-factoring if I hadn't been exposed to code and code writing
culture through TW.
Alex
> --
> You received this message because you are subscribed to the Google Groups
> "TiddlyWikiDev" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/tiddlywikidev/-/VmR3kF8ErhYJ.
webwarrior
But some points are unclear to me.
2. What about editing history? How is it supposed to be stored?
Jeremy Ruston
> project is alive.
along the way.
> But some points are unclear to me.
> 1. WYSIWYG - will it be main (or at least fully featured) editing mode? Any
> support for easy refactoring (I like how it works in Smallest Federated
> Wiki)?
moment. I think the success of MarkDown has shown that people are
starting to see that WYSIWYG is not the panacea it first seems: it may
be trivial to apply WYSIWYG to things like bold and italics, but
there's no easy way to handle things like macro calls with WYSIWYG. I
think of WYSIWYG as being pretty much a failure: look at Microsoft
Word, where the price of WYSIWYG is that your formatting commands are
attached to special invisible characters at the end of paragraphs. You
get unexpected behaviour when those invisible markers are pasted or
deleted.
Having said that, I have taken care to ensure that a WYSIWYG edit mode
could be added to TW5. The technical hassles are enough that I'm not
keen to do it myself. You'll appreciate that rich text editors on the
web work with HTML, not wikitext, and so one has to worry about round
trip conversion from wikitext to HTML and back again, which is
awkward.
I am very interested in being able to edit transcluded tiddlers
directly, though. The goal of TiddlyWiki is about improving the
reusability of content by breaking it up into small chunks, and using
transclusion and aggregation to thread those chunks together into a
narrative. I believe that direct editing will make it much easier to
work in that way.
I'm also tracking SFW with great interest. If you're referring to the
drag and drop refactoring, then, yes, that is very much on my roadmap.
There is a strong correspondence between SFW and TW5. What SFW calls
paragraphs correspond to tiddlers in TW5, and what SFW calls a page
corresponds to a story in TW5. It's not a primary focus, but I'd hope
to be able to use TW5 as a client for SFW.
> 2. What about editing history? How is it supposed to be stored?
be able to hook into.
I am also interested in implementing revision support within the
client, so that revisions are available when working as a single HTML
file. I'm hoping to use a diff-patch-merge function so that only the
differential changes between versions will need to be stored.
Mat
Why not both(!).
Why could todays "view" not be WYSIWYG for regular text and if you
wanted to code etc go into "edit" like today. (Actually, adopting
something like Tobias' linkifyplugin approach would even enable the
key concept of linking in WYSIWYG automatically! [1],[2]) While
everyone no doubt appreciated the coding aspects, actual applications
like "homepages" probably have more pure text than is reflected in the
google discussion groups (ie. where code is discussed).
I'd think WYSIWYG is particularly welcome (and expected) by people
coming in contact with TW for the first time.
<:-)
[1] http://linkify.tiddlyspot.com/index.html
[2] http://groups.google.com/group/tiddlywiki/browse_thread/thread/80569db4f950a96f
akira d ev
I agree. Has anyone thought about integrating notepad++ because it is such a wickedly powerfully editor and the pluging funtions are endless!
--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.