TiddlyWeb pre-alpha demo

[chris...@gmail.com]

I've been working on a bare-bones, back-to-basics implementation of a
server side for TiddlyWiki. Initially it was called mo'ass (mother of
all server sides), with special pronunciation instructions from
Jeremy, but I decided to give the project the easier to explain name
of TiddlyWeb.

The two primary goals of TiddlyWeb are:

1. Make a web-service style server-side that minimizes application
logic in the server[1], providing a resource oriented framework for
managing tiddlers. The final implementation will hew as closely as
possible to good HTTP behaviors and, with any luck, will be a useful
reference for an emerging standard for hosting and managing tiddlers
on the network.

2. Provide a simple but effective authentication and authorization
system for these tiddlers that are on the network. Jeremy has ginned
up this notion of recipes and bags. Recipes in TiddlyWeb are similar
to, but not quite the same as recipes in cook. Recipes are lists of
bags+filters. The result of processing a recipe is a workspace, which
is a useable list of tiddlers representing some purpose. A bag is a
container for tiddlers with specific access restrictions on who can
read and write from and to the bag. A filter is a query string that
limits which tiddlers from this bag are used in a recipe. These
concepts get a bit more clear when you play with it (see below).

In TiddlyWeb a list of tiddlers can be presented as a TiddlyWiki when
the right content-type is requested.

I have a working prototype of most of 1, and the bones of 2. There is
no authentication or authorization, it is a free for all, but recipes,
bags and filters operate to create useful workspaces. Here's one:
http://peermore.com:8080/recipes/TiddlyWeb.wiki

A friend of a friend once passed out the wisdom that the secret to
effective information technology is acknowledging that it is a simple
matter of moving data from here ... to here. I can already tell that
TiddlyWeb is going to be a very useful tool for moving data around and
presenting tiddlers and TiddlyWikis in new ways. I'm especially
intrigued by the possibilities of multiple people running multiple
workspaces operating on shared datasets.

If you find all this interesting, you can help out and determine where
next to apply work in three ways:

1. Check out the demo I have running on my server:

http://peermore.com:8080/recipes/TiddlyWeb.wiki#PlayAround

See what works, what's broken, what needs more explanation. Content
you edit there will be synchronized back to the server if a) you edit
existing tiddlers b) you create a new tiddler by linking from an
existing one.

2. Check out the code and run your own server.

http://svn.tiddlywiki.org/Trunk/contributors/ChrisDent/experimental/TiddlyWeb/

The code has been tested with Python 2.4 and 2.5. It requires selector
and simplejson. See README and RUN to see about making it go. There is
a script in the root of the checkout called 'demo.py' which will
startup a server using much of the same data I have running on
peermore.com.

3. Check out the code, have a look round, run the tests (see TESTING),
look at the TODO and maybe write some code.

In any case, respond here if you have any questions, comments or
thoughts. I warn you: this is pre-alpha code, it may blow up in your
face, it may operate in ways that make no sense and defy expectations.
Do not use this code to support any important work. Do not develop
code based on this code which relies on APIs in this code and be
disappointed when the APIs change later.

Key issues in the code right now:

* No support for file locking in the 'text' data store, which means
there could be write collisions.
* No support for revision handling in the model, meaning there's no
support for collision handling or reverting changes.
* No access control, you can write all over the place. (If this proves
to be a problem I'll take down my test server and implement a quick
HTTP Basic Auth hack)
* The client code does not think in terms of recipes and bags, so
workspace generation and synchronization is not as full-featured as it
could be.

In the not too distant future I hope to host TiddlyWeb's development
Wiki in TiddlyWeb.

[1] I've not done a full review, but my impression is that this is key
differentiator between TiddlyWeb and other server-side
implementations. My hope is that because TiddlyWeb is a simple box
with some knobs on it for getting and putting data and tries to be
RESTful that it will support unexpected use cases and expand the web.

[FND]

> The two primary goals of TiddlyWeb are: [...]

I really like this concept, and the implementation seems surprisingly
solid already (at least that's my first impression).

Obviously there's still a lot of work to be done until this can be used
as a multi-user collaboration platform, but it's very promising indeed.

Keep up the good work, Chris!

-- F.

[JayFresh - http://jayfresh.wordpress.com]

Chris,

This looks exciting. I like the feeling of being able to dip into a
TiddlyWiki-friendly data store on a server to deposit and retrieve
tiddlers. A Tiddler bank if you will. The support for requesting
different content-types makes me wonder whether this has a role to
play in fixing the SEO problem [1]. I'd welcome your thoughts on this!


J.

[1] http://groups.google.com/group/TiddlyWikiDev/browse_thread/thread/d14ec01af14e4f8f/e5643d40d0b031a8?#e5643d40d0b031a8

[chris...@gmail.com]

On Mar 26, 7:26 pm, "JayFresh - http://jayfresh.wordpress.com"

<jonny.jonat...@googlemail.com> wrote:
> This looks exciting. I like the feeling of being able to dip into a
> TiddlyWiki-friendly data store on a server to deposit and retrieve
> tiddlers. A Tiddler bank if you will. The support for requesting
> different content-types makes me wonder whether this has a role to
> play in fixing the SEO problem [1]. I'd welcome your thoughts on this!

A Tiddler bank, yeah, that's a good way to look at it. With some
accounts being shared.

One of the motivations for the multiple content-types is to make room
for dealing with the SEO problem. I've got no particular solution in
mind but the system is built in such a way that we can present
different stuff depending on what's looking for it and where the
request is coming from. There's a set of heuristics that are used to
determine how the requested resource (tiddler, bag, recipe, etc) is to
be serialized to the client. Those rules are stackable and the
serializer system is pluggable, so it should be possible to say:

if this_is_google:
do_something_awesome()

What that awesomeness is sort of beyond my area of expertise. If
something_awesome has been figured out and can be well described,
implementing it is probably not all _that_ hard. Once TiddlyWeb is a
bit more mature I'm happy to join a gang thinking on it.

[chris...@gmail.com]

On Mar 26, 4:46 pm, FND <F...@gmx.net> wrote:
> I really like this concept, and the implementation seems surprisingly
> solid already (at least that's my first impression).

I tried to be very disciplined about using test first principles with
all the code. I think that helped a lot.

I was able to stick with good test principles until I started
involving the TiddlyWiki client, and then I started being the tester
myself, rather than an automated tool. I found and fixed a lot of
issues that way, but I'm not scriptable and repeatable.

I hope to employ selenium in testing, further down the road.

> Keep up the good work, Chris!

Thank you very much.

[Xavier Verges]

>but I'm not scriptable and repeatable.

Could we open a ticket about this? :-)

-Xv

On Mar 26, 9:06 pm, "cd...@burningchrome.com" <chris.d...@gmail.com>
wrote:

[FND]

>> but I'm not scriptable and repeatable.
>
> Could we open a ticket about this? :-)

I'm afraid that's a wontfix; TPTB have backwards-compatibility concerns.

-- F.

[chris...@gmail.com]

On Mar 26, 5:17 pm, "cd...@burningchrome.com" <chris.d...@gmail.com>
wrote:

> 1. Check out the demo I have running on my server:
>
>    http://peermore.com:8080/recipes/TiddlyWeb.wiki#PlayAround
>
> See what works, what's broken, what needs more explanation. Content
> you edit there will be synchronized back to the server if a) you edit
> existing tiddlers b) you create a new tiddler by linking from an
> existing one.

I've made some adjustments to the demo:

1. There's a growing body of content that is trending in the direction
of documentation for the server.

2. I've written a TiddlyWebAutoSavePlugin which (optionally) causes
content to be saved to the server whenever 'done' is clicked after
editing a tiddler. I used some of the code from ccTiddly as an
example. This effectively turns the TiddlyWeb served tiddlers into a
'normal' online wiki [1]. It also does a handy job of demonstrating
some aspects of recipes and bags.

You can see the auto saving version at:

http://peermore.com:8080/recipes/AutoTiddlyWeb.wiki

The sync-only version of the same content (minus the auto save plugin)
is at:

http://peermore.com:8080/recipes/TiddlyWeb.wiki

The AutoTiddlyWeb recipe looks like this:

/bags/TiddlyWeb/tiddlers
/bags/TwAutoSave/tiddlers
/bags/NewContent/tiddlers

The TiddlyWeb recipe looks like this:

/bags/TiddlyWeb/tiddlers
/bags/NewContent/tiddlers

By placing the TwAutoSave bag in the AutoTiddlyWeb recipe, the tiddler
in it (TiddlyWebAutoSavePlugin) is included in the generated
workspace. By placing the TwAutoSave bag before NewContent we insure
that new tiddler content that is saved to the workspace without a bag
reference is saved to NewContent (as it would be using the TiddlyWeb
recipe to generate a workspace).

Each of the listed bags has no filters. By default a bag without a
filter or a restrictive security policy will accept any tiddler to be
saved to it. When new content is sent to a workspace without a bag
reference the first bag, from the bottom up of the recipe, that will
accept the tiddler is where the tiddler will be saved. Later when that
same tiddler is retrieved and edited again, it will have a bag
reference saying which bag it should be saved to.

In the long term there will be nothing preventing plugin authors and
TiddlyWiki users from altering the bag content can be saved to. It's
fairly easy to imagine interesting scenarios such as duplicating a
bunch of content to another location, or having a plugin which saves
every saved tiddler to its usual location as well as a backup server
somewhere.

In the even longer term recipes could look like this:

/bags/TiddlyWeb/tiddlers
/bags/TwAutoSave/tiddlers
/bags/NewContent/tiddlers
http://jeremysserver.example.com/bags/dailycontent?[tag[osmoblog]]
[sort[-modified] [count[10]
http://martinserver.example.com/bags/TiddlyNews

The first three bags comes from the current server, the later two from
others. Jeremyserver produces the 10 most recently modified tiddlers
tagged with osmoblog. Martinserver produces all the tiddlers in
TiddlyNews. Because he's old school like that, he uses cron and shell
scripts to manage the tiddlers in that bag so it always contains the
latest TiddlyWiki news[2].

What do people think is more important: getting these bits of
esoterica (such as remote bags) framed out for the sake of
experimentation, discovery and making sure we've got the bases covered
OR getting practical functionality such as bag security (access
control), revision handling and alternate data stores built? I'm very
interested in people's thoughts on these matters.

In the meantime give auto saving a try at http://peermore.com:8080/recipes/AutoTiddlyWeb.wiki

Please let me know if things seem odd. Thanks.

[1] Please note that there is not yet support for revision handling
and conflict detection. In auto mode, when you click done, your
changes are written to the server, even if the content on the server
has changed while you weren't looking. Conflict detection will be
coming in this coming month, with revision handling soon after that.

[2] Part of the TiddlyWeb purpose is to make it possible to manipulate
tiddlers with tools other than TiddlyWiki. Various things can pull
data from all over the place, slide, dice and tag as appropriate, and
then make it show up, via TiddlyWeb, in a TiddlyWiki. Similarly a
TiddlyWiki with the right plugins can slice, dice and tag data from
all over the place and then save it to a TiddlyWeb.

[paul.s...@gmail.com]

> I've been working on a bare-bones, back-to-basics implementation of a
> server side for TiddlyWiki

neat!

> In any case, respond here if you have any questions, comments or
> thoughts.

might be interesting to see if this deploys onto Google App Engine:

http://code.google.com/appengine/

which is Python based and has an interesting, if Google centric, story
for authentication:

http://code.google.com/appengine/docs/users/overview.html

Paul
--
http://blog.whatfettle.com