TiddlyWeb 0.8, getting closer to beta

[chris...@gmail.com]

To keep momentum going, I thought I would announce that there is a
TiddlyWeb 0.8 available for testing and use. Find it, previous
releases and some info on the CHANGES in:

http://tiddlyweb.peermore.com/dist/

To install the tarballs that are there you will need Python's
setuptools[1] and follow these instructions:

* unzip and untar the file: tar zxvf <filename>
* cd into the created directory and run: python setup.py install (you
either need to be root or use sudo with this if you want to install
into secure location)

This will put the tiddlyweb code into site-packages and the twanager
command line too into the usual location for scripts (/usr/local/bin/
on many systems). twanager now includes a command that makes it easy
to create a new instance of TiddlyWeb and start a server without
needing to do a lot of configuration.

If you don't wish to install the code, you can run it right from the
directory created from the tarball.

Read the README and the files in docs for more information.

You can find sample extensions/plugins for TiddlyWeb in subversion:

http://svn.tiddlywiki.org/Trunk/contributors/ChrisDent/experimental/tiddlyweb-plugins/

The 0.x series of release are alpha releases in preparation for a beta
before the end of this year. If you encounter difficulties or bugs
please report them here or on trac: http://trac.tiddlywiki.org/

[1] http://peak.telecommunity.com/DevCenter/setuptools

[ccahua]

On Nov 14, 5:43 am, "cd...@peermore.com" <chris.d...@gmail.com> wrote:
> To keep momentum going, I thought I would announce that there is a
> TiddlyWeb 0.8 available for testing and use. Find it, previous
> releases and some info on the CHANGES in:
>
>    http://tiddlyweb.peermore.com/dist/

Hi Chris,

Thank you for your ongoing efforts to keep the TiddlyWeb candle
burning. :-)

I added a modified version of your defaulter plugin (see below) to
handle the different loopback server ip addresses (Mac vs Windows) and
it worked fine for saving new tiddlers to TiddlyWeb 0.5 and appears to
work ok on tiddlyweb 0.8 when i save to a recipe, more specifically to
the system bag.

This new concept of keeping a system bag for plugins separate from a
content bag is new for me so it took a while to properly install as i
kept mistakenly creating new stores when i was not in the proper
directory. This is a great idea since the plugins in the system bag
can be applicable to other bags when combined in a recipe. Before on
0.5 I would add the plugins to my imported 'content' bag, but now
there are a separation of concerns.

Ideally saving new tiddlers to new content bags versus the default
system bag would be ideal as new content tiddlers would save to the
content bag, but I could not think of a way to specify the
config.defaultCustomField other than 'server.bag': 'system'.

It is easy to hard code the server.bag value to the name of the
predefined bag which will hold my content, but I can't see a way to
have a config.defaultCustomField for server.bag: <bag name at recipe
creation> ???

The new tiddlers saved to a recipe are saved to the system bag and
maybe there is a way to filter out non-plugin tiddlers for new recipes
which need the system bag plugins something like /bags/system/tiddlers?
[tag[systemConfig]] but do you have any thoughts on a filter or
workaround for saving tiddlers to their respective non-system bags as
in:

recipe1 = system bag + content bag1
with the TiddlywebDefaulterPlugin server.bag value changed to bag1
from system

As is new tiddlers are saved to system bag

so when

recipe2 = system bag + content bag2

with the TiddlywebDefaulterPlugin server.bag value changed to bag2
from system

recipe2 should really = system/tiddlers?[tag[systemConfig]] + content
bag2

but as far as I can make:

recipe2 = system bag + new tiddlers made from content bag1 + content
bag2

btw: I wrote this post in a tiddler saved in TiddlyWeb recipe, more
specifically in the system bag, ha ha!

Best,
tony

ps:
here's that bit from quirksmode that switches the server ip address to
0.0.0.0 when i move my store on the mac:
(it would be nice to know the variable name for the server.bag field;
i keep looking)

{{{
modifier: TonyChing
created: 200810240000
modified: 200810240000
type: None
tags: systemConfig excludeLists excludeSearch

/***
|''Name:''|TiddlyWebDefaulterPlugin|
|''Description:''|Set some defauls for the TiddlyWebAutoSavePlugin
when using |
|''Author:''|Chris Dent (cdent (at) peermore (dot) com)|
***/

//{{{
//# Ensure that the plugin is only installed once.
if(!version.extensions.TiddlyWebDefaulterPlugin) {
version.extensions.TiddlyWebDefaulterPlugin = {installed:true};

tiddler = store.getTiddler('TiddlyWebAutoSavePlugin');

// the name of the recipe/workspace is the same name as the bag
// in which the user's content is stored. Browser detect code from
quirksmode.org
var BrowserDetect = {
init: function () {
this.OS = this.searchString(this.dataOS) || "an unknown OS";
},
searchString: function (data) {
for (var i=0;i<data.length;i++) {
var dataString = data[i].string;
var dataProp = data[i].prop;
this.versionSearchString = data[i].versionSearch || data
[i].identity;
if (dataString) {
if (dataString.indexOf(data[i].subString) != -1)
return data[i].identity;
}
else if (dataProp)
return data[i].identity;
}
},

dataOS : [
{
string: navigator.platform,
subString: "Win",
identity: "Windows"
},
{
string: navigator.platform,
subString: "Mac",
identity: "Mac"
},
{
string: navigator.platform,
subString: "Linux",
identity: "Linux"
}
]

};
BrowserDetect.init();
if (BrowserDetect.OS == "Windows")
{
config.defaultCustomFields = {
'server.host': 'http://127.0.0.1:8080/',
'server.bag': 'system',
'server.workspace': '',
'server.type': 'tiddlyweb',
};
}
else
{
config.defaultCustomFields = {
'server.host': 'http://0.0.0.0:8080/',
'server.bag': 'system',
'server.workspace': '',
'server.type': 'tiddlyweb',
};
}
} //# end of 'install only once'
//}}}

}}}

[chris...@gmail.com]

On Nov 24, 11:25 pm, ccahua <cca...@gmail.com> wrote:
> Thank you for your ongoing efforts to keep the TiddlyWeb candle
> burning. :-)

And thank you.

> I added a modified version of your defaulter plugin (see below) to
> handle the different loopback server ip addresses (Mac vs Windows) and
> it worked fine for saving new tiddlers to TiddlyWeb 0.5 and appears to
> work ok on tiddlyweb 0.8 when i save to a recipe, more specifically to
> the system bag.

You can probably forego needing to use a different loopback server ip
address by only using 127.0.0.1 in your twanager command line OR using
the real network IP or hostname of your machine.

> This new concept of keeping a system bag for plugins separate from a
> content bag is new for me so it took a while to properly install as i
> kept mistakenly creating new stores when i was not in the proper
> directory. This is a great idea since the plugins in the system bag
> can be applicable to other bags when combined in a recipe. Before on
> 0.5 I would add the plugins to my imported 'content' bag, but now
> there are a separation of concerns.

Exactly. The idea is that you shouldn't need to duplicate some
tiddlers, but rather use recipes to assemble those that you need from
multiple places.

> Ideally saving new tiddlers to new content bags versus the default
> system bag would be ideal as new content tiddlers would save to the
> content bag, but I could not think of a way to specify the
> config.defaultCustomField other than 'server.bag': 'system'.

Handling the saving a new tiddlers is still not quite as well resolved
as I would like it to be. FND has been working on some solutions to
this problem but I don't think it is fully worked out yet. Maybe he
can comment and we can get a conversation rolling about it.

One thing you can try is to put a _different_ defaulter plugin in
different bags. That defaulter would have server.bag set to a
particular bag name (maybe the bag that it is in). Then, using
recipes, whichever bag is last in the recipe, it's default plugin
would be the one that would set the default for that recipe.

Which is pretty much what you describe here:

> It is easy to hard code the server.bag value to the name of the
> predefined bag which will hold my content, but I can't see a way to
> have a config.defaultCustomField for server.bag: <bag name at recipe
> creation> ???
>
> The new tiddlers saved to a recipe are saved to the system bag and
> maybe there is a way to filter out non-plugin tiddlers for new recipes
> which need the system bag plugins something like /bags/system/tiddlers?
> [tag[systemConfig]] but do you have any thoughts on a filter or
> workaround for saving tiddlers to their respective non-system bags as
> in:

There are a lot of mechanisms involved when saving tiddlers up to the
server that may or may not be interacting together as well as we would
like.

First and foremost: If a tiddler came from a tiddlyweb server in the
first place, it already has a server.bag field, so barring the action
of some plugin[1], when it is saved it will be saved back to the bag
it came from.

If a new tiddler is created _via a link from an existing tiddler_ it
will inherit fields from the existing tiddler, however at the moment
it does _not_ inherit server.bag, so barring the actin of some plugin
[1] it is put back to the recipe not the bag. In most cases this means
it will be written to the last bag in the recipe.

If a new tiddler is created from new tiddler or new journal macros, by
default the tiddler gets no fields and will not be saved to the
server. If there is something like the defaulter plugin[1] it will get
some default fields and be saved according to those rules.

Does that make much sense? I suspect we've all got some slightly
conflicting models of what's supposed to be going on that are
themselves conflicting with what is actually going on, so if we can
get what we want and what we see out on the table, we should be able
to work out some solutions.

[1] The whole notion of the fields being there as attributes is so
that they can be manipulated by plugins and the saving process can be
adjusted depending on context within the TiddlyWiki. Sometimes we want
all content to go back to a particular place on the server. Sometime
we want it go back to a personal override location, sometimes it
should go to the server at all, etc. Whatever plugins get created have
the option to control that in a very flexible way.

[FND]

> Ideally saving new tiddlers to new content bags versus the default
> system bag would be ideal

So you're suggesting that, just as a "system" bag is created initially,
there should be a second bag for content by default?
While it's a good idea for basic setups, we anticipate more elaborate
systems with several content bags.
However, come to think of it, such systems could still be customized
accordingly - no harder than if there were no default content bag.
Chris, what's your take on this?

> Handling the saving a new tiddlers is still not quite as well resolved
> as I would like it to be.

There are two ways to approach this:
a) set config.defaultCustomFields with a simple plugin (tiddler tagged
with "systemConfig"):
//{{{
config.defaultCustomFields = {
"server.host": "127.0.0.1",
"server.bag": "content"
};
//}}}
b) make the NewTiddler button (in the sidebar's SideBarOptions shadow
tiddler by default) set these fields:
<<newTiddler
fields:"'server.host':'127.0.0.1' 'server.bag':'content'"
>>
(cf. http://www.tiddlywiki.org/wiki/NewTiddler_(macro) )

The second solution gives you more flexibility to put content into
different bags as appropriate - e.g. you could provide multiple buttons
(e.g. "new journal", "new report" etc.).
Also see the notes here:
http://trac.tiddlywiki.org/ticket/755#comment:6

> maybe there is a way to filter out non-plugin tiddlers for new recipes
> which need the system bag plugins

I'm not sure I understand this - but I believe Chris has addressed it
already, so I probably don't need to worry about it.

-- F.

[ccahua]

On Nov 25, 7:10 am, FND <F...@gmx.net> wrote:
> b) make the NewTiddler button (in the sidebar's SideBarOptions shadow
>    tiddler by default) set these fields:
>      <<newTiddler
>          fields:"'server.host':'127.0.0.1' 'server.bag':'content'"
>      >>

>    (cf.http://www.tiddlywiki.org/wiki/NewTiddler_(macro) )

>
> The second solution gives you more flexibility to put content into
> different bags as appropriate - e.g. you could provide multiple buttons
> (e.g. "new journal", "new report" etc.).
> Also see the notes here:
>      http://trac.tiddlywiki.org/ticket/755#comment:6

Thank you, both for suggestions and guidance :-)

Your newTiddler macro approach kind of worked.

adding the newTiddler macro with the custom bag field value set to the
non-system bag:

<<newTiddler fields:"'server.host': 'http://127.0.0.1:8080/'
'server.bag': 'content'">>

But it is like catch 22 or the chicken and egg problem, in order to
create a new tiddler I had to save the new tiddler.
The side effect was I lost the ability to save to shadowed tiddlers
like sideOptions.
when the newTiddler macro is added to the sideoptions shadowed tiddler
it didn't save and for some odd reason i lost a long follow up to this
thread in a new tiddler, grr...

Edits to existing shadowed tiddlers which I thought were saved to the
system bag are actually saved to the content bag since this
redirection, 'server.bag': 'content'".
it seems like if i edit a shadow before i apply the redirection, then
I can still save to the shadow, but if I edit after redirection then
no go, ie I can save to SiteTitle but not SideOptions

I'm still getting mixed attempts at saving as I get the 'Saved
<tiddler>' dialog but if I exit out of 127.0.0.1 and go back in, saved
tiddler is gone! poof!
maybe I'm exacerbating the problem with multiple tabs/instances so
I'll keep testing to see if one of these approaches will stick.
I don't recall my trac login on hand, but I'll report back over there
at ticket 755

Best,
tony

[chris...@gmail.com]

On Nov 14, 1:43 pm, "cd...@peermore.com" <chris.d...@gmail.com> wrote:
> To keep momentum going, I thought I would announce that there is a
> TiddlyWeb 0.8 available for testing and use. Find it, previous
> releases and some info on the CHANGES in:
>
>    http://tiddlyweb.peermore.com/dist/
>
> To install the tarballs that are there you will need Python's
> setuptools[1] and follow these instructions:

There's an easier way to install TiddlyWeb now, if you are already
familiar with the README, COOKBOOK and other information.

It is now possible to install TiddlyWeb directly from PyPI, the python
package index. If you have easy_install on your system you can enter
the following:

sudo easy_install tiddlyweb

and the latest version, and its dependencies should be downloaded and
installed to your system.

On some systems you may have problems getting simplejson to install.
If you do, you will need to install that separately, and then do the
easy_install command again.

Let me know if people try this and experience trouble.

Thank you.

[FND]

> It is now possible to install TiddlyWeb directly from PyPI, the python
> package index.

That's huge, Chris!
How did you manage that - is there a lot of bureaucracy involved?

FWIW, these are the PyPI pages:
http://pypi.python.org/pypi/tiddlyweb/0.8
http://pypi.python.org/pypi/tiddlyweb-plugins/0.1

-- F.

[ccahua]

On Nov 25, 5:43 pm, "cd...@peermore.com" <chris.d...@gmail.com> wrote:
> On Nov 14, 1:43 pm, "cd...@peermore.com" <chris.d...@gmail.com> wrote:
>
> > To keep momentum going, I thought I would announce that there is a
> > TiddlyWeb 0.8 available for testing and use. Find it, previous

Hi again,

Finally have CRU, just missing D!

Here's some more observations:

:-( saving from a newTiddler macro
:-) saving from installing defaulter plugin with server.bag set to the
non-system bag
:-) was able to reinstall from PyPi using cygwin under WinXP Good job!
:-( Did not work: unable to delete
:-| updates to a bag are not reflected in the recipe unless the recipe
is rebuilt

thank you!

I'll try to login to trac and add these comments if they aren't there
already somewhere.

Background notes:
==============
Testing save using the newTiddler method

$twanager bag foo </dev/null
then must add a tiddler or get a 404 no content error
echo -e "modifier: Tony\n\nTest" | twanager tiddler TestTiddler foo

$ twanager recipe bar
/bags/system/tiddlers
/bags/foo/tiddlers

Unable to save to existing tiddler: SiteTitle from
http://127.0.0.1:8080/recipes/bar/tiddlers.wiki despite having
* TestTiddler
* TiddlyWebAdaptorPlugin
* TiddlyWebAutoSavePlugin
and clearing the browser cache
At first there was a upper right corner dialog indicating a "Save
<tiddler>" but going back to the html view shows no saved tiddler:
http://127.0.0.1:8080/recipes/bar/tiddlers

when in doubt startover

Testing save from injecting defaulter plugin to non-system bag

added the defaulter plugin with server.bag: 'foo' to bag foo

$ twanager tiddler TiddlyWebDefaulterPlugin foo < ~/tiddlyweb-0.8/
TiddlyWebDefaulterPlugin2.txt

but bar recipe is not updated with new tiddler so I made it again

$ twanager recipe bar
/bags/system/tiddlers
/bags/foo/tiddlers

inherently with the separation of system and non-system bags, bags in
effect just hold content and don't save ie just Read in CRUD

Bingo! we have save.

Testing TiddlyWeb from PyPi
I deleted my install from the tarball: $ rm -r tiddlyweb-0.8/

then ran
$ easy_install tiddlyweb
Searching for tiddlyweb
Best match: tiddlyweb 0.8
Processing tiddlyweb-0.8-py2.5.egg
tiddlyweb 0.8 is already the active version in easy-install.pth
Installing twanager script to /usr/bin

Using /usr/lib/python2.5/site-packages/tiddlyweb-0.8-py2.5.egg
Processing dependencies for tiddlyweb
Finished processing dependencies for tiddlyweb

also checked
$ find -name "tiddlyweb*"
./.python-eggs/tiddlyweb-0.8-py2.5.egg-tmp
./.python-eggs/tiddlyweb-0.8-py2.5.egg-tmp/tiddlyweb
./Desktop/tweb/tiddlywebconfig.py
./Desktop/tweb/tiddlywebconfig.pyc
./My Documents/tw/tiddlyweb_empty.html
./notebook/tiddlyweb-0.8.tar.gz
./Recent/tiddlyweb-0.8.tar.gz.lnk

I guess I forgot to delete my store before reinstall, but no matter as
they are independent.

I started up the server again: $ twanager server 127.0.0.1 8080

and was able to save a new tiddler. whew.

Unfortunately, I cannot delete the tiddler neither at recipe nor bag.

Best,
tony

[FND]

> Finally have CRU, just missing D!

I'm currently working on a TiddlyWiki plugin to improve the saving
mechanism. It's largely feature-complete, but hasn't been extensively
tested yet:
http://tinyurl.com/58g3fo
(http://svn.tiddlywiki.org/Trunk/association/plugins/ServerSideSavingPlugin.js)

Feel free to give it a try (replaces TiddlyWebAutoSavingPlugin); I'd
love to get some feedback...

> updates to a bag are not reflected in the recipe unless the recipe
> is rebuilt

I don't quite understand - do you mean simultaneous editing (whether in
separate tabs/windows or on multiple machines), with the changes not
being pushed to the client?

As for the remaining issues, I'll talk to Chris after Thanksgiving.

Again, many thanks for your great feedback!

-- F.

[FND]

> Feel free to give it a try (replaces TiddlyWebAutoSavingPlugin); I'd
> love to get some feedback...

I forgot a small, but important detail: The ServerSideSavingPlugin is a
generic plugin - so you need to tell it to use the TiddlyWeb adaptor first.
Create a tiddler, e.g. [[ConfigTweaks]], tag it with "systemConfig" and
add the following contents:
//{{{
config.extensions.ServerSideSavingPlugin.adaptor = config.adaptors.tiddlyweb
//}}}

> As for the remaining issues, I'll talk to Chris after Thanksgiving.

We've agreed that your suggestion of adding a default content bag is
sensible, so "twanager instance" will now create two bags ("system" and
"common"), as well as a "default" recipe.
http://trac.tiddlywiki.org/changeset/8014

We currently don't have a plugin to set the default custom fields to
match that default content bag - however, you can extend the above
ConfigTweaks plugin to do so:
//{{{
config.defaultCustomFields = {
"server.type": "tiddlyweb",
"server.host": "127.0.0.1", // just an example
"server.bag": "common"
};
//}}}

-- F.