Fixes for MonkeyGTD under TiddlyWeb (saving and deleting)

[Oveek]

I've been working on using these two excellent tools together. Out of
the box there are a few issues with running MonkeyGTD under TiddlyWeb.
I think the same issues apply to using MonkeyGTD under any TW
serverside that uses TiddlyWiki's adaptor mechanism.

First, sorry this post is so long, I actually wrote it over the course
of several days.

1) Saving
This is actually a problem with a simple fix.

Problem: Tiddlers created with MGTD's special buttons don't get saved
under TiddlyWeb. These are the '+ action', '+ project', and other
buttons found in the MGTD header area.

Cause: The custom buttons use the <<newSavedTiddler>> macro from the
NewSavedTiddlerPlugin. The problem arises due to the way the macro
works, and the way TiddlyWiki's adaptor mechanism works.

Clicking one of the buttons opens a special tiddler with the
appropriate GTD template applied to it. The problem is the field
attribute of the new tiddler is missing the server info which the
TiddlyWeb adaptor needs for saving. The necessary server details are
found in the config.defaultCustomFields array in the TiddlyWebConfig
plugin. They are server.type, server.host, and server.workspace

Solution: In the NewSavedTiddlerPlugin, in function
config.macros.newSavedTiddler.onClick, add one line of code:
merge(fields, config.defaultCustomFields, true);

This needs to be added before the following line:
var tiddler = store.saveTiddler(title,
title,text,config.options.txtUserName,new Date(),tags,fields);

How it works: Mostly for my own sake I'll describe what happens when
the new line is added and executed. As a sporadic tiddly hacker it can
be a challenge to unravel the chain of actions/function calls that are
occuring in TiddlyWiki.

As an example, if a new action tiddler is opened by clicking the '+
action' button:
*The newSavedTiddler macro's onClick handler is invoked. This is where
the new line of code is added.
*The new code takes the server info from config.defaultCustomFields
and adds it to the fields array.
*As seen above, the fields array is then passed to the
store.saveTiddler() core function where a new tiddler object is
created and added to the internal TiddlyWiki store object. Also in
saveTiddler() there is a call, tiddler.set
(newTitle,newBody,modifier,modified,tags,created,fields), that sets
all of the new tiddler's attributes. This is how the server details
from config.defaultCustomFields in the TiddlyWebConfig plugin end up
as attributes of the newly created tiddler.
*After the onClick handler executes, the new Action tiddler is visible
on the screen and exists in the store as a Tiddler object, but it has
not yet actually been saved to the server.
*The tiddler is saved back to the server only after it is edited and
the user presses 'done', or after edits are saved by pressing 'save
changes'. In either case the saving is accomplished by functions in
the ServersideSavingPlugin (starting with the core override,
saveChanges()). saveChanges() calls plugin.sync(), which calls
plugin.saveTiddler(), which in turn calls the TiddlyWeb adaptor's
putTiddler() function.
*In the adaptor's putTiddler() function the server info is finaly read
from tiddler.fields (server.host, and server.workspace) and used to
save the tiddler to the server via an http request.

2) Deleting
I was able to identify the problem and find a solution that works, but
I'm fairly sure it's not the right way.

Problem: The main problem is that when special GTD tiddlers are
deleted under TiddlyWeb, the tiddlers are not removed from the MGTD
variable config.indexedTags.tagLists which maintains a list of
tiddlers tagged with the special MGTD tags like action, Done, etc.

This causes two different problems depending on whether the tiddler is
saved back to the server or not, but the problem is most obvious in
the case where the tiddler is saved to the server so I'll focus on
that situation.

To replicate: create a new action, save it by pressing 'save changes',
then delete it.
The error can then be seen by looking at the 'Next Actions' tiddler,
and in other places the mgtdList macro is called.

Cause: This problem occurs due to a conflict between MgtdIndexedTags
and ServerSideSavingPlugin. MgtdIndexedTags hijacks the
Tiddlywiki.prototype.removeTiddlers() function, while
ServerSideSavingPlugin *overrides* the same function. The result is
the removeTiddlerHijack() function in MgtdIndexedTags never gets
called, and some important function calls that update the
config.indexed.tagLists variable are skipped.

Solution: Just as a test to see if the problem was what I thought, I
copied the following from the removeTiddlerHijack() function in
MgtdIndexedTags, and added it to the beginning of the removeTiddler()
function in ServerSideSavingPlugin:

var oldTags = tiddler ? [].concat(tiddler.tags) : null;
config.indexedTags.updateTagLists(title,oldTags);
config.indexedTags.updateIndexes(title);

This hack does remedy the problem since the functions that were being
skipped are executed, but It's not a good solution since it requires
adding MGTD specific code to the ServerSideSavingPlugin.

I guess the change will have to be on the MGTD end, but I'll leave it
to the TiddlyWeb devs and Simon Baird to decide the right solution to
both problems I mentioned.

Sorry if I was too detailed. For me debugging TiddlyWiki plugins feels
a lot like untangling a ball of yarn; writing everything down helped
me sort out what was happening on the inside.
Is it just me or are programs written in weakly-typed languages, like
javascript, tougher to follow along with?

Anyway, all the creativity that surrounds TiddlyWiki is excellent,
keep up the good work.

[FND]

That's some good investigative work there - thanks for putting so much
effort into this!

> Solution: In the NewSavedTiddlerPlugin, in function
> config.macros.newSavedTiddler.onClick, add one line of code:
> merge(fields, config.defaultCustomFields, true);

Yep, that sounds right - this fix should be fully backwards compatible too.
Simon, are you listening?

> Cause: This problem occurs due to a conflict between MgtdIndexedTags
> and ServerSideSavingPlugin. MgtdIndexedTags hijacks the
> Tiddlywiki.prototype.removeTiddlers() function, while
> ServerSideSavingPlugin *overrides* the same function.

That's quite likely.
Perhaps we need to force the ServerSideSavingPlugin to be loaded before
any other plugin - that's currently only possible by renaming the
tiddler though (as plugins are loaded in alphabetical order), so it's
not exactly an elegant solution...
We might also get by without a complete override there - will need to
investigate.

-- F.

[chris...@gmail.com]

On Feb 7, 3:54 pm, Oveek <mov...@gmail.com> wrote:
> I've been working on using these two excellent tools together. Out of
> the box there are a few issues with running MonkeyGTD under TiddlyWeb.
> I think the same issues apply to using MonkeyGTD under any TW
> serverside that uses TiddlyWiki's adaptor mechanism.

This is awesome work, thanks for posting all the details, it will be
useful for many people.

Most of the experiments thus far with getting so-called "verticals"
working with TiddlyWeb in the background have run into trouble in one
main area: collisions between methods that perform save related
actions that have been overridden or hijacked. Just as you've
discovered.

The general consensus in conversations I've been in is that the API
involved in saving activities is too broad, leading to too many places
where things can go wrong or collide.

> Sorry if I was too detailed. For me debugging TiddlyWiki plugins feels
> a lot like untangling a ball of yarn; writing everything down helped
> me sort out what was happening on the inside.
> Is it just me or are programs written in weakly-typed languages, like
> javascript, tougher to follow along with?

I have a similar experience debugging TiddlyWiki and javascript in
general but I don't think it is the weakly-typed nature, for me. I
think javascript has a tendency to get messy easily, TiddlyWiki
especially so. The extensive use of closures, events and asynchronous
calls, all very useful, can cause a lot of confusion. I find in
TiddlyWiki that the structure of the functions makes it so I'm never
quite sure what parameters are being passed to any function or if I
know their values, how they got those values in the first place.

I hope that Simon Baird and FND will negotiate some solution to the
problem. I'm sure they'll be able to figure out something good.