The Road to Jake Weary: Announcing jQuery in TiddlyWiki 2.5 Beta 2

[Jeremy Ruston]

I'm happy to announce the beta of TiddlyWiki 2.5. It's a big milestone
release for TiddlyWiki because it marks the first introduction of
jQuery into the TiddlyWiki code base, which we think will first make
life much easier and more productive for plugin authors, and
ultimately lead to a much sleeker, faster TiddlyWiki core for
everyone.

So, what is jQuery? For details see http://jquery.com/, but in brief
it is a JavaScript library that provides a nice, clean API for web
application developers that smooths out the differences and
incompatibilities between different browsers. It's widely used by many
organisations and individuals for it's ability to take the pain out of
web development.

Because jQuery wasn't around when TiddlyWiki was first created,
TiddlyWiki has included it's own big swathes of code to do roughly the
same thing, smoothing out the differences between different browsers.
Much of this code is completely generic from TiddlyWiki, but because
it was written from scratch as part of TiddlyWiki, it's also rather
brittle. A key issue for TiddlyWiki is that tracking the browser
incompatibilities is pretty tough, and testing this sort of code is a
big undertaking, needing access to lots of browser combinations and so
on. Replacing this compatibility layer with jQuery will give us a much
more reliable foundation, and allow us to benefit from the talents of
the many people who are contributing to jQuery.

So, switching to jQuery allows us to reuse existing work on the
intricacies of browser quirks. It also makes it easier for developers
familiar with that library to work on TiddlyWiki, and brings a library
of sophisticated UI widgets to TiddlyWiki.

jQuery isn't the only game in town, though; Dojo, Prototype and
Mootools are all fairly popular, too. In comparison jQuery has a
clear, concise scope (unlike say Dojo) which means it's stayed small
and lean. The jQuery chaining idiom has turned out to be cool and
productive, and the jQuery community is booming, and has generated an
impressive library of plugins and themes etc

This is a significant change for TiddlyWiki, but I think we can do it
in phases, with little or no impact on backwards compatibility. Here's
the plan in outline:

Step 1: Include jQuery in TiddlyWiki as release 2.5.0. At this point,
there will be no changes to the TiddlyWiki core code to take advantage
of jQuery, but it will become possible for plugin authors to start
using jQuery. There's many more TiddlyWiki plugin developers than
there are core developers, so this step alone will bring the
advantages of using jQuery to the widest group

Step 2: Rewrite certain TiddlyWiki core functions to take advantage of
jQuery (without changing the interface to that function). For example,
the relatively lengthy doHttp() function in TiddlyWiki could be easily
rewritten in a few lines to use the Ajax support built into jQuery.

Step 3: Replace certain calls to TiddlyWiki core functions with the
equivalent jQuery idiom. For example, most calls to
createTiddlyElement() or createTiddlyButton() could be replaced by
much more concise jQuery equivalents.

Steps 2 and 3 will take some time to go through the entire TiddlyWiki
code base. The idea is to prioritise carefully so that we get the most
benefit from this investment. We'll have a series of releases as we
gradually refactor the core code. We hope to have a release 2.5.1 soon
that does enough refactoring to return the size of empty.html to it's
2.4.3 levels.

For more information on the migration plans, see the roadmap on the
wiki <http://tiddlywiki.org/wiki/Dev:Roadmap>, and post any questions
here.

Cheers

Jeremy

--
Jeremy Ruston
mailto:jer...@osmosoft.com
http://www.tiddlywiki.com

[Saq Imtiaz]

In case anyone else is wondering what the link to the beta is...
http://tiddlywiki.com/beta/

Saq

[Saq Imtiaz]

Thank you for that insightful and detailed explanation for the new
beta release and where we go after it Jeremy, it's very much
appreciated. I think adopting jQuery could be a huge boost for
TiddlyWiki if done right.

I haven't had the chance to look at the new beta in much detail yet,
but I do have some immediate questions:

1) Have the changes to the macro handling described in the last post
( http://tinyurl.com/bxnde5 ) about TiddlyWiki-jQuery work been
dropped? I can't seem to find them in this release. It would be
helpful to know if these changes will now be introduced in a future
release or if they have been dropped altogether, and if so why? Being
privy to such information provides a better insight as to where the
TiddlyWiki code is headed and allows plugin writers to plan ahead.

2) In Steps 2 & 3 of the jQuery integration process that you describe,
it seems like a careful review of the entire TW code would be needed
to affect a proper re-factoring with jQuery in mind. What if any plans
have been made for this? Do you have some kind of mechanism in mind to
facilitate this? I do hope that whatever method is chosen, that it
will be done in a completely transparent and open manner which would
allow the community to contribute to the process rather than limiting
it to the few that work at Osmosoft. Whilst I have the utmost respect
for everyone there, the more eyes on this the better!

As of late I as a plugin writer have really struggled to understand
what's happening with regards to TiddlyWiki core development despite
monitoring Trac very closely and a little more communication like
today's post would go a long way towards helping with that.

Many thanks,
Saq

[Martin Budden]

Saq,

to answer your questions:

1) We took out the jQuery macro handling. The implementation was a bit
clumsy and rather than lock ourselves into something sub-optimal we
decided to take it out altogether. There are some fundamental clashes
between jQuery and tiddlywiki macros and it may not be possible to
combine them in a way that does not compromise one or the other. One
hope is that this might be possible when we get a bit more jQuery
experience, or if a person with jQuery experience learns enough about
TiddlyWiki to help us resolve the problems.

2) You are right, careful review of the code will be required. We'll
start this once the 2.5.0 release is out. And yes more eyes are
welcome. Especially from people who know jQuery well

The plans for core development are pretty simple:

i) more jQuery integration, as in (2). The main priority for this is
that compatibility is maintained. Our plans are mainly to replace
tiddlywiki core code with jQuery equivalents where possible. What we
will do will be influenced by feedback we receive and what we learn
about jQuery.

ii) Fix tickets as specified in trac.

Core development is pretty well ticket driven. Which tickets get in a
release is somewhat arbitrary, it depends on how important the
bug/feature is, how easy it is to fix, whether someone has submitted a
patch and if anyone in the community has expressed a particular need
for a feature or bug fix.

Martin

2009/3/3 Saq Imtiaz <lew...@gmail.com>:

[FND]

> Have the changes to the macro handling described in the last post
> ( http://tinyurl.com/bxnde5 ) about TiddlyWiki-jQuery work been
> dropped? I can't seem to find them in this release.

Those plans have indeed been dropped.
You're right that we should have posted an update in that previous
thread - and we'd be happy to elaborate on the reasons there.
To be fair though, we never got much of a response to those proposals,
so there didn't seem to be much interest in the issue.
The upshot is that for now, the macro and plugin interface will remain
unchanged.

> In Steps 2 & 3 of the jQuery integration process that you describe,
> it seems like a careful review of the entire TW code would be needed
> to affect a proper re-factoring with jQuery in mind. What if any plans
> have been made for this? Do you have some kind of mechanism in mind to
> facilitate this?

That's definitely something we're thinking about, and of course we'd
welcome any help and/or expertise in that process!
While we have to release 2.5.0 first, we'll start a thread about this
issue soon.

> As of late I as a plugin writer have really struggled to understand
> what's happening with regards to TiddlyWiki core development despite
> monitoring Trac very closely

Could you elaborate on what it is you're missing? Normally, every issue
considered for a release should be ticketed.
I agree though that posts like Jeremy's here are very helpful in mapping
out the future plans in a "big picture" kind of way.

-- F.

[Saq Imtiaz]

Martin: Thank you for explaining the reasons behind reverting the
macro handling changes. It makes sense not to rush that mechanism if
we are unsure as to the consequences.

With regards to code development being ticket driven, I'm afraid this
isn't entirely true in my experience and there are exceptions. Eg: I
can't remember having seen, nor can I find, a ticket for the macro
handling changes that had been proposed. So all I saw in the Trac
timeline was the changes being made and the post explaining them, and
then the changes being reverted... which was rather confusing.

Whilst monitoring Trac usually works well for specific and small
changes, more information is usually required for larger scale
changes. Also quite often keeping abreast of Trac only allows one to
offer an opinion on suggestions made and patches submitted etc., but
not to be a part of the deliberation process that lead up to them
(especially when it pertains to larger issues since the meat of the
discussion now takes place elsewhere and not on Trac or on this
group). All of which is why I really appreciate the time you and
Jeremy have taken today to provide us with this information and reply
to questions. Thank you.

I'll look forward to the Trac ticket/ group post for the core review
and refactoring that's planned for after the 2.5.0 release.
Cheers,
Saq

[Saq Imtiaz]

On Mar 3, 5:02 pm, FND <F...@gmx.net> wrote:
>
> Those plans have indeed been dropped.
> You're right that we should have posted an update in that previous
> thread - and we'd be happy to elaborate on the reasons there.
> To be fair though, we never got much of a response to those proposals,
> so there didn't seem to be much interest in the issue.

I'd like to point out that a lack of response and a lack of interest
are two completely different things. A lack of dissent is in fact an
approval of the proposal and as such it was entirely natural to expect
the lack of any further updates to mean those changes would be
implemented. Martin has been kind enough to explain the reasons behind
reverting that change, so no harm done. (Thanks again Martin)

> > In Steps 2 & 3 of the jQuery integration process that you describe,
> > it seems like a careful review of the entire TW code would be needed
> > to affect a proper re-factoring with jQuery in mind. What if any plans
> > have been made for this? Do you have some kind of mechanism in mind to
> > facilitate this?
>
> That's definitely something we're thinking about, and of course we'd
> welcome any help and/or expertise in that process!
> While we have to release 2.5.0 first, we'll start a thread about this
> issue soon.

Terrific, I'll be looking forward to it.
Cheers,
Saq

[Ton van Rooijen]

Hi Jeremy (and Martin?),
Thanks for this great announcement and the way you elaborated on this
and explained the way forward with TW. Will be very interesting.

What I would like to bring forward is the timely attention for the
consequences for the translators and translations of TW. I think e.g.
that it would be very nice if already in an early stage an updated
version of Martin's "locale.en.js" could be published.
Even better would be if it could be possible now and in the future to
publish a "diff" of any new "locale.en.js" and the previous version.
Until now it is often a tedious job to identify the differences.

Thanks for your attention for this international aspect of TW, which
in my humble opinion is not unimportant for TW and it's many alien
users.

Many thanks in advance and w/kind rgrds,

Ton van Rooijen

[Martin Budden]

Ton,

I'll do an update of locale.en.js shortly.

Your suggestion of doing a diff everytime locale.en.js is updated is a
good one, and I will do this.

Martin

2009/3/3 Ton van Rooijen <ton...@xs4all.nl>:

[Jeremy Ruston]

I've been thinking about translations a bit too, and am wondering
particularly about the way that we require our translators to write
and maintain syntactically correct javascript, with all the knobbly
bits about quote escaping. The obvious alternative might be for
translations to be stored as tiddler slices (ie, a 2 column wiki table
of identifier vs. translation), and I wonder if that might make life
easier for translators.

Cheers

Jeremy

[chris...@gmail.com]

On Mar 4, 11:54 pm, Jeremy Ruston <jeremy.rus...@gmail.com> wrote:
> I've been thinking about translations a bit too, and am wondering
> particularly about the way that we require our translators to write
> and maintain syntactically correct javascript, with all the knobbly
> bits about quote escaping. The obvious alternative might be for
> translations to be stored as tiddler slices (ie, a 2 column wiki table
> of identifier vs. translation), and I wonder if that might make life
> easier for translators.

This is pretty much exactly how it's done in Socialtext land (or at
least was): 2 column wiki table pairing identifier with string, no
weird escaping required. Later processing puts the "stuff" in the
"code" using "magic". In TiddlyWiki presumably different "code" and
"magic" could do much the same, on the fly.