tiddlywiki outputting <br> instead of <p>?

17 views
Skip to first unread message

dave

unread,
Oct 7, 2010, 10:56:34 AM10/7/10
to TiddlyWikiDev, david....@gmail.com
Hi, while working on a parser plugin, I'm using firebug to verify the
HTML displayed in the tiddlers. Basically what I'm noticing is that
tiddlers output <br>s where one would expect <p>'s. Such an example
is in blank lines between paragraphs of text. If you put 2 blank
lines between a pair of content lines, then you get 2 <br> tags. I
was expecting blank lines to make new paragraphs. Is there a reason
for this implementation?

FND

unread,
Oct 7, 2010, 11:03:04 AM10/7/10
to tiddly...@googlegroups.com
> tiddlers output <br>s where one would expect <p>'s [...]

> Is there a reason for this implementation?

The reasons for this are historic - unfortunately, it doesn't seem like
we can change this now without breaking backwards compatibility:
http://trac.tiddlywiki.org/ticket/34


-- F.

Colm Britton

unread,
Oct 7, 2010, 11:07:11 AM10/7/10
to tiddly...@googlegroups.com
> tiddlers output <br>s where one would expect <p>'s [...]
> Is there a reason for this implementation?

I have noticed this before too
 

The reasons for this are historic - unfortunately, it doesn't seem like
we can change this now without breaking backwards compatibility:
http://trac.tiddlywiki.org/ticket/34


That is a shame. <p> tags would be a lot nicer to work with. Not only are they more semantically correct but they also allow for greater degrees of control over the presentation of content.

colm

David Young

unread,
Oct 7, 2010, 11:13:55 AM10/7/10
to tiddly...@googlegroups.com
I see. old 'bug'. Would there be a technical breakage if I were to
add a few paragraph parsers for my plugin? Since I'm writing a new
parser, I have little concern for backwards compatability. The
example I'm starting with seems to throw <p> tags around when making
new !sections (or, in the example formatter's case, new =sections= but
they're both h1's). Maybe I can adapt this a bit.

http://www.martinswiki.com/%23ExampleFormatterPlugin&usg=AFQjCNHoSIQgIt4GQoIYukcaSoCsybrDmw

> --
> You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
> To post to this group, send email to tiddly...@googlegroups.com.
> To unsubscribe from this group, send email to tiddlywikide...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/tiddlywikidev?hl=en.
>
>

--
David Young

David Young

unread,
Oct 7, 2010, 11:15:19 AM10/7/10
to tiddly...@googlegroups.com
oops. I went and copied the URL out of groups and didn't clean up the
google-inserted cruft:

http://www.martinswiki.com/%23ExampleFormatterPlugin

--
David Young

chris...@gmail.com

unread,
Oct 7, 2010, 11:12:40 AM10/7/10
to tiddly...@googlegroups.com
On Thu, 7 Oct 2010, Colm Britton wrote:

>>> tiddlers output <br>s where one would expect <p>'s [...]
>>> Is there a reason for this implementation?
>>
>
> I have noticed this before too

Me too. One of the biggest warts in the wikifier as far as I'm
concerned, but I would guess that it would be very challenging to
clean up at this stage (at least in core).

--
Chris Dent http://burningchrome.com/
[...]

FND

unread,
Oct 7, 2010, 1:17:33 PM10/7/10
to tiddly...@googlegroups.com
> Would there be a technical breakage if I were to add a few paragraph
> parsers for my plugin? Since I'm writing a new parser, I have little
> concern for backwards compatability.

If you're not concerned about compatibility with existing tiddler
contents (i.e. TiddlyWiki markup), you can pretty much do whatever you
please.


-- F.

PMario

unread,
Oct 7, 2010, 1:25:55 PM10/7/10
to TiddlyWikiDev
Hi David
I think it is not that easy.
If you have a look at a "standard" html page it may look like this for
a "section"
<h1>header</h1>
<p>some more text<br />
second line
</p>

and not

<p>
<h1></h1>
</p>

But as FND sayd "you can pretty much do whatever you
please."

-m
On Oct 7, 5:13 pm, David Young <david.a.yo...@gmail.com> wrote:
> I see. old 'bug'.  Would there be a technical breakage if I were to
> add a few paragraph parsers for my plugin?  Since I'm writing a new
> parser, I have little concern for backwards compatability.  The
> example I'm starting with seems to throw <p> tags around when making
> new !sections (or, in the example formatter's case, new =sections= but
> they're both h1's).  Maybe I can adapt this a bit.
>
> http://www.martinswiki.com/%23ExampleFormatterPlugin&usg=AFQjCNHoSIQg...

David Young

unread,
Oct 7, 2010, 2:08:29 PM10/7/10
to tiddly...@googlegroups.com
yeah, I wouldn't want the h1-based thing. The standard way is to only
denote prose paragraphs with P's.

I'm thinking of something like a large regex accepting all text
without "block" "sectional" "table" or "list-like" markup... well that
and completely empty/whitespace lines. Then define it as an element
'p'. I see where the complexity comes in though.

When the parser is going through the text, does it search for all
matches of config.formatters[0] and replace text with the html tags,
then search the text (now with some tags) for all matches of
config.formatters[1]? I'm curious about how nesting/greediness are
handled.

PMario

unread,
Oct 7, 2010, 5:22:00 PM10/7/10
to TiddlyWikiDev
On Oct 7, 8:08 pm, David Young <david.a.yo...@gmail.com> wrote:
> When the parser is going through the text, does it search for all
> matches of config.formatters[0] and replace text with the html tags,
> then search the text (now with some tags) for all matches of
> config.formatters[1]? I'm curious about how nesting/greediness are
> handled.
I think it does the parsing in one run.
If you have a look at Formatter [1] it generates a very big xx|yyy|zzz
regexp, to identify a starting point.
If you have a look at config.formatters [2], you'll see that most of
them have a "termRegExp: /(\n)/gm," which specifies the end of one
recogniced block. There is also a lookaheadRegExp:

Wikification is done by the Wikifier

Have a closer look at the following two functions, to see what's going
on.
Wikifier.prototype.subWikifyUnterm = function(output)
Wikifier.prototype.subWikifyTerm = function(output,terminatorRegExp)

Hope this helps.

====================
@core devs.
Is there a detailed description, how tw syntax parsing is done,
somewhere out there??
====================

-mario

[1] http://www.tiddlytools.com/insideTW/#Formatter
[2] http://www.tiddlytools.com/insideTW/#config.formatters

Jeremy Ruston

unread,
Oct 8, 2010, 11:51:06 AM10/8/10
to tiddly...@googlegroups.com
> ====================
> @core devs.
> Is there a detailed description, how tw syntax parsing is done,
> somewhere out there??
> ====================

Just the code, I'm afraid.

When I first wrote TiddlyWiki the wikifier was very simple, it just
handled wiki links. Here's the prerelease version (view source and
search for the definition of the wikify function):

http://www.osmosoft.com/talkytalky/twiddlywiki.html

And here's the first version to be released, which also handled
external HTML links:

http://tiddlywiki.com/firstversion.html

Sometime before the second version was released at the end of 2004, a
chap called Isao Sonobe who was using TiddlyWiki at the Kyoto
University Theoretical Physics Group very kindly contributed changes
he had made to enrich TiddlyWiki with fuller formatting: bold,
italics, tables, etc.

At the time, TiddlyWiki still couldn't save changes, and so I still
regarded it as a bit of a demo, and not important in it's own right.
So, I folded Isao's code in pretty much untouched because it seemed to
make the demo a little more useful.

And then in April 2005 or so, we figured out how to make TiddlyWiki
handle saving itself, and GTDTiddlyWiki was released, incorporating
Isao's wikifier:

http://nathanbowers.com/gtdtw/

(Again, view source and search for the wikify() function)

At this point I slowly realised that I had a bit of a problem: Isao's
code needed refactoring to make it easier to extend and change, but
worse than that, I'd accidentally selected a wiki format that I was
stuck with. If I had known that TiddlyWiki was serious I'd have
started out with the wiki syntax that I prefer personally (for example
*bold* and /italic/).

Anyhow, over the next few months I undertook a complete rewrite of the
wikifier and formatters, to make them more extensible, and the code
more maintainable. The results are pretty much what you see today in
TiddlyWiki. I and other people like Martin Budden, have experimented
with retrofitting <P> support but have concluded that it cannot be
done in a backwards compatible manner. (But if anyone thinks otherwise
I'd love to know about it).

It's all very frustrating. I'm pretty proud of the wikifier code, but
the syntax is sometimes unfortunate.

Cheers

Jeremy

--
Jeremy Ruston
mailto:jer...@osmosoft.com
http://www.tiddlywiki.com

PMario

unread,
Oct 8, 2010, 4:14:04 PM10/8/10
to TiddlyWikiDev
Hi Jeremy,
Thanks for the links to the early versions. It is very interesting to
see TiddlyWiki's metamorphose. Having a short look at gtdtw source, I
understand the desire to rewrite it :)

== OT ==
@David
Sorry, for hijacking your thread a little bit.
@Jeremy
Are there some more links + timeline to early versions, that you know
of. I was thinking about, to make a little goodie. For this I need
some older versions. I know of tiddlythemes.com which seems to be
about 2007. For 2008 till now I think I have enough links. But V1 and
earlyer would be interesting. Especially if the layout has changed.
=== end OT ===

> ... I and other people like Martin Budden, have experimented
>with retrofitting <P> support but have concluded that it cannot be
>done in a backwards compatible manner.(But if anyone thinks otherwise
>I'd love to know about it).
I couldn't handle it with the formatter. But I think some HTML post
processing could do it.

>It's all very frustrating. I'm pretty proud of the wikifier code, but
>the syntax is sometimes unfortunate.

I think TW syntax is quite handy. I had a look at some other wiki
marpups, especially after I found martinswiki. Then looked at there
development sites/repos/blogs. And it seems that all of them have the
same problems: extensibility and macro including.

TW syntax is quite extensible and the macro and transclusion mechanism
is incredibly powerfull.
So I'd love to mix markdown syntax and TW macro handling :) And I'd
love to kick wysiwyg formats.

have fun!
Mario

Ton van Rooijen

unread,
Oct 9, 2010, 9:52:11 AM10/9/10
to TiddlyWikiDev
Hi Mario,
You wrote: "Are there some more links + timeline to early versions,
that you know of.".
Although this is "off topic" (may be you could open a new thread?),
the eldest TW I can help you with is TW v2.0.11 (2006).
And I have another suggestion for you, in case you really want even
older copies: If original early copies aren't available anymore, you
could have a look at the Trac-Translation page, that goes back as far
as several Verions 1 of TW. Potentially you could "reverse-engineer"
by undoing the translation.
http://trac.tiddlywiki.org/wiki/Translations

Best of luck,
Ton van Rooijen.

PMario

unread,
Oct 9, 2010, 11:41:53 AM10/9/10
to TiddlyWikiDev
Hi Ton,
Thats a good source, since there are links to the sites.
thx.

On Oct 9, 3:52 pm, Ton van Rooijen <tons...@xs4all.nl> wrote:
> Hi Mario,You wrote: "Are there some more links + timeline to early versions,
>
> that you know of.".
> Although this is "off topic" (may be you could open a new thread?),
> the eldest TW I can help you with is TW v2.0.11 (2006).
> And I have another suggestion for you, in case you really want even
> older copies: If original early copies aren't available anymore, you
> could have a look at the Trac-Translation page, that goes back as far
> as several Verions 1 of TW. Potentially you could "reverse-engineer"
> by undoing the translation.http://trac.tiddlywiki.org/wiki/Translations

FND

unread,
Oct 9, 2010, 12:51:34 PM10/9/10
to tiddly...@googlegroups.com
> Are there some more links + timeline to early versions

http://tiddlywiki.com/archive/


-- F.

Jeremy Ruston

unread,
Oct 9, 2010, 2:03:52 PM10/9/10
to tiddly...@googlegroups.com
I have kept a lot more older versions of TiddlyWiki (700MB of it, from
September 2004 to May 2006, which was when I think we established the
subversion repository). I will try to upload them to the archive this
weekend. As I was saving them I rather wanted to do one of those
stacked line graphs showing the growth of each of the subsystems over
time.

Cheers

Jeremy

> --
> You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
> To post to this group, send email to tiddly...@googlegroups.com.
> To unsubscribe from this group, send email to tiddlywikide...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/tiddlywikidev?hl=en.
>
>

--

David Young

unread,
Dec 4, 2010, 7:48:53 PM12/4/10
to tiddly...@googlegroups.com
I'm actually revisiting this now- as I ran into some odd truncation
problems with very large tiddlers.

Starting with Martin's exampleFormatter, there was a "paragraph" made
by finding all multiple repeat \n's and then:
createTiddlyElement(w.output,'p');
I think there's something different about tiddlywiki since this
example was created, because it looked ok visually, but not
semantically.

Semantically, every time there was a large block of empty space, a new
paragraph was created-- nested within the paragraph before. I
discovered when stress testing with 10k paragraphs of lorem ipsum
text, that the tiddler only displayed the first 193 paragraphs. Any
tiddler made of paragraphs would stop at 193. I think this is the
depth limitation on nesting tags on my version of firefox or
something. Firebug's html output was just this HUGE stack of
<P><P><P>.....</P></P></P>.

My solution is to instead replace multiple consecutive \n's with:
createTiddlyElement(w.output,"span").innerHTML = "<p></p>";

This makes a kludgy looking <span><p></p></span> in between proper
"paragraphs" but it delivers very consistent spacing. My custom
parser is based on txt2tags, which truncates multiple hard returns
into a new paragraph, block, etc. I tried to adapt do using <br>'s,
but making that look remotely consistent just wasn't working-
especially when you have a paragraph starting after some tag that has
its own idea of "spacing" like <h1>.

Reply all
Reply to author
Forward
0 new messages