TW too big

44 views
Skip to first unread message

Oscar

unread,
Apr 7, 2007, 4:03:05 AM4/7/07
to TiddlyWiki
Hi to everyone!
I use TW like a blog and I like it; but there is a problem: after one
year it growed its size and now it's near to 1MB; someone of my
readers has not a broadband, and opening my blog seems more hard any
day that passes.
I wouldn't change my blog to a common platform like wordpress or
blogger; but I'm searching a TW-based engine that works better for
blogging.
I saw that exist cctiddlywiki, phptiddlywiki and other platforms.
Do you know wich one is the simplest, fast and that permitt me to
import my posts and that supports plugins?

Thank you very much and pass a good Easter (or a good week-end,
anyway :-) )
Oscar
http://corax.altervista.org

jroeterd

unread,
Apr 7, 2007, 10:03:53 AM4/7/07
to TiddlyWiki
Or make a seperate TW for each archive month.

Greetings,
Jar

google....@dfgh.net

unread,
Apr 7, 2007, 10:23:28 AM4/7/07
to Tiddl...@googlegroups.com
cctiddly support import and some of the plugins but it does not have load on demand yet, the function that solves your problem

CC

Ken Girard

unread,
Apr 8, 2007, 6:23:53 PM4/8/07
to TiddlyWiki
Take a look at the IncludePlugin (http://tiddlywiki.abego-software.de/
#IncludePlugin).
It lets you 'include' tiddlers from other TWs by loading them
temporarily into your main TW.
An example would be to move all the tiddlers from the first 6 months
of your blog and put them in a different TW. The include plugin then
loads them later in the process, so that a visitor gets to see
something faster.
The one thing that I noticed doesn't work well is permalink as it
tries to open the tiddler before the tiddler is actually loaded.

Also, great looking TW.

Ken Girard

FND

unread,
Apr 9, 2007, 2:35:15 AM4/9/07
to Tiddl...@googlegroups.com
> Take a look at the IncludePlugin (http://tiddlywiki.abego-software.de/
> #IncludePlugin).
> It lets you 'include' tiddlers from other TWs by loading them
> temporarily into your main TW.

But IncludePlugin doesn't support load on demand (yet?) as far as I know.
That is, if include tiddlers from another TW, it will always load them
with the core TW, which results in the same filesize and loading time.
I might be wrong though... !?


-- F.

Udo Borkowski

unread,
Apr 9, 2007, 4:48:32 AM4/9/07
to Tiddl...@googlegroups.com
That is, if include tiddlers from another TW, it will always load them
with the core TW, which results in the same filesize and loading time.
If you add the filesize of the core TW and the included TW you will get roughly the same filesize as the "big one". If you add the load and include time you will also get the same time, possibly even a little more. But you may make it "feel faster" by using the "delay:" parameter
<<include "file://C:/Private/journals/2006-12.html" delay: 8000 >>
In this case the file is included 8000 ms (8 seconds) after TiddlyWiki is loaded. So the user can already start working with TiddlyWiki after the (smaller) core TW is loaded. After a while other files will access become available.

Another advantage of the include feature is: "Saving" gets faster since only the (smaller) core TW file needs to be saved after a change.

Udo

Oscar

unread,
Apr 9, 2007, 4:53:12 AM4/9/07
to TiddlyWiki
Thanks to everybody.
I'll try to use the includePlugin.
Fingers crossed ;-)


On Apr 9, 10:48 am, "Udo Borkowski" <udo.borkow...@googlemail.com>
wrote:


> > That is, if include tiddlers from another TW, it will always load them
> > with the core TW, which results in the same filesize and loading time.
>
> > If you add the filesize of the core TW and the included TW you will get
>
> roughly the same filesize as the "big one". If you add the load and include
> time you will also get the same time, possibly even a little more. But you
> may make it "feel faster" by using the "delay:" parameter
>
> <<include "file://C:/Private/journals/2006-12.html" delay: 8000 >>
>
> In this case the file is included 8000 ms (8 seconds) after TiddlyWiki is
> loaded. So the user can already start working with TiddlyWiki after the
> (smaller) core TW is loaded. After a while other files will access become
> available.
>
> Another advantage of the include feature is: "Saving" gets faster since only
> the (smaller) core TW file needs to be saved after a change.
>
> Udo
>

FND

unread,
Apr 9, 2007, 4:54:20 AM4/9/07
to Tiddl...@googlegroups.com
> But you may make it "feel faster" by using the "delay:" parameter

Good point, Udo - I hadn't thought of that!

> Another advantage of the include feature is: "Saving" gets faster since
> only the (smaller) core TW file needs to be saved after a change.

True - especially since it's unlikely that Oscar will retroactively
change any of the older blog entries, which aren't stored in the core
TW. (Other, less-linear/chronological TWs might be more problematic.)

While we're at it though: Is there any chance that load-on-demand
features will be available sometime in the foreseeable future? Or would
that require major changes in the TW core code?


-- F.

Udo Borkowski

unread,
Apr 9, 2007, 11:01:16 AM4/9/07
to Tiddl...@googlegroups.com
I think it is not very likely "load-on-demand" will be available soon because of the internal structure of TiddlyWiki. Things would be easier if the Tiddler's properties (like "text", "tags" etc) are always accessed through functions. But currently a lot of code, including plugin code, directly access these properties/variables (look for code like " tiddler.text" or "tiddler.tags"). So changing to a "always use a function to access a property" approach will probably lead to a lot of incompatibilities with existing (plugin) code.

Also: because of the current user interface (especially the "sidebar") this "load on demand" will probably happen fairly quickly after startup (for nearly every tiddler).  E.g. if the "Timeline" tab is open at least all titles, modifiers, mod-dates of tiddlers must be loaded immediately. When the "Tags" tab is selected, all tiddlers' tags must be loaded and for tabs like "Orphans" or "Missing" the complete tiddlers (esp. its texts and titles) must be loaded to check for orphan or missing tiddlers.

Udo

----------
Udo Borkowski
http://www.abego-software.de





On 4/9/07, FND <Ace_...@gmx.net> wrote:

FND

unread,
Apr 9, 2007, 1:07:45 PM4/9/07
to Tiddl...@googlegroups.com
> I think it is not very likely "load-on-demand" will be available soon
> because of the internal structure of TiddlyWiki.

I see - thanks for clearing that up.
(+5 Insightful: 70% Insightful, 30% Interesting ;) )


-- F.

Andrew Lister

unread,
Apr 9, 2007, 3:46:46 PM4/9/07
to TiddlyWiki
Udo's reply seems to suggest a change in policy or a difference in
opinion since the comments by Martin and Jeremy in this thread, from
Mar. 7, 8:

http://groups.google.com/group/TiddlyWiki/browse_frm/thread/64dbf9b3e4b09e69/47c91c9c60f3dc0e?lnk=gst&q=load-on-demand+lister&rnum=1#47c91c9c60f3dc0e

Has there been a change, or is there just some uncertainty or
disagreement about how things will / should evolve, or did I misread
those original posts?

Andrew

Jeremy Ruston

unread,
Apr 9, 2007, 5:09:36 PM4/9/07
to Tiddl...@googlegroups.com
We've got load-on-demand almost working in the latest betas for 2.2.

The new feature is kicks in when a link to a missing tiddler is
clicked. If the tiddler hosting the link is synced to a particular
server, then an attempt is made to asynchronously load the missing
tiddler from that server. There is also a mechanism to allow missing
tiddlers to be resolved when the host tiddler isn't synced.

One usage scenario is to pull in a single article from Wikipedia.
Clicking on links to other Wikipedia articles will cause the target
article to be automatically loaded.

Right now, the mechanism isn't super efficient in some scenarios - for
instance, if you use it to load tiddlers from an ordinary TiddlyWiki
file (whether loaded from file:// or http://), then the file is
reloaded afresh each time a missing tiddler is resolved. We'll get
that sorted out though.

Cheers

Jeremy.


--
Jeremy Ruston
mailto:jer...@osmosoft.com
http://www.tiddlywiki.com

Udo Borkowski

unread,
Apr 10, 2007, 3:31:31 AM4/10/07
to Tiddl...@googlegroups.com
OK, now I understand what you mean by "load on demand". Thanks for the clarification.

When I talk about "load on demand" I have in mind the vision a user should not see any difference between "normal" tiddlers and "load on demand" tiddlers (beside some delay when a tiddler is loaded "on demand"). E.g. a link to a "load on demand" tiddler should look like a normal link and not like a "missing" tiddler link. The "Timeline" should contain both kinds of tiddlers, a search should also return "load on demand" tiddlers etc.. I assume such a "seemless integration" of "load on demand" will require a lot of work and is beyond the scope of TW 2.2 (and even 2.3, I guess)

Nevertheless your new mechanism is definitely a great way to make TiddlyWiki files smaller and shorten initial load times. Looking forward to see the first applications.



Udo

----------
Udo Borkowski
http://www.abego-software.de





Jeremy Ruston

unread,
Apr 10, 2007, 5:56:02 AM4/10/07
to Tiddl...@googlegroups.com
Yeah, that's right. I wanted a design that would scale to very large
populations of tiddlers, and of course that constraint makes it
impractical to do the 'phantom tiddler' approach that we've discussed
before.

Cheers

Jeremy.

Andrew Lister

unread,
Apr 10, 2007, 10:28:19 AM4/10/07
to TiddlyWiki
Thank you, that's very helpful to have that explained. I have one
further question. I can see, intuitively, that load-on-demand for
links to missing tiddlers can accommodate an arbitrarily large number
of such tiddlers, because no information about the tiddler is loaded
until the missing link is clicked. In contrast, the 'phantom tiddler'
approach involves loading all of the metadata (tags, modification
date, etc.) for all tiddlers in advance, and then the body only on
demand. Still, would the phantom tiddler approach not permit largER
documents than are currently feasible? I suppose it depends on how
load-time works. If it's a linear function of the number of
characters, say, then not having to load the body of the tiddler could
be a big time saving, if each tiddler typically contains a good-size
paragraph. On the other hand, if there is significant overhead load-
time for each field, and the tiddler body is just one extra field, and
the size of the tiddler body isn't that important, then loading the
body on demand won't make a big difference.

My reason for wanting to understand this issue, and the likely course
of future developments, is that I am interested in being able to
accommodate large but not indefinitely large amounts of tiddlers and
text.


Jeremy Ruston

unread,
Apr 10, 2007, 11:00:40 AM4/10/07
to Tiddl...@googlegroups.com
I'm thinking about extending the current missing tiddler-based
mechanism to cover the phantom tiddler scenarios. The idea would be
precisely as you say: to be able to preload the metadata for a bunch
of tiddlers whose actual content is only loaded when needed.

Internally, the feature would be based on the convention that if the
'text' property of a Tiddler object were undefined, then the tiddler
would be either unloaded or in the process of being loaded, with a
further member 'loadStatus' tracking the precise loading status of the
tiddler.

On an attempt to display a tiddler that hasn't yet been loaded, an
asynchronous task would be kicked off to load the necessary data from
the server.

This is a slightly different approach than that proposed by Udo
earlier in this thread, because it requires code that access the
'text' property of tiddlers to be aware of the new behaviour. At first
sight, it seems that if we had enforced access to the 'text' property
via an accessor method, we could have seamlessly upgraded to the new
behaviour. Sadly, in practice, the new behaviour is asynchronous and
it is not in general possible or desirable to change an existing
synchronous function to asynchronous behaviour without breaking
dependent code.

Cheers

Jeremy

Oscar

unread,
Apr 15, 2007, 5:15:36 AM4/15/07
to TiddlyWiki
I used the includeplugin.
Now I'm in a testing phase, but it seem working.
Some little problem with permalink; but I think you resolved my
problem.
Thanmk you very much to all!

On 9 Apr, 10:48, "Udo Borkowski" <udo.borkow...@googlemail.com> wrote:
> > That is, if include tiddlers from another TW, it will always load them
> > with the core TW, which results in the same filesize and loading time.
>
> > If you add the filesize of the core TW and the included TW you will get
>
> roughly the same filesize as the "big one". If you add the load and include
> time you will also get the same time, possibly even a little more. But you
> may make it "feel faster" by using the "delay:" parameter
>
> <<include "file://C:/Private/journals/2006-12.html" delay: 8000 >>
>
> In this case the file is included 8000 ms (8 seconds) after TiddlyWiki is
> loaded. So the user can already start working with TiddlyWiki after the
> (smaller) core TW is loaded. After a while other files will access become
> available.
>
> Another advantage of the include feature is: "Saving" gets faster since only
> the (smaller) core TW file needs to be saved after a change.
>
> Udo
>

Reply all
Reply to author
Forward
0 new messages