TW5 node.js Running out of memory

[David Gifford]

Hi all


In node.js, using 5.0.6 beta, Windows 8.0, Firefox, I am starting to get the message below. Not sure what memory it is out of. This is a brand new computer with lots of memory. Tried clearing the browser history, etc, and that hasn't helped. Amazingly, though, the tiddlers did manage to save, despite the error message. Any ideas? I have just over 2000 tiddlers, many of them are image tiddlers. And I tweaked the tiddlerinfo panel so that there is an extra tab that displays the title, tags and text of all tiddlers tagged with that tiddler's title.

Here is the text of the error:

Internal JavaScript Error

Well, this is embarrassing. It is recommended that you restart TiddlyWiki by refreshing your browser

uncaught exception: out of memory

[David Gifford]

This is a pretty important question for me. If I don't get this resolved, node.js will not be worth investing my time and effort in.

Dave

[Jeremy Ruston]

Hi Dave

It's actually not Node.js that's running out of memory here, it's the browser. It's not normal behaviour, though, and I've not seen anything like this reported before. It's certainly not something that's associated with running Node.js.

You say that you've got 2000 tiddlers in there with many images, perhaps it's just that you're generating a sufficiently large TW file to confuse the browser.

If you save the wiki as a standalone HTML file, how big is it?

tiddlywiki --rendertiddler $:/core/save/all index.html text/plain

Another possibility is that you've inadvertently set up an infinite loop - TW tries to protect you from circular transclusion, but if you somehow managed it, the symptom would likely be an out of memory error.

Best wishes

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/groups/opt_out.

--
Jeremy Ruston
mailto:jeremy...@gmail.com

[David Gifford]

I thought that running it as node.js eliminated the issue of filesize. The tiddlers are separate files that are loaded when one opens the link, right? Or are all those tiddlers actually opened by the browser in the moment one opens the browser?

Dave

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/dIrdnKRfmj8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/groups/opt_out.

--
David Gifford
Christian Reformed World Missions, Mexico City

[David Gifford]

145MB is the filesize. And Firefox gave me an error when I tried to click on it.

Dave

On Tue, Jan 28, 2014 at 2:05 PM, Jeremy Ruston <jeremy...@gmail.com> wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/dIrdnKRfmj8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/groups/opt_out.

--

[David Gifford]

Still trying to drive trucks over your bridges, Jeremy! But seriously, that was my whole motivation for using node.js, because I thought that it saved the tiddlers as separate files and loaded them as needed, thus reducing filesize. But if node.js has the same limitation as standalone, then I am not sure what the advantage is, other than playing around with putting tiddlers in folders?

Dave

[Jeremy Ruston]

Hi Dave

At the moment, running under Node.js is more about having the convenience of managing your tiddlers as individual files. When a TW is built it includes a representation of all tiddlers, whether it is built for serving to the browser via tiddlywiki --server, or built as a standalone HTML file.

The reason that Firefox is crashing is because all of those images are packed into the TiddlyWiki file as base64 tiddlers. Unpacking and displaying them will be much less efficient than working with an ordinary independent image file.

TiddlyWiki5 does support lazy loading, where the original TiddlyWiki file is built with "skinny tiddlers" that omit the body, but include the metadata (tags etc), and then the body is loaded on demand. Lazy loading is currently only implemented when running on TiddlyWeb/TiddlySpace, but the plan is to support it natively too.

What's needed is a way of specifying tiddlers within a wiki folder that should be served skinny. For example, a new filter operation like "[is[image]]".

For better performance with images, it would be useful to be able to render images as ordinary <img src=...> tags, rather than base64.

I'll investigate some quick hacks that will get things working for you,

Best wishes

Jeremy

[David Gifford]

Hi Jeremy, that is somewhat bad news for me...I guess I will await your 'hacks'...

Question: if I do <img src=... can I do a partial filepath and store the images in a folder nearby? Or would I need the entire filepath? I would prefer to do the partial path so that if I transfer to a different computer in the future I won't have to redo filepaths

Dave

[Jeremy Ruston]

On Tue, Jan 28, 2014 at 8:23 PM, David Gifford <dgif...@crcna.org> wrote:

Hi Jeremy, that is somewhat bad news for me...I guess I will await your 'hacks'...

Question: if I do <img src=... can I do a partial filepath and store the images in a folder nearby? Or would I need the entire filepath? I would prefer to do the partial path so that if I transfer to a different computer in the future I won't have to redo filepaths

You can't get <img src> to work at the moment unless you can arrange for your image folder to be served over http. Unfortunately, browsers won't let you reference images on a file: URI from within a webpage that has been served over HTTP.

We need to get the TW server serving the tiddlers as separate image files.

Best wishes

Jeremy

[David Gifford]

Hi Jeremy,

Last question, I promise! If you wanted to store images online and not have those images inaccessible to the public and searches, yet use a TiddlyWiki to access them, what would you do to set it up?

Dave

[PMario]

On Tuesday, January 28, 2014 8:25:10 PM UTC+1, David Gifford wrote:

145MB is the filesize. And Firefox gave me an error when I tried to click on it.

Hi Dave,
Pushing it to the limits. That's cool :)

Did you optimize your images, to be used with your "app"?
eg:
they have a resolution, that is good enough to be displaied in a browser, but the browser dosn't need to resize them.
So there should be no CSS rules for image "width and height"

-m

[PMario]

I did a short test.
A 108kByte jpg file will end up in 145kByte base64 encoded text file.
-m

[David Gifford]

Hi Mario

I am just doing it for me not an adaptation this time around, so I left the images as is.

Dave

--

You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/dIrdnKRfmj8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/groups/opt_out.

[PMario]

On Wednesday, January 29, 2014 12:56:24 AM UTC+1, David Gifford wrote:

I am just doing it for me not an adaptation this time around, so I left the images as is.

ahh, ok.

Working with images is a big issue for web devs at the moment because of "high resolution displays" on some devices and low res (640 x 480) on others. To satisfy both worlds, there isn't a good solution yet.

I'd be interested, if one of your 140mb TWs would load on an ipad air, or if it would turn green and explode :)

-m

[David Gifford]

Well, it wouldn't open in my new Sony Vaio Fit 15....

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/dIrdnKRfmj8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/groups/opt_out.

[Jeremy Ruston]

I've got the basics of lazy loading working natively with TW5 for 5.0.8 with a demo that lazily loads images, but treats other tiddlers normally. The images are still being loaded as base64 data URIs, rather than referencing the image files directly on the server. But it still seems to work pretty well.

Best wishes

Jeremy

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/groups/opt_out.

--
Jeremy Ruston
mailto:jeremy...@gmail.com

[David Gifford]

Thanks Jeremy! Can't wait to try it out! Does it work with standalone, node.js, or both?

Dave

[Jeremy Ruston]

Hi Dave

Lazy loading just applies to the client-server edition - it is in effect a refinement of the interface between the browser and server, so it's the two components acting in concert.

Best wishes

Jeremy

[Jeremy Ruston]

Hi Dave

On Tue, Jan 28, 2014 at 8:46 PM, David Gifford <dgif...@crcna.org> wrote:

Last question, I promise! If you wanted to store images online and not have those images inaccessible to the public and searches, yet use a TiddlyWiki to access them, what would you do to set it up?

Did an extra "not" squeeze in there? If you mean storing images online without them being accessible to the public and searches, then I guess you just need some storage that is password protected.