[TW5] The best way to include PDFs

[Peter Smillie]

Hi,

I'm using TW5 with node.js. So far, I have been dragging and dropping PDFs to create appliation/pdf tiddlers. The issue with these is that they are slow to load into 'edit' mode, since they load a huge text file of gibberish. This is a problem mainly when I'm trying to add a tag.

It seems like there could be a lot of solutions for this; my favorite would be to link to pdf files using something like [img[photo.jpg]]. But I also wouldn't mind either A) a way to tag tiddlers without clicking 'edit' or B) a way to edit application/pdf tiddlers without opening the entire contents of the tiddler.

I would grateful for any tips!

Thanks,

Peter

[Xavier Cazin]

Hi Peter,

Not sure it's the best way, but with the node.js mode you may want to take advantage of decoupling content and metadata. Just move the PDF files into a subdirectory of tiddlers/ and create a .meta file from each PDF filename,  that you can then transclude in a tiddler that is easier to handle. Like so:

x2:server xavier$ ls tiddlers/invoices/
April 2014 invoice.tid            Invoice from supplier X.pdf.meta
Invoice from supplier X.pdf

x2:server xavier$ cat tiddlers/invoices/Invoice\ from\ supplier\ X.pdf.meta
title: $:/mypdfs/invoices/201404
type: application/pdf

x2:server xavier$ cat tiddlers/invoices/April\ 2014\ invoice.tid
created: 20140501163534829
modified: 20140501163640138
tags: [[pdf invoices]] due
title: April 2014 invoice
type: text/vnd.tiddlywiki

{{$:/mypdfs/invoices/201404}}

x2:server xavier$

Granted, it involves 3 files for each PDF, but it can certainly be scripted if you have a lot of pdfs.

Xavier Cazin.

-- Xavier Cazin

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[Peter Smillie]

Hi Xavier,

That's wonderful, thank you. If I may summarize, the brilliant solution is to make the file itself a shadow tiddler and then transclude it; the additional step of separating the pdf from the metadata is just good practice.

Peter

[Xavier Cazin]

Hi Peter,

I think maintaining a metadata file in this case is a bit more than good practice, because you also want to control the mimetype so that browsers know what you're actually transcluding.

Best,
Xavier.

-- Xavier Cazin

[Peter Smillie]

Thanks again Xavier for the solution.

I have two almost-related follow-up questions. One, I'm having no trouble with small one- or two-page pdfs, but trying to open a 2 MB 6-page pdf crashes my browser immediately (Chrome gives me "Aw snap!"). Is that to be expected? Is there a fix?

Two, it seems to me that lazy image loading has no effect on pdfs - is there a tweak to make pdfs load lazily as well?

Thanks,

Peter

[Xavier Cazin]

Hi Peter,

This was to be expected, I'm afraid :-)  You might try to add  -[type[application/pdf]] to the saveTiddlerFilter() macro of $:/core/save/lazy-images, but I don't think there is a reasonable workaround today, other than opening the resource file outside Tiddlywiki, for instance from a file:// href.

Regards,
XC.

-- Xavier Cazin

[andrew.j.harrison84]

What I do is include the pdf file in a framed window using a transcluded template and to make it searchable, I include the text from the pdf file compressed into a metafile comment block of text. Inside <--! and -->. My tiddlywiki has over 1000 tiddlers and still is small enough to be fast.

Sent from my Samsung Epic™ 4G Touch

[Peter Smillie]

The text comment is smart! Does your method let you view larger PDFS (like, > 2MB)? Can you share the template you use?

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/uzJK6mNQg6U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.

[Keith Ganey]

Peter,

I'm running lazy image loading on Node as well and it works on PDFs for me. Have you looked at the source code of your wiki to see if the gibberish for each PDF is in there?  If it is not cluttered with the gibberish, then the lazy loading is working. I agree that loading is a little slow.

Regarding the 2mb limit on PDFs, I just tested it and found the same problem. I expect that the real problem is the number of characters in the <textfield> html element. (See, for example, this discussion on different browser limits.) So if we could get intelligent editing for PDFs the way that images were changed to bitmap editing in 5.0.10 the problem might sort itself out.

Back dooring the PDFs via the meta method suggested by Xavier may avoid this problem. Does it also crash when just transcluding the PDFs?

Keith

[Danielo Rodríguez]

That way the PDF is not loaded into the HTML file?

Can you share your template? How do you generate the metafile? Is this under node?

Thanks in advance.

[Jeremy Ruston]

For 5.0.11 I've added a warning banner when attempting to edit binary tiddlers such as PDFs (see the image below).

A big problem with the performance of embedded PDFs is that they are rendered via conversion to a base64 data URI. That means we can hit the browser URL side limit (which I suspect is the cause of the 2MB file problem).

The plan is to add support for image tiddlers (including PDFs) to have a "_canonical_uri" which contains the (external) URL of the image. Then when rendering it with [img[myPDFtiddler]] the PDF is referenced remotely via the URL.

Best wishes

Jeremy

Jeremy Ruston
mailto:jeremy...@gmail.com

[Peter Smillie]

Hi Keith,

Yes, I've been transcluding the PDFs using Xavier's meta method, and it still crashes.

Also, it didn't occur to me to check if lazy loading was working by looking at the source. Thanks for the tip. But in fact my html includes the base64 PDFs. I tried looking at the source in Chrome and downloading a static copy, and I tried adding a [type[application/pdf]] in the natural place in the $:/core/save/lazy-images tiddler, but I always got gibberish. 

Best,

Peter

[Keith Ganey]

Danielo and Peter,

Keeping the PDFs out of the TiddlyWiki source is all done at the Node command. I've found that I have to specify all the different options when starting TiddlyWiki for it to work.

So my command to launch TiddlyWiki on node looks a little like:

tiddlywiki wikifoldername --server port-number "$:/core/save/lazy-images" "text/plain" "text/html" username password ipaddress

Filled in with example values:

tiddlywiki worknotebook --server 3000 "$:/core/save/lazy-images" "text/plain" "text/html" keith 45ZXqmyf 127.0.0.1

Specifying fewer than all those items did not work for me.

Peter,

When you save a local copy of the TiddlyWiki, then I think it pulls the images and PDFs into the source so you have a complete copy of the wiki even off-line. 

Thanks for letting me know that transcluding wasn't going to be a valid work around. Sounds like we'll just have to use other tools for now.

Keith