Programmer needed

246 views
Skip to first unread message

Edward Del Grosso

unread,
Oct 11, 2015, 8:33:20 PM10/11/15
to TiddlyWiki
I have a tiddlywiki running TW 5 under node.js.   My problem is that I created about 300 graphics .tid files that force it to run too slow to be useful.  Also have a few .pdf files that I would like to add to it as individual toddlers without bogging it down.  Since I only know enough about Tiddlywiki to be dangerous it is not complicated.  I have tried numerous times over the last year or so but just lack the basic knowledge even though otherwise tiddlywiki suits my needs perfectly.

I am looking to hire someone to do the conversion to canonical url or whatever it would take to make it responsive.  I also would like versions that would work as standalone,  node.js and under my own web server.

Please feel free to contact me directly at delg...@gmail.com if you might be able to help,  along with a feasible timeline and cost.

Thanks in advance.

ED

Jeremy Ruston

unread,
Oct 12, 2015, 6:30:29 AM10/12/15
to tiddl...@googlegroups.com
Hi Edward

I have a tiddlywiki running TW 5 under node.js.   My problem is that I created about 300 graphics .tid files that force it to run too slow to be useful.  Also have a few .pdf files that I would like to add to it as individual toddlers without bogging it down.  Since I only know enough about Tiddlywiki to be dangerous it is not complicated.  I have tried numerous times over the last year or so but just lack the basic knowledge even though otherwise tiddlywiki suits my needs perfectly.

I am looking to hire someone to do the conversion to canonical url or whatever it would take to make it responsive.  I also would like versions that would work as standalone,  node.js and under my own web server.

Great stuff, I’m sorry I’m a bit busy to help you myself, but I hope someone else will be able to step up. I think there’s quite a few people here with the necessary technical skill, and of course I’d be happy to advise as I can.

I think what needs to be done here is:

* Add/update build rules to your tiddlywiki.info file for working with external images
* Update your existing image .tid files with the “external-image” tag
* Add the new .pdf files as tiddlers with the “external-image” tag

Are you working on Windows?

Best wishes

Jeremy.


Please feel free to contact me directly at delg...@gmail.com if you might be able to help,  along with a feasible timeline and cost.

Thanks in advance.

ED

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/8dcafe5c-84e2-4c93-ae1b-43be864aba60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Tobias Beer

unread,
Oct 12, 2015, 8:33:21 AM10/12/15
to tiddl...@googlegroups.com
Hi Jeremy,

A quick test tells me that --rendertiddler doesn't allow to
convert an embedded base-64 encoded tiddler back to binary, correct?

If not, what is the purpose of ContentType?

Is it possible to use --rendertiddlers with a given template to achieve that?

Best wishes,

— tb

Tobias Beer

unread,
Oct 12, 2015, 8:48:49 AM10/12/15
to TiddlyWiki
Ah, just found this under ExternalImages, specifically...

Saving Separate Image Files
The following --savetiddlers command can be used to save the images of a wiki into an images subfolder:
--savetiddlers [is[image]] images
 
I feel like all node related instructions should not be intermingled with browser features.
I would have never suspected this information to be at the ExternalImages tiddler.

Best wishes,

— tb

Jeremy Ruston

unread,
Oct 12, 2015, 8:53:18 AM10/12/15
to tiddl...@googlegroups.com
Hi Tobias

 I feel like all node related instructions should not be intermingled with browser features.

The trouble is that a lot of the information is common between the Node.js and browser configurations.

I would have never suspected this information to be at the ExternalImages tiddler.

Interesting, given that you were explicitly looking at how to implement external images!

Best wishes

Jeremy


Best wishes,

— tb

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

Tobias Beer

unread,
Oct 12, 2015, 8:56:34 AM10/12/15
to TiddlyWiki
Hi Jeremy,

the “external-image” tag

I've never noticed any such yet. What is it?

Best wishes,

— tb

Jeremy Ruston

unread,
Oct 12, 2015, 9:00:01 AM10/12/15
to tiddl...@googlegroups.com
Hi Tobias

I've never noticed any such yet. What is it?

“external-image” is the tag that is used by convention to mark external images. See:


Best wishes

Jeremy.



Best wishes,

— tb

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

Jeremy Ruston

unread,
Oct 12, 2015, 9:03:19 AM10/12/15
to tiddl...@googlegroups.com
the “external-image” tag

I've never noticed any such yet. What is it?

I should have added that my advice to anyone wanting to understand TW5 at the level of the source code is to get yourself in the position of being able to search across the source repository.

You can do so on github.com:


But for many people it’s quicker and easier to download a copy of the TW5 repo and use a text editor to search across the files.

Just to be clear, I’m not recommending that end users need to do this. This is for people who wanting to understand the source code.

Best wishes

Jeremy



Best wishes,

— tb

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

Tobias Beer

unread,
Oct 12, 2015, 9:05:54 AM10/12/15
to TiddlyWiki
Hi Jeremy,
 
The trouble is that a lot of the information is common between the Node.js and browser configurations.

I would argue node commands are irrelevant if not confusing to standalone wiki users.

Interesting, given that you were explicitly looking at how to implement external images

Mhhh, other than content, there is no reference from the node documentation to the ExternalImages tiddler yet, afaics.

All I got to wonder was how to leverage node to transform image tiddlers that are not external images but rather embedded via base-64. I hadn't even gotten to the point of wanting to output image tiddlers setting the corresponding _canonical_uri field, yet.

Mind you, the only operation I've been using so far under node pretty much is tiddlywiki --server. ;-)

Best wishes,

— tb

Jeremy Ruston

unread,
Oct 12, 2015, 9:09:57 AM10/12/15
to tiddl...@googlegroups.com
Hi Tobias

On 12 Oct 2015, at 14:05, Tobias Beer <beert...@gmail.com> wrote:

Hi Jeremy,
 
The trouble is that a lot of the information is common between the Node.js and browser configurations.

I would argue node commands are irrelevant if not confusing to standalone wiki users.

We’ve discussed this before. One of the points I made was that actually Node.js is pretty straightforward to end-users. If we hide the Node.js docs off somewhere else we’ll be adding to the impression that Node.js is something esoteric for advanced users, and I don’t think that’s true.

Interesting, given that you were explicitly looking at how to implement external images

Mhhh, other than content, there is no reference from the node documentation to the ExternalImages tiddler yet, afaics.

We're always open to improvements to the docs. Simple tagging “ExternalImages” so that it appears in the TOC under Node.js might be a start.

All I got to wonder was how to leverage node to transform image tiddlers that are not external images but rather embedded via base-64. I hadn't even gotten to the point of wanting to output image tiddlers setting the corresponding _canonical_uri field, yet.

To answer your question from earlier, to save raw image tiddlers under Node.js you’ll need the savetiddler(s) commands, not rendertiddler(s) (the latter always parse and render their payload).

Mind you, the only operation I've been using so far under node pretty much is tiddlywiki --server. ;-)

Great stuff,

Best wishes

Jeremy


Best wishes,

— tb

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

Tobias Beer

unread,
Oct 12, 2015, 9:19:19 AM10/12/15
to TiddlyWiki
Hi Jeremy,
 
Just to be clear, I’m not recommending that end users need to do this. 
This is for people who wanting to understand the source code.

Mhhh, that's not quite correct.
For anyone wanting to get started and use TiddlyWiki under node,
  1. there's not necessarily a need to be a (node) developer
  2. there's a lot to be learned from the patterns used for all things around the core distro
Sure, a term like "build" is possibly not for grandma,
but then that's what we need to define in order to, well, build the kind of output
we want from a bunch of tiddlers in our node folders.

Best wishes,

— tb

Jeremy Ruston

unread,
Oct 12, 2015, 9:24:48 AM10/12/15
to tiddl...@googlegroups.com
Hi Tobias


I’m not quite sure what you’re getting at here.

If you’re arguing that some end users will benefit from perusing the source code, maybe I’d agree. But I still feel that we’ll have failed if end users are obliged to inspect the source code.

Best wishes

Jeremy



Best wishes,

— tb

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

Tobias Beer

unread,
Oct 12, 2015, 9:29:02 AM10/12/15
to TiddlyWiki
Hi Jeremy,
 
We’ve discussed this before. One of the points I made was that actually Node.js is pretty straightforward to end-users. If we hide the Node.js docs off somewhere else we’ll be adding to the impression that Node.js is something esoteric for advanced users, and I don’t think that’s true.

Not suggesting to remove node related info form TiddlyWiki.com,
just not to mingle node things with basic wiki topics too much.
Of course, ExternalImages should reference a tiddler to explain "Creating external images under Node.js".
  
To answer your question from earlier, to save raw image tiddlers under Node.js 
you’ll need the savetiddler(s) commands, not rendertiddler(s) 
(the latter always parse and render their payload).

Sorry for not quite been up to speed yet on all things node.
Always wanted to go through the node commands and test them out.
I've only today got to simply unzipping the repo somewhere that is not my fork
and run some commands against editions folders to see what happens.
Well, the things we want to do. :D

Best wishes,

— tb

Tobias Beer

unread,
Oct 12, 2015, 9:38:00 AM10/12/15
to TiddlyWiki
Hi Jeremy,
 
If you’re arguing that some end users will benefit from perusing the source code, maybe I’d agree. 
But I still feel that we’ll have failed if end users are obliged to inspect the source code.

In any case, the "external-image" tag handling in the build process is highly interesting to node users,
along with any possible ways for batch conversion / migration, mostly from base-64 to binary,
but possibly also the other way around to package external resources
into one standalone distro wiki w/o further dependencies.

Best wishes,

— tb

Mark S.

unread,
Oct 12, 2015, 11:57:02 AM10/12/15
to TiddlyWiki
Just some thoughts.

For anyone looking into this, reminder that there's a problem with using the "export" (I call it an export when you move embedded pictures into a separate file directory) and the _canonical_uri function under node.js

The export process is fairly straightforward. BUT, node.js can't serve up external images. To serve up the external images you need a second server that knows how to handle images. AFAIK, the two servers have to run at a different port or a different address. Which means that simple canonical addresses won't work. Instead you'll need to alter the $:/core/templates/canonical-uri-external-image tiddler to create a long path for the image (e.g. http://127.0.0.1:8084/myimage.jpg).

You would need to change the paths for all images anytime you change image server address or port, or if you create a stand-alone version.

Remember also that if some of the _canonical_uri's tiddlers already have a custom path inside the existing file structure they will be overwritten by the externalizing process. In that case, you might need to set up an additional flag field and filter to prevent those custom tiddlers from being over-written.

For these reasons, it might be preferable to abandon _canonical_uri entirely and display images via a global macro. The macro would provide the path infrastructure and could be quickly changed depending on how the TW is being used (stand-alone, local server, remote server). Different macros could be used for custom images outside the standard file paths.

Mark

Tobias Beer

unread,
Oct 13, 2015, 5:03:50 AM10/13/15
to tiddl...@googlegroups.com
Hi Mark,

Thanks for those very important details,
 
if some of the _canonical_uri's tiddlers already have a custom path inside the existing file structure they will be overwritten by the externalizing process. In that case, you might need to set up an additional flag field and filter to prevent those custom tiddlers from being over-written.

I have trouble understanding what you mean by this...
  1. What is the "externalizing process"?
  2. What gets overwritten doing what exactly?
  3. How and when and where use a "flag field / filter"?
For these reasons, it might be preferable to abandon _canonical_uri entirely and display images via a global macro. The macro would provide the path infrastructure and could be quickly changed depending on how the TW is being used (stand-alone, local server, remote server). Different macros could be used for custom images outside the standard file paths.

I very much agree, which is why there already is this:


The current `_canonical_uri` method appears a bit problematic in more than one way,
starting with the naming convention for that field.
Perhaps it's not the worst idea to deprecate this field and how it works today.

Best wishes,

— tb

Jeremy Ruston

unread,
Oct 13, 2015, 5:27:42 AM10/13/15
to tiddl...@googlegroups.com
Hi Tobias

The current `_canonical_uri` method appears a bit problematic in more than one way,
starting with the naming convention for that field.
Perhaps it's not the worst idea to deprecate this field and how it works today.

Correct me if I’m wrong, but doesn't all of the confusion around the _canonical_uri field stem from the limitations imposed by browsers? In all of the discussion I don’t think there’s been any proposals for anything that we could do differently that would meaningfully impact the limitations.

Best wishes

Jeremy.



Best wishes,

— tb

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

Tobias Beer

unread,
Oct 13, 2015, 7:02:31 AM10/13/15
to TiddlyWiki
Hi Jeremy,
 
I don’t think there’s been any proposals for anything that we could do differently that would meaningfully impact the limitations.

I have opened an issue on GitHub with a proposal...

#2016 routes to external resources

Best wishes,

— tb 

Mark S.

unread,
Oct 13, 2015, 10:42:54 AM10/13/15
to TiddlyWiki
Hi Tobias,

I'm referring to the "Externalising Image Tiddlers" section of the "ExternalImages" tiddler on tiddlywiki.com. It tells you how to "export" (externalize) images from TW, and how to reset the _canonical_uri field to the new directory of the external images. As part of process, you specify a filter [is[image]]. If you follow the directions as written, ALL images will have their _canonical_uri field overwritten, even if they have some custom path (to an image file outside that of the externalized images, or maybe even a web site). So you need a more complicated filter than [is[image]] and/or a way to mark those image tiddlers whose _canonical_uri paths should not be rewritten. I was suggesting an additional field (flag field) to separate image tiddlers that can have their _canonical_uri fields overwritten by the externalizing and those that can not.

HTH
Mark

Mark S.

unread,
Oct 13, 2015, 11:09:50 AM10/13/15
to TiddlyWiki
Hi Jeremy,

Starting with the naming convention, why is this field called "_canonical_uri" and not something friendlier and easier to type like "imagepath" ?

It seems to me that the _canonical_uri field and image works OK once you have it set up. The problem for Windows people is figuring out how the browser wants path to files that are not subdirectories of the TW.

The first actual problem is that node.js doesn't serve up images. Suddenly everything gets messy and you end up with long paths to images on a different server address that may have to change whenever the computer reboots.

The second problem is that there is no way to simply set a base address for all _canonical_uri images. Instead, you have to overwrite the _canonical_uri field of all the images every time you change servers, convert to stand-alone, or move the images. If there was a way to set a base address, then a user could simply change a configuration tiddler to point to the new base, and all the images would be available without having to write new paths.

Thinking about it more, it would be good if different sets of tiddlers could reference different base paths. So the image tiddlers might have an additional field "imageconfig" that would specify the name of the tiddler containing the base path for the tiddler.

This would solve another problem. Currently, if you want to use the externalizing process, all the files will end up in one directory. But someone like Edward, with 300 images, is likely to want to move them into sub-directories. Currently each image in a subdirectory would  need the relative path added to its title. So instead of "My_Little_Ponies", you end up with "XYZ201510/My_Little_Ponies". Not quite as friendly. You could of course have a _canonical_uri that points to a different place without changing the tiddler title, but then you could never run the external process without overwriting the custom _canonical_uri fields.

Thanks!
Mark

Jeremy Ruston

unread,
Oct 13, 2015, 1:39:36 PM10/13/15
to tiddl...@googlegroups.com
Hi Mark

Starting with the naming convention, why is this field called "_canonical_uri" and not something friendlier and easier to type like "imagepath" ? 

The field was first introduced in TiddlyWeb, and then adopted by TiddlyWiki5. It’s accidental that it has a unique naming convention, but not necessarily inappropriate in a way. The core logic to support the _canonical_uri field is much more deeply intertwined that any other field apart from the title and text fields.

It seems to me that the _canonical_uri field and image works OK once you have it set up. The problem for Windows people is figuring out how the browser wants path to files that are not subdirectories of the TW.

The first actual problem is that node.js doesn't serve up images. Suddenly everything gets messy and you end up with long paths to images on a different server address that may have to change whenever the computer reboots.

I agree that some limited static file serving facilities would be useful. There’s a ticket here:


The second problem is that there is no way to simply set a base address for all _canonical_uri images. Instead, you have to overwrite the _canonical_uri field of all the images every time you change servers, convert to stand-alone, or move the images. If there was a way to set a base address, then a user could simply change a configuration tiddler to point to the new base, and all the images would be available without having to write new paths. 

Tobias has made a proposal for one approach to doing this:


Thinking about it more, it would be good if different sets of tiddlers could reference different base paths. So the image tiddlers might have an additional field "imageconfig" that would specify the name of the tiddler containing the base path for the tiddler.

Yes, Tobias’ approach calls it a “route”.


This would solve another problem. Currently, if you want to use the externalizing process, all the files will end up in one directory. But someone like Edward, with 300 images, is likely to want to move them into sub-directories. Currently each image in a subdirectory would  need the relative path added to its title. So instead of "My_Little_Ponies", you end up with "XYZ201510/My_Little_Ponies". Not quite as friendly. You could of course have a _canonical_uri that points to a different place without changing the tiddler title, but then you could never run the external process without overwriting the custom _canonical_uri fields.

Good point,

Best wishes

Jeremy


Thanks!
Mark




On Tuesday, October 13, 2015 at 2:27:42 AM UTC-7, Jeremy Ruston wrote:
Hi Tobias

The current `_canonical_uri` method appears a bit problematic in more than one way,
starting with the naming convention for that field.
Perhaps it's not the worst idea to deprecate this field and how it works today.

Correct me if I’m wrong, but doesn't all of the confusion around the _canonical_uri field stem from the limitations imposed by browsers? In all of the discussion I don’t think there’s been any proposals for anything that we could do differently that would meaningfully impact the limitations.

Best wishes

Jeremy.



Best wishes,

— tb

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/571e911c-023a-4783-ba16-be170d2986f2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

Tobias Beer

unread,
Oct 14, 2015, 2:34:12 AM10/14/15
to tiddl...@googlegroups.com
Hi Mark,
 
I'm referring to the "Externalising Image Tiddlers" section of the ExternalImages tiddler on tiddlywiki.com...

It's is slowly dawning on me now. ;-)

It appears that it is not a widget that needs to specify what route an image tiddler specifying a _canonical_uri is to use (or, perhaps only as an override), but rather that it is the image tiddler itself that needs to qualify the route / base-url / path to which they are to be resolved, unless qualified with a full url (at which point an externalizing process should never touch them).

Then, to point to a different location all that needs changing is the tiddler that configures the path for any given route.

Best wishes,

— tb
Reply all
Reply to author
Forward
0 new messages