batch import/load images without filepath in title

[Donald Coates]

I'm beating my head against a wall on this one.  I want to move a directory of images into my wiki via the commandline.  I know I can download them then drag and drop but I would rather just do it all on the nodejs server.

using the load command ends up with a title that includes the entire path name.  Is there any way to get rid of that or some other way I'm just not seeing?

Thank you in advance!

[TonyM]

DOnald,

I am not so well versed on the server, but before you abandon drag and drop have a look at using the browse widget with `<$browse multiple/>` as it allows you to import multiple files / tiddlers / images /json at once.

Regards

Tony

[Birthe C]

Thank you for that one Tony, never thought of that. But sure it works well.

Birthe

[Donald Coates]

Aw man now you opened up another rabbit hole for me to jump down.  This did give me the idea to mount the server over sftp and then transfer multiple files that way so thank you.

I have a feeling if I was better at regular expressions I could figure some way to manipulate the title while using the load command but alas I'll leave that for another sleepless night.  

Thanks again for your insight.

[Beckstrom]

yeah that’s a good question! use the “load multiple” widget and would love a way to change the titles before importing. right now I import and then open each imported tiddler to change the title

[Jeremy Ruston]

Hi Donald

One way to achieve this would be to move your images folder into your wiki, and then create a tiddlywiki.files file to control how the image files are handled:

https://tiddlywiki.com/#tiddlywiki.files%20Files

The example there is for importing a folder full of PDFs, but you should be able to adapt it for other image types.

Best wishes

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/05e40971-35d4-42ca-9848-c1a0db326774%40googlegroups.com.

[Donald Coates]

Jeremy!!!!!!   That was the solution!!!!

Sorry for the exclamations but I have dumped a lot of time obsessively trying to figure this out.  It has been constructive playtime I hope because I'm starting to get a sense of just how powerful the server can be beyond just serving the files, but I was beginning to despair.  I have been playing with tiddlywiki.file in other contexts and have no idea why I did not apply it here.

You have truly saved my sanity.

This was part of a more complex task I was trying to achieve which I will write up later in case it might help others like Beckstrom.

Again - thank you for your advice and this wonderful tool.

Donald

[Atronoush]

Hi Donald,

 I have the same problem. See my file structure where there few tens of .jpg pictures are loctaed in a folder (picture) in the same folder with mywiki (wiki)

 

-- pictures
-- myWIki
-- -- tiddlywiki.info
-- -- tiddlers
-- -- -- images
-- -- -- -- tiddlywiki.files

ant my tiddlywiki.files is as below

    "directories": [
        {
            "path": "../../../pictures",
            "filesRegExp": "^.*\\.jpg$",
            "isTiddlerFile": false,
            "fields": {
                "title": {"source": "basename-uri-decoded"},
                "created": {"source": "created"},
                "modified": {"source": "modified"},
                "type": "image/jpeg",
                "tags": ["$:/tags/AttachedFile"],
                "text": "",
                "_canonical_uri": {"source": "filename", "prefix": "pictures/"}
            }
        }
    ]
}

When I start the wiki, I see all image tiddlers have been created successfully. but not image is shown.

This is the structure of one image tiddler

title:IMG_20160403_172207.jpg

_canonical_uri: pictures/IMG_20160403_172207.jpg

What do you think?

I appreciate to share your experience if you have been successful in loading several images from local disk in this way.

[Mohammad]

I tried the same example here:https://tiddlywiki.com/prerelease/#tiddlywiki.files%20Files

like you I get the pdf tiddlers created but nothing is displayed!

Seems something goes wrong!

Hope Donald share his finding if he was successful.

--Mohammad

[Donald Coates]

my use case is a bit different from yours in that I am using nginx to serve the image from a webserver.

when you click to open the tiddler I assume there is a 'broken picture' icon.  right click on that and look for something like open in a new tab or properties.  You want to see where the browser is looking for the file.  This usually puts me in the right direction.  You probably need to adjust the prefix of the canonical uri based on where the browser is being pointed.  Let me know if that helps or gives a clue.

[Donald Coates]

Assuming you are using locally and not from a web server I would assume it is the same answer as above.  I always have to fiddle with the canonical uri prefix to get it to point in the correct direction, since the tiddler is really just a placeholder that points the browser to where the file is actually located.

[Atronoush]

Donald,

 Yes, I am using Node.js on Windows machine.

  Tiddlywiki correctly scans the picture folder and create the image tiddlers with correct name (The same file name as in pictures folder). Also when I relocate pictures folder, on wiki startup I get the error complaining Tiddlywiki can not find the picture folder.

I see the broken image when I open an image tiddler. I did what you advised

This is address of image: http://127.0.0.1:8080/pictures/IMG_20160405_122242.jpg

Like you I think something wrong with _canonical_uri! But I don not know what is that?

[Mohammad]

Hi Donald,

Thank you! Yes, I also use Tiddlywiki on locale machine using Node.js

I see empty PDF tiddlers as explained by Atronoush for his image tiddlers!

The address is always created as localhost/pdfs/filename

where pdfs come from the prefix I set in tiddlywiki.files

--Mohammad

[Donald Coates]

This is my best guess:

The thing to remember is that the tiddlywiki nodejs server does not serve any other file type but tiddlers.  If you want to serve any other file type you have to use another server - whether it be something like nginx or Bob's plugin or Arlen's tiddlyserver.

What is happening is you are asking the tiddlywiki node js server at localhost:8080 to serve the file with the mime type pdf / jpg - which again is something it cannot do.

So either you have to use one of the above solutions - and I have used both bob and tiddlyserver and I love them both - or right click on the file and select 'open in browser' and make note of what it says.  it is going to be something like file:///file/path/file.jpg|pdf and then that will be what you put in 'prefix'.  So you are essentially creating a direct link to the file.  It is possible as well that that file path will be browser specific so keep that in mind.

I hope this helps and someone with more knowledge can confirm / correct what I am saying.

[Donald Coates]

See  my answer to Mohammad.

[Saq Imtiaz]

Try placing your files in a "files" subfolder of the wiki folder.

"Any files in the subfolder files of the wiki folder will be available via the route \files\<uri-encoded-filename>. For example: http://127.0.0.1:8080/files/Motovun%20Jack.jpg

This can be useful for publishing large files that you don't want to incorporate into the main wiki (PDFs, videos, large images, etc.)."

https://tiddlywiki.com/#Using%20the%20integrated%20static%20file%20server

[Donald Coates]

The tiddlywiki.files method is an excellent way to manipulate directories of tiddlers it is definitely worth checking out.  Also I noticed part of the TW Commander plugin the other day but have not used it.  According to the page it can "perform many kinds of batch operation like create or delete tiddlers, rename tiddlers, add/remove fields, tags, inspect tiddlers and do search and replace in all fields of tiddler."  Might be worth checking out.

[Donald Coates]

So the server can serve static files?!  Thank you for clarifying that.

[Donald Coates]

Okay Mohammad and Astronoush based on what Saq is saying I am going to say that your initial path is incorrect so that the tiddlywiki.files is not looking in the correct place.  It could be something to do with how the nodejs server is running on the windows machine and the availablity of the directory.  Ie if you are running it in cygwin or WSL you may be better off figuring out how to navigate to that directory and use the full path name in the initial path directive.  For example with WSL it would be something like /mnt/c/directory/where/files/are.  I always opt for full path names to avoid these headaches.

Or in the end just move the directory into the tiddler folder like Saq is saying.

[Mohammad]

In https://tiddlywiki.com/prerelease/#tiddlywiki.files%20Files

It says path is relative to where tiddlywiki.files is located!

Furthermore, the files are found and imported into Tiddlywiki. I see list of files in my Recent tab. The ur is created and everything is work.

but the image or pdf tiddlers show nothing!

--Mohammad

[Donald Coates]

okay my commute home brought some revelation.  The nodejs server isn't limited by what files it can server but by where it can serve files from.  So it can not serve files from outside the tiddler folder or a subdirectory of it.  As the static file page referenced by Saq indicates, if given the address localhost:8080/images/image.jpg the nodejs server will only look in a subdirectory of the tiddler folder called images.  Even though the tiddlywiki.files tells the server the files are outside the directory, the address it is giving points to a subdirectory inside the tiddler directory and the files are not there.  AFAIK there isn't any way to point it to outside of the tiddler directory to serve the file.

The answer would lie in that capability but I don't think it exists.

I know with bob you can direct his server to outside directories and with nginx you set up aliases in the configuration file.

I hope this helps.

[Donald Coates]

To add a little more to what I wrote above and address this specifically - the files are not imported.  A reference to their location is created when the server starts.  If you look in the directory that the tiddlywiki.files file is in there are no other files there.  At least this is what I remember from when I was playing with this before.  The only time tiddlywiki creates an actual file with this method is if you manually add a field or tag that is not referenced by the tiddlywiki.file parameters.  Then tiddlywiki has to create a meta file to contain that tag or field.

The only way to actually import files is through the methods on the importing tiddlers page or using the servers import or load command - which is what started me down this rabbit hole in the first place.

[Saq Imtiaz]

On Monday, March 9, 2020 at 1:15:44 PM UTC+1, Donald Coates wrote:

okay my commute home brought some revelation.  The nodejs server isn't limited by what files it can server but by where it can serve files from.  So it can not serve files from outside the tiddler folder or a subdirectory of it.  As the static file page referenced by Saq indicates, if given the address localhost:8080/images/image.jpg the nodejs server will only look in a subdirectory of the tiddler folder called images.  Even though the tiddlywiki.files tells the server the files are outside the directory, the address it is giving points to a subdirectory inside the tiddler directory and the files are not there.  AFAIK there isn't any way to point it to outside of the tiddler directory to serve the file.

This is my understanding as well. Regardless of the fact that tiddlywiki.files can "load" files from other paths, if you want to serve a static file using the node.js server it needs to be in the "files" subfolder of the wiki folder. In addition, the "images" subfolder is probably also supported as it is mentioned here: https://tiddlywiki.com/#ExternalImages

In other words, there is a different logic for what files can be "loaded" via tiddlywiki.files versus what static file paths can be served by the integrated static file server.

[Mohammad]

Saq, Donald

Lets summarize the findings here (I edit this post to add other conclusion with you and later add to TW-Scripts)

What is given below is tested and confirmed.

 External Images and PDFs

Mohammad 9th March 2020 at 4:53pm

Single file wiki

(e.g. testWiki.html)

1. External image can be put everywhere in local disk as can be embed in wiki using canonical URI. See full solution here: https://kookma.github.io/TW-Scripts/#Create%20External%20Images%20from%20A%20Directory

Node.js wiki

(e.g. myWiki) [The description here tested on Windows 10 + Tiddlywiki 5.1.22 + Node.js 12.13.1)

NO tiddlywiki.files is used here

1. the wiki started like tiddlywiki myWiki –server so the address is http://127.0.0.1:8080/
2. If you create a folder called files, inside wikiforlder, then you can get access to all objects (images, pdfs, subfolders ...)
   • through http://127.0.0.1:8080/files/obj-name
   • like http://127.0.0.1:8080/files/IMG_20160405_120320.jpg [WORK]
   • all sub folders in files work as above
3. you cannot access objects in a folder like pictures inside wikifolder [NOT WORK]

to be completed ....

[Mohammad]

On Monday, March 9, 2020 at 4:24:27 PM UTC+3:30, Saq Imtiaz wrote:

On Monday, March 9, 2020 at 1:15:44 PM UTC+1, Donald Coates wrote:

okay my commute home brought some revelation.  The nodejs server isn't limited by what files it can server but by where it can serve files from.  So it can not serve files from outside the tiddler folder or a subdirectory of it.  As the static file page referenced by Saq indicates, if given the address localhost:8080/images/image.jpg the nodejs server will only look in a subdirectory of the tiddler folder called images.  Even though the tiddlywiki.files tells the server the files are outside the directory, the address it is giving points to a subdirectory inside the tiddler directory and the files are not there.  AFAIK there isn't any way to point it to outside of the tiddler directory to serve the file.

This is my understanding as well. Regardless of the fact that tiddlywiki.files can "load" files from other paths, if you want to serve a static file using the node.js server it needs to be in the "files" subfolder of the wiki folder. In addition, the "images" subfolder is probably also supported as it is mentioned here: https://tiddlywiki.com/#ExternalImages

images was tested and it does not work! Seems only files folder work here. 

In other words, there is a different logic for what files can be "loaded" via tiddlywiki.files versus what static file paths can be served by the integrated static file server.

This seems true!

 

[Mohammad]

Thanks Donald

I gonna to summarize our findings here and hope Jeremy address this question: how node.js+tiddlywiki work with external images in a folder of choice using _canonical_uri.

--Mohammad

[Donald Coates]

I would clarify that the folder called files must be in the tiddler folder - the wikifolder may sound like it can be in the same folder as tiddlywiki.info and tiddlers.

[Donald Coates]

Just to clarify where tiddlywiki.files would come into this scenario.  If I dropped a folder of vacation pics into tiddlers/images/vacation then vacation1,jpg would be available at localhost:8080/images/vacation/vacation1.jpg and if you wanted to batch tag them all you would add a tiddlywiki.files file into that directory with the pictures that reads

{
    "directories": [

        {
            "path": "./",
            "filesRegExp": "^.*\\.(jpg|png|jpeg)$",
            "isTiddlerFile": false,
            "fields": {
               "tags": ["vacation"]
        }
    ]
}

and when the wiki loads it will add the tag to all those files.