Compression of embedded images

[Mal]

Tiddlywiki Folks,

I have a non-technical Tiddlywiki user with large (30-40MB) wikis of study notes containing lots of embedded images.  Ideally, I would like to provide a simple method of optimising these images during the process of dragging and dropping into the wiki.  Are there any good ideas out there for streamlining this process?

I know there are various on-line sites for image conversion and optimisation, like https://image.online-convert.com/convert-to-webp, and browsers also have some in-built image manipulation capability.

A plugin that automatically converted images when dropped into tiddlywiki would be ideal, but I fear this is a bit above my level of browser, javascript and Tiddlywiki knowledge.

In the past, I have used a simple bash script to run through the image files in a node.js implementation of the wikis and convert png images to jpg.  This provides some substantial reductions in Tiddlywiki file sizes.  However, it would be much cleaner to do the optimisation at the point of adding the images to the wiki, preferably with some options related to resulting quality and size.

I would love to read your thoughts on this.

Regards,

Mal

[PMario]

Hi Mal,

Image processing is a highly advanced art and libraries that can handle it in a sophisticated way are huge in size. .. So I _don't_ think it will ever be convenient to add an image processing library to the TW core. ... To deal with imperfections would be a maintenance nightmare.

IMO working with TW and images always depends on the "target group" you want to reach.

Print target)

    As soon as you want to print the content, you'll need "full" resolution, to get good results. ... So I would completely exclude the "print" target group, if I would work with embedded images. If you need to print something I think you will _have_ to work with "external images".

Computer screen shots)

    I still think, that GIF is still a format to go here. GIF was very popular in 1990 to 2000 but it had a really BAD problem. LZW lossless compression, which GIF uses was patented. In 2004 the relevant patents expired and since then GIF IMO is still a image format to go for screen-shots.

The nice thing is, it doesn't need any settings, when you save it. It always uses lossless compression. The only problem it has is the colour range it can handle. ... So if you have an image background ... forget about it. If you have a monochrome background (as I prefer) it's still a good format.

PNG was developed in the 2000th because of the GIF patent wars. IMO it's a bit better for handling modern colour palettes for screenshots, but it also has a bigger file size. _and_ it has a compression setting. :/ which has the potential to be set wrong by users.

With the screenshot I did create from my workspace the setting 9 creates a smaller image as setting 6. Where 9 is better quality. :/ The file size difference between PNG and GIF is about 150kByte. GIF (557k), PNG (703k c9)(710k c6 :/)

JPG should _NOT_ be used for computer screenshots. If you set quality to 100% it produces much bigger images as PNG. In my case 1016kByte instead of 703kByte for PNG

If 80% setting is used the jpg is 388kByte BUT it already produces very bad JPG-artefacts with text. Which can be seen if the screen-shot is ever printed. It doesn't look professional. ... ever.

Images)

    IMO JPEG is the format to go here. WEBP isn't supported by Apple, so chances are high, it produces problems, if embedded. JPG is already compressed and can _not_ be zipped. The second problem with images today is size.

Modern mobile pones easily create 5000x3000 pixel images. That's OK, because we want "good quality" and jpg always uses a "lossy compression" format, even if set to 100%!

The problem is, that those images already need 4-7MByte, depending on the content details. ... So if we want to embed JPG we need to do some "pre-processing".

IMO googles Squoosh-app is a nice way to play with different possibilities, that fits your needs. _but_ interested users should read this info first, to know what's going on. WebApp is: https://squoosh.app

Optimizing according to the usecase is nice ... BUT ... embedding images into TW converts them into base64 encoding, which throws a lot of the optimizations out of the window by blowing it up by 100%

SVGs)

For SVGs optimization we can use: https://jakearchibald.github.io/svgomg/

just some thoughts

have fun!

mario

[Mark S.]

I guess step one would be to locate any open-source, javascript-only code sources for converting PNG to JPG.

[@TiddlyTweeter]

Mal, partly backing up PMario & Mark but with a sharper intent ... :-)

It would definitely be easiest to use an external image editor with bulk functions to prepare images. Change resolution, palette color numbers & output format for LOWEST NICE LOOKING SIZE (do that interactively in the program). There are lots of them.

On Windows https://www.irfanview.net/ is a good free one to try. There are many others on all platforms.

Underlining your OP: You need make sure image resolutions are not higher than the viewport. Match them to the max resolution you'd display. That is the biggest issue.

Best wishes

TT

[Mal]

Mario / Mark / TT,

Thanks for all the helpful comments.

Mario, your post is a great reference - it's a pity this stuff tends to get buried in GG.

In my application, I think we are mostly dealing with screen shots and pretty much only view the wikis on screens, ranging from desktops to mobiles, so no need for printing.

I will look further into a suitable image manipulation application that will allow for a simple workflow to get from the screen capture through image optimisation to copying into Tiddlywiki.

Thanks again.

Regards,

Mal

[TonyM]

Mal

Actual screen shots, without photographic style images included, Actualy compress very well in jpg since a lot of the image is block colours. If you extract the images you would import and try some bulk converters you can get a quick idea if they are small enough in total to import or if you must go the external route.

If you go for external images you may loose the single file portability. I suppose it depends if its one or more users, needs to be private or not, needs to work off line etc that will determine the best solution. You could host images somewhere and reference them, the trick would be making an easy capture, save, reference workflow and this starts with how are the images acquired

[Mal]

Tony,

Thanks for your additional comments.

To answer some of your points:

• Yes, maintaining single file format is important - this makes it much easier to access on various device types.
• Yes, these wikis need to be kept private.

My previous batch approach was to process images that have already imported into the wiki, convert png images to jpg and retain whichever is smaller.  However, the streamlined workflow you mentioned would help to get the images optimised at the start.

Regards,

Mal

[PMario]

 Hi,

This may be interesting: https://support.mozilla.org/en-US/kb/firefox-screenshots  ... If a screenshot is done, you can use CTRL-V to import it into TW as PNG.

-m

[PMario]

On Sunday, November 10, 2019 at 4:08:15 AM UTC+1, Mal wrote:

In my application, I think we are mostly dealing with screen shots and pretty much only view the wikis on screens, ranging from desktops to mobiles, so no need for printing.

Hi,

As TT pointed out I also use IrfanView on Windows. I think I use it since 2000 or even earlier. It has a lot of filters ... as plugins AND batch processing!

With screenshots, I think the most important thing is. Consistent initial size.

Eg: If your wiki is viewed on a 1920x1080 full-hd screen, imo it is enough, to create full-screen screenshots with the same size maximum. Smaller may be better. Computer screens can handle that well and the file size will be about 200kByte GIF or PNG per page. ... IF you have text content.

On desktop browsers those images will need to be internally scaled down. .. So if your users are mainly PC users, you may even go down with initial size here. If images are scaled down by browsers, it always creates a "poor feeling". Scaling up is less of a problem. (That's a very personal point of view :)

Most mobile phones have higher native resolution than that, so browsers won't even need to internally scale them down if shown.

just some thoughts

mario

[Eric Shulman]

On Sunday, November 10, 2019 at 4:23:39 AM UTC-8, PMario wrote:

This may be interesting: https://support.mozilla.org/en-US/kb/firefox-screenshots  ... If a screenshot is done, you can use CTRL-V to import it into TW as PNG. 

In Chrome, you can take a screenshot by using ctrl-shift-I (to open the Developer Tools), followed by ctrl-shift-P (opens a list of Developer functions).

Then, type in "screenshot" to reduce the list to show only the screenshot-related functions.

There are four screenshot functions available:

1 Capture area screenshot

2 Capture full size screenshot

3 Capture node screenshot

4 Capture screenshot

Selecting option #1, "area screenshot", gives you a large "plus" (+) cursor that you can use to select an area of the current window display.  click-and-drag the cursor to highlight the desired display area .  When you release the mouse button, you will be prompted for a file to save the screen into.  Chrome then creates a PNG image file on your local harddrive.  Import that image into TiddlyWiki in the usual fashion.

Selecting options #2 or #4 automatically capture the entire window display without selecting an area, and then prompts for a file to save into.

To use option #3, "node screenshot", first right-click on a page element and select "inspect", then Ctrl-shift-P and type in "node screenshot".  This will take a screenshot of the visible portion of the currently inspected element.  Note that if you already have the developer tools window open, you can select ANY element directly from the developer window's "Elements" panel and then use ctrl-shift-P with "node screenshot" to capture a PNG image of the visible portion of that element.

enjoy,

-e