Issue 591 in webp: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess

[p.… via monorail]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 591 by p....@gmx.at: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591

## Environment

I use GraphicConverter (current version: 11.7.1 (build 5723) ) on macOS 11.7.2 Big Sur to generate WEBP files. The app uses the webp library (newest one I assume) to generate the WEBP files without any modifications/customizations according to the developer.


## What I noted:

- The Exif chunk is written at the very end of the WEBP file.
- Usually all data which is important for interpretation/rendering (ICC color profile embedded or as reference/flag, image resolution information, etc) should be at the file start, so that the browser engine can start drawing correctly right from the beginning and avoid re-processing / re-drawing again when file loading completed.


## My Questions

1. Is this Exif chunk at the end of the file (aka "tail") by design? If so, why?

2. Is this by co-incidence and no-one else noticed the possible downsides so far?

- I noticed this fact during a feature implementation discussion with the developer of GraphicConverter:
https://www.lemkesoft.info/forum/viewtopic.php?p=27608#p27608:~:text=But%20I%20have%20some%20remaining%20concerns

- To which the developer had replied:

> A WebP file is written with the public code from Google.
> I can't change the order.


## Followup / Proposal

- Therefore I wanted to bring this to your attention.
- After all the goal of WebP is to increase web performance.
- And if this aspect was not met in regards of Exif, and was indeed a mistake, then this should be corrected.

I'd very much appreciate a reaction. Thank you!

Attachments:
Exif in header of JPEG file.png 70.7 KB
Exif chunk in WebP file is at the file end.png 202 KB

--
You received this message because:
1. The project was configured to send all issue notifications to this address

You may adjust your notification preferences at:
https://bugs.chromium.org/hosting/settings

[ygu… via monorail]

Updates:
Owner: ygu...@google.com
Status: WontFix

Comment #1 on issue 591 by ygu...@google.com: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c1

Thank you for the report.

## What you noted

- The Exif chunk is written at the very end of the WEBP file.

Yes, see https://developers.google.com/speed/webp/docs/riff_container#extended_file_format

- Usually all data which is important for interpretation/rendering [...] should be at the file start

I agree.
The color profile is stored before the samples in the 'ICCP' chunk, see https://developers.google.com/speed/webp/docs/riff_container#color_profile.
Image dimentions etc. are stored in 'VP8X' and 'ANIM' chunks.
Image resolution is usually used for printing and not that much for displaying images *on the web* (pixels-per-inch do not impact WebP <img> scaling in Chrome, right?).

## Your questions

1. Is this Exif chunk at the end of the file (aka "tail") by design? If so, why?

Yes, because Exif and XMP metadata should not impact the rendering of the image. With that said, it is better to put it last, so that actual color and alpha samples are received/decoded/shown earlier to the user.
The exception would be Exif orientation, but it is not handled in a consistent way by all WebP renderers so use at your own risk.

2. Is this by co-incidence and no-one else noticed the possible downsides so far?

This was a design choice.

## Followup / proposal

- And if this aspect was not met in regards of Exif, and was indeed a mistake, then this should be corrected.

Even if it should be corrected, it is too late: the WebP format specification has been frozen for many years now, and modifying it now would be far more risky than beneficial.

[p.… via monorail]

Comment #2 on issue 591 by p....@gmx.at: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c2

Thanks for your detailed and well explained response!

The most important bit of your information

> … too late: the WebP format specification has been frozen for many years now.


Nevertheless for a possible future revision of WebP or any other image format that may come, you may take note of this aspect:

The "image resolution" in Exif could be facilitated quite well for responsive image solutions (e.g. 144 DPI being interpreted as @2x, etc).

So far there is no real standard how to denote what intended screen resolution an image file has. Many solutions outsource this to the filename with a "@2x" suffix/portion in the file name. Having quite technical portions in image in file names is not nice for beautiful permalinks (SEO) either… Would be more elegant to put this info into the metadata of the file.

But now I learned that this ship has sailed anyhow.

Maybe my feedback will at least resonate in image developer's minds as a note that encoding image resolution (e.g. screenshots or screen designs which are in @1x or @2x or @3x and shall be presented "in true size") in the current web technology landscape is not properly covered (=in the file) and instead currently need to be solved in markup / solutions on top, instead of simply traveling with the image file itself (as it did in the Desktop publishing / print world). Thanks!

[ygu… via monorail]

Comment #3 on issue 591 by ygu...@google.com: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c3

I understand your point of view.
However I remember the good practice of having width/height defined in the <img> (or CSS) rather than relying on the native dimensions of the loaded image for the following reason: the browser does not have to wait for at least a few bytes from the image payload to be received before knowing the final <img> size, making fewer or faster refreshes to the page layout.
For the same reason I would argue against storing display-impacting metadata directly in image formats, such as the resolution. Purely informational metadata is fine (but often useless for the web), and thus should be located after the pixels.

[jz… via monorail]

Comment #4 on issue 591 by jz...@google.com: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c4

Thanks for the feedback. Rotation is another area that might impact rendering, but as Yannis said, the we didn't put any strong restrictions on the EXIF chunk ordering and it's a little late to change it. EXIF can carry a lot of extra information making it quite large, which is why we decided to recommend it come after image data.

[jz… via monorail]

Comment #5 on issue 591 by jz...@google.com: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c5

> Maybe my feedback will at least resonate in image developer's minds as a note
> that encoding image resolution (e.g. screenshots or screen designs which are
> in @1x or @2x or @3x and shall be presented "in true size") in the current
> web technology landscape is not properly covered (=in the file) and instead
> currently need to be solved in markup / solutions on top, instead of simply
> traveling with the image file itself (as it did in the Desktop publishing /
> print world).

Having an all in one file solution would bring it's own complexity related to transfer. The client would need to be aware and able to download a portion of the file to reduce the transfer, rather than selecting a single entry from e.g., the <picture> tag.

[pasca… via monorail]

Comment #6 on issue 591 by pasca...@gmail.com: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c6

Note also that, by design, the codec should not attempt to understand the content of metadata (and just pass it around).
Making coding decisions (like, chunk order) based on the EXIF's content just opens a can of worms.
Starting, e.g., with how inconsistent is the rotation information is signalled. There's also a lot of proprietary syntax around.

thanks!
skal/

[p.… via monorail]

Comment #7 on issue 591 by p....@gmx.at: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c7

Ofc width/height and resolution should be in the HTML markup to avoid cumulative layout shift during the page loading/rendering lifecycle.

But it would have been so great if there's an image standard where I can write that "intended screen resolution" into the metadata of the image file, then just drop it at my content management system of choice, where the file gets parsed/analyzed, and the necessary dimension/resolution metadata written into the HTML markup accordingly.

As it is now, you have to either encode resolution info into the filename with a suffix like "@2x" as a means to communicate to the CMS "Hey this is a @2x" file, which adds unnecessary technical information to the filename rather than a clean SEO / human-readable filename. Or worse, in the CMS you have to select file(s) and tell it manually: This is a @2x file, these are @3x files, these are @1x files.

I wished that image format developers had thought of this. But it seems that responsive image solutions have developed in parallel to WebP, and these considerations did not make it into the standard in time.

[ygu… via monorail]

Comment #8 on issue 591 by ygu...@google.com: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c8

I found XResolution, YResolution and ResolutionUnit in Exif: https://exiftool.org/TagNames/EXIF.html#:~:text=0x011a-,XResolution,-rational64u%3A
WebP supports Exif, and that is the responsibility of the CMS to handle the Exif resolution properly. For that use case, it does not matter that Exif is at the beginning or at the end of the WebP file.

[p.… via monorail]

Comment #9 on issue 591 by p....@gmx.at: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c9

## 1) Resolution not needed for photo content as responsive images

The browser knows when the hardware resolution is satisfied and fetches the proper image from the image source set. And scales the image. A bit of interpolation happens. But only downscaling, because the browser selects the correct image dimensions with the necessary "interpolation headroom". All fine. No worries.

But it's a WHOLE different world, when you want to present screenshots/screendesigns on websites in "true to size". Where a thin hairline shall really appear as a thin hairline, and not fade away due to wrong interpolation. Or the whole UI shown way too small or too large. And elements shall be crisp. This use case is not yet covered well in any image format that I know of. At best approximations and hand tailored solutions are available. I will explain that further.


## 2) Exif resolution attributes - Thanks I knew those and used them so far

I once created a spreadsheet of resolution support in various image formats (see attached).
Just no CMS that I know of is utilizing the resolution metadata of JPEG / PNG / WebP for the purpose of responsive images as covered in #1.


## 3) Most likely due to the fuzzy nature inherent in the topic of "responsive images"

What is @1x? What is @2x? Most screen design software which have capabilities to export @1x @2x @3x … equate them to 72 144 288 DPI respectively, if they export the resolution into the metadata at all.

That is ofc only an approximation, never really corresponding to an actual hardware screen resolution but rather to "rough density classes".

Because in the 2000s the commonly found resolutions of flatscreen displays and laptop displays were around 90-120 PPI (see attached spreadsheet with screens over the years until now, only for "rough idea", not fact checked, may contains errors, data partially copied, partially self calculated).

Nevertheless for UI rendering purposes that whole range of hardware resolutions needed to be subsumed under an umbrella "standard UI resolution". On the Mac platform as 72 PPI (to be backwards compatible with what was established in the 1980ies Desktop Publishing world) and 96 PPI on the Windows platform (more true then, but also only an approximation).

With the advent of mobile devices and smart TVs we finally had screens with UIs watched at very different viewing distances. So what really counts is the density projected onto our retina in our eyes. So that at typical viewing distance a "16px CSS pixel font" gives the same size impression in the eye of the beholder as any other device which adheres to the usual rulesets. Well explained in:

- https://en.wikipedia.org/wiki/Retina_display
- https://en.wikipedia.org/wiki/Fovea_centralis#Angular_size_of_foveal_cones .

For app and website UIs to achieve "somehow uniform UI sizes" the device manufacturers need to tell the OS/browser the proper UI/CSS pixel density which is the appropriate ratio of typical viewing distance plus physical pixel density.

Hence PPI in image files is not fit for that purpose. At best an approximation. As such I handled it so far. I tag my image files in the density classes with the convention @1x = 72ppi, @2x = 144ppi, etc. And if a really appropriate standard comes one day I will batch convert from those "conventional density classes" to the real standardized attribute.

A modern image standard in the era of responsive images should provide means for these two different measurement units:

1) Density Class: Values would be @1x, @1.5x, @2x, @3x etc. Use case: A screenshot/screendesign tagged that way is shown "true to size" for the specified "density class". With up/down-scaling in full integers for perfectly crisp images.

2) A absolute physical metrics: Which allows to specify a value pair: viewing distance X shall yield density Y in the eye of the user. (I may not expressed this scientifically, but I hope you get the idea).

Attachments:
Resolution information PPI DPI in metadata of image file formats.pdf 41.8 KB
Screen Size + Resolution aka Pixel Density + Aspect Ratio - Smartphones Tablets Laptops.pdf 79.5 KB

[jz… via monorail]

Comment #10 on issue 591 by jz...@google.com: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c10

WebP is meant as a minimal web focused format. Formats like AVIF and HEIF might be better targets for this kind of suggestion as they are still evolving.

[p.… via monorail]

Comment #11 on issue 591 by p....@gmx.at: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c11

Coincidentally I think with my words may have reached the right person as @yguyon turns out to be the owner of webp2 -- which is experimental and upcoming. So maybe I landed something fruitful 😉

But yes, I may add these use case considerations/ideas to other upcoming image format projects too. Also thought of JPG-XL.

[jz… via monorail]

Comment #12 on issue 591 by jz...@google.com: Exif chunk is at end of WEBP file — Should be in file header for web engine efficiency I guess
https://bugs.chromium.org/p/webp/issues/detail?id=591#c12

WebP2 isn't currently under active development, it's been left as an experimental platform, work has moved to AVIF. I'd suggest looking into that format and then posting any issues to the github tracker [1].

[1] https://github.com/AOMediaCodec/libavif