Implementing srcset image optimization for mobile devices?

[Gwern Branwen]

Has anyone implemented a clean robust srcset feature for Hakyll or
understand how it's supposed to work? I tried and my version is hacky
and fragile.

The <img> attributes 'srcset'/'sizes' in theory let you serve
statically different-size images to desktop and mobile users, reducing
bandwidth to easily a quarter if you serve a screen-sized image to
both: https://developer.mozilla.org/en-US/docs/Learn/HTML/Multimedia_and_embedding/Responsive_images

As one of the main missing optimizations for gwern.net and potentially
very useful on image-heavy pages like https://www.gwern.net/Faces /
https://www.gwern.net/TWDNE / https://www.gwern.net/Red , and looking
fairly straightforward in the examples, I thought I'd implement it to
complement my existing lazy-loading optimization: with both on, images
are loaded only when necessary and only at the necessary size.

I could implement it as another Pandoc rewrite pass: for every Image
element, check that a smaller version exists, if not create it, and
then inline an appropriate srcset/sizes declaration to an updated
Image element; just another `Inline -> IO Inline` which can be
`walkM`ed over. As usual, the devil was in the details. It's hard to
simply create an image in _site because the static file phase runs
last so the directories don't exist (the easiest way is to create it
in the original directory tree, but now you have to make your VCS
ignore it or double the number of images tracked); the srcset/sizes
require the width of the original image, so you have to query both the
existence and size of the original (fortunately, I already have that
for adding height/width for layout hinting); you might as well lossily
optimize it or else it'll be twice as big as necessary; and then the
biggest problem proved to be getting the damn srcset/sizes to fire at
all! Watching in dev tools, I could see they weren't fetching the
smaller version no matter how many tutorials or docs I checked.
Eventually, I found the magic incantation '(max-width: 30em) 80vw,'
*seems* to be HTML valid, makes it fire in Chromium mobile simulator
but not in Chromium desktop mode, and Google Pagespeed Insights seems
to start fetching the small versions... but I'm not confident in this
at all.

My version (which you can see in action - for now - at the 3 links
above) goes like this:

-- Example: Image ("",["full-width"],[]) [Str "..."]
("/images/gan/thiswaifudoesnotexist.png","fig:")
-- type Text.Pandoc.Definition.Attr = (T.Text, [T.Text], [(T.Text, T.Text)])
imageSrcset :: Inline -> IO Inline
imageSrcset (Image (c, t, pairs) inlines (target, title)) =
do let ext = takeExtension $ T.unpack target
let target' = T.unpack target
(_,w) <- imageMagick $ tail target'

let smallerPath = (tail target')++"-768px"++ext
notExist <- fmap not $ doesFileExist smallerPath
when notExist $ do
(status,_,bs) <- runShellCommand "./" Nothing "convert"
[tail target', "-resize", "768x768", (tail target')++"-768px"++ext]
case status of
ExitFailure _ -> error $ show status ++ show bs
_ -> do if ext == ".png" then -- lossily optimize using
my pngnq/mozjpeg scripts:
void $ runShellCommand "./" Nothing
"/home/gwern/bin/bin/png" [tail target']
else
void $ runShellCommand "./" Nothing
"/home/gwern/bin/bin/compressJpg" [tail target']
void $ print ("Created smaller image: " ++ smallerPath)
let srcset = T.pack (target'++"-768px"++ext++" 768w, " ++
target'++" "++w++"w")
return $ Image (c, t, pairs++[("srcset", srcset), ("sizes",
T.pack ("(max-width: 30em) 80vw, "++w++"px"))])
inlines (target, title)
imageSrcset x = return x

Has anyone else tried this?

--
gwern

[Jasper Van der Jeugt]

I haven't thought too much about this but it does sound like a good
idea. I'd like to look into this and maybe even merge it into Hakyll if
it's "generic" enough. The version I would like to target:

- Doesn't depend on Pandoc but rather targets the HTML tree (because
not all Hakyll sites use Pandoc).
- Uses some sort of higher order function along the lines of
`Dimensions -> TempPath -> IO TempPath` to do the conversion, so users
can shell out to `convert` but also use other tools.
- Provides a convenience version that just uses `convert`.

I'll open an issue on GitHub to track this.

Warm regards
Jasper

> --
> You received this message because you are subscribed to the Google Groups "hakyll" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hakyll+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/hakyll/CAMwO0gzQd4qcieyaau9-fyxR-mjef6QmOeTVMLAAmEFrFkU1Qg%40mail.gmail.com.

[Gwern Branwen]

A generic version would, I think, be tricky.

`loading="lazy"` is so easy to add and universally desirable that I am
disappointed that there was any debate on upstreaming it to Pandoc,
and if Pandoc won't, it is an absolute no brainer to add to Hakyll.
Adding `<img>` height/width is not something Pandoc can do because
hrefs may refer to images that don't exist and documents may need to
be portable etc, but that's not a problem for Hakyll, and is something
that it should've'd a decade ago as one of the simplest and most
universally-recommended optimizations that a static site compiler is
capable of. `srcset` on the other hand...

AFAICT, there are at least *3* different ways of using srcset: using
it to pull in higher-resolution images for high-DPI displays, using it
to pull in smaller images for mobile devices, and pulling in
different-shaped images for devices which may be landscape vs portrait
aspect ratioed. It is not obvious how you'd support all 3. And just
getting my use-case, #2, was awful. There appear to be few
universally-agreed-upon sizes or critical points or how many to use
(you'll see sets from 2-4 images). The tutorials are complicated
enough that my eyes crossed, and none of the examples seemed to work
on gwern.net as they are supposed to. The code sample I gave above
works on gwern.net, I think, for now, but I have little faith in it.
Further, even if you assume it'll work on other Hakyll sites (which
will probably have completely different mobile CSS than gwern.net's
custom CSS so I have no idea what's required), there's still
additional things: if you care enough about performance & image
optimization for mobile to bother with srcset, you will want to use
lossy optimizers like `pngnq/advpng/mozjpeg/guetzli` to cut the srcset
image sizes by at least half from the fatty file `convert` produces...
but do you want to run those on every build to optimize the temp
files? `mozjpeg` is fast but `pngnq` takes a bit of time and `guetzli`
is notoriously slow at minutes & many gigabytes per image.

Getting it right probably requires input from a good web designer who
actually understands how all this stuff works under the hood and can
recommend how to do it right for Hakyll. >.<

--
gwern

[Gwern Branwen]

To update this: I've removed all srcset code from gwern.net.

The srcset approach has been nothing but misery. Browser support has somehow gotten worse and broken the srcset snippet which *used* to work (and possibly some demos like the MDN one which I think used to work), and further, managing the intermediate/small files has been a perennial PITA and complicated the backend a lot.

I advise you to not use it. loading=lazy has solved most of the performance problem, and has, in stark contrast to srcset, worked perfectly since I added it, and I haven't needed to touch it since.

--
gwern
https://gwern.net/design-graveyard#srcset-mobile-optimization