Round-trip rendering of images is distorted

[Lyle Kopnicky]

Hi folks,

I'm rendering diagrams to the Rasterific backend, and they include embedded images. The embedded images are slightly distorted in that they are not quite centered and bleed into adjacent pixels. I can't seem to find the right transformations to get them to render in place exactly as they were in the original diagram.

I would expect that if I have a PNG, and I ask diagrams to render just the image, using Rasterific, in exactly the same size as the original, I should get back exactly the original image.

I constructed a small example to show this: a 4x5 image with a red box and a black line through it:

It's small - you will want to magnify the image. I rendered it into a diagram with this code:

  encodePng $ renderDia Rasterific (RasterificOptions $ dims2D 4 5) tinyRectangleDiagram

  where

    tinyRectangleDiagram = image $ DImage (ImageRaster img) (dynamicMap imageWidth img) (dynamicMap imageHeight img)

After rendering it into a diagram, it comes out like this:

You can see that the top left of the box is clipped. In order to be able to see the whole thing, I added a bounding box around it (rect 10 11), so the whole image is 10x11. It comes out like this:

Now it is no longer clipped, but you can see it is not centered, but skewed a bit to the top left. Interestingly, the black bar does not bleed above or below, but only to the left and right. The red box seems to bleed on the bottom and right. On the top left it seems a bit faded.

So, what's happening here? Can someone explain the model as to how pixels are seen in diagrams? I would think that a pixel would just be a square (or rectangle if the aspect ratio is not 1:1), and that the center of the original image would be in the center of the diagram.

I then tried rendering the image at 10x the size in each direction, and got this:

Fascinating. It's not symmetrical at all like the original image, and it's not centered. The outside edges are sharp, but the interior is blurred and skewed up and to the left.

What's doing the blurring and skewing? Is that the Rasterific backend of Diagrams or is that Rasterific itself? Is there a way to turn that off? I want the output to look exactly as if I had pasted the original image into the center using Photoshop.

Thanks,

Lyle

[Sylvain Henry]

Hi,

It seems to be a Rasterific issue because the following program gives the same result:

import Codec.Picture
import Graphics.Rasterific

main :: IO ()
main = do
    i <- readPng "tiny-rectangle.png"
    case i of
        Left err -> error err
        Right img -> do
            let d = drawImage (convertRGBA8 img) 0.0 (V2 0 0)
            let img2 = renderDrawing 4 5 (PixelRGBA8 0 0 0 0) d
            writePng "tiny-rectangle-out.png" img2

In Graphics.Rasterific.Immediate.fillWithTexture, if we print "els" (the shape of the image we want to render) we get:

[LinePrim Line (V2 (-0.5) (-0.5)) (V2 3.5 (-0.5)),LinePrim Line (V2 3.5 (-0.5)) (V2 3.5 4.5),LinePrim Line (V2 3.5 4.5) (V2 (-0.5) 4.5),LinePrim Line (V2 (-0.5) 4.5) (V2 (-0.5) (-0.5))]

Notice the (-0.5,-0.5) translation.

It then gets clipped to:

fromList [LinePrim Line (V2 0.0 0.0) (V2 3.5 0.0),LinePrim Line (V2 3.5 0.0) (V2 3.5 0.0),LinePrim Line (V2 3.5 0.0) (V2 3.5 0.0),LinePrim Line (V2 3.5 0.0) (V2 3.5 0.125),LinePrim Line (V2 3.5 0.125) (V2 3.5 0.75),LinePrim Line (V2 3.5 0.75) (V2 3.5 2.0),LinePrim Line (V2 3.5 2.0) (V2 3.5 4.5),LinePrim Line (V2 3.5 4.5) (V2 1.5 4.5),LinePrim Line (V2 1.5 4.5) (V2 0.5 4.5),LinePrim Line (V2 0.5 4.5) (V2 0.0 4.5),LinePrim Line (V2 0.0 4.5) (V2 0.0 4.5),LinePrim Line (V2 0.0 4.5) (V2 0.0 0.0)]

The following patch seems to fix the issue:

diff --git src/Graphics/Rasterific.hs src/Graphics/Rasterific.hs
index 088248d..e60e2ac 100644
--- src/Graphics/Rasterific.hs
+++ src/Graphics/Rasterific.hs
@@ -823,8 +823,8 @@ drawImageAtSize img@Image { imageWidth = w, imageHeight = h } borderSize ip
         stroke (borderSize / 2) (JoinMiter 0)
                (CapStraight 0, CapStraight 0) rect'
         where
-          p = ip ^-^ V2 0.5 0.5
-          rect = rectangle (V2 0 0) rw rh
+          p = ip ^+^ V2 0.5 0.5
+          rect = rectangle (V2 (-0.5) (-0.5)) rw rh
           rect' = rectangle p reqWidth reqHeight

My guess is that "^-^ V2 0.5 0.5" was added to compensate for line sampling (hence rectangle sampling) which adds (0.5,0.5) to every sample coordinate (probably to get at the center of the pixels).

Texture sampling seems to use final image coordinates according to the comment in Graphics.Rasterific.Texture. So if we only do:

-          p = ip ^-^ V2 0.5 0.5
+          p = ip

We get the correct shape:

[LinePrim Line (V2 0.0 0.0) (V2 4.0 0.0),LinePrim Line (V2 4.0 0.0) (V2 4.0 5.0),LinePrim Line (V2 4.0 5.0) (V2 0.0 5.0),LinePrim Line (V2 0.0 5.0) (V2 0.0 0.0)]

But we don't compensate the (0.5,0.5) translation in samples so the output is still bad. We can do:

-          rect = rectangle (V2 0 0) rw rh
+          rect = rectangle (V2 (-0.5) (-0.5)) rw rh

to compensate, but then the coordinates are (-0.5,-0.5) off. So finally we do:

-          p = ip ^-^ V2 0.5 0.5
+          p = ip ^+^ V2 0.5 0.5

I'm not sure if it's the best fix. It's probably worth opening a ticket.

Cheers
Sylvain

--
You received this message because you are subscribed to the Google Groups "diagrams-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to diagrams-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/diagrams-discuss/e15c080c-4a16-4520-99d1-1c9117855e28n%40googlegroups.com.

[Lyle Kopnicky]

Great, thanks, Sylvain, for the investigation and patch!

Looking at the Rasterific github, it appears the line

p = ip ^-^ V2 0.5 0.5

was added in order to address complaints of blurriness reported in in Issue #15. Later someone else reported a similar issue, #22, and the same fix was proffered.

So, I'll create a new issue for Rasterific and post some new examples, along with your proposed fix. I'll try it out myself of course.

Thanks,

Lyle

You received this message because you are subscribed to a topic in the Google Groups "diagrams-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/diagrams-discuss/TRtnvrQaT-k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to diagrams-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/diagrams-discuss/9bd6e150-3384-8e8b-a202-55f65ba31184%40haskus.fr.

[Sylvain Henry]

Hi Lyle,

I've added a comment in #22: https://github.com/Twinside/Rasterific/issues/22#issuecomment-768922232

Cheers,
Sylvain

To view this discussion on the web visit https://groups.google.com/d/msgid/diagrams-discuss/CAH3XM1nk86e1%2B0Eir6turkq%2BdmNHAL9q3UvgxGcUfN8pZ-fcaQ%40mail.gmail.com.