CanvasRenderingContext2D code flow

[Paul Prechtel]

Hello,

I am trying to determine reasons why Canvas Fingerprinting is possible in the first place, or in other words: why there are differences in the rendering output after using CanvasRenderingContext2D.
To get behind this, I want to understand how chromium does the rendering, specific for CanvasRenderingContext2D.
To do this I studied the source code, but I can't wrap my head around it.

I searched around at https://www.chromium.org/developers/ and watched The Life of a Pixel (very nice presentation btw.), but it is not specific to CanvasRenderingContext2D.

Can someone direct me to some documentation or give a short code flow on how this works specifically for CanvasRenderingContext2D?
I am especially interested in how the general CanvasRenderingContext2D API stuff interfaces with the different "backends" (software rendering, skia, hardware rendering).

Thanks in advance,
Paul

[Justin Novosad]

There are many factors. Off the top of my head:

• Browsers use GPU acceleration to render canvas content, so there can be subtle differences between machines that have different GPU models or different graphics driver versions. In particular anti-aliased edges, image filtering and blending are all subject to numerical precision, which may vary between machines.  
• There are differences in numerical precision between different models of CPU.  Numerical precision for floating point arithmetic is only identical between machines when enabling a compiler option that enforces compliance with the IEEE-754 standard.  Because that has an impact on performance, we do not enable that option everywhere in the code.
• Depending on what fonts are installed on the user's computer, text can look slightly different from one system to another because there are different variants of the same font and also because of the fallback mechanisms in CSS font resolution, which applies to how the font attribute is interpreted in 2d rendering context.  
• Depending on the type of display device, sub pixel text anti-aliasing may be enabled or disabled. It gets enabled only on LCD screens that have a device pixel ratio of 1.

[Paul Prechtel]

Thanks Justin, that's a great list.
I have already found out about graphics changes through anti-aliasing, subpixel hining and so forth. These features could be turned off.
You directed my attention to the whole number precision/formats topic, which sounds very influential for rendering output. Unfortunately not simply toggleable.
Reminds me of <https://www.usenix.org/conference/usenixsecurity19/presentation/wu>. I will further look into this topic.

I still want to understand the source code used by the 2d rendering context to better understand the rendering process (the specific parts for this rendering context).
Does the 2d rendering context spit out OpenGL calls or how does it interface with the underlying rendering primitives?
I hope I can spot some differences between the different rendering backends.
What are these anyway? Are they both provided by Skia (software/cpu- and hardware/gpu-based)?

I think I have to sleep over it.
Thanks so far, I'll message you later on.

P.S.: If there is a better place to ask such questions just direct me there.

On 21/03/11 11:22, Justin Novosad wrote:
> There are many factors. Off the top of my head:
>

> - Browsers use GPU acceleration to render canvas content, so there can

> be subtle differences between machines that have different GPU models or
> different graphics driver versions. In particular anti-aliased edges, image
> filtering and blending are all subject to numerical precision, which may
> vary between machines.

> - There are differences in numerical precision between different models

> of CPU. Numerical precision for floating point arithmetic is only
> identical between machines when enabling a compiler option that enforces
> compliance with the IEEE-754 standard. Because that has an impact on
> performance, we do not enable that option everywhere in the code.

> - Depending on what fonts are installed on the user's computer, text can

> look slightly different from one system to another because there are
> different variants of the same font and also because of the fallback
> mechanisms in CSS font resolution, which applies to how the font attribute
> is interpreted in 2d rendering context.

> - Depending on the type of display device, sub pixel text anti-aliasing

[Justin Novosad]

On Thu, Mar 11, 2021 at 8:43 PM Paul Prechtel <paul.p...@uni-ulm.de> wrote:

Thanks Justin, that's a great list.
I have already found out about graphics changes through anti-aliasing, subpixel hining and so forth. These features could be turned off.
You directed my attention to the whole number precision/formats topic, which sounds very influential for rendering output. Unfortunately not simply toggleable.
Reminds me of <https://www.usenix.org/conference/usenixsecurity19/presentation/wu>. I will further look into this topic.

I still want to understand the source code used by the 2d rendering context to better understand the rendering process (the specific parts for this rendering context).

This part is quite complicated and has many layers. This page has links to lot of relevant documentation, but not all of it is up to date and not everything is documented: https://www.chromium.org/developers/design-documents/chromium-graphics

It's a bit overwhelming, but I'll tell you where to find the bits you are probably most interested in below.

 

Does the 2d rendering context spit out OpenGL calls or how does it interface with the underlying rendering primitives?

Not directly. Most of the 2d context entry points are implemented in these files:

https://source.chromium.org/chromium/chromium/src/+/master:third_party/blink/renderer/modules/canvas/canvas2d/base_rendering_context_2d.cc

https://source.chromium.org/chromium/chromium/src/+/master:third_party/blink/renderer/modules/canvas/canvas2d/canvas_path.cc

https://source.chromium.org/chromium/chromium/src/+/master:third_party/blink/renderer/modules/canvas/canvas2d/canvas_rendering_context_2d.cc

That code does not interact directly with the GPU. It calls into Skia.

 

I hope I can spot some differences between the different rendering backends.
What are these anyway? Are they both provided by Skia (software/cpu- and hardware/gpu-based)?

Yes, exactly. Skia provides the cpu and gpu backends that are used for rendering canvas contents.  The GPU backend does not interact directly with the graphics driver though, there are several layers in between that handle platform abstractions, provide workarounds for driver bugs and mitigate security vulnerabilities. For example, on Windows builds of Chrome, skia makes OpenGLES3 calls that get converted to Direct3D calls by a library called ANGLE.

The text rendering part of skia taps into a whole other ecosystem of open source libraries, which I am less familiar with.

I think that summarizes the parts of the graphics stack you'd be most interested in. The other big part I did not get into is the inter-process communication that is necessary to run all GPU calls in a separate process.  But that does not really affect fingerprinting.

I think I have to sleep over it.
Thanks so far, I'll message you later on.

P.S.: If there is a better place to ask such questions just direct me there.

Other than graphics developers, there are security folks who know a lot about fingerprinting.