2d Compositor

[Latarsha Dorrance]

Ittakes many years working in the animation industry to become a compositor. However, some companies have junior compositor roles, which give you the opportunity to develop into a senior compositor position. You might get into a junior compositor role straight after college or university or you might start in a related role, such as a roto artist or modeller, and work your way into the compositor role from there.

At school or college:

You can take A-levels or Highers in fine art, art and design, graphic design, or film studies. Or you might want to take any of the following Level 3 vocational qualifications:

Build a portfolio:

Learn animation and video editing software and start creating work that you can show to admissions tutors or employers. This is essential. Go to build your animation portfolio to learn how.

Take a short course:

Hone your skills in animation by taking a specialist course. Go to the list of training courses recommended by ScreenSkills and see if there is one that will improve your skills in compositing.

Network:

Get to know professionals in the animation industry by attending events. Meet with them and ask questions about their work, while demonstrating interest in and knowledge of the industry. Offer to provide them with your professional contact details and try to stay in touch with them. Go to how to network well to learn how to do this.

Early home computers and video games had 8-bit CPUs and only a few kilobytes of RAM. Some computers, like the Apple II, allocated some of the RAM as a frame buffer and used the CPU to fill it, but this approach is quite limiting. So designers of most other systems of the time got creative - in particular, they augmented the CPU with a graphics processing engine. This engine did a fair amount of processing during scanout, in other words on the fly as the system generated a video signal.

The combination of even a modest CPU and one of these video processing engines led to a creative explosion of games, with a unique aesthetic formed from the primitives provided by the video chip: colorful scrolling backgrounds formed from tiles, with the player avatar and other characters overlaid, also moving dynamically. One of the most popular (and best documented) such chips is the C64 VIC-II, and of course the NES PPU is another classic example. People still create software for these beloved systems, learning from tutorials such as VIC-II for beginners.

While the constraints of the hardware served as artistic inspiration for a diverse and impressive corpus of games, they also limited the kind of visual expression that was possible. For example, any attempt at 3D was primitive at best (though there were driving games that did impressive attempts, using the scrolling and color palette mechanisms to simulate a moving roadway).

An important aspect of all these systems, even those as underpowered as the Atari 2600, is that latency was very low. A well coded game might process the inputs during the vertical blanking interval, updating scroll and sprite coordinates (just a few register pokes), so they apply to the next frame being scanned out - latency under 16ms. Similar math applies to the latency of typing, which is why the Apple IIe scores so well on latency tests compared to modern computers.

The rendering of UI and 2D graphics continued to evolve during this time as well, with proportional spaced fonts becoming the standard, and antialiased font rendering slowly becoming standard as well during the 90s. (Likely the Acorn Archimedes was the first to ship with antialiased text rendering, around 1992)

A trend at the same time was the ability to run multiple applications, each with their own window. While the idea had been around for a while, it burst on to the scene with the Macintosh, and the rest of the world caught up shortly after.

When OS X (now macOS) first shipped in 2001, it was visually striking in a number of ways. Notably for this discussion, the contents of windows were blended with full alpha transparency and soft shadows. At the heart of this was a compositor. Applications did not draw directly to the screen, but to off-screen buffers which were then composited using a special process, Quartz Compositor.

Using the GPU to just bitblt window contents is using a tiny fraction of its capabilities. In the compositing process, it could be fading in and out alpha transparency, sliding subwindows around, and applying other effects without costing much at all in performance (the GPU is already reading all the pixels from the offscreen buffers and writing to the display surface). The only trick is exposing these capabilities to the application.

Core Animation was also made available in Mac OS 10.5 (Leopard). The corresponding Windows version was DirectComposition, introduced in Windows 8 and a core feature of the Metro design language (which was not very popular at the time).

On the desktop side, Windows 8.1 brought Multiplane overlay support, which seems to be motivated primarily by the needs of video games, particularly running the game at lower resolution than the monitor and scaling, while allowing UI elements such as notifications to run at the full resolution. Doing the scaling in hardware reduces consumption of scarce GPU bandwidth. Browers also use overlays for other purposes (I think mostly video), but in mainstream GUI applications their use is pretty arcane.

In order to avoid visual artifacts, these two paths must synchronize, so that the window frame and the window contents are both rendered based on the same window size. Also, in order to avoid additional jankiness, that synchronization must not add significant additional delay. Both these things can and frequently do go wrong.

Keep in mind that the existing compositor design is much more forgiving with respect to missing deadlines. Generally, the compositor should produce a new frame (when apps are updating) every 16ms, but if it misses the deadline, the worst that can happen is just jank, which people are used to, rather than visual artifacts.

For a simple tile, the compositor just uploads the metadata to the hardware. For a complex tile, the compositor schedules the tile to be rendered by the beam racing renderer, then uploads metadata just pointing to the target render texture. The only real difference between simple and complex tiles is the power consumption and contention for GPU global memory bandwidth.

-us/blog/2015/02/12/weston-repaint-scheduling/ - Good blog post with diagrams explaining how Weston reduces compositor latency by delaying compositing a configurable amount after the last vblank, and how game style rendering only gets the full latency improvement if it renders using the presentation time feedback rather than the normal frame callback.

- PR introducing the same model in Sway, except with a configurable per-app property to adjust when the frame callbacks are to trade off between available time in the deadline and latency. Seems to default to off.

-protocols/tree/stable/presentation-time/presentation-time.xml - Wayland presentation time feedback protocol with comments. Includes flags for if the presentation time is an estimate or a hardware timestamp from the display, as well as if presentation was done zero-copy.

In the meantime, applications need to make very difficult tradeoffs. Using the full capabilities of the compositor API can make scrolling, transitions, and other effects happen more smoothly and with less GPU power consumption than taking on the rendering tasks themselves. About a year ago, Mozilla made changes to leverage the compositor more on macOS, and was able to show fairly dramatic power usage improvements.

At the same time, exposing compositor capability in a cross-platform way seems harder than exposing GPU rendering, and as WebGPU lands that will be even more true. And, as mentioned, it may get in the way of the benefits of DirectFlip.

This is a task that tracks the current state of the Realtime Compositor project and lists some of its to-dos. For user-level overview and demos, see the [blog post](https:*code.blender.org/2022/07/real-time-compositor/) on the matter. For feedback...

Contributors to this topic are expected to provide feedback specifically regarding the aforementioned UI/UX concerns. For now, discussions should be limited in scope to the viewport compositor, while discussions regarding new features and long term goal should be avoided for the moment.

I really appreciate the reading of that document. The part you spoke about the future work " reuse of the compositor node tree " will make the editing faster.

Where can we download and try you patch .

On the Hue Saturation Value node, at present the Saturation is automatically clamped. There are cases (like implementing an additive keyer for fine greenscreen detail) where negative values generated earlier in the node tree need to be preserved through the HSV node.

I would prefer one shared node tree with options to split parts of it for viewport and for render. Similar to the shader nodes that have two different outputs that let you choose between Cycles and Eevee, but for Viewport and render.

The new compositor was designed to support infinite canvas compositing from the start, and it defines rules for how such things work, and we will likely add new nodes that helps in that area. I intend to write some documentation about this at some point since we only have technical documentation that describe the concept.

But in general, the compositor will never stretch or scale any images to fit other images, each image will be sampled from the exact area it occupies in the virtual compositing space, and the operation would only be interested in the areas of the inputs that intersects its area of interest, which is typically determined from its main input.

question, all the data used in the node editor needs to be stored somewhere, in the normal editor its the normal ram, in this viewport composite is it still saved in the ram or its stored in the vram?

3a8082e126