Progressive Decoding

[Deandra Uleman]

Bothspectral selection and successive approximation require that the encoder and decoder traverse the image multiple times. Each iteration is called a scan. Typically, encoding a progressive JPEG takes about 10 scans, which is why, during decoding, an image transitions from being blurry to being sharp after about 10 refinement steps.

The ability to convert a progressive JPEG to a baseline JPEG adds flexibility in image handling. By opening a progressive JPEG in any image editing program and saving it as a standard JPEG, one can easily switch between these two formats based on the needs of the application or website. This flexibility is particularly useful for designers and web developers.

A less obvious advantage of progressive JPEGs is that they tend to be smaller in size than nonprogressive JPEGs even though the final image looks exactly the same. Because similar DCT coefficients across multiple blocks in progressive JPEGs are encoded together, smaller files result; whereas nonprogressive JPEGs, whose blocks are encoded one by one, weigh more. Oftentimes, the extra compression is only a few percentage points, yet it still saves bandwidth and storage without affecting the image quality.

Additionally, it takes more CPU time to encode and decode progressive JPEGs because the algorithm must go over the image data multiple times, not in one single scan. Plus, the process incurs more memory since all the DCT coefficients must be stored in memory during decoding. In nonprogressive decoding, you need to store only one block of coefficients at a time.

By default, most progressive JPEG encoders define the segment of the image data in each of the 10 scans with a scan script. Advanced encoders like MozJPEG try out different scripts and pick the one that results in the best compression, which might require fewer or more scans, depending on the image.

You can combine the advantages of progressive and nonprogressive encoding by customizing the scan script. After some experimentation and adoption of a few ideas from an inspiring talk by Tobias Baldauf, Cloudinary came up with the following scan script for encoding progressive JPEGs:

The scan script leverages spectral selection, not successive approximation, which has a larger negative impact on the decode speed because the same coefficient is revisited multiple times. Also, since the decode time lengthens in proportion to the number of scans, the script runs only five scans.

In this example, the nonprogressive JPEG is 283 KB; the default progressive JPEG, 271 KB; and the semiprogressive, 280 KB. Note that since the default progressive JPEG takes more refinement steps to process, it delivers a high-quality preview faster than the semiprogressive JPEG.

With the default progressive and semiprogressive scan scripts, it can be hard to tell exactly when the image has completely loaded. Whether or not this is a problem is debatable: after all, the progressive mechanism is doing its job of producing a high-quality preview fast.

Toward that end, some websites progressively render images in two steps: load small, low-quality placeholder images first and then replace them with the actual versions. Given the large gap between a placeholder image and an actual one, you can readily tell when loading is complete.

For comparison, the video below shows the various scan scripts in action as they scan the same exact image, but in a different order. From left to right in the video: nonprogressive, steep-progressive, semiprogressive, and default progressive.

So, how to make semiprogressive or steep-progressive JPEGs, you ask? Most image programs do not offer that option. However, you can copy and paste either of those two scan scripts into a text file on the command line and then run the following command with the libjpeg or MozJPEG encoder:

Certain new image formats, such as JPEG XL, also support progressive decoding . In general, because progessive decoding makes little sense in video, that approach does not work with the formats that are derived from video codecs (WebP, HEIC, AVIF).

We at Cloudinary are fully aware that, as the web evolves to be more and more visual and most of the downloaded content on a page is images and videos, image optimization is key to user experience. In an ongoing effort to get the most out of every image format, we continuously enhance our image encoding and processing algorithms for the best possible end-user experience.

Concurrently, we strive to make life as easy as possible for developers. Do add q_auto,f_auto to your image URLs to automatically take advantage of the benefits from best practices and new image formats, now and in the future.

Progressive decoding provides the ability to incrementally decode and render portions of an image before the entire image has finished downloading. This feature greatly improves the user's experience when viewing images from the Internet, because the user does not have to wait for the entire image to download before decoding can begin. Users are able to see an image preview with available data long before the entire image is downloaded. This feature is essential for any application used to view images from the Internet or from data sources with limited bandwidth.

The Windows Imaging Component (WIC) in Windows 7 supports progressive decoding of popular image formats such as JPEG, PNG, and GIF. WIC also supports any WIC-enabled non-Microsoft codecs that implement progressive decoding. Progressive encoding is not supported in the current release of WIC. This topic outlines progressive decoding in Windows 7 and the procedure for enabling progressive decoding in your applications.

Progressive decoding is the ability to incrementally decode portions of an image from an incomplete image file. Traditional decoding requires a complete image file before decoding can begin. Progressive decoding starts after a progressive level of an image has finished downloading. The decoder performs a decoding pass on the image's current progressive level. It then performs multiple decoding passes on the image as each progressive level is downloaded. Each decoding pass reveals more of the image until the image is fully downloaded and decoded. The number of passes required to decode a full image depends on the image file format and the encoding process used to create the image.

Windows 7 provides built-in codecs that support progressive decoding for JPEG, PNG, and GIF image formats. Each of these Windows 7 codecs perform multiple decoding passes on an image. Each pass corresponds to a particular level and portion of the image that is decoded, eventually leading to a fully decoded image.

Each image format handles progressive decoding in a different manner. The following table provides information about the number of progressive levels and the decoding method supported by the Windows 7 progressive decoding formats.

In addition, progressive decoding can be implemented in codecs by providing support for progressive interfaces and methods. If progressive decoding is not supported in a codec, then appropriate error messages should be returned if these methods are called.

JPEG progressive decoding presents image data at increasingly higher resolutions for each level, until the full-resolution image is available. Each level of the image is set to provide a different resolution level. As more progressive levels become available, the image is displayed at higher resolutions, until the full resolution image is resolved.

The number of available levels and the resolution set at each level depends entirely on the encoded JPEG. The following two images show an example of JPEG progressive decoding at two progressive levels.

PNG image files provide seven progressive levels for decoding, as described in the PNG specification. PNG progressive decoding is implemented by decoding a specified pattern of pixels on each pass of the decoder. The pattern in the following table from the PNG specification is replicated over the entire image. Each number represents the progressive level in which the corresponding pixel will be decoded.

From the table above, you can determine the pixels that will be decoded with each pass of the decoder. Unlike the Windows 7 GIF codec, the Windows 7 PNG codec replicates the left-most available pixel on a scan line to populate empty pixels.

The image at the top left shows a PNG image decoded at progressive level 0. The top-right image shows the same PNG image decoded at progressive level 3. The bottom image shows the same image fully decoded after 7 progressive levels.

GIF image files provide four progressive levels for decoding, as described in the GIF specification. Each pass populates certain rows within an image, producing a full image after the fourth pass. The following table from the GIF specification shows which scan lines are decoded by each pass of the decoder.

The main progressive decoding interface is the IWICProgressiveLevelControl interface. To obtain a reference to the interface, query an image frame (IWICBitmapFrameDecode) for IWICProgressiveLevelControl. Progressive methods can then be accessed from the interface.

The preceding code provides the basic functionality necessary for implementing progressive decoding in most applications. Using the code, progressive levels can be accessed as image pixel data becomes available. The SetCurrentLevel function blocks execution until the level being requested is available.

Codec developers may choose to implement the IWICProgressiveLevelControl if their image formats support progressive decoding. Support for progressive decoding is not a requirement for discovery and arbitration by WIC. However, progressive decoding greatly enhances the user experience, and implementation should be considered if possible.

Progressive image decoding is an excellent way in which to accelerate page loads and hence improve the web-browsing experience. This post explains why and elaborates on the recent developments for that approach.

3a8082e126