Tbn Ffmpeg

[Frederic Laureano]

FFmpeg now implements a native xHE-AAC decoder. Currently, streams without (e)SBR, USAC or MPEG-H Surround are supported, which means the majority of xHE-AAC streams in use should work. Support for USAC and (e)SBR is coming soon. Work is also ongoing to improve its stability and compatibility. During the process we found several specification issues, which were then submitted back to the authors for discussion and potential inclusion in a future errata.

The FFmpeg community is excited to announce that Germany's Sovereign Tech Fund has become its first governmental sponsor. Their support will help sustain the maintainance of the FFmpeg project, a critical open-source software multimedia component essential to bringing audio and video to billions around the world everyday.

A new major release, FFmpeg 7.0 "Dijkstra", is now available for download. The most noteworthy changes for most users are a native VVC decoder (currently experimental, until more fuzzing is done), IAMF support, or a multi-threaded ffmpeg CLI tool.

This release is not backwards compatible, removing APIs deprecated before 6.0. The biggest change for most library callers will be the removal of the old bitmask-based channel layout API, replaced by the AVChannelLayout API allowing such features as custom channel ordering, or Ambisonics. Certain deprecated ffmpeg CLI options were also removed, and a C11-compliant compiler is now required to build the code.

The libavcodec library now contains a native VVC (Versatile Video Coding) decoder, supporting a large subset of the codec's features. Further optimizations and support for more features are coming soon. The code was written by Nuo Mi, Xu Mu, Frank Plowman, Shaun Loo, and Wu Jianhua.

The libavformat library can now read and write IAMF (Immersive Audio) files. The ffmpeg CLI tool can configure IAMF structure with the new -stream_group option. IAMF support was written by James Almer.

Thanks to a major refactoring of the ffmpeg command-line tool, all the major components of the transcoding pipeline (demuxers, decoders, filters, encodes, muxers) now run in parallel. This should improve throughput and CPU utilization, decrease latency, and open the way to other exciting new features.

This release had been overdue for at least half a year, but due to constant activity in the repository, had to be delayed, and we were finally able to branch off the release recently, before some of the large changes scheduled for 7.0 were merged.

Internally, we have had a number of changes too. The FFT, MDCT, DCT and DST implementation used for codecs and filters has been fully replaced with the faster libavutil/tx (full article about it coming soon).
This also led to a reduction in the the size of the compiled binary, which can be noticeable in small builds.
There was a very large reduction in the total amount of allocations being done on each frame throughout video decoders, reducing overhead.
RISC-V optimizations for many parts of our DSP code have been merged, with mainly the large decoders being left.
There was an effort to improve the correctness of timestamps and frame durations of each packet, increasing the accurracy of variable frame rate video.

A few days ago, Vulkan-powered decoding hardware acceleration code was merged into the codebase. This is the first vendor-generic and platform-generic decode acceleration API, enabling the same code to be used on multiple platforms, with very minimal overhead. This is also the first multi-threaded hardware decoding API, and our code makes full use of this, saturating all available decode engines the hardware exposes.

Those wishing to test the code can read our documentation page. For those who would like to integrate FFmpeg's Vulkan code to demux, parse, decode, and receive a VkImage to present or manipulate, documentation and examples are available in our source tree. Currently, using the latest available git checkout of our repository is required. The functionality will be included in stable branches with the release of version 6.1, due to be released soon.

As this is also the first practical implementation of the specifications, bugs may be present, particularly in drivers, and, although passing verification, the implementation itself. New codecs, and encoding support are also being worked on, by both the Khronos organization for standardizing, and us as implementing it, and giving feedback on improving.

A new major release, FFmpeg 6.0 "Von Neumann", is now available for download. This release has many new encoders and decoders, filters, ffmpeg CLI tool improvements, and also, changes the way releases are done. All major releases will now bump the version of the ABI. We plan to have a new major release each year. Another release-specific change is that deprecated APIs will be removed after 3 releases, upon the next major bump. This means that releases will be done more often and will be more organized.

New decoders featured are Bonk, RKA, Radiance, SC-4, APAC, VQC, WavArc and a few ADPCM formats. QSV and NVenc now support AV1 encoding. The FFmpeg CLI (we usually reffer to it as ffmpeg.c to avoid confusion) has speed-up improvements due to threading, as well as statistics options, and the ability to pass option values for filters from a file. There are quite a few new audio and video filters, such as adrc, showcwt, backgroundkey and ssim360, with a few hardware ones too. Finally, the release features many behind-the-scenes changes, including a new FFT and MDCT implementation used in codecs (expect a blog post about this soon), numerous bugfixes, better ICC profile handling and colorspace signalling improvement, introduction of a number of RISC-V vector and scalar assembly optimized routines, and a few new improved APIs, which can be viewed in the doc/APIchanges file in our tree. A few submitted features, such as the Vulkan improvements and more FFT optimizations will be in the next minor release, 6.1, which we plan to release soon, in line with our new release schedule. Some highlights are:

FFmpeg 5.0 "Lorentz", a new major release, is now available! For this long-overdue release, a major effort underwent to remove the old encode/decode APIs and replace them with an N:M-based API, the entire libavresample library was removed, libswscale has a new, easier to use AVframe-based API, the Vulkan code was much improved, many new filters were added, including libplacebo integration, and finally, DoVi support was added, including tonemapping and remuxing. The default AAC encoder settings were also changed to improve quality. Some of the changelog highlights:

Note that this filter is not FDA approved, nor are we medical professionals. Nor has this filter been tested with anyone who has photosensitive epilepsy. FFmpeg and its photosensitivity filter are not making any medical claims.

That said, this is a new video filter that may help photosensitive people watch tv, play video games or even be used with a VR headset to block out epiletic triggers such as filtered sunlight when they are outside. Or you could use it against those annoying white flashes on your tv screen. The filter fails on some input, such as the Incredibles 2 Screen Slaver scene. It is not perfect. If you have other clips that you want this filter to work better on, please report them to us on our trac.

We are not professionals. Please use this in your medical studies to advance epilepsy research. If you decide to use this in a medical setting, or make a hardware hdmi input output realtime tv filter, or find another use for this, please let me know. This filter was a feature request of mine since 2013.

This has been a long time coming but we wanted to give a proper closure to our participation in this run of the program and it takes time. Sometimes it's just to get the final report for each project trimmed down, others, is finalizing whatever was still in progress when the program finished: final patches need to be merged, TODO lists stabilized, future plans agreed; you name it.

Stanislav Dolganov designed and implemented experimental support for motion estimation and compensation in the lossless FFV1 codec. The design and implementation is based on the snow video codec, which uses OBMC. Stanislav's work proved that significant compression gains can be achieved with inter frame compression. FFmpeg welcomes Stanislav to continue working beyond this proof of concept and bring its advances into the official FFV1 specification within the IETF.

Petru Rares Sincraian added several self-tests to FFmpeg and successfully went through the in-some-cases tedious process of fine tuning tests parameters to avoid known and hard to avoid problems, like checksum mismatches due to rounding errors on the myriad of platforms we support. His work has improved the code coverage of our self tests considerably.

Umair Khan updated and integrated the ALS encoder to fit in the current FFmpeg codebase. He also implemented a missing feature for the ALS decoder that enables floating-point sample decoding. FFmpeg support for MPEG-4 ALS has been improved significantly by Umair's work. We welcome him to keep maintaining his improvements and hope for great contributions to come.

Jn Sebechlebsk's generic goal was to improve the tee muxer so it tolerated blocking IO and allowed transparent error recovery. During the design phase it turned out that this functionality called for a separate muxer, so Jn spent his summer working on the so-called FIFO muxer, gradually fixing issues all over the codebase. He succeeded in his task, and the FIFO muxer is now part of the main repository, alongside several other improvements he made in the process.

Jai Luthra's objective was to update the out-of-tree and pretty much abandoned MLP (Meridian Lossless Packing) encoder for libavcodec and improve it to enable encoding to the TrueHD format. For the qualification period the encoder was updated such that it was usable and throughout the summer, successfully improved adding support for multi-channel audio and TrueHD encoding. Jai's code has been merged into the main repository now. While a few problems remain with respect to LFE channel and 32 bit sample handling, these are in the process of being fixed such that effort can be finally put in improving the encoder's speed and efficiency.

c80f0f1006