Feb 25, 2022, 3:57:10 PM2/25/22
Looking at the libvpx project, I'm thinking that perhaps libvpx shouldn't use assembly language for anything. Using assembly language as opposed to intrinsics adds much complexity to the project.
For one, it complicates the build process. The requirement to use Yasm is a problem in some environments.
Additionally, supporting every ABI out there is difficult. As an example, the existing assembly code does not follow the Windows x64 calling convention fully: there is no exception metadata for stack unwinding. Crashes in the libvpx assembly code won't have a stack trace in Windows because of this.
The assembly code I've seen in libvpx looks like it would be better off as intrinsics. It likely would be slightly slower, but you'd gain in maintenance advantages.
Much of this is likely true of libdav1d as well, but I haven't checked.