Tracking CMake in the VFX Platform

[Neal Gompa]

Hey all,

One thing I noticed we do in the aswf-docker image is track and
upgrade CMake[1]. It makes sense that we do this because we rely on
CMake to be able to detect the other features and components for use
and CMake is the preferred interface to building CUDA enabled
applications (which the industry does a lot of).

Could we start explicitly tracking expected minimum CMake versions
too? That would help with assuring an experience around things like
C/C++ standards, CUDA capabilities, and so on.

For example, CMake introduced recognition of C17/C18/C23 and MSVC 2022
support in CMake 3.21. CMake 3.25 introduces support for nvtx3 and
nvPTX for CUDA. CMake 3.27 introduces an interactive debugger for
CMake scripts.

If we want to be able to ensure people have access to specific
features, capabilities, or dependencies, as well as ensure that
libraries know what to expect when trying to build against us, I think
it's worth us specifying this in the standard.

Thanks in advance and best regards,
Neal

[1]: https://github.com/AcademySoftwareFoundation/aswf-docker/blob/master/CHANGELOG.md

--
真実はいつも一つ！/ Always, there's only one truth!

[Jean-Francois Panisset]

For the aswf-docker containers, the unofficial "policy" is to pick the latest CMake available around the time work starts on a new yearly release, so for the 2024 containers, they will probably start with 3.27.5. For updated releases of a specific year, would likely increment the patch release (so 3.27.6 or newer). We just discussed that topic on the last ASWF CI WG call. Fortunately CMake has basically no external dependencies, so it's reasonably straightforward for a project to pick the version it wants.

So not an official VFX Platform mandated version, but perhaps something of a "de facto" standard?

JF

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAEg-Je9wXrkR6WHKAkU9Bmn5Z%2BnUhguZXiqYkDnymJe_phbqZw%40mail.gmail.com.

[Nick Cannon]

Since CMake does not directly contribute to Versionitis, we are not currently considering including it in the VFX Reference Platform.

I like JF's suggestion of a de facto standard based on the recipes created by the aswf-docker project. 

Nick 

[Neal Gompa]

My main reason for actually requesting it is because CMakeConfigs and
other types of CMake scripts used by dependencies and projects often
*do* depend on minimum levels of CMake functionality to work. It's
possible to get into a situation where CMake modules don't work
because features are missing in CMake or something similar.

This is why, for example, the OBS project takes great care of
supported CMake versions. And I think we need to be similarly mindful
so that projects circling this ecosystem have some idea of what to
expect people can use or want to use.

Newer versions of CMake are often strongly backwards compatible, but
having guidance on minimum expected CMake versions means that when
people are writing things for VFX, they can know what to expect is
available to them and do things like avoid features and functions that
don't exist versions that people are expected to be compatible with.

> To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/f7218d77-c25a-4842-829a-03cc5968a463n%40googlegroups.com.

[Larry Gritz]

I think it's important to emphasize that VFX Platform is not a standard, nor an endorsement. It's a list of shame. Historically, certain libraries were badly behaved (e.g., not properly namespacing or hiding their symbols), resulting in conflicts that made it hard for code that used the library to fit into the environments of multiple DCCs that didn't all agree on the same version of that library but exposed their symbols. So the list documents that several major DCC vendors have agreed to use the same versions of certain libraries and tools during a particular year, and then tell us what they agreed to so we know how to be compatible.

The goal of most libraries on the list ought to be to clean up their act enough that they can graduate and be removed from the list.

That's not to say that it wouldn't be helpful have some guidance on what minimum versions of build tools and whatnot we can expect studios to use. But VFX Platform is not attempting to solve that problem.

If you want to see what's widely available in practice, I usually refer people to this resource: https://docs.google.com/spreadsheets/d/1EwRlz5ZYObEOdBfIk8iTX5thlpTyEAfp3bxOgAfFOiU/edit

Most of the foundational open source packages in our ecosystem are on at least cmake 3.12, and some common ones everybody needs to build require 3.15, 3.18, or even newer. So you're pretty safe assuming ~3.15 or higher is generally available to everybody.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAEg-Je-KLa-s9L1_UYm7%2B1V9%2BAkjo35uz-CD4_0go_8yY1otHQ%40mail.gmail.com.

--

Larry Gritz

l...@larrygritz.com

[Neal Gompa]

On Sat, Sep 16, 2023 at 2:22 PM Larry Gritz <l...@larrygritz.com> wrote:
>
> I think it's important to emphasize that VFX Platform is not a standard, nor an endorsement. It's a list of shame. Historically, certain libraries were badly behaved (e.g., not properly namespacing or hiding their symbols), resulting in conflicts that made it hard for code that used the library to fit into the environments of multiple DCCs that didn't all agree on the same version of that library but exposed their symbols. So the list documents that several major DCC vendors have agreed to use the same versions of certain libraries and tools during a particular year, and then tell us what they agreed to so we know how to be compatible.
>
> The goal of most libraries on the list ought to be to clean up their act enough that they can graduate and be removed from the list.
>

So this is an interesting statement to make, because I wholly agree
with you and I am also seeing that important external stakeholders are
not treating it that way. They are treating it as a specification of
requirements for maintenance instead.

So I am starting to wonder again if we've been making any progress on
fixing the problems that led to this list being created?

[Larry Gritz]

I think the important guidance for external stakeholders boils down to:

"The major VFX DCC apps are going to coordinate to use these versions next year, so if you are building software (like plugins) that need to fit into those apps, or if you want your source code to build at the VFX studios against the compilers and libraries their pipelines need to use as a result, you should support those tool versions, too."

So I am starting to wonder again if we've been making any progress on
fixing the problems that led to this list being created?

As two examples:  OpenImageIO was never put on the list in the first place, didn't need to be, despite being very widely used. Imath (part of OpenEXR) has a plan moving forward to try to achieve a symbol and library stability that will make different versions fully interoperable and non-conflicting.

The DCCs themselves, also, have gotten better at the practice of, when they use these libraries internally, building them with custom namespaces (when supported by the libraries, which do in many cases) and/or hiding symbols in the libraries, to avoid conflicts. It's harder when a DCC not only embeds a library, but purposely exposes its types or methods in the public APIs of the DCC itself. That almost always forces the plugins to need the same version.

Realistically, some of the items on the list can probably be removed, or are close to it, but the community has been more focused on making sure the versions on the list are set correctly from year to year, than they have been about diligently removing the packages that are ready to "graduate."

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAEg-Je9kSbn5qgjiLeQygHaPPdpN2wpnFKdxzK7nFy5Ni4co6A%40mail.gmail.com.

--

Larry Gritz

l...@larrygritz.com

[Deke Kincaid]

It is really nice these days that I don’t need to have conversations with the main DCC makers anymore but I still have multiple conversations a year with the smaller plugin makers because their libraries will conflict when we try to extend things with their api’s.  Many plugins will pick the lowest common denominator, so if the lowest Maya version they support is v2018, all the libraries match that and the maya 2022, 23, etc are out of sync.  Some still have boost 1.61 and it is not namespaces.  Im often using this list as leverage which I can point to as a line in the sand.

Even if imath adds symbol stability I hope EXR itself stays on the list because there was also talk of symbol stability on that end but there has been rumbling of adding additional compressions to the format.  The other issue we run into that was really bad from the OpenExr 1.x to 2.x switch was that it took 5 years for many applications to have the ability to read 2.x files.  We could modify the writer in Nuke and other dcc’s to write 1.x compatible files but it was a pain.  In fact even dwa compression introduced in 2.2 took a fair number of years to get full support and we were often writing our own writers or recompiling libraries in order to add our own support for it since that was was pre-vfxplatform.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/5993B9FA-F884-48BE-86CD-8CE4F53D0D6C%40larrygritz.com.