CY2024 Draft Published

[Nick Cannon]

The Draft CY2024 VFX Reference Platform has been published at https://vfxplatform.com.

The significant change is the move to Qt 6 which we expect to be quite an undertaking for software providers, although not as challenging as the move from Qt 4 to 5 for those who were around for that.

The original plan was to move TBB and MKL to their oneAPI equivalents in CY2024 but the critical dependencies needed by the DCCs will not have all migrated in time for our cut-off date. So, our guidance is now that we will move to oneTBB and oneMKL for CY2025 which means all dependencies need to have released versions with full support by September 1st, 2024. If anyone needs additional help with the migration then the TBB team at Intel have been great partners and may be able to provide additional guidance and support.

To provide feedback on this Draft Platform, reply to this thread or respond privately to the Working Group by emailing feedback at vfxplatform.com.

We are looking to lock down the Final version at the start of September after discussing it at the virtual Birds of a Feather event at SIGGRAPH in August.

Nick

[Michael Rochefort]

Awesome, thanks for the notification!

Regarding TBB and MKL, is there a list of the blocking software dependencies (outside of those that are already part of the spec)? I wonder if there's anything we could contribute (at least those more knowledgeable in C++) to accelerate the progress of those projects.

Cheers,

Mike

[Jesse Y]

The oneAPI delay is frustrating and unfortunate.  Why did the reference platform not follow up throughout the year to ensure that the one bad project (who's not even in the reference platform) was on track to get things done?

[Deke Kincaid]

Pixar contributes a significant portion of it’s development.  We have all hooked ourselves to the USD wagon so unless someone else can pony up development for this it seems like we are stuck.

From the OpenUSD mailing list there are issues with changes to OneTBB and Pixar’s internal software which is a requirement before Pixar can commit resources to move OpenUSD to it.  Here are the tickets Spiff mentioned are wip towards this goal.

https://github.com/PixarAnimationStudios/USD/pull/2087

https://github.com/PixarAnimationStudios/USD/pull/1908

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/3b45d054-81c3-405f-868d-9772379c8752n%40googlegroups.com.

[Michael Rochefort]

Separating out into a different subthread, but regarding GCC selection, is there a particular reason vendors are choosing to stick with GCC 11.2 instead of moving to a newer variant like 12.x (or 13 if they're feeling super adventurous)?

Seeing as how the draft is sticking with what is essentially an EL8 compatible build environment, the following GCCs (gcc/gcc-c++) and their history are currently provided in the EL8 platform (as of 8.8.0):

gcc.x86_64                    8.2.1-3.5.el8
gcc.x86_64                    8.3.1-4.5.el8
gcc.x86_64                    8.3.1-5.el8
gcc.x86_64                    8.3.1-5.1.el8
gcc.x86_64                    8.4.1-1.el8
gcc.x86_64                    8.5.0-3.el8
gcc.x86_64                    8.5.0-4.el8_5
gcc.x86_64                    8.5.0-10.el8
gcc.x86_64                    8.5.0-10.1.el8_6
gcc.x86_64                    8.5.0-15.el8
gcc.x86_64                    8.5.0-16.el8_7
gcc.x86_64                    8.5.0-18.el8
gcc-toolset-9-gcc.x86_64      9.1.1-2.4.el8
gcc-toolset-9-gcc.x86_64      9.2.1-2.2.el8
gcc-toolset-9-gcc.x86_64      9.2.1-2.3.el8
gcc-toolset-10-gcc.x86_64     10.2.1-2.el8
gcc-toolset-10-gcc.x86_64     10.2.1-8.2.el8
gcc-toolset-10-gcc.x86_64     10.3.1-1.2.el8_5
gcc-toolset-11-gcc.x86_64     11.2.1-1.1.el8
gcc-toolset-11-gcc.x86_64     11.2.1-1.2.el8_5
gcc-toolset-11-gcc.x86_64     11.2.1-9.1.el8
gcc-toolset-12-gcc.x86_64     12.1.1-3.2.el8
gcc-toolset-12-gcc.x86_64     12.1.1-3.4.el8_7
gcc-toolset-12-gcc.x86_64     12.2.1-7.4.el8

If vendors are interested in GCC 13, it should be available as an additional gcc-toolset in the distribution as late as RHEL 8.9, which would be released approximately in November.

GCC 12 release notes: https://gcc.gnu.org/gcc-12/changes.html

GCC 13 release notes: https://gcc.gnu.org/gcc-13/changes.html

I would be curious to see if, besides performance and optimization enhancements provided by the newer compilers, there are any new features that would be advantageous to use for our projects.

Cheers,
Mike

On Tuesday, May 16, 2023 at 11:48:04 AM UTC-4 nick....@gmail.com wrote:

[Neal Gompa]

The big advantage of GCC 13 is that it has much of the C23 and C++23
standards implemented ahead of GCC 14 next year.

As a heads-up to everyone: GCC 14 will be updating the default
behaviors for C code compilation to align more with C23:
https://fedoraproject.org/wiki/Changes/PortingToModernC

I think it would be a good idea to also indicate C standard minimums
alongside C++ standard minimums. I don't know how aggressive this
community is, but I know that KDE requires C++20 and C17/C18 now with
the upcoming KDE 6 Platform (KF6, Plasma 6, etc.). I think it'd be a
good idea to apply similar guidance for the VFX community. This will
be especially helpful once C23 and C++23 become fully available with
GCC 14, as the two standards are largely alignment standard revisions
(making the two more interoperable and compatible).

> --
> You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/4438a30a-c738-4b5a-b727-f14e8a339905n%40googlegroups.com.



--
真実はいつも一つ！/ Always, there's only one truth!

[Nick Cannon]

On Tue, May 16, 2023 at 2:19 PM Jesse Y <jes...@gmail.com> wrote:

The oneAPI delay is frustrating and unfortunate.  Why did the reference platform not follow up throughout the year to ensure that the one bad project (who's not even in the reference platform) was on track to get things done?

The VFX Reference Platform is a community initiative run by volunteers working on their own time so bandwidth is unfortunately limited. The idea is that all community members are encouraged to provide feedback to projects and hold each other accountable for aligning on a common platform and synchronizing disruptive migrations like this one.

In this case, earlier intervention is unlikely to have resulted in a different outcome. For complex software products like USD and the DCCs which have exacting performance requirements, the move to oneTBB is non-trivial, and achieving equivalent performance to TBB can require significant additional code. This is something the TBB team at Intel has been working on, and they are planning improvements in upcoming releases. The purpose of the delay is not only to accommodate those dependencies that do not yet support oneTBB, but also to allow the TBB team time to deliver these improvements.

Nick

[Nick Cannon]

Good questions about the GCC version, and there has also been further discussion and feedback on this topic over on the ASWF Slack.

 is there a particular reason vendors are choosing to stick with GCC 11.2 instead of moving to a newer variant like 12.x (or 13 if they're feeling super adventurous)?

It primarily comes down to balancing the amount of effort involved in changing compiler version against competing priorities for product features. Some of the software vendors have a preference for only changing compiler versions every 2 to 3 years rather than every year.

Over in that Slack thread, Larry Gritz nicely encapsulated the case for upgrading the compiler more frequently:

(a) performance -- the compilers actually get better with each major release, and that translates to real $$ at the scale we operate;
(b) developer productivity gains from being able to use the new language and compiler features;
(c) access to libraries, tools, documentation, and published techniques that are developed outside our industry, not beholden to the VFX Platform, and therefore liberally use features we can't be compatible with for years to come;
(d) increased ability to attract and retain developers to our companies and our open source projects without having them seduced away by places or projects where they would be allowed to use tools from the current decade.

I'll take that feedback back to the Working Group for us to reconsider the GCC version decision in our next meeting. In the meantime, if anyone else here has a strong opinion about whether we should or shouldn't change GCC version for CY2024 then please let us know.

Nick

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAEg-Je8ZdwuQ4mcHyS3pZg01t6GL7DLh36wdOwYRH28YSESm6w%40mail.gmail.com.

[ma...@pixar.com]

First off, thanks to Nick, Francois, and others on the VFX Platform team.  You all go to a lot of hard work, and what you do is very appreciated!!

For those of you that don't know me, I'm the Director of Product Management for RenderMan and Tractor.  I have personally done several of the past benchmarks we use to monitor changes in performance when we move from one version of the VFX Reference Platform to another.

There are always two sides (more than two sides?) to every conversion, and in a complex landscape such as VFX software, there are the studios who use it, the vendors who supply it, and the open source community -- figuring out the right way forward is full of compromise.  So I thought I'd give our perspective as RenderMan.

While we believe in moving the platform definition forward as fast as we can, there is the reality of how much effort it takes to move to it, as well as the opportunity cost of doing so.  While the benefits have all been outlined here and in previous conversations, there are also drawbacks.

* Upgrading compilers can have unintended performance compromises, and there is just as much risk of having the compromises as there are benefits.  Most of the time, when we upgrade compilers it is neutral, or we see only very very minor gains.  However, we had one instance several years ago (maybe 2017) where on many of our test scenes, we saw what I recall as a 10% performance degradation.  Not on all of the test scenes -- but on enough of them to raise eyebrows.  There was nothing we could do to easily get it back, resulting in real costs for studios.

* Upgrading compilers can cause previously working software to fail in strange and hard-to-debug ways.  Humans make mistakes, and as much as I'd like to say RenderMan has the best code in the world, the reality is that humans work on it.  Humans work on the compilers too.  We've had areas of code that are coded properly, but when we upgrade compilers, that previously working code has new bugs.  Perhaps it is something with the optimizer that doesn't work any more.  Or that we were relying on an unintended side-effect that got fixed by the new version of the compiler.  Either way, we need to spend time tracking the issues down and solving them.

* The opportunity cost of the effort.  Upgrading compilers is one of the more intensive things we do in the tick-tock of the VFX Reference Platform upgrade.  It takes a huge amount of effort.  It takes our developers away from working on artist-facing productivity and creativity-enhancing features.  That feature work can be extremely beneficial to studios, so we have to work on the right balance of upgrading our compile + library infrastructure compared to working on new features and use cases.

I don't want to sound like we should never move forward.  We should.  We must.  And I do hear the concerns from developers about wanting to improve their productivity.  These are all good things.  Moving forward does help improve the quality of what we offer, as well as bringing new feature goodness to everyone.

But from RenderMan's perspective, taking time to breathe between things like compiler upgrades is a good thing.  We feel that the tick-tock nature of the VFX Reference Platform gives us the best chance to stay (reasonably) current, while at the same time giving us more space to innovate in other ways to provide value (and hopefully happier lives) for the artists and studios that use RenderMan -- ultimately allowing artists to raise their level of creativity, and for productions to come in on-time and on-budget.

Mark

[Robert Fanner]

I work closely with Foundry's engineering teams (as a Director of Engineering) in relation to VFX Reference Platform work.

Mark's input beautifully captures many touchpoints that are equally germane from our perspective, and for similar reasons.

All the best,

Rob

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/060cd76f-3811-4bd1-b348-0ca93672f0bfn%40googlegroups.com.

--

Rob Fanner

Senior Director of Engineering

Web: www.foundry.com

The Foundry Visionmongers Ltd.  -  Registered in England and Wales No: 4642027  -  Address: 5 Golden Square, London, W1F 9HT  -  VAT No: 672742224

[Pierre Jasmin]

Resending, I think I used the wrong email address for this list

Sorry

Can someone refresh me about what g++ version breaks what again?
I vaguely remember something about strings but was not paying attention :)
I would like to test and regress to old apps to see if something breaks here.
Already have a dev whose road laptop can't dual boot windows 11 and latest Rocky... his other road laptop is a macbook and parallels is not good as some apps require GPU libs...

Note for us (plug-in developer for 20 apps) hosts like Foundry and BMD apps still have centOS 7 still as minimal requirement...
We regularly parked old versions in previous archive as as we move ahead for people who don't upgrade their host permanent license or for other reasons but we have to work with whatever products like Nuke, Baselight, Autograph or Fusion/Resolve etc  require.

Pierre

[David Aguilar]

On Thu, Jun 1, 2023 at 11:07 AM Pierre Jasmin <pierre...@gmail.com> wrote:

Resending, I think I used the wrong email address for this list

Sorry

Can someone refresh me about what g++ version breaks what again?
I vaguely remember something about strings but was not paying attention :)
I would like to test and regress to old apps to see if something breaks here.
Already have a dev whose road laptop can't dual boot windows 11 and latest Rocky... his other road laptop is a macbook and parallels is not good as some apps require GPU libs...

Note for us (plug-in developer for 20 apps) hosts like Foundry and BMD apps still have centOS 7 still as minimal requirement...
We regularly parked old versions in previous archive as as we move ahead for people who don't upgrade their host permanent license or for other reasons but we have to work with whatever products like Nuke, Baselight, Autograph or Fusion/Resolve etc  require.

Pierre

Hi Pierre,

From the perspective of the reference platform, the switch from gcc9 in vfx2022 to gcc11 in vfx2023 is an ABI-breaking change.

If you're on rhel/centos7 and using compilers from the devtoolset packages it's important to note that -D_GLIBCXX_USE_CXX11_ABI=0 is the default behavior because all of the centos/rhel7-era devtoolset compilers (even gcc9) were built with these settings:

"""

$ gcc -v

Using built-in specs.
COLLECT_GCC=/opt/rh/devtoolset-9/root/usr/bin/gcc
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-9/root/usr --mandir=/opt/rh/devtoolset-9/root/usr/share/man --infodir=/opt/rh/devtoolset-9/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --with-default-libstdcxx-abi=gcc4-compatible --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-9.3.1-20200408/obj-x86_64-redhat-linux/isl-install --disable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC)

"""

The key bit is --with-default-libstdcxx-abi=gcc4-compatible option which makes it so that the older pre-c++11 "gcc4" ABI is used by default. Without that flag newer gcc versions would default to the new ABI.

Once you're on centos8 or centos9 the compilers are no longer built with that flag so they effectively default to -D_GLIBCXX_USE_CXX11_ABI=1 aka the "new ABI".

The simplest way to think about it is that you should be compiling on centos7 / old ABI for software built against vfx2022 and earlier.

For vfx2023 and newer you should be compiling on centos8 or newer, where the new ABI is the default.

https://developers.redhat.com/blog/2015/02/05/gcc5-and-the-c11-abi

https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html

The articles above mention gcc5 as the breaking point, but remember that the usage of gcc6 and gcc9 in the reference platform is with builds of these compilers and #defines that enable the old ABI.

As far as vendor adoption of vfx2023, here's my current understanding. I don't represent these companies, but I'm just sharing what I heard from the grapevine.

Maya 2024 (currently available in alpha form IIUC) will be the vfx2023-compliant Maya.

Houdini 20 (alpha expected at the end of June, final sometime in Q4) will be the vfx2023-compliant Houdini.

Can someone from Foundry share rough dates around when a vfx2023-compliant Nuke (15?) might be available?

It's best for everything to be built with the same compiler versions and -std=c++XY settings. I've run into issues when mixing gcc6 c++14 shared libs / plugins with gcc9 c++17 tools (both using the old ABI) and vice-versa, and ditto between c++14 and c++11. It's important for vendor software to use the same ABI, -std=c++XY compiler flags, compiler versions and library versions as specified in the reference platform to avoid interop issues. This does mean that your dev/build environments will have to upgrade beyond centos7 in order to start developing for the newer vfx2023-era apps.

Hope that helps,

--

David

[Robert Fanner]

Can someone from Foundry share rough dates around when a vfx2023-compliant Nuke (15?) might be available?

No firm dates to share at the moment, but our porting efforts are already well underway. We'll be testing all Foundry products supporting VFX Reference Platform CY2023 on Rocky 9.1, with basic testing on matching versions of Alma and RHEL. Alpha builds of VFX RP CY2023 for Nuke, Katana and Mari will be available for testing in the coming months ahead of releases later in the year.

Best,

Rob

[Brock Taylor]

Robert, feel free to contact me if you run into any issues with Rocky. We're working on packaging the VFX platform on Rocky to better support activities like this.

Thanks,

Brock

[Pierre Jasmin]

Back to this.

Context: In OpenFX we can if need be put two shared libraries (plug-ins) in the same file and on load plug-in can select the one to load. 

What is the simplest, reliable simple check ( that >= this) that will work for all Linux distros (there are some hosts built against Ubuntu and even one using SUSE and we have seen users on Fedora too - not something we can control here).The API is classless and maps everything to a C core API so c++11 / 14 should not be an issue I think.

Pierre

[Robert Fanner]

Thanks Brock - I'd missed off replying to this kind note earlier.

I'm confident that would be a boon to many organisations, and so I'm encouraged by this initiative.

We might not be able to directly leverage this work for product releases, as they're currently backed by a cross-OS Conan package management stack, but I look forward to seeing where this work goes in future!

All the best,

Rob

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/b085daaa-7552-4bf8-b5d5-77fa4d1913d5n%40googlegroups.com.

[Raphaël Ortiz]

Another thing to maybe consider regarding the GCC version is compatibility with nvcc (NVIDIA compiler). For example CUDA 11.x works with GCC 6-11, CUDA 12 with GCC 6-12. https://gist.github.com/ax3l/9489132

More generally is there any plan to specify anything related to GPU support in the VFX platform? I guess that the fact that it's mostly closed source might be a problem.

Best,
Raphael

[Larry Gritz]

Closed source is not a problem. It's not a list of open source projects, nor is it a list of recommendations or endorsements of what software to use.

It's just a list of software that have historically caused problems for studios and vendors when everybody doesn't use the same version, and for each one, the version to use in each year if you want to be on the same page as the major software vendors in order to avoid those conflicts.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/63baf92e-e82b-4298-a2f1-7b698d44ebb9n%40googlegroups.com.

--

Larry Gritz

l...@larrygritz.com

[Nick Cannon]

On Tue, Jul 11, 2023 at 9:05 AM Raphaël Ortiz <raphae...@disneyresearch.com> wrote:

Another thing to maybe consider regarding the GCC version is compatibility with nvcc (NVIDIA compiler). For example CUDA 11.x works with GCC 6-11, CUDA 12 with GCC 6-12. https://gist.github.com/ax3l/9489132

More generally is there any plan to specify anything related to GPU support in the VFX platform?

The CUDA compatibility note is a good one. Does this present real compatibility issues within studios? If so, it would be great to have a bit more detail to better assess whether the benefit of adding nvcc or CUDA to the Platform would outweigh the added complexity.

 

There are no plans to add GPU or other hardware requirements to the VFX Reference Platform. If anyone thinks that should change then the Working Group would welcome suggestions and proposals.

 

I guess that the fact that it's mostly closed source might be a problem.

Larry was spot on as usual with his response. The only factor for considering whether to add something to the Platform is if it contributes to major version compatibility issues for VFX/animation studios or DCC software providers. 

Nick

[Larry Gritz]

I can't speak authoritatively about Cuda, especially for old versions, but our experience at SPI is that we build for multiple environments corresponding to various VFX Platform years, and we currently use Cuda 11.6 across the board. As far as I know, in practice we have not thus far found a need to lock down a version for the sake of platform consistency.

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CABiBYoz--p7nk1YBKHpHG7V3kw822xWs_TLfuffJPRNRZtNqig%40mail.gmail.com.

--

Larry Gritz

l...@larrygritz.com