CY2021 Draft Published

[Nick Cannon]

The Draft CY2021 VFX Reference Platform has been published at https://vfxplatform.com.

Our approach to CY2020 was to minimize changes other than the major upgrade to Python 3, so CY2021 is an opportunity to catch up in other areas with the most significant proposed change being the compiler upgrade to gcc 9.3.1 and C++17 support.

We are continuing to look into what the VFX Reference Platform support may look like on macOS and Windows and hope to share more information soon.

Please send any feedback on this Draft Platform either to feed...@vfxplatform.com or reply here at vfx-platform-discuss, and as usual we are looking to lock down the Final version around SIGGRAPH time in August. Despite SIGGRAPH going virtual this year, we still plan to host a Birds of a Feather session and hope the online format may allow a broader base of the community to join.

Nick

[Larry Gritz]

Very nice!

--

Larry Gritz

l...@larrygritz.com

[Tom Cowland]

Hey Nick,

Great to see C++17 in there. Was there any discussion of Python 3.8 this time around?

Many thanks in advance,

Tom

On Tuesday, 16 June 2020 07:20:25 UTC+1, Nick Cannon wrote:

The Draft CY2021 VFX Reference Platform has been published at https://vfxplatform.com.

[Nick Cannon]

Hi Tom,

On Tue, Jun 16, 2020 at 1:12 AM Tom Cowland <to...@cinesite.com> wrote:

Great to see C++17 in there. Was there any discussion of Python 3.8 this time around?

Yes, and we are very open to feedback on 3.7 vs 3.8. We had heard early reports of some issues between Python 3.8 and PySide, plus with studios having ongoing migration efforts we are leaning towards Python stability rather than keeping up with the latest for CY2021.

If anyone here has a strong preference for staying on 3.7 or moving to 3.8 then please share on this thread or contact the working group privately at feed...@vfxplatform.com. The 3.9 release planned for October comes a little too late in our annual cycle to be considered for next year as it doesn't allow enough time for the vendors to carry out integration and testing.

Thanks,

Nick

[Brecht Van Lommel]

For the Blender project, we'd like to upgrade to Python 3.8 and even
3.9. We have users that care about VFX platform compatibility, but
also those who care about using Blender together with other parts of
the Python ecosystem that move more quickly, and so we'd like to have
those in sync as much as possible. I understand that the Python 3
migration is ongoing and can see the arguments for stability as well
though.

Regarding the C++ ABI, my understanding is that this is not affected
by the GCC or C++ version upgrades. The old C++ ABI continues to be
required by RHEL / CentOS 7. Is that correct? It's a bit unclear now
since there is a note for GCC 6 but not GCC 9.

Thanks,
Brecht.

> --
> You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CABiBYoz41b59z1hiMpW%3Dee_cZ5AyJrQZVt9oVD47HADZ_k53kg%40mail.gmail.com.

[Nick Cannon]

On Tue, Jun 16, 2020 at 10:17 AM Brecht Van Lommel <brechtv...@gmail.com> wrote:

Regarding the C++ ABI, my understanding is that this is not affected
by the GCC or C++ version upgrades. The old C++ ABI continues to be
required by RHEL / CentOS 7. Is that correct? It's a bit unclear now
since there is a note for GCC 6 but not GCC 9.

Good question, you are correct, we plan for CY2021 to still use the old C++ ABI (by setting _GLIBCXX_USE_CXX11_ABI=0). We'll add a note for GCC 9 to clarify before we publish CY2021 Final.

Thanks,

Nick 

[Sean Wallitsch]

Since python is now doing yearly releases, delaying a year is only going to put us further behind.

The API breaking changes between 3.7 and 3.8 are minimal, and all have been deprecated for prior to removal. While I don't think the industry absolutely needs any of the features that are new in 3.8, I believe it'd be best for the industry to try and keep pace now that we've done the investment to get to Python 3. This has us station keeping with the 2019 release for the 2021 year (and the 2020 release for the 2022 refplat), which is right around the time the last bug fixes will be released for that version (1.5 years after initial release, then 3.5 years of security only fixes). 

We had heard early reports of some issues between Python 3.8 and PySide 

This does concern me. Do we think this is a python thing (and thus likely to be present in 3.9 as well) or a PySide thing (where we're waiting for a compatible version of PySide)?

Thanks for the hard work that went into this!

[Christian López Barrón]

Hi all, great to see the draft!

It's nice to see the industry moving forward with Python 3 support. Hopefully all the pain will be worth it.

I agree with Sean's concerns about timing with Python releases.

Searching for 3.8 current support, on the main Qt for Python docs (PySide2) they mention this about Python-3.8 on Windows:

> ***: 5.14 will not work on Windows with Python 3.8.0, please use Python 3.8.1 or greater.

From the Qt for Python Development Notes, last mention about Python-3.8 was in March and they've started testing Python-3.9:

5. March 2020

• CI Update
  • Including Python 3.8

20. February 2020

• Starting testing Python 3.9
  

Probably the Pyside / Python 3.8 issues have been tackled?

[Tom Cowland]

Hey folks,

> On Wednesday, June 17, 2020 at 12:07:27 PM UTC-7 Sean Wallitsch wrote:
> Since python is now doing yearly releases, delaying a year is only going to put us further behind.
>
> The API breaking changes between 3.7 and 3.8 are minimal, and all have been deprecated for prior to removal. While I don't think the industry absolutely needs any of the features that are new in 3.8, I believe it'd be best for the industry to try and keep pace now that we've done the investment to get to Python 3. This has us station keeping with the 2019 release for the 2021 year (and the 2020 release for the 2022 refplat), which is right around the time the last bug fixes will be released for that version (1.5 years after initial release, then 3.5 years of security only fixes).

Yeah, this was the reason I originally asked. Not withstanding any major compatibility issues, keeping the momentum would be great.

Gaffer is, by GitHub metrics, 50% python (I believe certain other apps in this sphere may share a similar split too). Though none of it is essential, for me, there are features in Python 3 that could really help with the long-term maintenance of such a code base. It's kind of moot for us right now, as though we have a 3.7 build running today, we also get imported into a whole host of other environments that still require Python 2 compatibility. Many of these are steered by the VFX Platform though, so hopefully there will come a day when we can leave 2 behind. At that point, it'd be great if we weren't then back in a similar situation, but with an ageing release from the 3.x series. Given the lag between the platform standard being published and adoption in an application, the gap becomes greater in practice than the timeline of the platform editions themselves.

'best/2p
Tom

[Nick Cannon]

Thank you for all the feedback so far on the CY2021 Draft.

Regarding the Python version, the preference seems heavily towards moving to 3.8 rather than the stability of staying on 3.7. If you feel differently then please make your voice heard either on this thread or you can contact the Working Group directly at feed...@vfxplatform.com.

There has been one minor change. Due to a bug found in Boost 1.72 on Windows we have bumped CY2021 to 1.73.

Nick

[jerry...@gmail.com]

Mr van Lommel was probably referring to https://bugzilla.redhat.com/show_bug.cgi?id=1546704 which is worth adding to the VFX page as explication why the ABI stays at C++11.

[Ashley Whetter]

I put this comment in the ASWF slack channel but I'll comment here because it's more permanent.

I don't know what usually gets a version into the platform but if I was to point out the things that I'm excited about as a developer:

• TypedDict would be my #1
• importlib.metadata would be a big win for us because the slowness of pkg_resources is a constant source of pain that trips us up. We already use the backport in places.
• The f{expr=} syntax is awesome, although just a convenience
• Positional-only parameters would allow us to clean up some areas of our APIs
• shlex.join would give me a lot more confidence in how we try to do this now in our code
• There's lots of other small things but these are the bigger things for me.

Talking more generally, there's a few things that 3.8 removes that were introduced in 3.x so having those be removed sooner after the switch to Python 3 would be a good thing I think. There's no changes in the Python API or the C API that look particularly disruptive.
Overall, there's a lot of good things and not anything that gives me great concern (but I'll add a disclaimer that I'm usually an advocate for change and more willing to accept risk than many).

Not that "being old" usually factors into the decision, but the last bugfix version of Python 3.7 was just released and it'll be getting security updates until 2023: https://pythoninsider.blogspot.com/2020/06/python-378-and-3611-now-available-last.html?m=1 With it comes the recommendation to upgrade to 3.8 "as soon as practical".

However I'm sympathetic towards the vendors. If Python 2 support does indeed end up sticking around for another year then I can understand why we wouldn't want to update this year.

Keeping Python 2 support isn't something I want to happen. The longer we keep supporting Python 2, the longer it's going to take places to switch to Python 3 because realistically there are a lot of places that won't update until vendors drop support for it. But this is a separate issue outside of the reference platform and something that we're actively discussing in the Python 3 working group.

[Nick Cannon]

Thank you for all the feedback on the merits and considerations of Python 3.8 vs 3.7.

For those that have been unable to join our recent events, we are leaning towards Python 3.7 for CY2021 despite the clear support on this thread for 3.8. The primary reason for this is that the software vendors are already incurring significant overhead in having to support two build sets for their products while customers are migrating from Python 2 to Python 3.7. To add a third build set for Python 3.8 in 2021 would add even more overhead and take resources away from developing new product features.

We recognize that there is a strong desire to keep up to date with Python in future so vendors are encouraged to look into dynamic loading of Python in their products to avoid the need for multiple build sets. There is clear intent in future for the VFX Reference Platform to be more progressive in staying up to date with Python, but we are not there yet and recognize the need for some stability into next year as studios continue their work to migrate away from Python 2.

Please keep feedback coming, particularly if you think the benefits of moving to Python 3.8 outweigh the downside of it taking resource away from vendors developing new features in their products.

As Ashley mentions, there is a Python 3 Working Group hosted by the Academy Software Foundation which is a great place to learn more about transitioning from Python 2 to Python 3. For more information, see:

https://github.com/AcademySoftwareFoundation/wg-python3/

Nick

[Sean Wallitsch]

Following your assurance in the ASWF session that we'd be targeting 3.9 in 2022 and skipping 3.8 entirely, I'm much less concerned about sticking with 3.7 in 2021.

--

You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/3fc7e80d-d0da-4daf-b52c-6c3e59331552n%40googlegroups.com.

[Nick Cannon]

On Tue, Aug 25, 2020 at 8:34 AM Sean Wallitsch <shid...@alphamatte.com> wrote:
>
> Following your assurance in the ASWF session that we'd be targeting 3.9 in 2022 and skipping 3.8 entirely, I'm much less concerned about sticking with 3.7 in 2021.

The target is for us to be more progressive for CY2022 and hopefully
there are no obstacles to us adopting the latest stable Python release
but I just want to be clear that this isn't something that is a
cast-iron guarantee. Please continue to let your software vendors know
if this is important to you, and please do what you can to help
studios make good progress with their migration to Python 3.

Thanks,
Nick

[Sean Wallitsch]

I definitely didn't take it as a guarantee, just that it is what we'd be hoping to do. :)

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CABiBYoyZAtQ0u38M9X8cMsDdH2e%3Dh9Ai2qMD9rWxDmWxkBmwnA%40mail.gmail.com.

[Paul Miller]

I have already done the working of porting my API + scripts to Python 3,
so I'm not particular about the Python 3 version myself.

What is most time consuming for me is the updating/building of the
proper versions of Qt, Python, and PySide. I'm betting that once there
are containers/bundles of the recommended set of dependencies available
for all three platforms, I'll just use those instead and it'll save a
ton of time.

BTW, regarding Qt on Mac, I still build from source because I prefer
shared libs to Frameworks. Anyone have thoughts on that?

[Jean-Francois Panisset]

On Linux you can likely find what you need with the aswf-docker containers:

https://github.com/AcademySoftwareFoundation/aswf-docker

Delivering something equivalent for Windows and macOS is a stated goal of the ASWF CI working group, especially now that the Reference Platform includes explicit guidance for those systems.

JF

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/29824258-4524-c589-13e0-ea21d2f5f5db%40fxtech.com.

[Robert Vinluan]

On 2020-08-25 12:44 p.m., Paul Miller wrote:
>
> BTW, regarding Qt on Mac, I still build from source because I prefer
> shared libs to Frameworks. Anyone have thoughts on that?
>
>

This brings up an interesting point for discussion.

We build Qt from source but we build Qt as frameworks instead of plain
shared libs.  We used to build shared libs but then we ran into
conflicts at runtime if our application loaded something that was built
against the system's Qt frameworks.  So we switched to frameworks to
match the system.

For improved compatibility among software, I think we need to agree on
building frameworks vs building shared libraries.

Similarly, we also build Python as a framework to match the system's
Python but this means our Python is not compatible with Python distros
built as shared libraries such as Conda Python.

Cheers,
Rob

[Wayne Arnold]

For Maya our build of Qt is using shared libraries, not frameworks

Changing to frameworks has been brought up a couple of times, but hasn't been something that needed to be done or scheduled

For Python we use framework.

--
Wayne

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.

To view this discussion on the web visit https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msgid_vfx-2Dplatform-2Ddiscuss_ed3d431b-2D3025-2Db231-2Df0ca-2Dcc49fe6b16b1-2540gmail.com&d=DwIFaQ&c=76Q6Tcqc-t2x0ciWn7KFdCiqt6IQ7a_IF9uzNzd_2pA&r=GXild4_VNtcQ0xI2TKQFvvYynQM3zqA42N5y8EpO-kM&m=AeSykiC6lHmfqcgb0VZEhSUtY_qRZ2qaiWNfvy4KYSo&s=RtlcC-YerZHxd6llGHNJXgGJ0iZ15b7C9eE5Za8Uw5I&e= .

[Robert Fanner]

Foundry tends to use frameworks for Python and Qt.

A possible OS-specific advantage here is that you can package up the headers as part of the framework, instead of needing to bundle them separately or needing to point users at additional downloads, but this is arguably a lesser consideration vs compatibility.

Kind Regards,
Rob

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/7D8962CD-D664-445C-A50D-E7CEE5F79927%40autodesk.com.

--

Rob Fanner

Engineering Manager - Platform

Web: www.foundry.com

The Foundry Visionmongers Ltd.  -  Registered in England and Wales No: 4642027  -  Address: 5 Golden Square, London, W1F 9HT  -  VAT No: 672742224