Are we trying to solve the VFX Platform issue correctly?
1,138 views
Skip to first unread message
Neal Gompa
Jan 17, 2022, 6:52:53 PM1/17/22
to vfx-platfo...@googlegroups.com
Hello all,
I've spent some time over the weekend looking over the survey report,
reading through some of the older discussions on this list, and
looking over the VFX Reference Platform artifacts released by the
ASWF.
There were a couple of things that caught my eye about the current situation:
* ASWF is attempting to maintain a development platform and runtime alone
* ISVs are predominantly bundling/vendoring these libraries anyway
These two statements lead to an interesting third statement: it's
*really hard* for the VFX community to ramp up into new technologies,
because of the byzantine bundling and lack of synchronization with the
platform.
I certainly understand that during film production, studios try to
lock down the stack to minimize disruption during the development of a
project. However, this method causes major heartache when trying to
on-ramp to a newer Linux platform for better performance, hardware
support, etc.
I understand that part of the point of the VFX Reference Platform is
to make it easier for everyone to get in sync and roll forward
together, but I question some of the current approach. Perhaps I'm a
bit naive here, but I've always had a much better time with this sort
of thing by working together with the Linux platform to ensure the
user's capabilities are available. This leads to some interestingly
positive outcomes:
* The Linux platform vendor and the industry work together to resolve
issues that neither could do alone
* The *voice* of the industry is strengthened when having
conversations with ISVs
* The larger community can help support the needs of today and
anticipate the needs of tomorrow (e.g. Vulkan, HDR, etc.)
It's my firm belief that one of the reasons why stuff like HDR is
taking so long is because there's a dearth of folks who can work with
the developers to implement what is actually *needed*. In order to
explain why I believe that, I want to talk about how the audio stack
has evolved on Linux in the last few years.
About seven years ago, Wim Taymans started on the project that would
eventually become PipeWire. The project was slow-going initially, but
that changed in 2018 when all the major audio stack developers and
significant consumers came together for a hackfest[1] to hammer out
the final architecture and functionality for audio through PipeWire.
After that point, the development seriously accelerated, to the point
that in late 2020, it was proposed to switch Fedora Linux 34 to use
PipeWire for all audio[2]. The proposal brought even more ecosystem
stakeholders to the table to flesh out the remaining bits for
integration, and I helped Wim architect the integration of PipeWire
into Fedora Linux. The end result was that Fedora Linux 34 shipped
with PipeWire with great accolades by the Linux user community. We had
successfully consolidated the Linux audio entry-points into PipeWire,
significantly improved the user experience for audio on Linux, and
drastically lowered the bar for people to do professional audio on
Linux. Six months later, we repeated our success with the introduction
of WirePlumber[3], and I worked with Wim and other folks at Red Hat to
bring this into CentOS Stream 9 as part of my work in the CentOS
Hyperscale SIG[4]. This work is now present in the latest release of
the Red Hat Enterprise Linux 9 Beta, including full support for
building and shipping applications that use the JACK API by binding
them to PipeWire.
On the graphics side with Wayland, the story is very similar. The big
difference is that the major players only started coming to the table
about two years ago instead of four years ago. For the longest time,
the major drivers in the ecosystem were Red Hat and Collabora. Valve
came onto the scene a few years ago as part of trying to significantly
reduce input lag and increase the rate of perfect frames for games and
simulations on Linux. The legacy X11 stack was identified as the
bottleneck and Valve started funding work in SDL, KDE, Wine, and other
parts of the stack to uplift everything to Wayland (ideally running on
Vulkan). With everyone pushing forward together, insurmountable issues
became surmountable. KDE Plasma's Wayland session is now one of the
most advanced Wayland-based desktop environments out there, and Fedora
Linux 34 switched KDE Plasma to Wayland by default[5]. NVIDIA has
finally started properly supporting Wayland, and Fedora Linux 36 (and
soon CentOS Stream 9) will have NVIDIA users experiencing GNOME on
Wayland by default[6]. Even SDL applications (like Unreal Engine) are
able to get in on the Wayland fun because the necessary enablement is
now in RHEL 9, courtesy of yours truly. The ecosystem rapidly evolved
once people came together to work together.
I mention these stories in order to circle back to an interesting
problem: the VFX Reference Platform doesn't currently have Linux
vendor involvement to enrich the platform. In my experience working at
an ISV in a different industry, I've observed that building Linux
software platforms with the Linux vendors tends to be much more
successful because they know how to help you solve your problems in an
optimal fashion. In the years I've been doing this, I've always
appreciated being able to work with folks at Red Hat and SUSE to not
only make things work, but make things better. That's a big part of
why I do so much in the Fedora Project, the CentOS Project, and the
openSUSE Project. Now, I'm not saying that everyone needs to do it
exactly like how I did, but I think we can take a fresh look at the
problem based on some key facts:
* Both Enterprise Linux vendors are considerably more open to
community collaboration and co-development than they were 10 years ago
* Red Hat's CentOS Stream 9 project is designed to give you tight
feedback loops as you build into and around it
* SUSE's openSUSE Leap project offers direct paths to contribute
features and packages to SUSE Linux Enterprise
* CentOS Stream and openSUSE Leap move at the speed of contribution,
user needs, and business objectives
* Already at least half of the VFX reference platform components are
already included in both distributions
CentOS Stream 9 currently ships GNOME 40 and openSUSE Leap 15.4 ships
GNOME 41, which both provide a much more refined user experience than
what many are used to experiencing with GNOME 3 and include many
improvements around latency and precision input handling. People who
had trouble with GNOME 3 before should consider taking a fresh look
again and they might be surprised to see how much better it is now.
But if GNOME isn't your bag, both distributions ship KDE Plasma
through community repositories: KDE Plasma 5.24 will be available in
EPEL for EL9 and KDE Plasma 5.18 for Leap 15.4. Qt 5.15 is present in
both *now*.
Now, as the survey indicates that a whopping 82% of studios use CentOS
Linux, I think it makes sense for me to concentrate on CentOS Stream
and RHEL here (though this will equally work with openSUSE). Now,
here's my crazy radical idea: what if we collaborated with the CentOS
Project to *build* reference workstation and headless platforms based
on CentOS Stream? If we worked with Fedora and CentOS to package and
ship the remaining libraries and tools (presuming they're all open
source), we could assemble containers, server disk images, and
workstation live media leveraging these components to provide
something everyone could target and rely on. This has several major
benefits:
* VFX Reference Platform components benefit from shared maintenance
between RHEL and the community
* ISVs who work with these libraries and discover issues or need
enhancements can either contribute or request it, to the benefit of
everyone
* The barriers of entry for folks building applications for this
audience drops precipitously
* The commons is enriched for the ecosystem
* The quality of the Linux environment for ISVs and users goes up
This also avoids the pitfalls of building and maintaining a custom
Linux distribution because the actual effort by folks here would be
around the components they care about, and the recipes created to
build images for a CentOS Stream spin could be reused with RHEL,
AlmaLinux, or Rocky Linux too. ISVs can build and test around this
CentOS Stream spin, and if studios (and more importantly in my view,
newcomers to the industry) want to use it for production, they
absolutely could too. I know this strategy can work because that's
what I do at work: I build the software packages on CentOS Stream and
release it to users on RHEL, AlmaLinux, and others with them none the
wiser (I've talked about this recently[7])! At this point, I've got
plenty of experience contributing to CentOS Stream, and I'm planning
to give a talk about my experiences at the upcoming CentOS Dojo[8]
(schedule pending).
There are a couple of ways this could be done in CentOS: a dedicated
SIG could be spun up or a partnership with an existing SIG could be
established to pull it off. I could see it working out well either
way, though if you wanted to partner with an existing SIG, I could
certainly arrange some kind of meetup with Hyperscale SIG folks and
CentOS VFX folks to do it together (as I think other Hyperscale folks
would be interested in this too).
So, tell me: am I crazy for suggesting this? What am I not thinking
about? Or is it an awesome idea and people are interested? Or are
people iffy about it and want to dig into the idea more?
[1]: https://blogs.gnome.org/uraeus/2018/10/30/pipewire-hackfest/
[2]: https://fedoraproject.org/wiki/Changes/DefaultPipeWire
[3]: https://fedoraproject.org/wiki/Changes/WirePlumber
[4]: https://blog.centos.org/2022/01/centos-hyperscale-sig-quarterly-report-for-2021q4/
[5]: https://fedoraproject.org/wiki/Changes/WaylandByDefaultForPlasma
[6]: https://fedoraproject.org/wiki/Changes/WaylandByDefaultOnNVIDIA
[7]: https://youtu.be/HHOsMK8i-9s?t=787
[8]: https://hopin.com/events/centos-dojo-fosdem-2022
--
真実はいつも一つ！/ Always, there's only one truth!
I've spent some time over the weekend looking over the survey report,
reading through some of the older discussions on this list, and
looking over the VFX Reference Platform artifacts released by the
ASWF.
There were a couple of things that caught my eye about the current situation:
* ASWF is attempting to maintain a development platform and runtime alone
* ISVs are predominantly bundling/vendoring these libraries anyway
These two statements lead to an interesting third statement: it's
*really hard* for the VFX community to ramp up into new technologies,
because of the byzantine bundling and lack of synchronization with the
platform.
I certainly understand that during film production, studios try to
lock down the stack to minimize disruption during the development of a
project. However, this method causes major heartache when trying to
on-ramp to a newer Linux platform for better performance, hardware
support, etc.
I understand that part of the point of the VFX Reference Platform is
to make it easier for everyone to get in sync and roll forward
together, but I question some of the current approach. Perhaps I'm a
bit naive here, but I've always had a much better time with this sort
of thing by working together with the Linux platform to ensure the
user's capabilities are available. This leads to some interestingly
positive outcomes:
* The Linux platform vendor and the industry work together to resolve
issues that neither could do alone
* The *voice* of the industry is strengthened when having
conversations with ISVs
* The larger community can help support the needs of today and
anticipate the needs of tomorrow (e.g. Vulkan, HDR, etc.)
It's my firm belief that one of the reasons why stuff like HDR is
taking so long is because there's a dearth of folks who can work with
the developers to implement what is actually *needed*. In order to
explain why I believe that, I want to talk about how the audio stack
has evolved on Linux in the last few years.
About seven years ago, Wim Taymans started on the project that would
eventually become PipeWire. The project was slow-going initially, but
that changed in 2018 when all the major audio stack developers and
significant consumers came together for a hackfest[1] to hammer out
the final architecture and functionality for audio through PipeWire.
After that point, the development seriously accelerated, to the point
that in late 2020, it was proposed to switch Fedora Linux 34 to use
PipeWire for all audio[2]. The proposal brought even more ecosystem
stakeholders to the table to flesh out the remaining bits for
integration, and I helped Wim architect the integration of PipeWire
into Fedora Linux. The end result was that Fedora Linux 34 shipped
with PipeWire with great accolades by the Linux user community. We had
successfully consolidated the Linux audio entry-points into PipeWire,
significantly improved the user experience for audio on Linux, and
drastically lowered the bar for people to do professional audio on
Linux. Six months later, we repeated our success with the introduction
of WirePlumber[3], and I worked with Wim and other folks at Red Hat to
bring this into CentOS Stream 9 as part of my work in the CentOS
Hyperscale SIG[4]. This work is now present in the latest release of
the Red Hat Enterprise Linux 9 Beta, including full support for
building and shipping applications that use the JACK API by binding
them to PipeWire.
On the graphics side with Wayland, the story is very similar. The big
difference is that the major players only started coming to the table
about two years ago instead of four years ago. For the longest time,
the major drivers in the ecosystem were Red Hat and Collabora. Valve
came onto the scene a few years ago as part of trying to significantly
reduce input lag and increase the rate of perfect frames for games and
simulations on Linux. The legacy X11 stack was identified as the
bottleneck and Valve started funding work in SDL, KDE, Wine, and other
parts of the stack to uplift everything to Wayland (ideally running on
Vulkan). With everyone pushing forward together, insurmountable issues
became surmountable. KDE Plasma's Wayland session is now one of the
most advanced Wayland-based desktop environments out there, and Fedora
Linux 34 switched KDE Plasma to Wayland by default[5]. NVIDIA has
finally started properly supporting Wayland, and Fedora Linux 36 (and
soon CentOS Stream 9) will have NVIDIA users experiencing GNOME on
Wayland by default[6]. Even SDL applications (like Unreal Engine) are
able to get in on the Wayland fun because the necessary enablement is
now in RHEL 9, courtesy of yours truly. The ecosystem rapidly evolved
once people came together to work together.
I mention these stories in order to circle back to an interesting
problem: the VFX Reference Platform doesn't currently have Linux
vendor involvement to enrich the platform. In my experience working at
an ISV in a different industry, I've observed that building Linux
software platforms with the Linux vendors tends to be much more
successful because they know how to help you solve your problems in an
optimal fashion. In the years I've been doing this, I've always
appreciated being able to work with folks at Red Hat and SUSE to not
only make things work, but make things better. That's a big part of
why I do so much in the Fedora Project, the CentOS Project, and the
openSUSE Project. Now, I'm not saying that everyone needs to do it
exactly like how I did, but I think we can take a fresh look at the
problem based on some key facts:
* Both Enterprise Linux vendors are considerably more open to
community collaboration and co-development than they were 10 years ago
* Red Hat's CentOS Stream 9 project is designed to give you tight
feedback loops as you build into and around it
* SUSE's openSUSE Leap project offers direct paths to contribute
features and packages to SUSE Linux Enterprise
* CentOS Stream and openSUSE Leap move at the speed of contribution,
user needs, and business objectives
* Already at least half of the VFX reference platform components are
already included in both distributions
CentOS Stream 9 currently ships GNOME 40 and openSUSE Leap 15.4 ships
GNOME 41, which both provide a much more refined user experience than
what many are used to experiencing with GNOME 3 and include many
improvements around latency and precision input handling. People who
had trouble with GNOME 3 before should consider taking a fresh look
again and they might be surprised to see how much better it is now.
But if GNOME isn't your bag, both distributions ship KDE Plasma
through community repositories: KDE Plasma 5.24 will be available in
EPEL for EL9 and KDE Plasma 5.18 for Leap 15.4. Qt 5.15 is present in
both *now*.
Now, as the survey indicates that a whopping 82% of studios use CentOS
Linux, I think it makes sense for me to concentrate on CentOS Stream
and RHEL here (though this will equally work with openSUSE). Now,
here's my crazy radical idea: what if we collaborated with the CentOS
Project to *build* reference workstation and headless platforms based
on CentOS Stream? If we worked with Fedora and CentOS to package and
ship the remaining libraries and tools (presuming they're all open
source), we could assemble containers, server disk images, and
workstation live media leveraging these components to provide
something everyone could target and rely on. This has several major
benefits:
* VFX Reference Platform components benefit from shared maintenance
between RHEL and the community
* ISVs who work with these libraries and discover issues or need
enhancements can either contribute or request it, to the benefit of
everyone
* The barriers of entry for folks building applications for this
audience drops precipitously
* The commons is enriched for the ecosystem
* The quality of the Linux environment for ISVs and users goes up
This also avoids the pitfalls of building and maintaining a custom
Linux distribution because the actual effort by folks here would be
around the components they care about, and the recipes created to
build images for a CentOS Stream spin could be reused with RHEL,
AlmaLinux, or Rocky Linux too. ISVs can build and test around this
CentOS Stream spin, and if studios (and more importantly in my view,
newcomers to the industry) want to use it for production, they
absolutely could too. I know this strategy can work because that's
what I do at work: I build the software packages on CentOS Stream and
release it to users on RHEL, AlmaLinux, and others with them none the
wiser (I've talked about this recently[7])! At this point, I've got
plenty of experience contributing to CentOS Stream, and I'm planning
to give a talk about my experiences at the upcoming CentOS Dojo[8]
(schedule pending).
There are a couple of ways this could be done in CentOS: a dedicated
SIG could be spun up or a partnership with an existing SIG could be
established to pull it off. I could see it working out well either
way, though if you wanted to partner with an existing SIG, I could
certainly arrange some kind of meetup with Hyperscale SIG folks and
CentOS VFX folks to do it together (as I think other Hyperscale folks
would be interested in this too).
So, tell me: am I crazy for suggesting this? What am I not thinking
about? Or is it an awesome idea and people are interested? Or are
people iffy about it and want to dig into the idea more?
[1]: https://blogs.gnome.org/uraeus/2018/10/30/pipewire-hackfest/
[2]: https://fedoraproject.org/wiki/Changes/DefaultPipeWire
[3]: https://fedoraproject.org/wiki/Changes/WirePlumber
[4]: https://blog.centos.org/2022/01/centos-hyperscale-sig-quarterly-report-for-2021q4/
[5]: https://fedoraproject.org/wiki/Changes/WaylandByDefaultForPlasma
[6]: https://fedoraproject.org/wiki/Changes/WaylandByDefaultOnNVIDIA
[7]: https://youtu.be/HHOsMK8i-9s?t=787
[8]: https://hopin.com/events/centos-dojo-fosdem-2022
--
真実はいつも一つ！/ Always, there's only one truth!
Larry Gritz
Jan 17, 2022, 11:36:47 PM1/17/22
to Neal Gompa, vfx-platfo...@googlegroups.com
Can you clarify what you mean by "development platform" and "runtime"?
On Jan 17, 2022, at 3:52 PM, Neal Gompa <ngom...@gmail.com> wrote:* ASWF is attempting to maintain a development platform and runtime alone
Neal Gompa
Jan 18, 2022, 6:47:31 AM1/18/22
to Larry Gritz, vfx-platfo...@googlegroups.com
On Mon, Jan 17, 2022 at 11:36 PM Larry Gritz <l...@larrygritz.com> wrote:
>
> Can you clarify what you mean by "development platform" and "runtime"?
>
>
Sure. I'm primarily referring to the existence of aswf-docker[1].
>
> Can you clarify what you mean by "development platform" and "runtime"?
>
>
[1]: https://github.com/AcademySoftwareFoundation/aswf-docker
Brent Villalobos
Jan 18, 2022, 2:29:42 PM1/18/22
to Neal Gompa, Larry Gritz, vfx-platform-discuss
Interesting insights Neal. I think one of the challenges in the VFX and feature animation worlds is that we are often managing multiple reference platforms simultaneously. That makes it difficult for an OS distribution to deeply integrate a specific set of libraries. It's common for a studio to have a spread of VFX platform years both between productions and even within a single production. That makes the container-based solution, or some sort of abstraction layer, palpable. I would love to hear if there are technologies or practices within the Linux communities on how to address this challenge.
-Brent
--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAEg-Je9bxEz4yhSaOfqwdGmOidtiqzC2wNGB-b64CquTnd9y8g%40mail.gmail.com.
Neal Gompa
Jan 18, 2022, 2:55:57 PM1/18/22
to Brent Villalobos, Larry Gritz, vfx-platform-discuss
On Tue, Jan 18, 2022 at 2:29 PM Brent Villalobos
<brent.vi...@dreamworks.com> wrote:
>
> Interesting insights Neal. I think one of the challenges in the VFX and feature animation worlds is that we are often managing multiple reference platforms simultaneously. That makes it difficult for an OS distribution to deeply integrate a specific set of libraries. It's common for a studio to have a spread of VFX platform years both between productions and even within a single production. That makes the container-based solution, or some sort of abstraction layer, palpable. I would love to hear if there are technologies or practices within the Linux communities on how to address this challenge.
There are two relatively common practices for libraries, and a third
<brent.vi...@dreamworks.com> wrote:
>
> Interesting insights Neal. I think one of the challenges in the VFX and feature animation worlds is that we are often managing multiple reference platforms simultaneously. That makes it difficult for an OS distribution to deeply integrate a specific set of libraries. It's common for a studio to have a spread of VFX platform years both between productions and even within a single production. That makes the container-based solution, or some sort of abstraction layer, palpable. I would love to hear if there are technologies or practices within the Linux communities on how to address this challenge.
newcomer solution from Fedora:
* Solution 1: Ensure libraries and tools are designed for parallel
installability. That means libraries get shared object versions
(soversions) that make it so libraries with binary interface
incompatibilities can be parallel installed. For example: OpenSSL
1.1.x provides libssl.so.1.1 and libcrypto.so.1.1 and OpenSSL 3.x
provides libssl.so.3 and libcrypto.so.3. Tools would get versioned
names (e.g. Python 3.x interpreters are python3.X).
* Solution 2: Namespace libraries and tools. That means that libraries
and tools that would ordinarily conflict would get de-conflicted by
shuffling files around and possibly also using alternatives. This
isn't too common anymore now that Solution 1 is done more by projects
these days. Basically libraries would get put into a subdir in
/usr/lib64 and things that link to it get an RPATH set for it. It's
kludgy and kind of difficult to manage, hence why projects just adapt
to Solution 1 instead.
* Solution 3: Modularity[1]. This is a technology that also exists in
RHEL/CentOS as Application Streams[2], and can be used to define a
collection of components into a kind of "sub repository" that offers
content that overshadows the main repository content. Modularity can
be used in combination with Solution 1 to have parallel installable
modules too. The advantage of these modules is that you can define
supported content, support timeframes, etc. into the metadata. The
disadvantage is that there's not much tooling for this yet. This also
can adapt well for building appliances or containers, since it
simplifies access to stacks.
I would aim for Solution 1 and maybe work with RHEL folks for Solution
3. I think some of the challenge with the spread of VFX reference
platform versions is because of how difficult it is to plan and ramp
up to the next one. Building a partnership and a pipeline for the
platform to evolve along with all the other stakeholders together will
eventually result in an equilibrium in terms of evolution cadence and
simplify things for longer term maintenance.
[1]: https://docs.fedoraproject.org/en-US/modularity/
[2]: https://access.redhat.com/support/policy/updates/rhel8-app-streams-life-cycle
Deke Kincaid
Jan 18, 2022, 2:56:42 PM1/18/22
to Brent Villalobos, Neal Gompa, Larry Gritz, vfx-platform-discuss
Yes, to pile on what Brent brought up, Neil I think you are missing that big piece of the pie with how vfx works. We use solutions such as Rez or other homegrown package management solutions in order to use different configurations of packages based on a show's needs.
We can have dozens of shows all going at the same time which can down to the department level use a different version of VFX Platform depending on which DCC's they are using on that current show. We have some reliance on the libraries which ship with the host operating system but often times it is only for very low-level ones such as glibc and some base ones such as Windows Managers. At runtime, though most production-facing tools are using our packaged up libraries/tools all installed on a central server. Very little is actually locally installed in /usr/local or /opt unless it is part of some kind of caching mechanism to speed things up. So we don't have to compile complicated tools from scratch or there are tools with cyclic dependencies we will use the rpm or deb to seed our system for building packages.
Currently, we have shows using VFX Platform 2017 all the way up to cy2021 going on at our facility right now depending on the current lifecycle of the show.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAAoNwLSnYrdwxGrG1qb%3DH3wzVD6SRDdy3Yu6P3H7OG9QiRyjfg%40mail.gmail.com.
Deke Kincaid
Jan 18, 2022, 3:00:46 PM1/18/22
to Neal Gompa, Brent Villalobos, Larry Gritz, vfx-platform-discuss
I don't think vfx is looking for the OS companies to solve package management, there are already many solutions and some new ones coming down the line. Moving to Wayland, great, package management, not so much.
--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAEg-Je_4S0tHzf4%2BhV%3DPr%2BTKe92k_apFL2-bjj7kPWTbrxq%2ByA%40mail.gmail.com.
Sean Wallitsch
Jan 18, 2022, 3:04:33 PM1/18/22
to Deke Kincaid, Brent Villalobos, Neal Gompa, Larry Gritz, vfx-platform-discuss
Currently, we have shows using VFX Platform 2017 all the way up to cy2021 going on at our facility right now depending on the current lifecycle of the show.
And this is not as awful as it seems at first glance. What the refplats do here is allow our studios and vendors to maximize their efforts on the work they sell. If a show is on refplat 2017, that's ok because the list of software that is compatible with that is easy to find. You know what you're going to get. You shouldn't start a new show on 2017, but the refplat has taken what was a continuous cycle of chaos and created defined, compatible groups.
This isn't for Linux vendors to solve, as studios need to be able to run 2017 alongside 2021. The major issue we face is that we still need a baseline, stable linux distro to put these refplats on top of and the loss of a stable CentOS release has removed that.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAOOm49RNdjGkQ087ABdnMSAoK8t_sX35pNhvNPzt8PjkbNcRuA%40mail.gmail.com.
Neal Gompa
Jan 18, 2022, 3:10:28 PM1/18/22
to Deke Kincaid, Brent Villalobos, Larry Gritz, vfx-platform-discuss
On Tue, Jan 18, 2022 at 3:00 PM Deke Kincaid <dekek...@gmail.com> wrote:
>
> I don't think vfx is looking for the OS companies to solve package management, there are already many solutions and some new ones coming down the line. Moving to Wayland, great, package management, not so much.
>
The problem is that they are interlocked. One of the problems with
>
> I don't think vfx is looking for the OS companies to solve package management, there are already many solutions and some new ones coming down the line. Moving to Wayland, great, package management, not so much.
>
your existing solution is that you're working completely out of sync
with everyone else trying to get this working. Take SDL2 in games as
an example. Most bundled copies of SDL2 are never getting updated by
the game developer. But because Fedora and RHEL ship an SDL2 that
actually *works* and has the Wayland integration in place, it is
possible for users to force that version to be used and benefit from
it. Take that and scatter it across all the projects that do things
with graphics and you've got a nightmare scenario that has taken over
a decade to work through.
(Actually, in the SDL2 case, there's a bit of a special trick: if the
bundled copy of SDL2 detects a systemwide copy, it'll switch to that
transparently. But most libraries don't have one of the most creative
engineers trying to work around these problems implementing things
like that.)
You have this problem with Qt, for example, and this continues to be a
problem with applications that bundle in toolkit libraries that
provide interfaces for these things.
Chad Dombrova
Jan 18, 2022, 3:17:12 PM1/18/22
to Sean Wallitsch, Deke Kincaid, Brent Villalobos, Neal Gompa, Larry Gritz, vfx-platform-discuss
It would be very convenient to `yum install vfxplat-2020`, and I assume vendors would love to be able to rely on a set of approved and standardized libs provided by the OS rather than compile and distribute these libs themselves. However, that particular problem does seem less pressing than "what will be the officially supported Linux OS for the platform?" Perhaps I'm mistaken, but it seems like these suggestions are predicated on using CentOS Stream or other rapidly changing variants when our industry needs much more stability; since there's no longer a stable variant of CentOS we have to look elsewhere.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAOZ7%3DV3DzAaO3SX%2Bo7B0VxrVE%2B_4wNy4Qu7KvA6awZRx2M0r7w%40mail.gmail.com.
Neal Gompa
Jan 18, 2022, 3:18:31 PM1/18/22
to Sean Wallitsch, Deke Kincaid, Brent Villalobos, Larry Gritz, vfx-platform-discuss
On Tue, Jan 18, 2022 at 3:04 PM Sean Wallitsch <shid...@alphamatte.com> wrote:
>>
>> Currently, we have shows using VFX Platform 2017 all the way up to cy2021 going on at our facility right now depending on the current lifecycle of the show.
>
>
> And this is not as awful as it seems at first glance. What the refplats do here is allow our studios and vendors to maximize their efforts on the work they sell. If a show is on refplat 2017, that's ok because the list of software that is compatible with that is easy to find. You know what you're going to get. You shouldn't start a new show on 2017, but the refplat has taken what was a continuous cycle of chaos and created defined, compatible groups.
>
> This isn't for Linux vendors to solve, as studios need to be able to run 2017 alongside 2021. The major issue we face is that we still need a baseline, stable linux distro to put these refplats on top of and the loss of a stable CentOS release has removed that.
>
Okay, I'll bite. Running across multiple platforms itself is fine, but
>>
>> Currently, we have shows using VFX Platform 2017 all the way up to cy2021 going on at our facility right now depending on the current lifecycle of the show.
>
>
> And this is not as awful as it seems at first glance. What the refplats do here is allow our studios and vendors to maximize their efforts on the work they sell. If a show is on refplat 2017, that's ok because the list of software that is compatible with that is easy to find. You know what you're going to get. You shouldn't start a new show on 2017, but the refplat has taken what was a continuous cycle of chaos and created defined, compatible groups.
>
> This isn't for Linux vendors to solve, as studios need to be able to run 2017 alongside 2021. The major issue we face is that we still need a baseline, stable linux distro to put these refplats on top of and the loss of a stable CentOS release has removed that.
>
clearly there's an issue because despite you all wanting to grow your
Linux usage, your ISVs aren't playing ball. If this was okay, then the
sheer might of your industry should have convinced Adobe and Epic to
cave in and fully support Linux 5 years ago.
What you've done is created a different kind of chaos: ISVs can't
reliably build software for you in a way that is compatible with both
broad-base usage and specialty deployments. If I were making software
for you folks, I'd be pretty cheesed off because I have no means to
stabilize, test, and validate in a way that allows people to lifecycle
continuously (broad-base usage). All of you have created a world in
which things are so frozen that it's now seriously expensive to
support you as customers.
Put it more simply: It's very easy for me to build a package for RHEL
and ship it to broad-base customers by relying on what RHEL provides
to minimize my support burden. It's very difficult for me to build a
package on the VFX platform and ship that to specialty customers
because not only do I have to take ownership for the software I made,
but I have to take *sole* ownership over everything it uses instead of
sharing that with the vendor.
Neal Gompa
Jan 18, 2022, 3:23:41 PM1/18/22
to Chad Dombrova, Sean Wallitsch, Deke Kincaid, Brent Villalobos, Larry Gritz, vfx-platform-discuss
On Tue, Jan 18, 2022 at 3:17 PM Chad Dombrova <cha...@gmail.com> wrote:
>
> It would be very convenient to `yum install vfxplat-2020`, and I assume vendors would love to be able to rely on a set of approved and standardized libs provided by the OS rather than compile and distribute these libs themselves. However, that particular problem does seem less pressing than "what will be the officially supported Linux OS for the platform?" Perhaps I'm mistaken, but it seems like these suggestions are predicated on using CentOS Stream or other rapidly changing variants when our industry needs much more stability; since there's no longer a stable variant of CentOS we have to look elsewhere.
>
So, in my experience working with CentOS Stream 8, it doesn't *really*
>
> It would be very convenient to `yum install vfxplat-2020`, and I assume vendors would love to be able to rely on a set of approved and standardized libs provided by the OS rather than compile and distribute these libs themselves. However, that particular problem does seem less pressing than "what will be the officially supported Linux OS for the platform?" Perhaps I'm mistaken, but it seems like these suggestions are predicated on using CentOS Stream or other rapidly changing variants when our industry needs much more stability; since there's no longer a stable variant of CentOS we have to look elsewhere.
>
change that much. The churn rate is roughly equivalent to what you see
across RHEL point releases. The difference is that you get them in
bite-sized chunks rather than a huge glut at once every six months.
Now, CentOS Stream 9 is still in the rapid development phase, but I
expect that to end within a month or two as we get closer to the
spring release of Red Hat Enterprise Linux 9.0.
From an ISV and user perspective, I *like* CentOS Stream because it
gives me the same stability I had with legacy CentOS Linux while also
giving me the ability to *fix* broken things and get those fixes
shipped quickly. That's pretty useful when doing development or
working on time-critical projects. Outside of paying for RHEL to get
RHEL folks to give out of band fixes for you, that's pretty much not a
thing in the long-term-maintained Linux distro world.
I'm curious where this perception that CentOS Stream is "rapidly
changing" comes from, though. Was there something you saw that made
you think that?
Larry Gritz
Jan 18, 2022, 3:30:34 PM1/18/22
to Neal Gompa, vfx-platform-discuss
But that's not what it's for. The Docker containers are for our GitHub Actions CI for the projects. To the extent that others might find them useful for other purposes, more power to them. But the goal of those containers is not to make a distribution platform for use in production.
Larry Gritz
Jan 18, 2022, 3:41:51 PM1/18/22
to Neal Gompa, Brent Villalobos, vfx-platform-discuss
We already do those things. All the well-run projects name their libraries properly with so versioning, as well as C++ version-specific namespacing on the symbols, etc.
The problem is that some things are not easily versionable -- for example, .h files for a library, cmake config files, and many other auxiliary components conflict in their naming, placement, or contents. And different versions may support different features or file formats, so merely having libraries installed in parallel is not enough, for example if you have one application linked against OpenEXR 3.1, it had better not write a file with the DWA compression that can't be read by a second application linked against OpenEXR 2.2.
So you have a very complicated environment, beyond merely one set of mutually linked libraries and executable, that all needs to be in sync.
And while we can rebuild any of the open source packages (and do, in multiple configurations and versions), we're stuck with what the big DCC vendors supply (and how well they hide or if they purposely expose any of their dependencies in particular versions). And you may have 5 shows in house, which may not all be on the same versions of the DCC apps, and within one show may have locked to a version of Houdini and a version of Maya, and because of their staggered release schedules, those may not be on the same VFX Platform year at all.
--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAEg-Je_4S0tHzf4%2BhV%3DPr%2BTKe92k_apFL2-bjj7kPWTbrxq%2ByA%40mail.gmail.com.
Alan Fregtman
Jan 18, 2022, 3:52:46 PM1/18/22
to Larry Gritz, Neal Gompa, Brent Villalobos, vfx-platform-discuss
Out of curiosity, has anybody explored rootless container runtimes as a way of running DCCs in hyper-controlled environments?
Is anything holding this back technologically? (Is the GPU support very poor for example? I remember Nvidia had some containerized GPU passthrough support years ago, but never tried running anything too advanced on it.)
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/172C617C-4827-47C5-B1A3-ED6DD01106DE%40larrygritz.com.
Neal Gompa
Jan 18, 2022, 3:59:13 PM1/18/22
to Alan Fregtman, Larry Gritz, Brent Villalobos, vfx-platform-discuss
On Tue, Jan 18, 2022 at 3:52 PM Alan Fregtman <al...@felixandpaul.com> wrote:
>
> Out of curiosity, has anybody explored rootless container runtimes as a way of running DCCs in hyper-controlled environments?
>
> Is anything holding this back technologically? (Is the GPU support very poor for example? I remember Nvidia had some containerized GPU passthrough support years ago, but never tried running anything too advanced on it.)
>
That's basically Flatpak in a nutshell. If Red Hat expanded the RHEL
>
> Out of curiosity, has anybody explored rootless container runtimes as a way of running DCCs in hyper-controlled environments?
>
> Is anything holding this back technologically? (Is the GPU support very poor for example? I remember Nvidia had some containerized GPU passthrough support years ago, but never tried running anything too advanced on it.)
>
UBI package set (which I keep hearing rumors of it happening...) and
made the Flatpak runtime freely accessible[1], it'd be somewhat
doable.
There are two big problems:
1. Binary objects in the container and on the host for drivers need to
match exactly for Mesa and the NVIDIA driver. The current
solutions for this are very fragile and prone to breakage.
2. Access to the GPU device is usually mediated by the compositor or
something else. This is mostly handled by the Flatpak runtime
environment setup. We don't have mediation for all the fancy stuff
applications may want from GPUs yet, though.
[1]: https://developers.redhat.com/blog/2020/08/12/introducing-the-red-hat-flatpak-runtime-for-desktop-containers
Jeff Bradley
Jan 18, 2022, 6:19:11 PM1/18/22
to vfx-platform-discuss
First, welcome Neal! It's great to have new perspectives. I appreciate questioning assumptions--particularly as technology moves forward in areas that I'm not keeping up with every day.
As you point out, one of the major challenges the studios face is the with common libraries like Qt, versions of which make their way into our environments through the system, DCC installations, and independent installations. We frequently hit build and runtime issues with other system level tools like boost, python, zlib, tiff,... Your examples of ssl and crypto ring linkage issue bells for me, too. It seems likely that with sufficient versioning, namespacing and controls throughout the tool chain, we could have the entire range of DCCs and custom tools run against a single system. I can't speak for the studios, but that level of intense version control could result in more maintenance than is palatable--particularly in the short and medium term. I'm also curious about the value proposition.
We have DCC's running within Nvidia-enabled docker containers with performance equivalent to a standard studio workstation. One of the valuable aspects is customized container's complete isolation from the host system, which allows us to reduce the chance of library confusion. Another valuable feature is how the image provides a high level of assurance that we can run the same container on many systems without additional work to those new systems. e.g CentOS 7 image running on CentOS 8, Ubuntu or Mac (some use cases).
It's been a while since I last looked at Flatpak and I didn't look deeply, but it seemed great at isolating one application from another. The shared runtimes seemed like a potential challenge. I never looked into it far enough, but I'm curious what the development environment would look like for building against an application in a Flatpak. (It's an exercise for me to follow up on another day.)
One more note: even if two DCC's are on the same year of the VFX Reference Platform, it doesn't mean that their choice of library versions within that VFX spec are always 100% compatible. (If memory serves, we hit something with Qt recently.)
Thanks,
Jeff
fede...@gmail.com
Jan 18, 2022, 7:41:06 PM1/18/22
to vfx-platform-discuss
On Wednesday, 19 January 2022 at 07:52:46 UTC+11 Alan Fregtman wrote:
Out of curiosity, has anybody explored rootless container runtimes as a way of running DCCs in hyper-controlled environments?Is anything holding this back technologically? (Is the GPU support very poor for example? I remember Nvidia had some containerized GPU passthrough support years ago, but never tried running anything too advanced on it.)
We have been exploring this. For almost a year now, we have had worked with hybrid rez/singularity (apptainer now) environments for development. Basically, our environments are composed of a base container image with the base OS libraries plus a dynamic layer of the rest of the software stack controlled by rez - note that we have rezyfied all 3rd party low-level libraries where we need different versions (i.e openssl) and all the components of the VFX platform
(.ei, OpenEXR, python, QT, boost, etc). That gives us:
(.ei, OpenEXR, python, QT, boost, etc). That gives us:
- A stable base with all the benefits of reproducible builds (as we control all the software that goes into builds)
- RnD independence from the host OS updates that the System Engineer team wants to deliver to the artists. For example, we were able to upgrade all RnD and all-new productions to CentOS-7.8, but since we still have a show locked to CentOS-7.4, we still transparently build, test and release from rez using a CentOS-7.4 base apptainer container.
- Ability to experiment with new Nvidia drivers ( we do not bundle the Nvidia drivers in the apptainer image, we passthrough the one installed in the host, but we have the ability to mount different versions)
Regarding your question about runtime, all our initial performance tests have been great (again using a passthrough to the GPU/ Nvidia drivers).
The performance of running interactively the DCC apps (Maya, Nuke, Houdini, RV) is the same as bare metal (same frame rates). Downloading the image takes 3-5 seconds and then it is cached so users do not perceive any difference. All the I/O devices like Wacoms work with no lag, and audio works fine as well.
This year we are trying to deploy this as a default for all our shows. This, in my view, gives us some extra benefits:
- Matching build/runtime environments which is paramount for reproducibility
- Allow us to maintain different shows under different OS versions (we can control the runtime OS environment with just an env variable)
- We can transition and adopt newer OS versions faster, as productions that are afraid to move forward can test and rollback easily (without the need of massive re-imagine deployments)
The next step we are exploring is to move all the more infrequent moving parts into the container and have only the fast pace moving parts ( in-house software) managed dynamically by rez. That is, build specialized images with the 3rd party libraries needed by a given version of the DCCs and the DCCs on it.
If this is successful, I think it would
- Improve startup and runtime performance (because more software is locally cached and not pulled from the network base rez central repository, and because we will shorten the length for lookup paths)
- Increase stability (because we will be installing software packages in their native location. Currently, we are doing lots of surgery in some 3rd party libs/app)
- Reduce rez package count -> reduce resolve times -> reduce conflicts
The 2 main container image options we exploring are formed as:
- OS-base + OS-runtime + VfXplatform-202X (i.e for 2021 gcc-9.3.1, Qt-5.15, OpenEXR-2.4, boost-1.73, python-3.7) + DCCAPP-Version
- SO-base + OS-runtime + DCCAPP-Version (with his preferred libs Qt, OpenEXR, opnessl, pyton, etc)
I'm more inclined to 2, because even the VFX platform has been very helpful in setting a common target platform, the reality is that not all software vendors ship their products with complete vfx reference platform adherence for the given year. There is always a transition period. Some vendors release early access versions at the beginning of the year adopting some but not all the recommendations, and others will just release the end product at the end of that given year with different subsets of those recommendations.
Paolo Berto
Jan 20, 2022, 1:20:44 AM1/20/22
to vfx-platform-discuss
>
For almost a year now, we have had worked with hybrid rez/singularity (apptainer now)
Quick side question about this: what advantages do you have in using apptainer compared to the the more popular os-level virtualizations such as docker or podman?
P
Nicholas Yue
Jan 20, 2022, 11:39:22 AM1/20/22
to Paolo Berto, vfx-platform-discuss
Hi Paolo,
This may shed some light
For Singularity(apptainer) and Shifter, there is no daemon [root!] running (smaller/different attack surface?)
Singularity/Shifter was born in the HPC environment so native (GPU) performance is not an afterthought, HPC environment has standardise way for user to run jobs/container (e.g. SLURM), HPC admin definitely do not allow root running on their HPC cluster.
Supporting complicated software stack is designed for e.g. Genomic sequencing requires running a large collection of very different software written over time by many different people each with different dependencies.
I recall the folks at Animal Logic use Singularity, hopefully they are able to share their expertise and experience.
Cheers
--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/fb6e4784-61ef-413d-ac73-8826faae54c0n%40googlegroups.com.
Mike Rochefort
Jan 20, 2022, 12:23:39 PM1/20/22
to vfx-platform-discuss
On Thu, Jan 20, 2022 at 1:20 AM Paolo Berto <pbe...@gmail.com> wrote:
> Quick side question about this: what advantages do you have in using apptainer compared to the more popular os-level virtualizations such as docker or podman?
Deploying Singularity workloads was part of my job when I was a member
of the UConn HPC admin team. This was a few years ago, but at the time
rootless container workloads were still not up to snuff for end-users,
and the use of GPU passthrough required the NVIDIA docker setup or
similar OCI runtime hooks. On a cluster, root privileges are not
allowed in the slightest, so that kind of ruled out that ecosystem.
The main benefit of Singularity for us was that container images are
actually executable SquashFS binaries that can be copied around and
run like traditional applications (e.g. ./myApp) without root
privileges. Similar to the OCI space, the build recipe allows you to
specify a default command and optionally additional "applications". A
similar structure can be achieved by using CMD and a custom ENTRYPOINT
script for OCI images. So in terms of workflow, a user on their own
system could create a Singularity image, copy it directly to the
cluster and run it out of the box like a native app, no registry
required (which was a bonus considering how many nodes we had and
Slurm jobs could access the binary directly). If additional parameters
were required, then you could use the Singularity cli in conjunction
with the image path. There are other ways of transporting OCI images
without using a registry (like generating tarballs, etc), but those
are extra steps.
For the small render farm I was managing, I also used Singularity to
deploy Blender as at the time it wasn't VFX Reference Platform
compliant and wasn't possible to run on EL7 using the official
binaries.
Ultimately, Singularity provided a convenience factor that other
tooling didn't (being able to natively pass in NVIDIA GPUs with just
--nv was very nice, it did the binding for you). It was a more
"batteries included" approach compared to other solutions and simple
to work with, both for admins and users. Some other cluster admins I
talked with online weren't fans of it and chose other HPC oriented
container solutions and authored their own workflows for users.
How it compares to the container tooling landscape of today, I can't
really say. I haven't used Singularity (now Apptainer) in a few years
and for the work I do now I get along fine with the rootless and
daemon-less Buildah/Podman/Skopeo toolset. Haven't tried doing GPU
passthrough in a while...
--
Mike Rochefort
Solution Architect, Red Hat
> Quick side question about this: what advantages do you have in using apptainer compared to the more popular os-level virtualizations such as docker or podman?
Deploying Singularity workloads was part of my job when I was a member
of the UConn HPC admin team. This was a few years ago, but at the time
rootless container workloads were still not up to snuff for end-users,
and the use of GPU passthrough required the NVIDIA docker setup or
similar OCI runtime hooks. On a cluster, root privileges are not
allowed in the slightest, so that kind of ruled out that ecosystem.
The main benefit of Singularity for us was that container images are
actually executable SquashFS binaries that can be copied around and
run like traditional applications (e.g. ./myApp) without root
privileges. Similar to the OCI space, the build recipe allows you to
specify a default command and optionally additional "applications". A
similar structure can be achieved by using CMD and a custom ENTRYPOINT
script for OCI images. So in terms of workflow, a user on their own
system could create a Singularity image, copy it directly to the
cluster and run it out of the box like a native app, no registry
required (which was a bonus considering how many nodes we had and
Slurm jobs could access the binary directly). If additional parameters
were required, then you could use the Singularity cli in conjunction
with the image path. There are other ways of transporting OCI images
without using a registry (like generating tarballs, etc), but those
are extra steps.
For the small render farm I was managing, I also used Singularity to
deploy Blender as at the time it wasn't VFX Reference Platform
compliant and wasn't possible to run on EL7 using the official
binaries.
Ultimately, Singularity provided a convenience factor that other
tooling didn't (being able to natively pass in NVIDIA GPUs with just
--nv was very nice, it did the binding for you). It was a more
"batteries included" approach compared to other solutions and simple
to work with, both for admins and users. Some other cluster admins I
talked with online weren't fans of it and chose other HPC oriented
container solutions and authored their own workflows for users.
How it compares to the container tooling landscape of today, I can't
really say. I haven't used Singularity (now Apptainer) in a few years
and for the work I do now I get along fine with the rootless and
daemon-less Buildah/Podman/Skopeo toolset. Haven't tried doing GPU
passthrough in a while...
--
Mike Rochefort
Solution Architect, Red Hat
fede...@gmail.com
Jan 20, 2022, 8:37:05 PM1/20/22
to vfx-platform-discuss
Hi Paolo,
Yue and Mike mentioned what probably are the most important reasons why we choose singularity (I tried first buildah/podman but our Centos version and some other internals at AL put some roadblocks, singularity was simpler)
Target a single application or workflow (often with very complicated stacks), software that has difficult dependencies
Guiding principles
Make container as transparent as possible to user.
Run as much as possible as a normal user process, change environment only enough to provide portability of the application.
Level of isolation
Favours integration rather than isolation. (still preserves security restrictions on the container, and providing reproducible images)
The container had access to the host filesystem automatically
Global config shared across all images ie. Mounts/env variables /defaults
Easy GPU/nvidia integration (single parameter --nv) . Native access, native speed.
Transparent sharing host peripherals/network/driver
Yue and Mike mentioned what probably are the most important reasons why we choose singularity (I tried first buildah/podman but our Centos version and some other internals at AL put some roadblocks, singularity was simpler)
Here are some more points from our container investigation:
Use case FocusTarget a single application or workflow (often with very complicated stacks), software that has difficult dependencies
Guiding principles
Make container as transparent as possible to user.
Run as much as possible as a normal user process, change environment only enough to provide portability of the application.
Level of isolation
Favours integration rather than isolation. (still preserves security restrictions on the container, and providing reproducible images)
The container had access to the host filesystem automatically
Global config shared across all images ie. Mounts/env variables /defaults
Easy GPU/nvidia integration (single parameter --nv) . Native access, native speed.
Transparent sharing host peripherals/network/driver
No daemon process, no sudo need to run it. No root privilege escalation
The container runs as the current user automatically, no sudo user inside the container
Performance
Easy to use
It is a single easy distributable executable
Can build from existing docker images
Ser Heang Tan
Jan 21, 2022, 10:46:34 AM1/21/22
to vfx-platform-discuss
Hi Fede,
Very well said! I always love the integration of singularity/apptainer. Just want to point out that you will still need to have singularity/apptainer installed before you can run the singularity executable file (usually named with extension 'sif''[sif]), and you will need GO.
By the way, there are 2 diff forks of singularity:
1. singularity by apptainer.org
2. singularity by sylab.io
The version prior to 3.9 would be identical but it might be slightly different once apptainer release version 1.0.
[sif]: SIF (singularity image format) container that uses squashfs for the file system. SIF files are compressed and immutable making them the best choice for reproducible, production-grade containers
Regards,
Ser Heang TAN
Pierre Jasmin
Jan 25, 2022, 3:50:18 PM1/25/22
to vfx-platform-discuss
My 2 cents - with the caveat that I am a developer not an IT
person, last thing I want to have is an opinion on containers...
That said, whatever you decide, as we support like 15-20
applications with maybe over half of them with some linux version,
so eventually what you decide indirectly affects us, SO might as well have an opinion, seems the
best idea is to stick to a downstream version of RHEL. Right now I know of but have not used them, Alma and Rocky Linux that showed up to fill the gap of CentOs going upstream from RHEL. That
is already 3 guaranteed to be binary compatible choices including
RHEL. All I care at first degree I think is total binary compatibility.
Let them deal with upstream distros bugs and variants.
Since CentOS is the most supported in this neck of the world and developed on, seems a bad idea to go to Ubuntu/Debian etc... for a reference platform. Maybe (?) actually the better idea is to work with IBM/RHEL on a plug-and-play distro that includes the VFX platform tested downstream (as in even pre-install gcc 6.3.1 and 9.3.1 if they are in last 5 years supported as opposed to the version that is default of a current dot release with standard distribution)? .... I also see RHEL now has 16 seats free dev licenses: https://developers.redhat.com/articles/faqs-no-cost-red-hat-enterprise-linux . Maybe I should go back on RHEL? I mean I pay for Parallels so I can multi-OS on a mac laptop on the road, and it if you are a developer as opposed to an IT/pipeline person makes life so easy like that. Which is my next point, For deployment I understand the value of having Linux packages particularly for render farms and cloud equivalent, to quickly push a new workstation live - the biggest value of linux is deployment, not development. I am still curious how you sliced in year slots Windows though :) [KEEP THIS A SHORT EMAIL]
SO, again some of this thread is above my head - all I need
is what flavor.version to work in...
On a meta-level, I guess there would an advantage for market acceptance (and actually support in my case) to have only one desktop, and at this point (it's not 2002 anymore), I don't care if it's KDE or gnome... Thinking like that would also substantially increase the usefulness of all this (maybe make it worthwhile for drivers etc to pay attention,someone mentioned HDR as an example).
Neal Gompa
Jan 26, 2022, 6:34:50 AM1/26/22
to Pierre Jasmin, vfx-platform-discuss
On Tue, Jan 25, 2022 at 3:50 PM Pierre Jasmin <pierre...@gmail.com> wrote:
>
>
> My 2 cents - with the caveat that I am a developer not an IT person, last thing I want to have is an opinion on containers...
>
> That said, whatever you decide, as we support like 15-20 applications with maybe over half of them with some linux version, so eventually what you decide indirectly affects us, SO might as well have an opinion, seems the best idea is to stick to a downstream version of RHEL. Right now I know of but have not used them, Alma and Rocky Linux that showed up to fill the gap of CentOs going upstream from RHEL. That is already 3 guaranteed to be binary compatible choices including RHEL. All I care at first degree I think is total binary compatibility. Let them deal with upstream distros bugs and variants.
>
> Since CentOS is the most supported in this neck of the world and developed on, seems a bad idea to go to Ubuntu/Debian etc... for a reference platform. Maybe (?) actually the better idea is to work with IBM/RHEL on a plug-and-play distro that includes the VFX platform tested downstream (as in even pre-install gcc 6.3.1 and 9.3.1 if they are in last 5 years supported as opposed to the version that is default of a current dot release with standard distribution)? .... I also see RHEL now has 16 seats free dev licenses: https://developers.redhat.com/articles/faqs-no-cost-red-hat-enterprise-linux . Maybe I should go back on RHEL? I mean I pay for Parallels so I can multi-OS on a mac laptop on the road, and it if you are a developer as opposed to an IT/pipeline person makes life so easy like that. Which is my next point, For deployment I understand the value of having Linux packages particularly for render farms and cloud equivalent, to quickly push a new workstation live - the biggest value of linux is deployment, not development. I am still curious how you sliced in year slots Windows though :) [KEEP THIS A SHORT EMAIL]
>
While I don't work for Red Hat, I actually agree that going to RHEL
>
>
> My 2 cents - with the caveat that I am a developer not an IT person, last thing I want to have is an opinion on containers...
>
> That said, whatever you decide, as we support like 15-20 applications with maybe over half of them with some linux version, so eventually what you decide indirectly affects us, SO might as well have an opinion, seems the best idea is to stick to a downstream version of RHEL. Right now I know of but have not used them, Alma and Rocky Linux that showed up to fill the gap of CentOs going upstream from RHEL. That is already 3 guaranteed to be binary compatible choices including RHEL. All I care at first degree I think is total binary compatibility. Let them deal with upstream distros bugs and variants.
>
> Since CentOS is the most supported in this neck of the world and developed on, seems a bad idea to go to Ubuntu/Debian etc... for a reference platform. Maybe (?) actually the better idea is to work with IBM/RHEL on a plug-and-play distro that includes the VFX platform tested downstream (as in even pre-install gcc 6.3.1 and 9.3.1 if they are in last 5 years supported as opposed to the version that is default of a current dot release with standard distribution)? .... I also see RHEL now has 16 seats free dev licenses: https://developers.redhat.com/articles/faqs-no-cost-red-hat-enterprise-linux . Maybe I should go back on RHEL? I mean I pay for Parallels so I can multi-OS on a mac laptop on the road, and it if you are a developer as opposed to an IT/pipeline person makes life so easy like that. Which is my next point, For deployment I understand the value of having Linux packages particularly for render farms and cloud equivalent, to quickly push a new workstation live - the biggest value of linux is deployment, not development. I am still curious how you sliced in year slots Windows though :) [KEEP THIS A SHORT EMAIL]
>
would be a good move. As for the idea of a plug-and-play distro with
integrated VFX platform components in there as "plug-and-play", that
was essentially the crux of my idea that started this thread. :)
I think that it's a good idea to do it, not just for deployment
purposes, but also to make it so ISVs actually *want* to support your
industry on Linux. My suggestion is to work with Red Hat and the
community in a CentOS SIG to produce something with CentOS Stream that
people could use and develop against, and leverage that same work in
production with RHEL to make a plug-and-play deployment system.
And CentOS Stream *is* still binary compatible with RHEL in every way
that matters (except for some third party kernel modules). Just think
about it as getting some of the updates a few months early. I know
this because at $DAYJOB, we build everything on CentOS Stream and
release it to customers running CentOS Linux, AlmaLinux, Rocky Linux,
and RHEL. :)
> SO, again some of this thread is above my head - all I need is what flavor.version to work in...
>
> On a meta-level, I guess there would an advantage for market acceptance (and actually support in my case) to have only one desktop, and at this point (it's not 2002 anymore), I don't care if it's KDE or gnome... Thinking like that would also substantially increase the usefulness of all this (maybe make it worthwhile for drivers etc to pay attention,someone mentioned HDR as an example).
>
the way you folks want to do things basically makes it currently very
unappealing for someone like me as an ISV to make anything for you.
Doing something like what I suggest would make market acceptance much
easier, and allow vendors, ISVs, and experts to work together to
implement things like HDR.
David Aguilar
Mar 10, 2022, 2:01:51 PM3/10/22
to Neal Gompa, Pierre Jasmin, vfx-platform-discuss
On Wed, Jan 26, 2022 at 3:34 AM Neal Gompa <ngom...@gmail.com> wrote:
On Tue, Jan 25, 2022 at 3:50 PM Pierre Jasmin <pierre...@gmail.com> wrote:
>
>
> My 2 cents - with the caveat that I am a developer not an IT person, last thing I want to have is an opinion on containers...
>
> That said, whatever you decide, as we support like 15-20 applications with maybe over half of them with some linux version, so eventually what you decide indirectly affects us, SO might as well have an opinion, seems the best idea is to stick to a downstream version of RHEL. Right now I know of but have not used them, Alma and Rocky Linux that showed up to fill the gap of CentOs going upstream from RHEL. That is already 3 guaranteed to be binary compatible choices including RHEL. All I care at first degree I think is total binary compatibility. Let them deal with upstream distros bugs and variants.
>
> Since CentOS is the most supported in this neck of the world and developed on, seems a bad idea to go to Ubuntu/Debian etc... for a reference platform. Maybe (?) actually the better idea is to work with IBM/RHEL on a plug-and-play distro that includes the VFX platform tested downstream (as in even pre-install gcc 6.3.1 and 9.3.1 if they are in last 5 years supported as opposed to the version that is default of a current dot release with standard distribution)? .... I also see RHEL now has 16 seats free dev licenses: https://developers.redhat.com/articles/faqs-no-cost-red-hat-enterprise-linux . Maybe I should go back on RHEL? I mean I pay for Parallels so I can multi-OS on a mac laptop on the road, and it if you are a developer as opposed to an IT/pipeline person makes life so easy like that. Which is my next point, For deployment I understand the value of having Linux packages particularly for render farms and cloud equivalent, to quickly push a new workstation live - the biggest value of linux is deployment, not development. I am still curious how you sliced in year slots Windows though :) [KEEP THIS A SHORT EMAIL]
>
While I don't work for Red Hat, I actually agree that going to RHEL
would be a good move. As for the idea of a plug-and-play distro with
integrated VFX platform components in there as "plug-and-play", that
was essentially the crux of my idea that started this thread. :)
I think that it's a good idea to do it, not just for deployment
purposes, but also to make it so ISVs actually *want* to support your
industry on Linux. My suggestion is to work with Red Hat and the
community in a CentOS SIG to produce something with CentOS Stream that
people could use and develop against, and leverage that same work in
production with RHEL to make a plug-and-play deployment system.
Something that could be really helpful IMO would be multiple prebuilt component sets that track the major vfxplatform releases.
This means not one set of packages, but several with perhaps redundant packages across them.
As a consumer of these third-party applications it's really super annoying that Nuke requires exactly Qt 5.12.1 and Maya+Houdini don't work with that version but require 5.12.6+.
This inevitably leads to geometric growth in the permutations of builds that a studio has to perform.
Wouldn't it be great if there was a neutral, vfxplatform2021-qt5 package that all three of these apps could reliably target so that they wouldn't have to ship their own redundant dependencies? As a consumer I personally think that would be great.
I don't think any of the ISVs particularly cares to build and provide their qt5, python3.7 and other platform components.
It happens because there's no other option.
It's also important that the individual components are separable. eg. someone might want to use just a subset and replace some slice of it, so there's some benefit to having things like the compiler or other bits in package-specific installation prefixes so that the user can compose things just-in-time.
> Solution 1: Ensure libraries and tools are designed for parallel
installability.
installability.
That seems like an important part of it, and would probably be needed one way or another.
> Solution 2: Namespace libraries and tools. That means that libraries
and tools that would ordinarily conflict would get de-conflicted by
shuffling files around and possibly also using alternatives.
and tools that would ordinarily conflict would get de-conflicted by
shuffling files around and possibly also using alternatives.
This is effectively what some places do in practice.
I don't think "alternatives" is a viable solution, though. It's global state, but we need the superset of all states, all at the same time, so doing stuff like playing games with symlinks in eg. /usr/bin is not going to fly
> Solution 3: Modularity[1]. This is a technology that also exists in
RHEL/CentOS as Application Streams[2], and can be used to define a
collection of components into a kind of "sub repository" that offers
content that overshadows the main repository content.
RHEL/CentOS as Application Streams[2], and can be used to define a
collection of components into a kind of "sub repository" that offers
content that overshadows the main repository content.
IMO solution this sounds like a dead end to me. We don't want to overshadow the main repository content.
Just my .02.
Kevin Constantine
Mar 10, 2022, 8:39:33 PM3/10/22
to vfx-platform-discuss
On Thursday, March 10, 2022 at 11:01:51 AM UTC-8 david....@disneyanimation.com wrote:
On Wed, Jan 26, 2022 at 3:34 AM Neal Gompa <ngom...@gmail.com> wrote:On Tue, Jan 25, 2022 at 3:50 PM Pierre Jasmin <pierre...@gmail.com> wrote:
>
>
> My 2 cents - with the caveat that I am a developer not an IT person, last thing I want to have is an opinion on containers...
>
> That said, whatever you decide, as we support like 15-20 applications with maybe over half of them with some linux version, so eventually what you decide indirectly affects us, SO might as well have an opinion, seems the best idea is to stick to a downstream version of RHEL. Right now I know of but have not used them, Alma and Rocky Linux that showed up to fill the gap of CentOs going upstream from RHEL. That is already 3 guaranteed to be binary compatible choices including RHEL. All I care at first degree I think is total binary compatibility. Let them deal with upstream distros bugs and variants.
>
> Since CentOS is the most supported in this neck of the world and developed on, seems a bad idea to go to Ubuntu/Debian etc... for a reference platform. Maybe (?) actually the better idea is to work with IBM/RHEL on a plug-and-play distro that includes the VFX platform tested downstream (as in even pre-install gcc 6.3.1 and 9.3.1 if they are in last 5 years supported as opposed to the version that is default of a current dot release with standard distribution)? .... I also see RHEL now has 16 seats free dev licenses: https://developers.redhat.com/articles/faqs-no-cost-red-hat-enterprise-linux . Maybe I should go back on RHEL? I mean I pay for Parallels so I can multi-OS on a mac laptop on the road, and it if you are a developer as opposed to an IT/pipeline person makes life so easy like that. Which is my next point, For deployment I understand the value of having Linux packages particularly for render farms and cloud equivalent, to quickly push a new workstation live - the biggest value of linux is deployment, not development. I am still curious how you sliced in year slots Windows though :) [KEEP THIS A SHORT EMAIL]
>
While I don't work for Red Hat, I actually agree that going to RHEL
would be a good move. As for the idea of a plug-and-play distro with
integrated VFX platform components in there as "plug-and-play", that
was essentially the crux of my idea that started this thread. :)
I think that it's a good idea to do it, not just for deployment
purposes, but also to make it so ISVs actually *want* to support your
industry on Linux. My suggestion is to work with Red Hat and the
community in a CentOS SIG to produce something with CentOS Stream that
people could use and develop against, and leverage that same work in
production with RHEL to make a plug-and-play deployment system.Something that could be really helpful IMO would be multiple prebuilt component sets that track the major vfxplatform releases.This means not one set of packages, but several with perhaps redundant packages across them.As a consumer of these third-party applications it's really super annoying that Nuke requires exactly Qt 5.12.1 and Maya+Houdini don't work with that version but require 5.12.6+.This inevitably leads to geometric growth in the permutations of builds that a studio has to perform.Wouldn't it be great if there was a neutral, vfxplatform2021-qt5 package that all three of these apps could reliably target so that they wouldn't have to ship their own redundant dependencies? As a consumer I personally think that would be great.I don't think any of the ISVs particularly cares to build and provide their qt5, python3.7 and other platform components.It happens because there's no other option.It's also important that the individual components are separable. eg. someone might want to use just a subset and replace some slice of it, so there's some benefit to having things like the compiler or other bits in package-specific installation prefixes so that the user can compose things just-in-time.
I always wondered how effectively we'd be able to leverage something like a RedHat SCL that provided the VFX reference platform dependencies every year. It might be constructed in such a way that you could install the entire reference platform, or just the few packages that you needed (the compiler and all its dependencies). Something like that might solve the "there's no other option". It certainly doesn't solve the case generically for Ubuntu or Debian, though maybe they have something similar, but since Centos/RHEL is 80% of the market according to the survey, maybe focusing on Centos/RHEL is sufficient to move the needle. Apologies if it's been talked about before and found to not be sufficient.
-kevin
Mike Rochefort
Mar 10, 2022, 9:14:28 PM3/10/22
to vfx-platform-discuss
On Thu, Mar 10, 2022 at 8:39 PM Kevin Constantine
<kevin.co...@gmail.com> wrote:
> I always wondered how effectively we'd be able to leverage something
> like a RedHat SCL that provided the VFX reference platform
> dependencies every year.
As a question on that, who would the target consumers of the SCL be?
<kevin.co...@gmail.com> wrote:
> I always wondered how effectively we'd be able to leverage something
> like a RedHat SCL that provided the VFX reference platform
> dependencies every year.
I toyed around with creating a VFX Platform SCL back for CY2018 or so.
Besides the issue of distribution portability, a larger issue that
comes up with such a task is asking for industry standardization for
every main component from all vendors and consumers.
Vendors and studios may choose different compile options depending on
what they need, or apply custom patches beyond the projects' released
sources. An SCL effectively forces a shared opinion on every consumer
of the given package (whether that's a good thing or not is a
different debate). Every vendor is going to have their own QA process
and pipeline for dependency updates, including bumps to x.y.Z versions
of software, and may not appreciate losing that control. And that's
not touching on the actual delivered application schedules.
Relating to delivery schedules, this would also call into play
determining an appropriate lifecycle for updating the platform SCL
over time, to account for patch releases, custom patches, security
updates, etc. On paper it's a relatively simple prospect, but when
looking at the larger picture it would require a lot of buy-in and
communication from the broader ecosystem.
Regardless, it should totally be possible to create a generic Platform
SCL and I might just be overthinking things.
Kevin Constantine
Mar 11, 2022, 12:58:21 PM3/11/22
to vfx-platform-discuss
On Thursday, March 10, 2022 at 6:14:28 PM UTC-8 Mike Rochefort wrote:
On Thu, Mar 10, 2022 at 8:39 PM Kevin Constantine
<kevin.co...@gmail.com> wrote:
> I always wondered how effectively we'd be able to leverage something
> like a RedHat SCL that provided the VFX reference platform
> dependencies every year.
As a question on that, who would the target consumers of the SCL be?
I imagined the consumers being the DCC vendors who currently need to provide many of their own dependencies. It's maybe naive of me, since I don't have any experience on the vendor side of things, but right now there's no easy "paved road". What we have is a list of suggested package versions that make up the reference platform. That was a big step forward to be sure, but it still leaves dependency management up to each vendor. I'd like to think that if there was a bunch of RPMs that a vendor could download, install, and build/test their software, that'd be preferable to having to maintain their own set of dependencies.
Ultimately the users of the DCC products would also be the consumers as well since you'd need the SCL installed to provide dependencies for the software built on top.
I toyed around with creating a VFX Platform SCL back for CY2018 or so.
Besides the issue of distribution portability, a larger issue that
comes up with such a task is asking for industry standardization for
every main component from all vendors and consumers.
Vendors and studios may choose different compile options depending on
what they need, or apply custom patches beyond the projects' released
sources. An SCL effectively forces a shared opinion on every consumer
of the given package (whether that's a good thing or not is a
different debate). Every vendor is going to have their own QA process
and pipeline for dependency updates, including bumps to x.y.Z versions
of software, and may not appreciate losing that control. And that's
not touching on the actual delivered application schedules.
In terms of customization, I think one of the goals of the VFX reference platform is to reduce the amount of variation that exists in the software ecosystem, both for the vendors and for the studios who need to build their own software in each of the vendored permutations. While reduction of choice is in some cases problematic, it can also beneficial, and can likely be viewed as a design goal of the vfx-reference-platform.
For those studios who want to make modifications, the source-rpms should be available for the vfx-platform packages and could be modified and deployed. You'd then be on your own to support the vendor supplied software that runs on top of it.
For the vendors who want a version change to the reference-platform supplied packages, there's already a yearly cadence and a process for altering versions. Presumably that process would continue to be used. The same with compiler flags. Right now it's a free-for-all. It may be tricky to make everyone happy (especially at first), but presumably negotiations can occur to arrive at a set of options and flags that make everyone equally unhappy ;-)
Providing an SCL also doesn't prevent a studio from building all of the products in the reference-platform as they presumably do today, in support of the various DCC's. This would just create a path of least resistance that became more and more useful as it gained adoption.
Relating to delivery schedules, this would also call into play
determining an appropriate lifecycle for updating the platform SCL
over time, to account for patch releases, custom patches, security
updates, etc. On paper it's a relatively simple prospect, but when
looking at the larger picture it would require a lot of buy-in and
communication from the broader ecosystem.
Yeah, agreed that there probably needs to be a cadence for change or something like that. Right now the platform changes once a year. Typically you're able to install multiple SCL's on a host, so it ought to be possible to support older productions/projects who can't upgrade for a year or two. Honestly, I'm not sure how security fixes would be handled in the existing vfx-reference-platform "framework". It seems like it might be an "exercise for the reader" at this point. It would be additional work, but providing security fixes for the various "supported SCL's" (maybe SCL's are supported for 2 years or something), as long as they don't break ABI compatibility would be feasible, and also help security efforts.
At the end of the day, if no one uses it, it was done for naught, but my thought is that centrally providing an easy path for vendors to obtain their dependencies and provide them to their customers so that they don't need to maintain and ship them individually is beneficial. It also provides some additional consistency for the studios/community because they don't need to support a private copy of Qt5 provided by VendorA who applied their own patches, and another copy of Qt5 with some other set of patches for VendorB.
-kevin
Aloys Baillet
Mar 11, 2022, 7:35:35 PM3/11/22
to vfx-platform-discuss
Hi David,
I agree with you, and that's what I've started there: https://github.com/AcademySoftwareFoundation/aswf-docker/tree/master/packages/conan/recipes using Conan as the package manager, with binaries available for each VFX platform here: https://linuxfoundation.jfrog.io/ui/repos/tree/General/aswf-conan/aswf/boost. I've got these recipes working and uploaded and used to build the linux ASWF docker images with prebuilt binaries: alembic, boost, clang, cmake, cppunit, glew, glfw, gtest, imath, log4cplus, ninja, openexr, pybind11, pyside, python, qt.
And these are available for VFX-2019 up to VFX-2022.
N.B. that I have a recipe for Qt that works locally but can't be uploaded as it takes too long to build on the free GitHub action machines...
And the other issue is that I haven't had the time to test these on windows, that will take a bit more time which I don't have enough of these days unfortunately!
Cheers,
Aloys
Edward Lam
Mar 11, 2022, 7:55:02 PM3/11/22
to David Aguilar, Neal Gompa, Pierre Jasmin, vfx-platform-discuss
Hi David,
On Thu, Mar 10, 2022 at 2:01 PM David Aguilar wrote:
I don't think any of the ISVs particularly cares to build and provide their qt5, python3.7 and other platform components.It happens because there's no other option.
We ship these components also because we *require* building our own patches on top of them. This is for a variety of reasons:
- patches are applied from upstream that fix bugs as our customers encounter them
- perform local patches that fix bugs experienced by our customers (and hopefully upstream accepts them)
- rollback breaking changes from version to version. eg. Qt 5.15 regressions from Qt 5.12
The DCC in this respect is very much like a distro in this sense. We don't live in that ideal world where the third party components are bug free nor where I can tell a customer that the bug they're experiencing is in a third party component and therefore cannot do anything for them.
Best regards,
-Edward
Reply all
Reply to author
Forward
0 new messages