[rez] Project as Rez package

[Marcus Ottosson]

Hi all,

Just getting started with integrating Rez and was wondering what your thoughts and experiences were with regards to having projects as regular old Rez packages.

At my current studio, we’ve got a system much like Rez and it’s being used in this way with one exception; projects have what you might call sub-packages. That is, for a project called Avengers and an application called Maya, there is an embedded maya, maya:render and maya:py (and more), each of which having its own dependencies and environments. It’s quite likely these could just as well have been separate packages, I’m just about to look into this now, and if so, then it’d make sense to continue the tradition into Rez as well.

I’m expecting a few bumps in the road along the way, and that’s my question to you. Has anyone done this, or are doing this? What should I expect?

Best,
Marcus

[Blazej Floch]

We had 'project' and 'studio' (location) as rez packages.

This allowed us to create some interesting variants like having a 'license' package manage different license severs depending on studio.

As to project we used this since (naively) we did manage requests with rez packages. I would not recommend this route since rez does not have a concept of overrides and you end up with a lot of micromanagement.

But even along the lines of inhouse tooling that relied on some kind of project variable, we moved that variable to the project package. This way we can guarantee it's existence since rez took care of it.

Just be careful to check that there is an explicit version in the request, else you get an arbitrary project if not specified.

[Marcus Ottosson]

Cool, thanks for that.

As to project we used this since (naively) we did manage requests with rez packages. I would not recommend this route since rez does not have a concept of overrides and you end up with a lot of micromanagement.

Can you elaborate on this, what does overrides mean in this context? E.g. projectA has one set of environment variables that projectA_animation overrides? Is there an explicit order to packages Rez resolves, or do they come randomly? If there’s an order, would overrides not work?

--
You received this message because you are subscribed to the Google Groups "rez-config" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.
To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

[Blazej Floch]

The way I see it there is two ways you can operate with rez:

- via resolved requests as rxt (could be within a suite or not)

- via dynamically solving requests (may be frozen in state via timestamp + memcached)

The latter gives you the benefit that it still is a solve that takes care of implicit package changes (os, platform, arch).

However you will need to kind of introduce two systems to have controllable environments:

A) A system that manages the requests representing a toolset

B) A system that manages the timestamp in order to freeze states of environments

Ideally system A) can reuse core technology requests while still allowing you to override specific packages (note: request overrides). This way you can manage multiple shows but still override specific packages as necessary.

I implemented system B) and designed system A), but for the time being I chose to create a rez package that just has all of the requests as requirements.

Don't do it. You loose the priority of a requirement vs. a request within variant selection.

What I meant with you can not have overrides means that you can not have a overrides for requests within rez. This was just a poor choice but it worked as duct tape as long we did not have many environments to manage.

Hope it makes sense.

On Wednesday, 17 April 2019 11:22:32 UTC-4, Marcus Ottosson wrote:

Cool, thanks for that.

As to project we used this since (naively) we did manage requests with rez packages. I would not recommend this route since rez does not have a concept of overrides and you end up with a lot of micromanagement.

Can you elaborate on this, what does overrides mean in this context? E.g. projectA has one set of environment variables that projectA_animation overrides? Is there an explicit order to packages Rez resolves, or do they come randomly? If there’s an order, would overrides not work?

On Wed, 17 Apr 2019 at 16:17, Blazej Floch <blaze...@gmail.com> wrote:

We had 'project' and 'studio' (location) as rez packages.

This allowed us to create some interesting variants like having a 'license' package manage different license severs depending on studio.

As to project we used this since (naively) we did manage requests with rez packages. I would not recommend this route since rez does not have a concept of overrides and you end up with a lot of micromanagement.

But even along the lines of inhouse tooling that relied on some kind of project variable, we moved that variable to the project package. This way we can guarantee it's existence since rez took care of it.

Just be careful to check that there is an explicit version in the request, else you get an arbitrary project if not specified.

On Wednesday, 17 April 2019 02:31:25 UTC-4, Marcus Ottosson wrote:

Hi all,

Just getting started with integrating Rez and was wondering what your thoughts and experiences were with regards to having projects as regular old Rez packages.

At my current studio, we’ve got a system much like Rez and it’s being used in this way with one exception; projects have what you might call sub-packages. That is, for a project called Avengers and an application called Maya, there is an embedded maya, maya:render and maya:py (and more), each of which having its own dependencies and environments. It’s quite likely these could just as well have been separate packages, I’m just about to look into this now, and if so, then it’d make sense to continue the tradition into Rez as well.

I’m expecting a few bumps in the road along the way, and that’s my question to you. Has anyone done this, or are doing this? What should I expect?

Best,
Marcus

--
You received this message because you are subscribed to the Google Groups "rez-config" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-c...@googlegroups.com.

[Marcus Ottosson]

Thanks Blazej!

A system that manages the requests representing a toolset

This makes sense, and now I see what you meant by..

you can not have a overrides for requests within rez

However, I would have thought that having a broad requirement - such as maya>=2015,<2019 - from e.g. a facility-level package could get “specialised” by a show package, e.g. maya>=2017,<2019 and then further specialised by an asset, e.g. maya==2018

$ rez env alita alita_1500

Would that qualify as overrides? Or what would an override do differently?

A system that manages the timestamp in order to freeze states of environments

This goes over my head at the moment; do you mean that you’d only resolve an environment anew if an existing one was old enough? How old is old, 1 minute, 1 hour, 1 week? Is it for performance reasons, or something functional? Spontaneously, it sounds like an artist could end up with an old environment, even though the developer swears by it.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.

[Marcus Ottosson]

Hi all,

Here’s something I’m struggling with in attempting this.

1. I’ve got a facility package, with variables like GITLAB_URL=https://gitlab.here.com
2. I’ve got a alita package for a project, with requirements like facility and core_pipeline>=1.0.2, great
3. I’ve got a maya2018 package, with variables e.g. MAYA_ENABLE_LEGACY_VIEWPORT=1, great

But now I’ve got special requirements for Maya, if used with alita.

1. So I made a alita_maya2018 package, with maya2018 as a requirement, along with project-specific ones and an environment

This kind of works, I think, except it’s quite verbose. I’d end up with lots of duplicate maya2018 packages for each project.

What do you think?

Best,
Marcus

[Dani Asztalos]

Hi Marcus,

First I think you should have a maya package, with 2016/2017/2018 versions, not a maya2018 package.

Then you could make one alita package, with different variants for different maya versions. I wouldn't want an alita_maya2018 package, as it could easily become unmanageable, but of course I don't see the full image yet ;) 

So you should look into variants.

Dani

[Marcus Ottosson]

Thanks Dani!

First I think you should have a maya package, with 2016/2017/2018 versions, not a maya2018 package.

Agreed, wanted to keep things simple for this example.

Then you could make one alita package, with different variants for different maya versions.

Great, will look into that.

[Dani Asztalos]

Out of curiosity, what are you doing differently if alita is used with maya-2018?

Dani

[Marcus Ottosson]

Not 2018 specifically, but if used with Maya versus Houdini, for example. E.g. a modeling tool for Maya, or a rendering plug-in for Houdini.

[Dani Asztalos]

Oh, okay, you are right.

So yeah, you are probably better off with the alita_maya2018 package, because you want to define different requirements per dcc.

We built a system for that on top of rez, with gui and database, where we have environments (essentially list of packages, like maya-2017, mtoa-x.y.z, someplugin, etc.), and you can select the DCC from gui, then select the project you want to work on which will resolve the requested environment and launch the app.

Dani

[Marcus Ottosson]

Ok, I’ll revert to alita_maya2018, but mostly because I can’t actually figure out how to actually use variants. :/ They looked promising though!

• https://github.com/nerdvegas/rez/wiki/Variants

This one doesn’t have an actual example, but says it triggers a build multiple times, but my project package has build_command = None as it doesn’t need building.

Anyone have a reproducible example I could see, i.e. a package.py and (if necessary) a rezbuild.py?

[Allan Johns]

Hey Marcus,

First, please see the previous thread "Opinion request: the proper way to document and validate dependencies outside of the core Rez packages". It gives a sense of how we manage this at Method.

Rez was not designed to manage production environments; it resolves package dependencies, and then lets those resulting packages configure the target runtime env. I think it makes more sense to go the route that Dani describes (and is not dissimilar to what we do at Method, and other studios). Ie, use some other system to define the packages that you require for a given show ad tool; and then use rez to resolve the subsequent environment.

For example, you may have some config that defines the packages you want to use if you're running maya. You may then have another config that partially overrides the global "maya" config, and you use that to refine your requirements for a given show (such as "alita"). Expressing overrides like this is not well suited to rez - if you had some "maya" rez package, and an "alita_maya" package (requiring "maya"), and they both required some "anim_tool" package, then you'd immediately run into trouble if you wanted alita to use a different version of anim_tool than the default specified in "maya" - this would produce a resolve conflict.

It's interesting that newcomers to rez often think about using it in terms of defining a production runtime env, rather than a package's. In rez, a "maya" package should configure the env to run maya, and nothing else - it should not be used to configure an env to use "maya and whatever other tools we want, in this given situation." The reason I started on rez in the first place was because I'd come from two different studios where the problem of package management in production environments had not been solved well. In both studios, if you released a new version of a package, then you also had to update the relevant config files to ensure that that package was then used on the right shows, and that the dependencies stayed correct for those shows. This can quickly become a very difficult and maintenance-heavy problem, and it wasn't uncommon to have runtime conflicts (ie crashing libs, API incompatibilities etc) as a result. The emphasis behind designing rez was to split the problem into two separate parts. In one part (the part that rez manages), you define packages' dependencies, and rez is responsible for resolving an env that contains no version conflicts, given a request for a set of package from a user. In the other part, you define what packages you want to use in a given situation (tool, show, site, or some combination of those); then, once you have the resulting request, you pass it onto rez, which is able to resolve and configure the matching env in realtime (or ahead of time, if you're using contexts/suites).

On another topic, you asked earlier about the order of packages in a resolve. For any given request, the resolve (the list of resulting packages, and the order in which their commands are sourced) is always deterministic. See here for more details: https://github.com/nerdvegas/rez/wiki/Package-Commands#order-of-command-execution

Furthermore, packages listed earlier in a request get priority. For example, consider 4 packages: A-1, A-2, B-1, B-2. Let us also say that both (A-2, B-1) and (A-1, B-2) are valid environments. If you ran "rez-env A B", you would get (A-2, B-1) - because A was requested first, the highest possible version of that package is selected in the resolve. This section of the docs gives more detail, however I do need to update it to emphasise this version priority behaviour: https://github.com/nerdvegas/rez/wiki/Basic-Concepts#dependency-resolving

Hth

A

 

[Marcus Ottosson]

Hey Allan,

if you had some “maya” rez package, and an “alita_maya” package (requiring “maya”), and they both required some “anim_tool” package, then you’d immediately run into trouble if you wanted alita to use a different version of anim_tool than the default specified in “maya” - this would produce a resolve conflict.

I must admit I’m not fully appreciating the problem just yet. Here’s how I would approach this example.

	package.py	Requires
A	anim_tool/package.py
B	vanilla_maya/package.py	requires = ["anim_tool>0.5,<3.0"]
C	vfx_maya/package.py	requires = ["anim_tool>2.2,<2.7"]
D	alita_maya/package.py	requires = ["anim_tool>2.6,<2.6.3"]
B	alita_vector_maya/package.py	requires = ["anim_tool==2.3.6"]

Likewise, with overly and underly permissive requirements:

	package.py	Requires
	anim_tool/package.py
	vanilla_maya/package.py	requires = ["anim_tool>0.5,<3.0"]
x	vfx_maya/package.py	requires = ["anim_tool>2.0,<3.5"]
y	alita_maya/package.py	requires = ["anim_tool>0.1,<2.5"]
z	alita_vector/package.py	requires = ["anim_tool==2.3"]

I expect there’s something I’m not yet seeing, but this is what I’m investigating at the moment.

If you ran “rez-env A B”, you would get (A-2, B-1) - because A was requested first

This goes over my head at the moment, but sounds like it might throw a wrench in the theory above; will investigate!

It’s interesting that newcomers to rez often think about using it in terms of defining a production runtime env, rather than a package’s.

Defining packages was quite straightforward I thought, so that might be one reason it doesn’t come up as often in questions and conversation. Projects and assets on the other hand doesn’t seem as common, and from my point of view - having written several launchers like the ones you describe in the past - having one less system to juggle is an appealing proposition; especially when the two systems solve almost identical problems.

For any given request, the resolve (the list of resulting packages, and the order in which their commands are sourced) is always deterministic

This has been my experience so far, which is really great!

[Alexandra Lefève-Gourmelon]

Hi All,

I agree with Marcus on this one, having already designed a system using only packages of diffferent types : either simple ones as the maya package or config ones as, for example, a custom maya_config package setting the desired values of the versions for maya and our tools' packages). It worked pretty well as Marcus described. The thing is, to use Allan's example, the maya package doesn't even know about the anim_tool package existence. It's the anim_tool package that knows which version(s) of maya to use. And the maya_config package knows which anim_tool version and which maya version to use. And then, you have a show_config package to know which version of maya_config to use.

Alexandra Lefève-Gourmelon

Lead R&D

www.superprod.net

107 Avenue Parmentier 75011 Paris France 

[Marcus Ottosson]

Thanks for sharing Alexandra!

[Jean-Christophe Morin]

I concur with what Allan said: "Rez was not designed to manage production environments; it resolves package dependencies, and then lets those resulting packages configure the target runtime env. I think it makes more sense to go the route that Dani describes (and is not dissimilar to what we do at Method, and other studios). Ie, use some other system to define the packages that you require for a given show ad tool; and then use rez to resolve the subsequent environment.". At least it's what we (RodeoFX) quickly noticed/discovered when we started to use Rez.

For me, using Rez to define your projects environments would be like if you would create RPMs that would configure an entire machine by specifying a ton of rpms as dependencies (I'm pushing the limit here of course, and it's extreme, but it's a similar thing). For this task, there are tools that deploy packages and configure machines, take care of installing the right dependencies and all. Again, it's maybe not the best comparison ever. I could do the same parallel with pip actually. Pip manages dependencies but doesn't manage environments. It can "configure" (to some extent, the runtime env, or kind of by giving you commands defined by packages). But environments are managed by virtualenv/venv/pipenv/poetry/anaconda, etc.

Jean-Christophe Morin
Software developer

Office:   418 523-9996
_____________________

      

[Marcus Ottosson]

Thanks Jean-Christophe!

At least it’s what we (RodeoFX) quickly noticed/discovered when we started to use Rez.

Would it be possible to go into more detail on what prompted you to leave Rez, and build another system instead?

One of the key motivations for me personally wanting to use Rez is because I’ve built the kind of system you mention, many times. And the reason for building it over and over is precisely why I’m looking into Rez to solve (pun!) this once and for all; because (a) it’s a hard problem and (b) the result is often proprietary or too specific; and yet they all solve the same problem. At Rodeo. At Method. You name it. One of the benefits of using Rez isn’t just that it solves a particular problem, but because there exists a community around it, and extra pairs of hands to help advance and enhance it. Just imagine being able to come to a community to ask for help about not just resolving software environments, but projects (and assets!) too. Think of the incentive for developers - in-house and open source alike - to build and enhance available tooling around such a system, both technically and graphically. And the kind of adoption such a think could have. With that kind of community, one could fathom make a good living off of building tools around Rez that enhance the lives of others.

I put together a small example of how one might use projects and software with Rez; in this repo, I’d like to try and capture the requirements of my current client, but also identify exactly the kind of problems lie with in, and whether (a) it’s a lost cause, (b) there’s a way or (c) Rez can be made to support it.

• https://github.com/mottosso/rez-for-projects

There are a few things missing from this already, which I’ve added to the bottom of the README. As of this writing, I’m looking into:

• Relative Requirements I’d like to associate Pyblish with Maya, but only when run alongside alita, not lotr.
• Overwriting Installs Every install overwrites a prior install, that’s bad. I’d like for an argument to be passed to rez build to explicitly allow overwriting.
• Rezutils Requirement I’d like for standalone packages, like Pyblish, to not depend on an internal rezutils package to be built.

Let me know your thoughts.

Best,
Marcus

[Fede Naum]

There are a few things missing from this already, which I’ve added to the bottom of the README. As of this writing, I’m looking into:

• Relative Requirements I’d like to associate Pyblish with Maya, but only when run alongside alita, not lotr.

You can achieve that (conditional requirements) with late bindings functions, see here https://github.com/nerdvegas/rez/wiki/Package-Definition-Guide#late-binding-functions

• Overwriting Installs Every install overwrites a prior install, that’s bad. I’d like for an argument to be passed to rez build to explicitly allow overwriting.

I think you are mixing here build and install (rez release)

You should use `rez build` while developing your packages. when you use `rez build -i`, it DOES override your local package ( you can  pass -c as well to clean the build directory as well)

Once you are happy you just `rez release` your packages to your main/production environment.

I' guessing you do not want to override a released package right? t can be done, but I think this is a bad idea

• Rezutils Requirement I’d like for standalone packages, like Pyblish, to not depend on an internal rezutils package to be built.

I had a look at your repo, we do not need that because we build everything with Cmake (even python packages) but either way, you should not add `rezutils` to your requirements, but to your private_build_requires 

Fede

[Marcus Ottosson]

You can achieve that (conditional requirements) with late bindings functions

Yes, that’s it!

@late()
def requires():
    global in_context
    requires = this.build_requirements[:]

    if in_context() and "maya" in request:
        requires += this.maya_requirements

    if in_context() and "nuke" in request:
        requires += this.nuke_requirements

    return requires

However I did find this note in the wiki a little disconcerting;

if this function … [then] no code evaluation has to happen at runtime then, so it’s cheaper

How and when could @late impact the system? Is there any way I can stress-test the overhead, to get a sense of whether the cost outweighs the gain? Spontaneously, I would have thought the cost of calling a Python function - even hundreds to thousands of times - would fade in comparison to the amount of IO happening elsewhere.

$ python test.py
Open/read file: 273.5/sec  # NFS mounted drive, on Windows 10
Calling function: 1,280,023.4/sec

• Reproducible

• I think you are mixing here build and install (rez release)

That is true, I never got to actually release anything yet, which would have solved this problem. Thanks.

you should not add rezutils to your requirements, but to your private_build_requires

Genius, this is great; although, the OCD isn’t too keen on mixing an internal package, like rezutils in with an otherwise vanilla external package, like pip. It would make this otherwise generic package into something specific to my system. Though I can’t think of another way, since I wouldn’t want pip to duplicate the rather involved build script.

I’ve updated the repo with these additions; the next missing bit is overrides. That is, if /projects/alita/rez is on the REZ_PACKAGE_PATH, then the contained maya/package.py should override the studio-wide Maya configuration for this project.

Is there any way of achieving this at the moment? Be it hacky or otherwise. :)

--

[Allan Johns]

Ok, so I'll keep on with this thread a little longer as I think I have some solid examples of stuff we do at Method that could further explain why we don't model projects with rez.

So first, Marcus you mentioned that a separate config system is a lot of extra work that isn't necessary if you just use rez instead. At Method that does not describe our situation. We already had a config system that we use for hundreds of other cases, and so it made sense to use it to define the package versions we want when running a tool in a given show/shot/location/other. For us it was a choice between rez and this existing config system, not a choice between rez and building a config system.

We use our config system to define "profiles", which are just a package request list. We tend to have one profile per tool, or per tool+department. For example, we might have an "rv" profile, or a "maya_anim" profile, or a "maya_fx" profile.

What typically happens is, we have a profile defined at the "facility" level, which is what any new show will default to. If a show wants to override anything (like a package version), it does so using our config system. Typically, this override is not a narrowing of the version range specified at the facility level (as your illustration suggested in previous email - but please correct me if I have misinterpreted). Instead, a new show will typically want to use a newer version of a package (or often DCC), which isn't necessarily stable yet.

There is also a common case where a show is nearing completion, and does not want to be exposed to any changes at the facility level. For example, the "maya_anim" profile might specify "anim_plugin-3.4" at facility level, and a show may not have overridden that. However, because the show is almost finished, they do not want to suddenly pick up "anim_plugin-3.5" when someone changes the facility-level version. We can manage this, because we have something called a "profile lock." We just drop in an extra config setting for the show which says "read facility-level profiles up until date X (ie ignore changes after that date), and continue merging with our own show-level overrides." If we didn't have this, we'd be forced to copy the entire profile from facility down into show, in an attempt to lock everything down. Even this wouldn't work however, since a new package could be added to facility level too.

Another important feature we have is "package locks." What a show often does not want, is the latest version of packages as soon as they are released; but you don't want to avoid this by overspecifying your profiles either (ie, listing every version of every package you want, down to the patch level). Overspecification creates a big maintenance burden, and a bottleneck, as it becomes someone's entire job to sift through hundreds of packages in an attempt to determine what the long and specific version list should be (I have seen this happen). So, we have a "package lock" instead. With this, we can say "ignore all packages released after date X." An example of how you use this - and how we do - is to lock off packages to say, 6am every Tuesday and Thursday. This can give a good compromise between stability and features/bugfixes.

There are other things we can do too, that simply don't have an analogy in rez packages (or would be comparatively complicated to implement). For example, I mentioned earlier that we might have a "maya_anim" and "maya_fx" profile. If we want both maya instances to be visible in an artist's console, then we need a way to differentiate. We can do this by specifying a "profile prefix". So, we can expose the maya for animators as "anim_maya", and maya for fx artists as "fx_maya".

As well as all of the above, we can override profile definitions per:

* show

* sequence

* shot

* location

* user (for local testing)

All the features outlined above do not lend well to analogies in rez. Imo, specifying what packages you want for a specific situation is a configuration issue; Determining what packages you need in order to satisfy such as request, and configuring the resulting env in a way that satisfies this resolved package list, is a rez issue.

Now, having said all of the above - if the requirements of your studio don't include the kind of more advanced features that I've outlined above, and if you don't have an existing config system to drive the sort of thing that I've described, then it could very well make sense to use rez to solve this problem instead. I certainly have no issue with a studio doing so! Here I am just going into further detail on the features we've ended up with, and I can say that they do give us a flexible and powerful way of controlling our production environments. If you see your own studios getting to the point where you need the same degree of control, you may start to find the pure rez approach too limiting. Having said that though, bear in mind that the config system that we have was in and of itself quite a lot of work, so there's that too (I do wish I could open source it!).

Hth

A

[Marcus Ottosson]

Thanks Allan, you raise many of good points; some of which clearly fall outside the scope of Rez. Others I can see handled with Rez, albeit in a round-about-way. And some I might not necessarily want a specific feature for.

1. Rez versus Existing Config System
2. Application and Department Profiles
3. Override package to newer-than-current-maximum
4. Time-based requires
5. Cascading overrides

We already had a config system

This is likely true for a lot of situations, but does it not require maintenance? I’d imagine that being more expensive than building one.

Typically, this override is not a narrowing of the version range specified at the facility level

This was a little surprising, I would have thought the facility would encapsulate all available versions of e.g. Maya, in which case any config using facility would have to be a narrowing.

requires = [
  "~maya>2015,<=2020.pr102"
]

Why would facility specify anything other than the latest available at a facility?

read facility-level profiles up until date X

This does sound like a solution to the problem you specify, but I wonder whether it isn’t too “magic”? Or at least overly convenient. That is, once locked at a particular date, how do you go about debugging an unexpected/erroneous package becoming part of your profile? Sounds like you would almost need an additional tool(chain) to visualise (and edit?) a profile at a given time; like a change timeline, or Git history.

If we didn’t have this, we’d be forced to copy the entire profile from facility down into show, in an attempt to lock everything down.

From the outside, it does read like this is both more explicit and a reasonable compromise. A locked facility is a locked facility, having a (static) copy of it reside alongside a project seems fair, and more importantly debuggable.

Even this wouldn’t work however, since a new package could be added to facility level too.

Wouldn’t it though? Once copied and added to the Rez packages path, wouldn’t it be picked up instead of the global facility package, in which case any changes to the original would go unnoticed? Locked is locked.

$ export REZ_PACKAGES_PATH=/packages  # global `facility`
$ export REZ_PACKAGES_PATH=/projects/alita/rez  # containing /facility/package.py
$ rez env ...

ignore all packages released after date X

Not having experienced the problem first-hand, I couldn’t say whether this is a good or bad solution to it. But it still does sound magical. What kind of problems have you experienced with this kind of “time-based requires”, and has it ever inspired a separate/better solution that you are unable to implement for whatever reason, e.g. (1) it’s good enough or (2) we’ve got too much riding on it already?

In practice, due to these red flags, I likely wouldn’t implement this in a pipeline solely based on the experience of others. I would need to experience the problems first-hand, despite..

Overspecification creates a big maintenance burden, and a bottleneck

..because on the other side of the coin, overengineering and creates a maintenance burden too. (Not to imply the system at Method is overengineered, as it sounds to have been built to address an existing problem, but rather that it would have been to use it for a studio not having experienced the problem)

lock off packages to say, 6am every Tuesday and Thursday

Though I will admit, there is something about this though that puts the artist in me at ease; having been through too many “but it worked yesterday!?” situations myself. :)

If we want both maya instances to be visible in an artist’s console, then we need a way to differentiate.

Do you mean instead of opening up a second console session?

Terminal 1

$ rez env alita maya_fx
$ # maya available, with an fx "profile"

Terminal 2

$ rez env alita maya_anim
$ # maya available, with an anim "profile"

I would have though that to be more clear and expected, than mixing what is effectively two independent environments. Maybe I’m not reading that right.

we can override profile definitions per: show, sequence …

This is what I’m looking into for Rez at the moment, will put something together about this in the coming days, but in short I’ve found a few ways to go about it using plain packages that I’ve found either good-enough or to require a feature addition to Rez itself.

One idea builds on this: https://github.com/nerdvegas/rez/issues/21#issuecomment-484924799

Imo, specifying what packages you want for a specific situation is a configuration issue; Determining what packages you need in order to satisfy such as request, and configuring the resulting env in a way that satisfies this resolved package list, is a rez issue.

You may be right, but at this point in time - despite the unique additions of your configuration system - I honestly can’t tell the difference. If you turned that sentence around, I wouldn’t have batted an eye.

Imo, specifying what packages you want for a specific situation is a rez issue; Determining what packages you need in order to satisfy such as request, and configuring the resulting env in a way that satisfies this resolved package list, is a configuration issue.

Now, having said all of the above - if the requirements of your studio don’t include the kind of more advanced features that I’ve outlined above, and if you don’t have an existing config system to drive the sort of thing that I’ve described, then it could very well make sense to use rez to solve this problem instead.

I’ve got a few handful of requirements left to implement, but indeed it’s looking like we would at the very least be able to start off using just Rez, and in time build another system on-top as problems arise better suited for a separate system.

the config system that we have was in and of itself quite a lot of work

It does sound quite involved.

Takeaway

1. Existing features of Rez cover current requirements, of this mid-sized studio of 120
2. But may need either a new system in the future, or features added to Rez
3. Time-based requirements seem possible, and within scope, for Rez; especially considering it already does timestamps (and has a concept of time)
4. Overrides seems like a missing core feature of Rez

For time-based requirements, I’d imagine something along the lines of:

name = "alita"
requires = [
  "facility-3:<2019-4-23",  # Locked to a specific max-date
  "core_pipeline-1:==tuesday,thursday"  # Picked up from only these days of the week
]

Although I wonder whether this isn’t an argument against semver, and for calendar versioning. https://news.ycombinator.com/item?id=19658969. Especially considering we (in VFX) rarely (ever?) follow the “major release when introducing breaking changes” anyway and are more likely to follow a “2 weeks == ancient history” mentality. :)

For overrides I’d imagine:

/shows/alita/rez/maya/package.py

name = "maya"
version = "1.0"
override = True
requires = [
  "fur_tools==1.6.3",
]

/packages/maya/package.py

name = "maya"
version = "2018.5.1"
requires = [
  # lots of requirements,
]

Where the “override” is discovered first, and added to whatever was found next.

[Fede Naum]

Hi Marcus, 

There is a lot of things on this thread, but I'll try to address some of your points.

First of all, at AL, we have a very similar setup described by Allan, (instead of calling profiles we have presets). So, we have a layer on top of rez that let specify a set of high-level packages and versions organized by facility/show/department. That same tool allows overrides at any level, which the main thing it does is to add a different version range of a package or pass a global timestamp. 

There is a nice feature in rez env (-t --timestamp) that let you lock the resolve to a particular date, giving you the peace of mind you mention before. So that freezes the whole resolve to a particular date, not for just a given package. And that is deterministic if you ask for the same set of packages locked to a particular date you get the same resolve. 

Remember also that you can lock a particular package using the exact version range, that is packageA==1.2.3 if you need.

So from your Takeaway points, I will say that the only thing you need (to start with) is a small layer on top of rez that spits something like:  

rez env maya-2018 alita-1.2+<2 foo-2 bar-3 blah-1.4 --timestamp 1555933692   (or calling the API equivalent)

Then you can release new versions of alita, foo, bar, blah, and they won't affect the solve you give to production. 

When you release new versions of those tools, then you can test without the --timestamp, and if you are happy with the solve, you then bake a new timestamp for your rez env line.

A common scenario that you will quickly find is that when you lock resolves to a specific date is that you for some reason you want the exact same resolve but you want to add the bug fix of packageB that was released after the locked timestamp, but at the same time you do not want to risk getting all other tools that where released in between that could change your whole resolve. In that case, rez also offers another feature that let's you do that, which is, lock all the packages to a specific data, but for this package let all version (or a constrained range) be part of the choices of the solver regardless the date lock.

I'll say that with those two features, you can go a long way. and I would not want to put the information of an override nor the time locked inside the package itself.

Hope that helps

Fede

 

Thanks for sharing Alexandra!

Alexandra Lefève-Gourmelon

Lead R&D

www.superprod.net

107 Avenue Parmentier 75011 Paris France 

--
You received this message because you are subscribed to the Google Groups "rez-config" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-c...@googlegroups.com.
To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rez-config" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rez-c...@googlegroups.com.
To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

--

...

[Marcus Ottosson]

Thanks Fede, this is invaluable information!

rez env maya-2018 alita-1.2+<2 foo-2 bar-3 blah-1.4 —timestamp 1555933692

That’s really neat. I think at this point, given the predominance of the two-systems approach to project configuration, the main gap in my understanding is this:

1. How do you configure that second system?

If I didn’t know any better, I’d think you had a configuration file of sorts, with a name = "my_preset" and requires = ["alita", "maya"] variable along with timestamp = 1555933692. Something you developed locally whilst testing, versioned independently and released without the risk of conflict. I’d think that you developed and maintained such a configuration file similarly to how you develop a Rez package. Maybe you even called on Rez to bring everything together and spit that string out.

$ rez env second_system-1.0 -- second_system -c "alita animation"

rez env maya-2018 alita-1.2+<2 foo-2 bar-3 blah-1.4 --timestamp 1555933692

What does your configuration file look like, and how do you develop it? Specifically, how does it differ from a Rez package?

To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.

[Fede Naum]

Hi Marcus, 

We do not use rez for this second system. We had a system that pre-dates our rez era. It is a java GUI with a database that stores key-value pairs of settings and has different types (strings, int, paths, and we extended to PACKAGES). packages type will take the form 

PACKAGENAME=VERSION RANGE

So a preset (or in Allans term a profile) will look like

maya=2018 (type package)

bla=1.5<2 (type package)

foo=  (type package)

timestamp=155512312345 (type string)

then this tool bakes those settings into a python script that calls the rez API (rez resolve context with the list of packages and the timestamp as arguments).

that gives you a resolved context (if it does not fail to resolve) and then you apply the context. From that point on you can call your os.system(MAYA_EXE) with whatever startup script as a parameter.

R&D releases tools (like 50 a day), TD's per production/department are in charge of changing the version ranges in this second system that makes sense for their production/department.

Our second system, since is backed with a DB,  allows as to rollback and give us the settings as they were x days ago.  ( we use that for debugging purposes)

In this second system, the TD's maintain (automatically with a schedule) three level of presets. Beta, preproduction, production. The only thing that differs on this preset level is the timestamp variable. Beta presets do not have any timestamp variable, they resolve most of the time to the latest and greatest released by R&D, and from time to time they may break (not resolve).

 At some point lets say every Monday, the preproduction preset get the today's timestamp. then TDs verify that everything is ok under those presets for some days, and a later day, let's say Wednesday that timestamp is transferred to the production preset. 

Artists/end users should only use production preset to start apps and send render to the farm. 

The process repeats every week, or two weeks, according to the stage of production. At some final stage in the movie usually, they do not modify production presets, but we still can add in this second system another key-value pair of a different type (PATCH) that allow what I described before, lock everything to the timestamp except for this particular package where it considers versions released after that date.

The farm uses those generated python scripts to set up the environment prior to any farm job (render or any other job), for production presets the rez request will 95-98% of the time hit the memcache and you'll get a resolve applied in ~3 seconds. That is a good price to pay for a stable/reliable environment.

Fede

You received this message because you are subscribed to a topic in the Google Groups "rez-config" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rez-config/ni5CK2pxj38/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rez-config+...@googlegroups.com.

[jcm...@rodeofx.com]

Hey Marcus

Would it be possible to go into more detail on what prompted you to leave Rez, and build another system instead?

At RodeoFX we in fact never went with Rez for managing the production environment. We considered it though. I won't write a too long answer as it would be repetitive with what Allan and Fede already said.

At the time we started to use Rez, we had no package manager already in place, and no configuration system in place. We considered multiple options and for us configuring the production environment with rez was dropped because it wouldn't allow us to link with our asset manager (Shotgun). Linking to our exists asset manager was really important for us. That allows us to easily see inside Shotgun what installed on what and query all packages for a given entity straight from the Shotgun entity without using glue between the packages list and Shotgun. So we built a configuration system using Shotgun. We created a custom entity that we use to store the list of packages that should be used to run a rez-env for a particular "level" (what we call a level is facility, show, seq, shot, asset, app). With that we have a little shell CLI that artists use to start their applications. They simply type the show on which they work and the level + app and it starts the app, or simply enters the rez-env without starting an the said app.

In the end, our system is similar to what Method and AL have right now. We can set packages at the facility for a particular app or for all apps, and same for all the levels. Each level inherits their parent. We also have the ability to freeze an entire show in time (though that's hardcore but useful. It bakes all packages at every single level on each entity on a show and level where there was a package set). The main difference with what Allan and Fede said is that we don't use timestamps. So that means a little bit more manipulation is required in order to get the bug fixes and new features out in production, but that's what the productions teams wanted. They prefer stability over anything else.

I hope it answers your question.

As for your rezutils thing, I'd be curious to know why you don't want o have a dependency to build you packages? That's a route I've taken at RodeoFX. We have a rdo_package_utils package that is used by all our packages to remove any boiler plate code from our package and have the build/install/release process be handled in a central place. We can also import it from any early binded functions of our packages. As Fede said, it's normally something you will want to add to your private_build_requires. I've personally set some quite strong rules our around utils packages. They are quite simple: 1) Only use it during a build process, 2) Never ever use it in situation that would make a package to interactively changes at runtime when resolving the packages, 3) Using it from a package.py should result in a backed result inside the released package.

Inside this package, we have a build module that we use to build out packages, that way we only have to call one function with parameters to build/install/release. We also have a module to get the path to our DCCs (that function is used inside the preprocess function of some packages). And we also have a module to manipulate our versions (handle the beta, non beta, etc, that is also used in preprocess and or in the build script). That package just made our life easier and let us control they way our packages are built and released in a really simple way.

Thanks for sharing Alexandra!

Alexandra Lefève-Gourmelon

Lead R&D

www.superprod.net

107 Avenue Parmentier 75011 Paris France 

--
You received this message because you are subscribed to the Google Groups "rez-config" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-c...@googlegroups.com.
To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rez-config" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rez-c...@googlegroups.com.
To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

--

Jean-Christophe Morin
Software developer

Office:   418 523-9996
_____________________

      

--
You received this message because you are subscribed to the Google Groups "rez-config" group.

[Marcus Ottosson]

So we built a configuration system using Shotgun

Thanks JC, this was a real Aha-moment and makes a lot of sense! Using Shotgun - or any GUI for that matter - for project configuration means you’re able to manage high-level requirements for high-level aspects of a show, like the show itself and assets, without resorting to command-line and Git. Especially when the crew is already familiar and involved with the system, like they would be for Shotgun/ftrack.

[Marcus Ottosson]

It is a java GUI with a database that stores key-value pairs of settings

Thanks for sharing Fede, the advantages to this sound similar to what JC mentioned about using a production tracker which makes sense.

These were my takeaway from your setup (primarily meant for future readers, including myself).

1. Rez came second - Which meant an integration of Rez into an existing system, rather than a system built on Rez
2. Many moving pieces - The release cycle is long with many steps; as it should, as the alternative is time consumed by figuring out and cleaning up mistakes
3. It’s not for us - In some cases, even the best system isn’t something you should necessarily replicate, and I think this is a good example. Scale look like a major factor in some of the decisions made in this design, and what I’m working with at the moment is 100-150, <20 tds, <5 developers.

Just wanted to add, the contributions to this thread so far has been amazing, thanks to everyone!

[Marcus Ottosson]

I’d be curious to know why you don’t want o have a dependency to build you packages?

Sorry, forgot to answer this one!

After some thought and some more experience going this route, I think you’re right; it hasn’t been an issue so far and is actually quite intuitive.

[Jean-Christophe Morin]

Thanks JC, this was a real Aha-moment and makes a lot of sense! Using Shotgun - or any GUI for that matter - for project configuration means you’re able to manage high-level requirements for high-level aspects of a show, like the show itself and assets, without resorting to command-line and Git.

To be clear, we don't use the UI per se to manage the packages. We use our launcher to do that. And there is a really specific reason why we don't use the Shotgun UI to manage the packages: When we "install" a new package or a new version, we need to make sure there won't be any conflict. So our command line does a rez-env with the requested changes (with --no-local) to ensure everything will still resolve once "installed". 

So we use Shotgun only for its DB and to be able to link against the production assets (show, seq, shot, etc). This was the simplest method for us at the time we built that system. We might change it in the future to use its own database to make the operations faster and more Shotgun agnostic.

On Tue, Apr 23, 2019 at 2:50 AM Marcus Ottosson <konstr...@gmail.com> wrote:

So we built a configuration system using Shotgun

Thanks JC, this was a real Aha-moment and makes a lot of sense! Using Shotgun - or any GUI for that matter - for project configuration means you’re able to manage high-level requirements for high-level aspects of a show, like the show itself and assets, without resorting to command-line and Git. Especially when the crew is already familiar and involved with the system, like they would be for Shotgun/ftrack.

--

You received this message because you are subscribed to the Google Groups "rez-config" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.

To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

[Marcus Ottosson]

Ah, gotcha.

At least then you'll benefit from having a single "definition" of a project and asset in Shotgun, rather having one as a separate Rez package on disk, or additional database. Other than that though, not sure I see much benefit to leveraging Shotgun for ensuring package uniqueness; aren't you already getting that guarantee from git tags?

[Jean-Christophe Morin]

We are not using Shotgun for that. We use Shotgun as a configuration, not as a package manager ;) So we can configure a set of packages for any given entity. But the packages themselves are not registered in Shotgun as entities. They only thing we store is a list of packages as strings. So it's really just a configuration.

On Tue, Apr 23, 2019 at 10:01 AM Marcus Ottosson <konstr...@gmail.com> wrote:

Ah, gotcha.

At least then you'll benefit from having a single "definition" of a project and asset in Shotgun, rather having one as a separate Rez package on disk, or additional database. Other than that though, not sure I see much benefit to leveraging Shotgun for ensuring package uniqueness; aren't you already getting that guarantee from git tags?

--

You received this message because you are subscribed to the Google Groups "rez-config" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.
To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.