Optional package in request

[Dani Asztalos]

Hi,

Can I make a package request optional? I want to include a package if it is resolvable, but I don't mind skipping it.

E.g:

I have these houdini versions

houdini-15.5.532
houdini-15.5.623
houdini-16.0.312

Then I have a package testpackage, with only houdini-16.0 variant.

When running rez-env testpackage houdini-16 it should resolve both of them, but I would expect rez-env testpackage houdini-15 to skip the testpackage.

Something like Weak references, but this does not seem to do the trick. Of course I could make an empty houdini-15 variant for the testpackage, but I hope there is a better solution.

Thanks,

Daniel

[Francois Lord]

Yeah I also had trouble with weak references for that situation.

You can use the @late() decorator on the requires() function of your package.

@late()
def requires():
    packages = ['package_to_load']
    if in_context() and 'houdini' in request and request['houdini'] == "houdini-16.0.312":
        packages.append('package_to_load_only_for_houdini_16')
    return packages

But know that this method will make the commands rez-depends and rez-search behave differently for that package. I'm still hesitating to use this in production yet for that reason.

--
You received this message because you are subscribed to the Google Groups "rez-config" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.
To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

[Allan Johns]

So, what you're talking about is essentially late-binding requirements, as Francois has pointed out.

There are 2 issues with this:

1. What Francois points out - what is the 'correct' package definition, if the package isn't within a resolve?

2. This is the big one - it breaks the resolve caching.

Resolve caching in rez is a big deal. Resolves are expensive, but with a cache running, and in a typical production scenario, you'll typically see ~98% hit rate on the cache (and when you hit the cache, the resolve is almost free). But the cache doesn't know what your late-binding requirements might be doing, so if you did something weird (like change requirements based on an env-var, or the time of day, or current moon phase) then you're gonna trick the cache, and get back stale results.

However, if your late-binding requirements are only affected by other info from the request (as in this example), then you avoid that problem. So Francois' suggestion will work fine, IF you ensure that you aren't using any external factors to influence your requirements list.

Hth

A

To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+unsubscribe@googlegroups.com.

To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rez-config" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+unsubscribe@googlegroups.com.

[Christian]

hi, I'm using this workaround for a similar usecase ("only include an optional package dependency if the package exist"):

package.py

import os
import platform

name = "foo"
version = "1.0.0"
requires = ["bar"]
optional_requires = ["moo"]
build_requires = []
variants = []
uuid = "7d685c22175b41e4a7471b0f4484eceb"

# cheap imp for 'optional_requires': only include
# opt pkgs if they exist (a bit whacky as it runs
# another 'rez-env' as shell cmd)
silencer = r"> nul 2>&1" if platform.system().lower() == "windows" else r"> /dev/null 2>&1"
for optional_require in optional_requires:
    try:
        p = os.popen("rez-env -q %s -c \"exit 0\" %s" % (optional_require, silencer))
        ret = p.close()
    except:
        ret = 1
    if ret == None:
        requires.append(optional_require)

def commands():
    pass

Best,

Chris

[Marcus Ottosson]

Thanks for sharing Christian; that is hacky to say the least haha!

--

You received this message because you are subscribed to the Google Groups "rez-config" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.

[Allan Johns]

When running rez-env testpackage houdini-16 it should resolve both of them, but I would expect rez-env testpackage houdini-15 to skip the testpackage

That is not at all what I would ever expect rez to do. Rez gives you the packages you asked for, and you asked for `testpackage`. The result should either be a valid resolved env, or failure.

If what you want is "require this package [version range] only if it exists", then I could see that this might be useful. I would manage this by introducing a new syntax (maybe `~~foo`?). But bear in mind that this could introduce a lot of extra complexity (you have consider the changes to the solver, and ambiguous cases that may arise). It isn't clear to me if this feature is worth the potential development cost.

A

--

[Blazej Floch]

Why not ~foo? Isn't this currently a noop?

I am not in favor of having this functionality but I saw TDs in the past writing ~foo and in expectations it would do exactly this: require a package if it is available. To my best knowledge it does nothing.

Cheers,

Blazej

On Saturday, 25 May 2019 21:29:02 UTC-4, allan.johns wrote:

When running rez-env testpackage houdini-16 it should resolve both of them, but I would expect rez-env testpackage houdini-15 to skip the testpackage

That is not at all what I would ever expect rez to do. Rez gives you the packages you asked for, and you asked for `testpackage`. The result should either be a valid resolved env, or failure.

If what you want is "require this package [version range] only if it exists", then I could see that this might be useful. I would manage this by introducing a new syntax (maybe `~~foo`?). But bear in mind that this could introduce a lot of extra complexity (you have consider the changes to the solver, and ambiguous cases that may arise). It isn't clear to me if this feature is worth the potential development cost.

A

On Thu, Apr 19, 2018 at 8:03 PM Dani Asztalos <asztal...@gmail.com> wrote:

Hi,

Can I make a package request optional? I want to include a package if it is resolvable, but I don't mind skipping it.

E.g:

I have these houdini versions

houdini-15.5.532
houdini-15.5.623
houdini-16.0.312

Then I have a package testpackage, with only houdini-16.0 variant.

When running rez-env testpackage houdini-16 it should resolve both of them, but I would expect rez-env testpackage houdini-15 to skip the testpackage.

Something like Weak references, but this does not seem to do the trick. Of course I could make an empty houdini-15 variant for the testpackage, but I hope there is a better solution.

Thanks,

Daniel

--
You received this message because you are subscribed to the Google Groups "rez-config" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-c...@googlegroups.com.

[Thorsten Kaufmann]

Weak references are different, they are not optional. They only force a package to be within a specified range if it is part of the request, not if it exists.

So rez-env ~package-1.0 foo will not add package to the request. But if foo adds package it will be forced to 1.0.

I do agree that optional requests sounds like a useful thing to have. But i can't really comment on the dev work needed to implement and also agree that it is not worth crazy amounts of dev work.

Cheers,

Thorsten

[Blazej Floch]

I totally get what weak refs do when they specify a version. But what do they do if there is no version:

`~package`?

I am merely saying that I saw people doing this and expecting to do exactly what is being discussed here.

If this indeed currently is a noop then it certainly does seem a better syntax then `~~package`, no?

If not I am curious what it does.

[Fede Naum]

2 years later answer... 

a weak ref without a version `~package` makes no sense, and does nothing