Any successful run of Shotgun hook rez_app_launch in Win10?

[Daniel Wong]

Hi all, 

I'm trying to use Bleeding-Rez, a fork of nerdvegas/rez, to setup environment variables when launching software in Shotgun, but with no succeed. Seeing that the Shotgun hook in that repo, rez_app_launch, is identical to that in nerdvegas repo, and it hasn't been updated for long, I'm not sure if anyone could use it in Win10 successfully.

I use the given hook to replace the default app_launch hook in tk_multi_launchapp. I get the follow issues when running the hook:

1. The following error is prompted in stdout when running

Found Rez: %REZ_REZ_ROOT%
Adding Rez to system path...
...
ImportError: No module named rez.resolved_context

Cause: It seems that for L134 in the hook, the ECHO command failed to output the value of EnvVar REZ_REZ_ROOT:

rez_cmd = 'rez-env rez -- echo %REZ_REZ_ROOT%'

I found that using SET command can output the value, but need some string operations to remove the EnvVar name and the "=" sign.

2. After fixing #1, another issue comes when launching. The following dialog is prompted when executing the shell command:

Cause: Seems there's problem manipulating the command line argument when called by execute_shell() in L101-L107 in the hook:

            proc = context.execute_shell(
                command=cmd,
                parent_environ=n_env,
                shell=shell_type,
                stdin=False,
                block=False
            )

I've no idea why for this part.

Could someone please help if you manage to launch it successfully in Win10? Thanks!

[Joseph Yu]

Hey,

This might not directly answer your questions but recently the Shotgun hooks have started to be migrated out and updated.

Check out https://github.com/nerdvegas/rez-shotgun/tree/from-rez-pr-823

I can't test on Windows at the moment but do you know if your Shotgun configurations look more like tk-config-default2 or tk-config-default (legacy)?

[Vitor Hugo]

Hi. I'm using the exact same setup as you.

For the first problem I just hardcoded the path to the rez executable. I don't see the problem with this since it's a standardized installation anyway.

I see the code in the repository, but I really don't know what this "App" argument is for. Googling didn't help there, it doesn't seem to be any default cmd command. I suspect this line was took from a old thread on this forum about someone trying to launch XSI, which also had this "App" argument. Fixing that is easy, just take off the "App".

However, that's not the actual problem. The real problem is that execute_shell doesn't seem to correctly spawn a subshell on Windows. If you were to delete the "app" part, you will notice that even then your package doesn't work. None of the package variables are set.

The work around I found was to change block to True. In that case the environment is correctly configured. Of course, this is less than ideal because it's now blocking execution, which could be solved by spawning a new shell by either calling execute_shell with the start_new_session parameter or by using start to spawn a new shell on the Windows side. Unfortunately strangely even doing this the Shotgun python interpreter is still blocked.

For us this setup is fine because the only hook that the execution really blocks is the `before_app_launch` hook and we didn't use that hook too much.

[Daniel Wong]

Hey Joseph, 

Thanks for your reply!

My config is cloned from legacy config. I've tested the URL you've given me. The 1st issue doesn't occur, but the 2nd one has similar result (The Error dialog in my first post).

In order to get more verbose, I try to launch the software in command prompt, and it prints similar error message: 

Joseph Yu於 2020年4月17日星期五 UTC+8下午7時34分39秒寫道：

[Daniel Wong]

Hi Vitor, 

Could you please tell me your software environment? 

- Which OS?

- Which Python version?

- Which Rez repo?

I've tested your suggestion with Bleeding-Rez, under Win10 and Python3, but it returns similar error (Windows cannot find "\"C:\Program", seems it splits the arguments followed after wrongly).

And I'd like to have a try with Nerdvegas Rez-2.54.0 also, but I've had a hard time to run it, and till now I'm not manage to run it successfully. 

Cheers, 

Daniel

Vitor Hugo於 2020年4月18日星期六 UTC+8下午9時25分26秒寫道：

[Vitor Hugo]

Sure. We're using Windows 10, Python 2.7.16, bleeding-rez 2.40.3.

Here's the execute and background_shell_type_methods that worked for me:

def execute(self, app_path, app_args, version, **kwargs):
    """Start the required application using rez if required.

    Notes:
        - Define variables used to bootstrap tank from overwrite on
          first reference
        - Define others within ``tk-multi-launchapp.yml`` file in the
          ``extra:rez:parent_variables`` list.

    Args:
        app_path (str):
            The path of the application executable
        app_args (str):
            Any arguments the application may require
        version (str):
            version of the application being run if set in the "versions"
            settings of the Launcher instance, otherwise ``None``

    Returns:
        dict[str]:
            Execute results mapped to 'command' (str) and
            'return_code' (int).
    """
    multi_launchapp = self.parent
    rez_info = multi_launchapp.get_setting("extra", {}).get("rez", {})
    cmd, shell_type = self.background_cmd_shell_type(app_path, app_args)

    # Execute App in a Rez context
    rez_py_path = self.get_rez_path()
    if rez_py_path:
        if rez_py_path not in sys.path:
            self.logger.debug('Appending to sys.path: "%s"', rez_py_path)
            sys.path.append(rez_py_path)

        # Only import after rez_py_path is inside sys.path
        from rez.resolved_context import ResolvedContext
        from rez.config import config
        rez_parent_variables = rez_info.get("parent_variables", [])
        rez_packages = rez_info.get("packages", [])
        self.logger.debug("rez parent variables: %s", rez_parent_variables)
        self.logger.debug("rez packages: %s", rez_packages)

        config.parent_variables = rez_parent_variables
        context = ResolvedContext(rez_packages)
        self.build_venv_environ()
        current_env = os.environ.copy()

        env_kwargs = {'suffix': '-prev-env.py', 'delete': False}
        with NamedTemporaryFile(mode='w+', **env_kwargs) as env_file:
            env_file.write(pformat(current_env))
            self.logger.debug(
                'Copied existing env for rez. See: "%s"',
                env_file.name
            )

        with TemporaryFile(mode='w+') as info_buffer:
            context.print_info(buf=info_buffer)
            info_buffer.seek(0)
            self.logger.info(
                "Executing in rez context [%s]: %s\n%s",
                shell_type,
                cmd,
                info_buffer.read(),
            )

        # Originally this was non-blocking. But it seems rez on Windows
        # doesn't create the subshell correctly, so if we don't block
        # we don't get the subshell configured
        launcher_process = context.execute_shell(
            command=cmd.strip(),
            parent_environ=current_env,
            shell=shell_type,
            stdin=False,
            block=True
        )
        exit_code = launcher_process[0]
    else:
        # run the command to launch the app
        exit_code = subprocess.check_call(cmd)

    return {"command": cmd, "return_code": exit_code}

def background_cmd_shell_type(self, app_path, app_args):
    """Make command string and shell type name for current environment.

    Args:
        app_path (str): The path of the application executable.
        app_args (str): Any arguments the application may require.

    Returns:
        str, str: Command to run and (rez) shell type to run in.
    """
    system = sys.platform
    shell_type = 'bash'

    if system.startswith("linux"):
        # on linux, we just run the executable directly
        cmd_template = "{path} {flattened_args} &"

    elif self.parent.get_setting("engine") in ["tk-flame", "tk-flare"]:
        # flame and flare works in a different way from other DCCs
        # on both linux and mac, they run unix-style command line
        # and on the mac the more standardized "open" command cannot
        # be utilized.
        cmd_template = "start /B {path} {flattened_args} &"

    elif system == "darwin":
        # on the mac, the executable paths are normally pointing
        # to the application bundle and not to the binary file
        # embedded in the bundle, meaning that we should use the
        # built-in mac open command to execute it
        cmd_template = 'open -n "{path}"'
        if app_args:
            app_args = app_args.replace('"', r'\"')
            cmd_template += ' --args "{flattened_args}"'

    elif system == "win32":
        # on windows, we run the start command in order to avoid
        # any command shells popping up as part of the application launch.

        # Usually I would use Powershell instead of cmd
        # but Shotgun really doesn't like it
        # this shouldn't cause any problems
        shell_type = 'cmd'

        # cmd_template = 'start /B "App" "{path}" {flattened_args}'
        # This start /B thing is here as a workaround for the blocking
        # problem on Windows. Unfortunately, in Shotgun, even with this
        # it still blocks. So it's irrelevant.
        cmd_template = '{path} {flattened_args}'

    else:
        error = (
            'No cmd (formatting) and shell_type implemented for "%s":\n'
            '- app_path:"%s"\n'
            '- app_args:"%s"'
        )
        raise NotImplementedError(error % (system, app_path, app_args))

    cmd = cmd_template.format(path=app_path, flattened_args=app_args)
    return cmd, shell_type

Sorry if the formatting breaks, it looks fine for me.

Vitor Hugo Pipeline Engineer São Paulo | Brasil wildlifestudios.com

--
You received this message because you are subscribed to the Google Groups "rez-config" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rez-config/8c9b80b6-107a-4735-bcb7-4978d4a3f82a%40googlegroups.com.

[Joseph Yu]

I have a sneaky suspicion even Shotgun was wrong when they included "App" in their official, example tk-multi-launchapp app_launch.py from reading the official start command documentations

For the environment variables, try using the extra.rez.parent_variables list to see if you can get them work.

I just merged my PR so you guys can fork, test and submit fixes to the nerdvegas/rez-shotgun repository

To unsubscribe from this group and stop receiving emails from it, send an email to rez-c...@googlegroups.com.

[Daniel Wong]

Hi Vitor, 

Thanks for your code Vitor! I've tested it, and it gets similar result as mine.

However, I've tried to hardcode an application path without any space, and remove all double-quotes in the command, both yours and mine work. It seems that the API can't handle the double-quotes correctly.

Do your application paths have space? In windows, "Program Files" have space. 

Cheers, 

Daniel

Vitor Hugo於 2020年4月20日星期一 UTC+8下午9時42分39秒寫道：

To unsubscribe from this group and stop receiving emails from it, send an email to rez-c...@googlegroups.com.

[Daniel Wong]

Hey Joseph, 

The double-quoted "App" is actually the <title> argument. Here is the help doc printed in cmd:

C:\Users\danielwong>start /?
Starts a separate window to run a specified program or command.


START ["title"] [/D path] [/I] [/MIN] [/MAX] [/SEPARATE | /SHARED]
      [/LOW | /NORMAL | /HIGH | /REALTIME | /ABOVENORMAL | /BELOWNORMAL]
      [/NODE <NUMA node>] [/AFFINITY <hex affinity mask>] [/WAIT] [/B]
      [command/program] [parameters]


    "title"     Title to display in window title bar.
    path        Starting directory.
    B           Start application without creating a new window. The
                application has ^C handling ignored. Unless the application
                enables ^C processing, ^Break is the only way to interrupt
                the application.
    I           The new environment will be the original environment passed
                to the cmd.exe and not the current environment.
    MIN         Start window minimized.
    MAX         Start window maximized.
    SEPARATE    Start 16-bit Windows program in separate memory space.
    SHARED      Start 16-bit Windows program in shared memory space.
    LOW         Start application in the IDLE priority class.
    NORMAL      Start application in the NORMAL priority class.
    HIGH        Start application in the HIGH priority class.
    REALTIME    Start application in the REALTIME priority class.
    ABOVENORMAL Start application in the ABOVENORMAL priority class.
    BELOWNORMAL Start application in the BELOWNORMAL priority class.
    NODE        Specifies the preferred Non-Uniform Memory Architecture (NUMA)
                node as a decimal integer.
    AFFINITY    Specifies the processor affinity mask as a hexadecimal number.
                The process is restricted to running on these processors.


                The affinity mask is interpreted differently when /AFFINITY and
                /NODE are combined.  Specify the affinity mask as if the NUMA
                node's processor mask is right shifted to begin at bit zero.
                The process is restricted to running on those processors in
                common between the specified affinity mask and the NUMA node.
                If no processors are in common, the process is restricted to
                running on the specified NUMA node.
    WAIT        Start application and wait for it to terminate.
    command/program
                If it is an internal cmd command or a batch file then
                the command processor is run with the /K switch to cmd.exe.
                This means that the window will remain after the command
                has been run.


                If it is not an internal cmd command or batch file then
                it is a program and will run as either a windowed application
                or a console application.


    parameters  These are the parameters passed to the command/program.


NOTE: The SEPARATE and SHARED options are not supported on 64-bit platforms.


Specifying /NODE allows processes to be created in a way that leverages memory
locality on NUMA systems.  For example, two processes that communicate with
each other heavily through shared memory can be created to share the same
preferred NUMA node in order to minimize memory latencies.  They allocate
memory from the same NUMA node when possible, and they are free to run on
processors outside the specified node.


    start /NODE 1 application1.exe
    start /NODE 1 application2.exe


These two processes can be further constrained to run on specific processors
within the same NUMA node.  In the following example, application1 runs on the
low-order two processors of the node, while application2 runs on the next two
processors of the node.  This example assumes the specified node has at least
four logical processors.  Note that the node number can be changed to any valid
node number for that computer without having to change the affinity mask.


    start /NODE 1 /AFFINITY 0x3 application1.exe
    start /NODE 1 /AFFINITY 0xc application2.exe


If Command Extensions are enabled, external command invocation
through the command line or the START command changes as follows:


non-executable files may be invoked through their file association just
    by typing the name of the file as a command.  (e.g.  WORD.DOC would
    launch the application associated with the .DOC file extension).
    See the ASSOC and FTYPE commands for how to create these
    associations from within a command script.


When executing an application that is a 32-bit GUI application, CMD.EXE
    does not wait for the application to terminate before returning to
    the command prompt.  This new behavior does NOT occur if executing
    within a command script.


When executing a command line whose first token is the string "CMD "
    without an extension or path qualifier, then "CMD" is replaced with
    the value of the COMSPEC variable.  This prevents picking up CMD.EXE
    from the current directory.


When executing a command line whose first token does NOT contain an
    extension, then CMD.EXE uses the value of the PATHEXT
    environment variable to determine which extensions to look for
    and in what order.  The default value for the PATHEXT variable
    is:


        .COM;.EXE;.BAT;.CMD


    Notice the syntax is the same as the PATH variable, with
    semicolons separating the different elements.


When searching for an executable, if there is no match on any extension,
then looks to see if the name matches a directory name.  If it does, the
START command launches the Explorer on that path.  If done from the
command line, it is the equivalent to doing a CD /D to that path.

C:\Users\danielwong>

The "App" has got no problem when running in a vanilla Shotgun config.

And I've set extra.rez.parent_variables in my shotgun .yml:

  extra:
    rez:
      parent_variables:
        - PYTHONPATH
        - HOUDINI_PATH
        - NUKE_PATH
        - HIERO_PLUGIN_PATH
        - KATANA_RESOURCES
      packages:
        - maya
        - mtoa-4.0.2.1
        - htoa
        - maya_utils

Cheers, 

Daniel

Joseph Yu於 2020年4月20日星期一 UTC+8下午10時10分13秒寫道：

[Vitor Hugo]

Hello, Daniel.

 

You’re correct. Opening a vanilla cmd and executing:

 

start “New title” cmd

 

Indeed works. Unfortunately, for me, with Shotgun it does not. It’s a bit weird to put this command here since you’re executing with the /B flag anyway.

 

But answering your previous question, yes, in my solution I can execute paths with spaces, no problem.

 

I remember testing the parent variables solution and it didn’t work. I’ll give it another try.

 

Thanks.

Hi Vitor, 

ImportError: No module named rez.resolved_context

 

Cause: It seems that for L134 in the hook, the ECHO command failed to output the value of EnvVar REZ_REZ_ROOT:

 

rez_cmd = 'rez-env rez -- echo %REZ_REZ_ROOT%'

I found that using SET command can output the value, but need some string operations to remove the EnvVar name and the "=" sign.

 

 

2. After fixing #1, another issue comes when launching. The following dialog is prompted when executing the shell command:

 

 

Cause: Seems there's problem manipulating the command line argument when called by execute_shell() in L101-L107 in the hook:

 

            proc = context.execute_shell(
                command=cmd,
                parent_environ=n_env,
                shell=shell_type,
                stdin=False,
                block=False
            )

 

I've no idea why for this part.

 

Could someone please help if you manage to launch it successfully in Win10? Thanks!

--
You received this message because you are subscribed to the Google Groups "rez-config" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rez-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rez-config/8c9b80b6-107a-4735-bcb7-4978d4a3f82a%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "rez-config" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rez-config/223c337e-bd87-4e2a-9928-a15849d4822a%40googlegroups.com.

 

[Joseph Yu]

Oh, oops. My bad. That's why I need you guys, I won't know and can't test these things otherwise

[Joseph Yu]

Ah, I think the parent_variable might only be for tk-config-default2 which will need to be back ported to the old legacy config's hook

Currently the legacy one's parent variables are hard coded here

To view this discussion on the web visit https://groups.google.com/d/msgid/rez-config/223c337e-bd87-4e2a-9928-a15849d4822a%40googlegroups.com.

 

[Daniel Wong]

Hi Vitor, 

I think the "App" placed here is to make sure the following double-quoted app_path will not be mistakenly parsed as <title>. If there's no space in the app_path and double-quote for the app_path is not a must, then the "App" will not be necessary.

Thanks for telling me about your setup. I'll try a bit more. Thanks!

Cheers, 

Daniel

Vitor Hugo於 2020年4月21日星期二 UTC+8下午10時24分29秒寫道：

To view this discussion on the web visit https://groups.google.com/d/msgid/rez-config/223c337e-bd87-4e2a-9928-a15849d4822a%40googlegroups.com.

 

[Daniel Wong]

Hi Joseph, 

I'm using the hook in tk-config-default2 and I've added the required parent variables in the .yml. Seems that part of code runs fine for me.

Cheers, 

Daniel

Joseph Yu於 2020年4月21日星期二 UTC+8下午10時38分26秒寫道：