[Pipeline Organisation] Binary Plugins

[Timm Wagener]

Hey guys,

this is my first topic started on the mailing list and i use it to ask around about your habits concerning binary plugins (.mll) in your pipeline?

I recently felt like they are a huge pain to replace / version up since you cannot just delete and replace them as you could do with a python package as long as anybody in production has his Maya open and therefor the program has a handle to it.

A few solutions that quickly come to my mind:

1. Control an env. var. that points to the binary plugin dir. You can swap the directory in there for production and wait a few days (weeks...) till everybody closed his current Maya session.......Pretty unacceptable!

2. Force the handle to be removed with another program. (Unlocker etc.). That doesnt seem like a clean solution too me. What implications will that have to currently open Maya sessions?

3. Announce "Update Times" for the production where people know they should close their Maya session for updates to be deployed and if they dont their session might crash because handles are removed forcibly.......Hmm, not too elegant either..

Maybe i am missing something very evident here? How do you handle the replacement/update of your binary plugin suite? (Not just for Maya but also Houdini, Nuke etc.)

Cheers,

Timm

--

Timm Wagener
Technical Artist / 3D Generalist

www.timmwagener.com

Email:
wagen...@gmail.com

Phone:
0178-2810920

[Marcus Ottosson]

You can swap the directory in there for production and wait a few days (weeks…) till everybody closed his current Maya session…….Pretty unacceptable!

I was in an environment once in which they restarted all computers every day. As an artist, I couldn’t understand it and felt it was a huge pain, but after a few weeks it was quite soothing, starting with a fresh slate each day. And now I realise the benefit from a developer/maintainers point of view too.

​

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CA%2Bj65ips9_s_w4-q%2BHPZ57RfBkajbgcYuq0FC5%3DGFv6e_Qzyxg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Timm Wagener]

I was in an environment once in which they restarted all computers every day. As an artist, I couldn’t understand it and felt it was a huge pain, but after a few weeks it was quite soothing, starting with a fresh slate each day. And now I realise the benefit from a developer/maintainers point of view too.

That seems like a good way of dealing with the problem at first, but at a second glance its not that convenient too.

What if the deployment you rolled out turns out to be incompatible in some way and users slowly find out.

By the time the error is pinpointed already a lot of machines are running again.

Also doing this on a daily basis seems pretty inconvenient for a render farm, that probably shares

the same plugins as production.

Maybe it would be the best if pipeline TDs could enforce a global restart with a 10min warning period for artists to react!?

Apart from that:

I know the current CG Content Creation Tools (Maya, Houdini, Max etc.) are really old. I've seen videos of Tools (Unreal Editor)

where they do on-the-fly replacement of compiled code while the engine is running. I'm lacking the details of how this is done,

but could this also be an option for a plugin system of a 3D app.? (Not Maya probably but modern apps currently being designed...)

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOBXb8BK5C2iBBAeXW9hnM95nuw6Xe5tEwNk1b%2BFnZWJAg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[Paul Molodowitch]

As a caveat, we don't use windows much at work, and I think Linux is more forgiving, in that once a maya plugin is loaded, you can delete the underlying binary (.so, in Linux-land) without much issue (or maybe it's our NFS-based filesystem which allows this)?

Anyway, while we haven't had this exact issue, the solution we use for this general class of problems is a combination of versioned release dirs, and package-based environment-variable management to control them.  The idea is that every time you release a new version of the plugin, it goes into an upticked version release dir:

.../maya_plugins/windows-7/maya-2015/TheAwesominator/1.3.4.1038

Using your package management system, you then set the global default version for "TheAwesominator" to be 1.3.4.1038.*  It is the responsibility of the package management system to ensure that the correct directories is added onto the appropriate environment variables - for instance, if you're using "rez" as your package manager, if you do:

rez-env

..then you might configure your "TheAwesominator" plugin to execute these commands (apologies if my memory of windows shell syntax is wrong):

    

set MAYA_PLUGIN_PATH=%MAYA_PLUGIN_PATH%;%MAYA_PLUGINS_ROOT%/windows-%WIN_VER%/maya-%MAYA_VER%/TheAwesominator/%AWESOMINATOR_VER%/bin

set MAYA_SCRIPT_PATH=%MAYA_SCRIPT_PATH%;%MAYA_PLUGINS_ROOT%/windows-%WIN_VER%/maya-%MAYA_VER%/TheAwesominator/%AWESOMINATOR_VER%/scripts

Then you just make sure that every time someone launches maya, they're actually running a wrapper which uses your package management tool to setup the environment, then launch maya.  You can do this by controlling both the gui shortcuts that are distributed to artists, and the startup scripts for artists' shells.

The upshot is that, at any given time, different artists across the studio might be using different versions of the same plugin - ie, at one time, all these versions might be in use across the studio:

TheAwesominator/1.2.1.0016

TheAwesominator/1.3.4.1038

TheAwesominator/1.3.4.1041

TheAwesominator/2.0.1.0486

It does mean that you'll need to keep around a bunch of old versions of your plugins... but since the amount of disk space most plugins take up is small in comparison to, say, your average exr render, this hopefully won't be much an issue.

The main problem I can see for you is that I don't know of a good package management solution available for windows.  At Luma, we use a proprietary system that has rudimentary windows support.  I would normally recommend rez (https://github.com/nerdvegas/rez); but while I know it's 2.0 branch has an architecture which supports different shells, including windows terminals, the actual implementation for windows isn't finished yet.  I'm not sure how far along it is - you can contact Allan Johns and find out, or give it a shot yourself.  If anyone else has a package management solution they'd recommend for windows, I'd be curious to hear it...

Best of luck,

- Paul

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOBXb8BK5C2iBBAeXW9hnM95nuw6Xe5tEwNk1b%2BFnZWJAg%40mail.gmail.com.

[Marcus Ottosson]

Also doing this on a daily basis seems pretty inconvenient for a render farm, that probably shares the same plugins as production

Ah yes, not sure this relates to render farms. What size of studio are we talking about here? If it’s about 20 ppl, all running Windows then things are likely a little different then if we’re talking 500 ppl running Linux.

In the case of 500 workstations, I think each machine on a render farm is typically capable of running a variety of versions on the fly, via environment configuration based on which job is coming in.

For example, if a job is sent out to render a scene with the following requirements.

"app": "maya-2013x64",
"render": "arnold-1.0",
"plugins": ["somePlugin-10.1.12", "someOtherPlugin-1.0.0"]

Then the machine could set environment variables (i.e. paths to where these versions reside) for these to get picked up during the boot cycle of Maya prior to initiating a render.

Ultimately, it depends on how flexible you need to be. If you render all projects using the same version and software, then life will be a little easier.

Maybe it would be the best if pipeline TDs could enforce a global restart with a 10min warning period for artists to react!?

This was the case for us (at Framestore), typically after-hours and if you worked late you could cancel the restart, which would try again a few hours later.

I’ve seen videos of Tools (Unreal Editor) where they do on-the-fly replacement of compiled code while the engine is running.

I think the key is to keep your dependencies modular so that you can configure your environment “on the fly” - i.e. by configuring the target render machine prior to launching. It’s conceptually similar to setting the PYTHONPATH, regardless of which tool or OS you use but if you do have a look at Rez you’ll get a sense of how complex it can get.

​

[David Moulder]

Our system is just a python module and has a startup method much like maya's initialise().  It handles copying plugins  and python / mel files from the repository using robocopy.  The repository has configs that are like manifests files that describe the folder, it' contents, what version of maya it should run on etc.  Depending on this manifest we add unpack these as python modules, plugins etc on the users local machines.  We've just moved this logic to Perforce and it's was painless, sped up the users boot time.  We adde a tray app and icon in windows to tell the user updates are available.  They run the Perforce update from the tray icon.  Then reboot maya.  The perforce stuff is still robocopied to a single local location.  So plugins are not locked.

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOD3KgCXbcDp%3DX%2B98MUtwyGqfk9M4CtxW83Ti1VXWYg31Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

David Moulder Technical Animator / Artist squi...@gmail.com Professional Profile	Mobile: +44 (0)7814033242
See who we know in common	Want a signature like this?

[Justin Israel]

Our solution at my studio more our less matches what Paul described. We have a 1000+ workstations and a custom environment management system. Our plugins are deployed as versioned packages and the environment describes which ones to load up. We would never delete old plugins, because if you went to rerun old shots from an archived show, they would need to run under their original environment again.
The problem with people having stale shells and Maya sessions is quite common, but we don't do anything so invasive as forcing reboots. It is up to the individual to watch for pertanent release notes or have a good habit of reloading every so often.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CABD4PkTsBXhRqCc1EryaguzPi8FPgSkm2mjTRDC%3DcjvDfQygTg%40mail.gmail.com.

[Jesse Kretschmer]

As Paul and Justin have pointed out, version specific release directories are a huge help. I would like to note though that transparency is important with renders. My studio uses rush but we have a custom render package that contains all the relevant environment information. The plugin versions are readily apparent. I leave the policing to the TD on the job. 

Most of the time, our projects will lock to a specific version early on and won't ever change unless there is a huge bug or fix that comes through. For this reason, during the rare transitions the TDs can be very attentive. Outliers are found and corrected quite quickly.

Your mileage may vary.

Good luck; Have fun.

[Timm Wagener]

Hi guys,

thanks for your insights, great explanation Paul.

What size of studio are we talking about here? If it’s about 20 ppl, all running Windows then things are likely a little different then if we’re talking 500 ppl running Linux.

The pipeline i'm concerned with right now is layed out for approx. 20 ppl. running on Windows. Not yet a studio pipeline, its for our diploma movie, but nevertheless trying to run this show as organized as possible :D

As a caveat, we don't use windows much at work, and I think Linux is more forgiving, in that once a maya plugin is loaded, you can delete the underlying binary (.so, in Linux-land) without much issue (or maybe it's our NFS-based filesystem which allows this)?

So far Houdini (our Fx & Lighting/Shading Tool) also runs under Linux within our pipeline. Will test  if i can delete .so files while an instance of Houdini is running next time i fire it up under CentOS.

Anyway, while we haven't had this exact issue, the solution we use for this general class of problems is a combination of versioned release dirs, and package-based environment-variable management to control them.  The idea is that every time you release a new version of the plugin, it goes into an upticked version release dir:

That seems like the best solution. I will have to look into package management solutions and set it up like this next time.

Our system is just a python module and has a startup method much like maya's initialise().  It handles copying plugins  and python / mel files from the repository using robocopy.

Before i reorganized our pipeline with wrapper tools to point to a central location on the server for plugins etc., i was actually doing the same thing with a startup script that would copy/setup the local environment.

I'm gravitating more towards the env. var. solution with a wrapper that points to a central location though, feels a little more elegant to me.

Cheers and in case i'm not writing till tomorrow:

Happy new year everybody! ;)

Timm

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CANESWi0f-RdLQrgBBooZgLvPxY5TqdszqrrmMapFzWYPjJ0JTg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[Todd Widup]

myself and my manager have been writing a setup of powershell tools as our previous team supported various projects from various studios here at MS.  as a result, we were always running into situations where studio X would be using Maya 2012, and studio Y would use 2014, but both would have completely different configurations and scripts and plugins to use for their game engines as well as animation needs.

so we setup a few folders that contain their data, and then our powershell tools simply map the folders (on the network) to the correct drive letter that that studio used and setup all the appropriate env var before launching Maya. 

This was about the simpliest way we could deal with it here. 

as for userSetup files, I set ours up so that we all share a common one per project that sources all the correct tools in addition to all our custom tools we have developed for our team. 

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CA%2Bj65irGHpX%3DDqLyS4zv_TW0wK6RQPCmQiDaEHm8%2BKvE5HbkKg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Todd Widup
Creature TD / Technical Artist
to...@toddwidup.com
todd....@gmail.com
www.toddwidup.com

[Todd Widup]

I should preface for version control most of our studios use TFS or perforce, so we just run with that. 

Might look at something for handling plugin versions viaa perforce and maya a bit more in the new year, some way to tage and associate a certain perforce version of the plugin with the scene.

[damon shelton]

I have used cvs, perforce, and git to manage syncing all the tools in the different pipelines I have worked in.

you could use sitecustomize.py to handle updating plugins for all your programs running python interpreters. I have used it before for maya, motionbuilder, nuke, houdini and a few other programs. What I have done before is copy all the plugins on startup into a temp folder and add the temp folder to the plugin path instead of the main branch. Do this for each session of the program you run, therefore updating plugins doesn't get blocked. The programs run the latest version when launched. you can do this in many ways, powershell tools, cygwin environments, etc. the simplest of these to implement was sitecustomize.py(the py file living in your %PYTHONPATH% directory). you can check the sys.executable to figure out the program, versions, etc. and you can set up all your environment paths in the same file. This will still only setup this information for the running instance leaving you the ability to launch different programs under different environments with the proper scripts/plugins. This breaks down when your program doesn't run python but there are workarounds. A lot of them include executing python scripts through cygwin, tcsh etc to do the same thing.

-cheers

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CABBPk35TCTcnkc-mNauh5LYrznM3nEdoGh4ZWSMpc%2BjrGvJiKw%40mail.gmail.com.