Speed Up

[Count Zer0]

Gang,

I seem to recall a recent post somewhere about addressing eval time as
part of a future release. Any hints on how you're planning on speeding
up PyMEL?

I'm giving a talk at GDC on Python, PyMEL and IDE benefits and I'd
like to placate any worries about PyMEL's speed issues.

Thanks,

-jason

[Paul Molodowitch]

Well, one large reason for slowdown is the need to constantly convert between strings and python-api mobjects; we're currently having discussions with autodesk about providing flags (or entering a mode, perhaps) where api objects (mobjects, MSelectionLists) are returned from / accepted by maya.cmds calls, instead of string names, and they're open to the idea.  If any of you folks feel this would be a really desired feature - for your own use, or for pymel speed reasons - please, speak up and let autodesk know.

In the meantime, we're looking at providing alternate 'fast' versions of certain core / common methods... ie, we might provide a 'fastLs' and 'fastListRelatives' - or alternatively, a 'fast' flag to these functions.  The fast versions would be stripped down versions of these commands that would use api methods under the hood to try and replicate some of the functionality of the mel commands, and avoid extra un-needed mobject-string conversions.  For instance, 'ls' is essentially taking a string, using an MSelectionList to get a list of mobjects, then casting all those to strings, and then pymel comes along and re-casts them all to mobjects.  So, instead, we implement our own version of 'ls' which uses the MSelectionList directly, and goes straight to the mobjects.

It would be possible to try and write a complete re-implementations of ls that would then completely replace pymel.core.ls, without the need for a separate mode / function; but, given that this is a very important and widely-used function, we'd want to try match the behavior of the normal ls as closely as possible.  But, given that it's also a fairly complicated function, with a lot of different flags, edge cases, etc, and that we don't have access to the exact source code for how ls is implemented under the hood, we'd probably never get it EXACTLY right... and besides, it would be way more work than I'd want to put into it what is essentially just a stop-gap solution.

We also hope to be able to do some miscellaneous tightening of screws to help provide speed boosts throughout our code.

Hope that helps answer some of your questions,

- Paul

--
You received this message because you are subscribed to the Google Groups "pymel-dev" group.
To post to this group, send email to pyme...@googlegroups.com.
To unsubscribe from this group, send email to pymel-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pymel-dev?hl=en.

[Chad Dombrova]

i know it isn't exactly the answer you were looking for, but the unfortunate reality is that we can't do a whole lot without Autodesk's help. if you want real maya objects and not just strings there is a time penalty for conversion, and that conversion cannot be delayed until some later time (i.e. when the object is actually used), because by that time it's name could have changed and it will be invalid. so, it has to happen immediately for every node. functions that return lots of nodes have to do lots of conversions.

these are our best options:

1) educate users on efficient practices
2) add additional features to avoid wasted conversions, where possible
3) profile and look for inefficiencies
4) lobby autodesk for changes

for #2, one simple thing we could do is to create generator versions of common listing functions. since these only return results one-at-a-time, as they are requested, if you break out of a loop early, the remaining results are not cast to PyNodes. the problem is, how common is this really?

-chad

[Count Zer0]

That is a good list of options. I think it is acceptable, especially
if you can follow-through with #1 and let people know where they're
going to have issues and possibly a couple of tricks to work around
them. (Guess 'ls' is one to look out for, and any medium to heavy
vertex iteration, I've noticed).

I was hoping for a wink and a nudge on progress w/ #4. Guess we'll get
the latest when we meet w/ Autodesk pre-GDC.

Can I list the points in my talk? I'm hoping to just have something on
a slide so I can acknowledge the issue, state my "rarely an issue in
practice" line and let the public know you have a plan and we'll keep
working on it.

Thanks fellas,

-jason

[Chad Dombrova]

>
> Can I list the points in my talk? I'm hoping to just have something on
> a slide so I can acknowledge the issue, state my "rarely an issue in
> practice" line and let the public know you have a plan and we'll keep
> working on it.

yes of course. essentially, if you'd like to convey this message, it all comes down to api to string conversions (and vice versa). every PyNode starts its life by acquiring an API object (for the reasons mentioned in the previous email). if it is instantiated from an API object then that is very fast -- many people don't realize you can instantiate a PyNode from an MObject, MDagPath, or MPlug -- however, if it is instantiated from a string, it's quite a bit slower.

the problem is compounded when we take this PyNode, which has already done the work of getting an API object, and we pass it to an API-ignorant function list listConnections. the PyNode must now use its API object to convert *back* to a string to pass to listConnections. the results from listConnections are also strings, which must be converted *back* to PyNodes.

if listConnections were API-aware, then POOF, no more conversions. API objects go in, API objects come back out. as an experiment, today i wrote a simple replacement for listConnections using the maya python API. in my test, it doubled the speed. not too shabby. the problem, as paul pointed out, is that these functions are exceedingly complex. we might be able to pull of listConnections, but cmds.ls is just too massive.

my success with listConnections today makes me hopeful that we could offer an environment variable to switch to these API-only alternatives, for the brave at heart. of course we will have tests in place for these functions, but i would still be very worried about replacing something as fundamental as listConnections without months of field testing. after all, tests are often a collection of things that went wrong, which are there to ensure that they don't go wrong again.

-chad

[Seth Gibson]

I'd be 100% on-board for testing/assisting with implenetation for something like that.  Keep me in the loop, please!

[Alexander Morano]

Same here.

I absolutely will go bang on ADs door right now with a please consider this more email.

Cheers.