Flushing PyQT classes from Maya's Memory

[md]

Is there a way to flushing all prior QT instances from memory when developing PySide/Qt scripts for Maya?

I am constantly exiting Maya and reloading it when my forms don't seem to refresh properly.

Thanks 

M

[Justin Israel]

There isn't any formal kind of global "flush" for Qt. If you are leaking memory then you would need to fix that. Since Maya is a persistent environment for all of the tools you run, tools have to take care to clean themselves up.
With Qt, even more so.  Because if you create widgets with parent to the main window, they don't get deleted unless you do it. An example is a dialog window. When you make a dialog and parent it to Maya's main window, and then close the dialog, you are most likely leaking it if you are just creating a new one from scratch each time.

For any widget you have widget.deleteLater()  which will delete it once control gets back to the event loop.

If you are not keeping a global ref to your dialog to reshow each time, and your dialog is being parented to Maya, then you can use QtCore.Qt.WA_DeleteOnClose and set it with dialog.setAttribute()
This will tell the dialog to delete when you close it. All widgets in the parent child chain will delete.
http://qt-project.org/doc/qt-4.8/qt.html#WidgetAttribute-enum

This may not be exactly the problem you are having so you might need to explain the unwanted behavior a bit more

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/ff0065a9-e32e-4cac-b163-d9103c9a1f0f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tony Barbieri]

We've also made use of weak references.  Cyclical references can keep Qt, and especially PySide, from correctly garbage collecting.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA0trNa%3Djj8ujws8M0XJA4usAO_Wccq9PaHCn2%2BGmKCWhw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Tony

[md]

Thanks Justin ...

Prior to reading this , I thought it was something like this ... I have the following in my Class __init__

self.setAttribute(QtCore.Qt.WA_DeleteOnClose)

On Tuesday, July 15, 2014 4:00:15 PM UTC-4, Justin Israel wrote:

There isn't any formal kind of global "flush" for Qt. If you are leaking memory then you would need to fix that. Since Maya is a persistent environment for all of the tools you run, tools have to take care to clean themselves up.
With Qt, even more so.  Because if you create widgets with parent to the main window, they don't get deleted unless you do it. An example is a dialog window. When you make a dialog and parent it to Maya's main window, and then close the dialog, you are most likely leaking it if you are just creating a new one from scratch each time.

For any widget you have widget.deleteLater()  which will delete it once control gets back to the event loop.

If you are not keeping a global ref to your dialog to reshow each time, and your dialog is being parented to Maya, then you can use QtCore.Qt.WA_DeleteOnClose and set it with dialog.setAttribute()
This will tell the dialog to delete when you close it. All widgets in the parent child chain will delete.
http://qt-project.org/doc/qt-4.8/qt.html#WidgetAttribute-enum

This may not be exactly the problem you are having so you might need to explain the unwanted behavior a bit more

On 16/07/2014 4:22 AM, "md" <acco...@mdonovan.com> wrote:

Is there a way to flushing all prior QT instances from memory when developing PySide/Qt scripts for Maya?

I am constantly exiting Maya and reloading it when my forms don't seem to refresh properly.

Thanks 

M

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

[Marcus Ottosson]

Have you guys tested memory usage on widgets sticking around after getting closed? I can't imagine it ever becoming a problem, even in long-running sessions of Maya (e.g a whole day). The only issue I've had with the persistence of variables in Maya is in debugging and making sure you're actually using the latest of your modules as most of them would have been reused rather than reloaded; and even if you reload all of them, you'll have to do it in the order they are implicitly loaded which isn't always possible if you're having cyclical imports.

So, in general, I'd say there isn't any recommended way to clear Qt classes or any classes for that matter from Maya's Python, or Python overall. The recommended approach for Python in general is to either restart the interpreter, which in the case of Maya is to restart Maya, or to handle each deletion yourself, which again, I think is of infinitesimal importance.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/9e20ea8d-4852-489f-a050-84c684f48654%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Mike Donovan]

the thing I have noticed is that when I updated my UIs and covert them to .py modules for import ... I ALWAYS 
need to restart Maya. Even if I have closed all my QT_Dialogs/Widgets

Sent from my iPhone

You received this message because you are subscribed to a topic in the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python_inside_maya/sNqxa2yDt5s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOD1L7yH2E12zioO2iax0pU_MK9eecTBvVRA%2BCMBWJBezw%40mail.gmail.com.

[Marcus Ottosson]

Yes, but this isn't unique to Maya. It's similar to having a Python interpreter running, importing your module and then expecting the interpreter to know it's been updated. Modules affect global state and can have a great effect on the behavior of interpreter. Since the interpreter in Maya is running for as long as Maya is, the only safe option (besides keeping track of changes and flushing by hand) is to reload Maya. This is different in Softimage where the interpreter is running as a separate process to Softimage (at least that's my impression) and is thus restarted at each execution of scripts.

The overall workflow for me, and others I think, is to develop separately, and test in mayapy if you can. Testing in Maya can be a last resort and shouldn't need as many iterations as can be tested externally.

I do wish though that Maya did what Softimage does. I really don't see the benefit of doing it this way, the interpreter should really be a separate process.

Off the top of my head though, you might have better luck running your scripts through subprocess if you really do need more iterations within Maya. It might act similar to Softimage, but I haven't tested much and can't say for certain. For one, you'd be dealing with threading issues as you can't (or shouldn't) call upon maya.cmds from threads other than the main one. It's something you can work around, but it wouldn't be as transparent as with Softimage.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/315E4F7F-F0DD-4130-B336-5DC3C35E7B1C%40mdonovan.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Justin Israel]

@Mike - It sounds like the issue was that you might have only been reloading the main script, as opposed to "deep reloading". First all of the dependencies have to be reloaded, and then the dependants have to be reloaded. Also I think it also makes a difference how you are importing things. It might require that you are importing the module, as opposed to doing a "from" import to bring in the class. Because once you have the class reference, I don't think reloading will work. 

@Marcus 

First off I will say that I agree with the mess that can come from trying to reload() tools in a production situation, as opposed to doing it for debugging purposes. 

My view on the matter is that leaking memory is leaking memory. 

Will it every become a problem in a long-running Maya session? Well that depends on what kind of Qt applications/tools are being written. I doubt you would see an impact in any reasonable amount of time if you were leaking a simple dialog that has some buttons that do calls to Maya and whatnot. But that is one particular application. If an application consumes noticeable resources as part of its function, then leaking those could have a big impact on how long you can run that Maya session, rerunning that tool, before you encounter issues and have to restart it. So my opinion is that it should be considered an impacting situation. If you can figure out ways to avoid leaking, even with the small harmless stuff, you would end up thinking the same way when you are doing something that will have an impact.

The parent-child relationship in Qt means that if you loose a reference to some parent QObject/QWidget, without deleting it, then none of the child objects will get cleaned up either. It is not just the top level object that you lose. Pythons garbage collection might clean up the python objects, but Qt isn't doing anything to clean up yet.

I threw together a super-duper contrived example of how this could potentially be a problem. Lets say you have some app that loads in and displays tons of images for review (or video if you prefer). You may have a ton of QLabel objects that have a pixmap set on each one, or  basically any possible situation where you are loading up data into QWidgets/QObjects. Then you show it as a dialog to your user. They use it for a bit, and then close it, but the dialog doesn't get deleted. So that dialog still continues to live, along with all of the QLabel's with the QPixmap that each one owns:

https://gist.github.com/justinfx/811902fc2aadce31f631

This particular example is just a mock up to make it easier to show. It creates a temporary 10MB file to pretend it is an image, and then reads it 10 times into a buffer (or QPixmap being then set into a QLabel). If you run that dialog and click the button, you should notice your RSS increase by about 100MB each time. Not only is the memory increasing, but it is leaking file descriptors as well ( lsof -p <pid> | wc -l ). When you close the dialog, the resources are not freed. And then you can rerun that dialog and do it some more. At some point you would either run out of memory, or have too many open file descriptors, and have to restart Maya. 

My point is that I think it should be considered a potential issue, since anyone is capable of writing anything and we all have to use the tools within the same environment. Everyone is at the mercy of everyone else :-/

-- justin

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOCtBngEbSTsxdFL9cto74hYq7BCTv9n1d73r7eW3eoGtA%40mail.gmail.com.

[Erdinç Yılmaz]

(If i understood correctly)

Let's say we have a myQtApp script or class, 

and also, maybe subclasses which we import in myQtApp.

After we make some changes in our code,

using this piece of code and then reimporting works.

You don't need to exit from maya. 

But you need to use it for all your modules or files which you make changes!

import sys

def flushModule(name):

    if sys.modules.has_key(name):

        del(sys.modules[name])

     

flushModule("myQtApp")

flushModule("someClassYouImportedToMyQtApp")

flushModule("sub.mySubLineEdit")

flushModule("sub.myTreeModel")

# and then reimport myQtApp in maya.

import myQtApp

[Fredrik Averpil]

I never got the sys.modules deleting approach to work (tried it about a year ago in python 2.6). As far as I understand you’re not supposed to delete imported modules. That’s not how Python is designed to work.

Instead I just always (yes, always) import modules with a reload() as well:

import myModule
reload(myModule)

Then I make sure to always delete the UIs (and dockControls in Maya) properly before loading up the UI. This makes it possible to reload the whole pyqt/pyside app without issues and without having to restart Maya/Nuke. Example of this in Maya/Nuke here.

Regards,
Fredrik

​

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/0f26a930-db5a-4656-87dd-fc0de62df0e3%40googlegroups.com.

[Cesar Saez]

I've been manually removing modules from python cache during years without issues, I use it so often that I made a GUI helper to ease the process :)
http://cesarsaez.me/2014/08/hard-reloader.html

[Justin Israel]

But this approach only deletes the direct module you refer to. Any other existing modules that might have imported the library will still have a live reference to the old one. To really do a "deep reload" you have to recurse down all of the dependencies that had been imported, and reload anything else that might have loaded your original target module. On top of that, compiled extensions can't be reloaded.

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPamJi-oj10Ld5ojC%2BQYQD6AVpwF0HE_C-eL%2Bay9h%3DE%3DpDM5BQ%40mail.gmail.com.

[Cesar Saez]

Hi Justin,
I assumed we were talking about reloading during the development stage where you will most likely code your library as a package (there's no need to reload all dependencies) and have access to the source code as you're the one developing it, under those conditions removing modules from python's cache works way better than any kind of import hook (we managed the cache through a hook at my previous job and it was a nightmare!).

Cheers!

[Justin Israel]

Actually in development I have had the same need. If I am working on a tool and it imports a few other modules in the same tool, and I make a change to those modules, I have to kill the changed dependencies and the top level tool to get the changes reflected.  If you are testing a tool that consists of one file, and you only make changes to one file, then you wouldn't need to worry about killing multiple imports.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPamJi9AG-PJAF9ntTwKP5AER1M_jvpPDL19jS8pYM_vMWuUzg%40mail.gmail.com.

[Justin Israel]

Sorry I wasn't clear before. I am only talking about development. I wouldn't use any deep reloading tricks in a production situation.

[Cesar Saez]

I don't know if you actually saw the code I linked before, but it 'reloads' the entire package (all submodules included).

import sys

def hard_reload(package_name):
    for k in sys.modules.keys():
        if k.startswith(package_name):
            del sys.modules[k]

Anyway... it was just a quick suggestion, it's really not a big deal :)

Cheers!

[Justin Israel]

Ah missed that part. All good.  Lots of options for dev.

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPamJi_zUpPC0c9VAR9%3DzsbNjk%3D%2BZ8FwZqJUVNoKwsjm-xgBVA%40mail.gmail.com.