fatal error in Maya when iterate opening files

[Benjam901]

Hello all,

I am having some trouble with iterating over files.

I am opening Maya and passing in a script to it that iterates over a large list of Maya files and performs some operations on the nodes inside and then moves onto the next.

I get to a certain file and Maya throws a fatal error crash. I have some debug lines that allow me to see which file has been attempted to open. If I open this file normally all is fine and similarly if I open it using the script editor on its own it is fine.

I have checked the system resources and the only thing being maxed out when I open the file is the CPU, the mem and disk mem is all fine.

When I load the file by itself the script editor complains about Mayatomr being needed for the scene would this impact the iteration of opening files in such a way to fatal crash Maya?

I am using openFile:

pm.openFile(mayaFilePath, force=True, loadReferenceDepth='none')

I have tried this out also but to no avail:

pm.flushUndo()
pm.clearCache(all=True, c=True)
pm.openFile(mayaFilePath, force=True, loadReferenceDepth='none')

Cheers,

Ben

[Marcus Ottosson]

Iterating through many files within a single instance of Maya is much too volatile I think. A safer thing to do would be run your script in a new process per file via subprocess. It will add a few seconds to each file-open, but I can’t see another way without walking on glass.

When doing this however, you may still encounter this issue. It’s happens too when Maya has been initialised and Python is about to quit. You can work around it by instead of quitting normally, such as letting the script finish, you quit by hand via sys.exit() or force-quit with os._exit(0). The latter command basically shuts down the process without giving it any chance of performing clean-up and is the safest way I’ve found to get a reasonable return-code. You’d think there are other problems attached to this, but so far I’ve been using it for continuous integration which is running this way tens of times a day in multiple versions of Maya and still haven’t encountered an issue.

# Example
import os
import sys
import inspect
import tempfile
import subprocess

# This code runs in each file
my_script = """
import maya.cmds
import maya.standalone
maya.standalone.initialize()
maya.cmds.file( {fname} )

# some code..

os._exit(0)

"""

# Iterate over each file
for fname in ("/some/file1.ma", "/some/file2.ma"):

  # Create a script per file
  with tempfile.NamedTemporaryFile(suffix=".py") as pyfile:
    pyfile.write(my_script.format(fname=fname)
    pyfile.close()

    # Run script in file
    CREATE_NO_WINDOW = 0x08000000
    popen = subprocess.Popen(["maya", pyfile.name],
                             stdout=subprocess.PIPE,
                             stderr=subprocess.STDOUT,
                             creationflags=CREATE_NO_WINDOW)

    popen.communicate()  # Block till finished

    assert popen.returncode == 0, "An error occurred"

​

[Ben Hearn]

Hey Marcus,

Thanks for the help. Looks like an interesting solution. I will be implementing and testing tomorrow.

I forgot to mention however that I need the Maya GUI open during this process. If I use this process and don't use the CREATE_NO_WINDOW flag will the process fail or will it still run as intended?

Cheers,

Ben

--
You received this message because you are subscribed to a topic in the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python_inside_maya/rOnS_4vY5sE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOAF0p3TnUp%3DCyR_vYeAa0pOd2tACRNnzBA7NTsNKq5%3D7A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Tel - +46 76245 92 90 (Sweden)

LinkedIn: http://www.linkedin.com/pub/ben-hearn/50/a64/33b

[Marcus Ottosson]

Ah, yes this won't work if you need the GUI to remain open between opening files. But the fact that you need this is what I think is part of the problem in the first place.

You could run the above an launch a GUI per file, if you really really need it (for some things it's difficult to avoid, for example if the script you need to run depend on the Qt event loop), it would just add more seconds to each invocation.

As an optimisation, if it does take too long, you could run multiple Maya's simultaneously in this way as well; something not possible with a single instance. Depending on the amount of memory you have, and requirements of the files you are opening, you could potentially run 10+ instances simultaneously and go ~10x faster under ideal circumstances. 

Basically what I'm saying is, this approach is more difficult, but scales better, is more safe and potentially (most likely) faster.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAM2ybkUBUxMo%3D6qh3EX7GFmhsqa9uwmoMa-KA93sHACmmPW2_w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Ben Hearn]

I had suspected that the opening of the Maya gui was causing an issue also but there is no way around it. I am using certain shaders and the Hypershade functions which need the maya GUI window open.

"You could run the above an launch a GUI per file"

Do you mean on the instance of each file being opened in batch force the GUI to boot?

Hmm running multiple Maya's could be a solution that I could consider

-- Ben

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOCm%2BrWytxFtEe_9gwpz9_ciko2axwa_MvamtgcGAYG9Lw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Marcus Ottosson]

Hmm running multiple Maya’s could be a solution that I could consider

I’d start with one, and once you have that up and running, it will be clear how to launch 2, and then 3 and so on..

Do you mean on the instance of each file being opened in batch force the GUI to boot?

I noticed a typo in the previous call to subprocess.Popen, it should have been mayapy, not maya, which would launch the Python interpreter within which you initialize Maya and run your script.

In the case of launching the GUI however, you could use maya and pass it a Maya file. It’s possible you could pass in a startup-script this way as well, but otherwise it’s safe to generate a userSetup.py and append it’s parent directory to the PYTHONPATH when running Popen. It would cause the Maya GUI to open, and then run userSetup.py. Basically the same as above, but without the need to maya.standalone.initialize().

You could generate the userSetup.py in the same way as above, and launch maya in the same way as well.

I’m typing the code in the browser btw to don’t rely on syntax too much, just hoping to get the general idea across!

​

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAM2ybkWKnjQkcoAJjo6%2BbudORF2fYA_cx7U63oQcTCTaKjHLjg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Geordie Martinez]

Marcus is right. subprocess is the best way. me likey.
However. you’re using pymel so before you begin be sure and set an environment variable to turn off initializing plugins. Chad Vernon sussed out this crash a week or two ago on this group.

so set this variable if you’re on widows in your environment vars:
set PYMEL_SKIP_MEL_INIT = 1;

or on linux:

setenv PYMEL_SKIP_MEL_INIT 1

test it before hand to make sure it’ll work in subprocess. you don't want to hunt down two different issues. 
in a shell type:
mayapy

you should get a python prompt then test this to make sure you don't get a traceback. then you're ready to convert your code below to a subprocess call. 

import maya.standalone
maya.standalone.initialize()
import os
import pymel.core as pm

# you should see this next line if you set the environment variable correctly.
#pymel.internal.startup : INFO : Skipping MEL initialization
cube = pm.polyCube()

## your code goes here 

pm.saveAs("someFile.ma")
# result: Path('someFile.ma')
os._exit(0)

​

[Ben Hearn]

Hey guys,

I have tried the solution to launch multiple Maya's. At first I split my iteration dictionary into 2 and ran the 2 Mayas in parralell which produced the same fatal error. So I split it into 4 and made ach Maya wait for the other to finish but I am still getting the same error on the same file in any case regardless of how many Maya processes I open up. 

As I stated earlier the file can be loaded normally or by script on it's own. Is there any way I can test files for fatal errors or any suggestions in debugging these particularly troublesome files? 

d = UTILS.linch_dict_divider(mayaFiles, 4)

mayaExe = os.environ['MAYA_EXE']

mayaRunScript = os.environ['BATCH_CONVERT']

mayaSplitDict1 = d[0]

mayaSplitDict2 = d[1]

mayaSplitDict3 = d[2]

mayaSplitDict4 = d[3]

p1 = subprocess.Popen([mayaExe, '-script', mayaRunScript,str(mayaSplitDict1)],

                           stdout=subprocess.PIPE,

                           stderr=subprocess.STDOUT)

p1.communicate()

p2 = subprocess.Popen([mayaExe, '-script', mayaRunScript, str(mayaSplitDict2)],

                           stdout=subprocess.PIPE,

                           stderr=subprocess.STDOUT)

p2.communicate()

p3 = subprocess.Popen([mayaExe, '-script', mayaRunScript,str(mayaSplitDict3)],

                           stdout=subprocess.PIPE,

                           stderr=subprocess.STDOUT)

#subprocess.Popen.communicate(subprocess.Popen)

p3.communicate()

p4 = subprocess.Popen([mayaExe, '-script', mayaRunScript, str(mayaSplitDict4)],

                           stdout=subprocess.PIPE,

                           stderr=subprocess.STDOUT)

p4.communicate()

--

You received this message because you are subscribed to a topic in the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python_inside_maya/rOnS_4vY5sE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CABPXW4jTXuTzP3VMbqLq3yXB9RgP%2B_9W2B10%2B3Xk9wt2Uw5%3DPA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Ben Hearn]

UPDATE:

I dug around inside the troublesome files and saw that a couple of them were applying metadata with defunct file paths so I looked inside the hierarchy and found some file nodes swimming around in there. I deleted the nodes and hey presto. The process goes off without a hitch.

Some of the file nodes were pointing to PSDs that do not exist, jpegs or TGAs. Why would this cause Maya to crash though? Can file nodes become corrupt or would Maya choke on defunct pathing under heavy iteration? Some of the nodes were perhaps under File and were referencing PSDs which could be an issue or vice versa.

Thoughts?

[Marcus Ottosson]

Hey Ben,

If Maya is crashing due to something in the scene, the things I’ve suggested is void. The issue I was tackling was whether Maya crashed inbetween runs, which can sometimes cause a segmentation fault.

If you are still getting a crash, even when launching via subprocess, then it can’t have anything to do with the clean-up that normally occurs when closing one file and opening another, because each subprocess will be its own unique process that couldn’t possibly affect the following or prior processes.

As far as the code you posted goes, I’m not sure what’s going on.

1. What is d?
2. What is in the MAYA_EXE variable?
3. What is in the BATCH_CONVERT variable?
4. Why are you keeping code from us? :)

Oh and stdout and stderr of Popen are only necessary if you intend to print their output, as in my previous example, and if you are calling .communicate() you can replace Popen with call which basically does Popen followed by .communicate(). Less code == happier Marcus.

​

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAM2ybkUTZq-gpvpcO1zKJGxSm0aZAUbmU10sRYOxmt%2Bk7WTv_Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Ben Hearn]

Hey Marcus,

Sorry to keep code from you guys. I am not sure how much I generally should post online since it is for work and the code is split up into serveal chunks in batch and separate files, but I will make it a bit clearer for you guys :) Hope it is a bit clearer now

Also, my reply about sketchy file nodes, things like this happen a lot during development when people throw things around quickly or carelessly but being able to open it individually and not have it crash really threw me on it...

set BATCH_CONVERT=%CGROOT%folder\export_units_wrapper.mel
set MAYA_EXE=C:\Program Files\Autodesk\Maya%MAYA_VERSION%\bin\maya.exe

The MAYA_EXE variable is simply the executable path to Maya2015

The BATCH_CONVERT variable is a mel wrapper that I pass into Maya when it boots that runs the python code I want it to :)

mel code:

python("import maya_run");

python("maya_run.main()");

d is the result of splitting my dictionary up into chunks to process each bit individually:

def linch_dict_divider(raw_dict, num):

	list_result = []
	len_raw_dict = len(raw_dict)
	if len_raw_dict > num:
		base_num = len_raw_dict / num
		addr_num = len_raw_dict % num
		for i in range(num):
			this_dict = dict()
			keys = list()
			if addr_num > 0:
				keys = raw_dict.keys()[:base_num + 1]
				addr_num -= 1
			else:
				keys = raw_dict.keys()[:base_num]
			for key in keys:
				this_dict[key] = raw_dict[key]
				del raw_dict[key]
			list_result.append(this_dict)

	else:
		for d in raw_dict:
			this_dict = dict()
			this_dict[d] = raw_dict[d]
			list_result.append(this_dict)

	return list_result

[Chad Dombrova]

Hi Ben,

I just thought I'd mention that there are very few situations that I have encountered where you need a maya gui session instead of a batch session (you can even do openGL playblasts from a batch session).  You said that you are using "certain shaders and the Hypershade functions which need the maya GUI window open".  Can you tell us exactly what those are?  In some cases there will be more direct routes than using hypershade functions (for example, there is a hypershade proc for assigning shaders, but the 'sets' command will also work), and in other cases you simply need to source the appropriate hypershade mel script to ensure that the mel proc is available.  Just a thought.

-chad

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAM2ybkUa3AVtweJkHCkxOVP683y4t_ba6BHBTDEtr22%3DoVM9pQ%40mail.gmail.com.

[f.michal]

W dniu 2015-12-01 o 01:58, Chad Dombrova pisze:

> Hi Ben,
> I just thought I'd mention that there are very few situations that I
> have encountered where you need a maya gui session instead of a batch
> session (you can even do openGL playblasts from a batch session).

Hi Chad
I don't want to steal the thread, but how do you do a batch playblast?
I thought thats not possible.

[Marcus Ottosson]

I don’t want to steal the thread, but how do you do a batch playblast? I thought thats not possible.

Maybe open up a separate thread about it. That way that the answer won’t get lost as easily for future searchers.

​

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/565CF1DF.8030000%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Benjam901]

Hello Chadrik,

Yes absolutely.

The certain shaders I mention are CGFX shaders and for some reason Maya crashes when trying to apply said shader in Batch although I may be using the wrong call to apply them. The full function below illustrates use-age of both the hypershade command and the CGFX application:

def createReplacementCgfx(material='', texturePaths={}, cgfxChannels={}, cgfxShaderPath=''):
	""" If we find a shaderFxShader type material we need to replace wiuth CGFX for export purposes """
	print 'Replacing shader with CGFX'
	#mel.eval('cgfxShader -fx "%s" ' %cgfxShaderPath)
	matName = str(material)
	shader = pm.listConnections(material, type='shadingEngine')[0]
	members = list(set(shader.members(flatten=True)))
	pm.delete(matName)
	print 'got members'

	if 'diffuse_layer0_texture' in texturePaths:
		newMat = pm.shadingNode('cgfxShader', asShader=True, name=matName)
		mel.eval('cgfxShader -e -fx "%s" %s;' %(cgfxShaderPath,newMat))

	for key in texturePaths:
		slot = cgfxChannels[key]
		texture = texturePaths[key]
		newFileNode = createFileNode()
		pm.setAttr('%s.fileTextureName' %newFileNode, texture, type='string')

		pm.connectAttr('%s.outColor'%newFileNode, '%s.%s'%(newMat,slot), force=True)
		print 'connected', newMat

	pm.select(members)
	pm.hyperShade(assign=newMat)
	print 'CGFX function complete'

Cheers,

Ben

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

[Chad Dombrova]

try replacing these lines:

pm.select(members)
pm.hyperShade(assign=newMat)

with this:

pm.sets(newMat, edit=True, forceElement=members)

​

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAM2ybkUa3AVtweJkHCkxOVP683y4t_ba6BHBTDEtr22%3DoVM9pQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/e8f82c8e-a92e-465a-b838-52b18219f72d%40googlegroups.com.

[Ben Hearn]

Hey Chad,

Oh damn that worked an absolute treat!

Thank you for the heads up!

--
You received this message because you are subscribed to a topic in the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python_inside_maya/rOnS_4vY5sE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAGq9Q7EpTi7mySZGBaMrF1WJ3Yw9UfR2rvyzy9fZfVsAz%2B2KRg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--