[Maya-Python] List referenced files

[Marcus Ottosson]

I’m looking for a way to list all nodes that somehow reference an external file; be it references, textures, or some obscure custom node with an attribute for paths.

I was looking at the cmds.filePathEditor which is close.

>>> cmds.filePathEditor(query=True, listDirectories="")
[list of paths]

But it doesn’t show me which nodes are responsible for these paths, and doesn’t show me the full filenames.

Any ideas?

​

--

Marcus Ottosson
konstr...@gmail.com

[Marcus Ottosson]

Also, importantly, I’d like their unresolved entry, no the resolve one. The command above only returns resolved paths.

# Right
$ROOT/parent/file.mb

# Wrong
/server/projects/myproject/parent/file.mb

​

--

Marcus Ottosson
konstr...@gmail.com

[Arvid Schneider]

I did a similiar thing..but it wasnt a one function solution. 

Its tool which catches all paths ( files ) and copies them to a given directory and remaps all the paths to the new set directory.

To get it to work though, I created a dictionary with nodeType and path attribute eg. file:filepath etc. And then I did for p in pmc.ls(type = nodeType) and then append all the nodes and paths to a dict. 

But once you get it setup its really easy to add new nodes to it. Alembic fbx aiimage they all have their own attributes.

Hope it did help 

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOA8yE2SWch_%3D-_jAG-uEUY6r52CA3tYg%3D00hDnb8REjEg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Arvid Schneider

Look Develpopment | Lighting TD | Show TD

www.arvidschneider.com

Mobil DE: (+49) 170 321 0074

Mobile UK: (+44) 744 824 6973

Mail: arvidsc...@gmail.com

[damon shelton]

I would look at the referenceQuery cmd. this will let you query the unresolved path as well as use this command to tell if a particular node is referenced or not. I think this command is exactly what you need.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CADrk2uQHWZ-wseM4haxmC4x7Wa2EzrSFt2Hbik-KUXnHdQU4qA%40mail.gmail.com.

[Fredrik Averpil]

I ended up making a dictionary of nodes and attributes and query these "manually" since there is (to my understanding) no way to really just find all filepaths existing in some shape or form in the scene.

Just think about plugins. You can never know if the developer properly registered the attribute so that filepaths will be found using the filepath editor.

I would love to be proved wrong here.

Regards,
Fredrik

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CADrk2uQHWZ-wseM4haxmC4x7Wa2EzrSFt2Hbik-KUXnHdQU4qA%40mail.gmail.com.

[Fredrik Averpil]

Oh, you're just talking about Maya references?
I thought you wanted to find e.g. ies files or vrmesh files etc and other files which are "referenced" (without being a "Maya reference").

[Fredrik Averpil]

> Oh, you're just talking about Maya references?

Sorry, getting tired :) re-read your initial question and I still don't think you can do what you want to achieve without parsing all nodes and all their attributes and then filter the value manually in order to determine whether the value is a folder path or file path.

Since that could become a bit heavy I made a dictionary like I mentioned before...

// Fredrik

[Robert White]

So if you want the texture file nodes in the scene, you can get them from pymel with:

import pymel.core as pm
file_nodes = pm.ls(type=pm.nt.File)
file_paths = [fyle.fileTextureName.get() for fyle in file_nodes]

with cmds:
import maya.cmds as cmds
file_nodes = cmds.ls(type='file')
file_paths = [cmds.getAttr(fyle + '.fileTextureName') for fyle in file_nodes]

Unresolved reference paths can be gotten pretty easy in pymel :
unresolved_paths = [ref.unresolvedPath() for ref in pm.listReferences()]

with cmds:
[cmds.referenceQuery(pth, unresolvedName=True, filename=True) for pth in  cmds.file(q=True, reference=True)]

[Marcus Ottosson]

Thanks guys for a quick reply.

@Arvid and Fredrik: Would it possible to share this dictionary?

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/a0cfa12a-90cb-4890-8e3f-6b46cfc34ffd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[damon shelton]

Also, you can use listAttr to list attrs that are usedAsFilename. Command only returns attribute name but then you can loop through getting all objects with attribute name and querying it and checking the path. you can also list string attributes if you want to take it further, you would have to turn relative paths into resolved paths by adding the current working directory to it.

cmds.listAttr(cmds.ls('*'),usedAsFilename=True)

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOBgtVoNmxD1XOCNbFu3jrkShgYx5ro_ePL1-uKL7-C%2Bfw%40mail.gmail.com.

[Sivanny Selvakumar]

`file -q -list` will return all the file reference paths and string attributes marked as usedAsFilename. So it saves you from having to iterate over the nodes in the scene yourself. This is how Maya gets a list of files for the File > Archive Scene feature. You can also access this functionality from outside Maya with the `maya.exe -archive <scene_file>` flag. 

As long as your plugin nodes mark their file path attributes as usedAsFilename, or override the MPxNode::getFilesToArchive function, it should pick up the paths.

[Geordie Martinez]

pymel is super useful for this

import pymel.core as pm

# GET A LIST OF REFERENCES
refs = pm.listReferences()

# LOOP THROUGH AND PRINT THE PATH
# 
for ref in refs:
    # PATH 
    print("path: %s"%  ref.path)
    
    # NAMESPACE
    print("namespace: %s" % ref.namespace)

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/1ebbe8ae-aada-4a31-8e71-2eddd8f02e93%40googlegroups.com.

[Geordie Martinez]

there used to be a script on creative crash called "File Path Manager" that did this with textures

[Roy Nieterau]

Ok, so the filePathEditor command does allow you to retrieve the attribute belonging to a certain file. So I assume it would also allow you to query the unresolved path?

from maya import cmds

def grouper(iterable, n, fillvalue=None):
    "Collect data into fixed-length chunks or blocks"
    # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx
    from itertools import izip_longest
    args = [iter(iterable)] * n
    return izip_longest(fillvalue=fillvalue, *args)

files = cmds.filePathEditor(query=True, listDirectories='', withAttribute=True)
for file in files:
    file_with_attrs = cmds.filePathEditor(query=True, listFiles=file, withAttribute=True)
   
    for filepath, attribute in grouper(file_with_attrs, 2):
        attr_value = cmds.getAttr(attribute)
        print attr_value

I remember that originally the filePathEditor got implemented under the premise that also plug-in nodes would tag there attributes as filenames so it could always find all files, even for custom nodes. Yet most nodes still aren't doing that, eg. Alembic, gpuCache, V-ray proxies. That's likely why it has a 'registerType' method now so you can 'try it yourself'.

Another way of listing files used in the scene is the cmds.file command. That will list references and textures (and probably some others) but won't find you the attribute. Probably together with the filePathEditor command it might.
The file command is also the one used when performing an Archive Scene with Maya's built-in functionality. (It's a python script in Maya/Python/site-packages/maya/app/general/zipScene.py)

According to the code in there it's this line:

    # get a list of all the files associated with the scene
    files = cmds.file(query=1, list=1, withoutCopyNumber=1)

This returns the current scene, note that it will return a non-existing scene if the current scene isn't saved. Because it will list {workspace_rootdirectory}/untitled as the first argument.
Though note that it does not find Alembic caches, Yeti fur caches, gpuCaches, vray Proxies, etcetera.
I think `cmds.file` allows querying both unresolved and resolved filenames.

Maybe this could also be interesting information: http://around-the-corner.typepad.com/adn/2015/06/registering-a-plug-in-to-work-with-maya-archive.html
Yet it seems to be Maya 2016 SP1 only to be fully functional. But at least I think it shows Autodesk's intentions.
Probably the Archive Scene command was updated for Maya 2016 SP1 to support this, so maybe have a look there to see what commands they use to retrieve the file list.

Cheers,
Roy

[Marcus Ottosson]

Thanks everyone for your replies!

It certainly looks more challenging than I would have thought, the cmds.file(query=True, list=True) is close, and the listFiles of filePathEditor almost got it, and though it looks to work with file nodes, it doesn’t seem to work with reference nodes, and I wonder what else it won’t work with. Alembic files probably, as you say, Roy.

What’s more, it doesn’t look like reference nodes contain their unresolved filename, maybe this is resolved by the editor itself?

# Always returns a resolved path?
cmds.getAttr("ben02_RN.fileNames[0]")

The goal of this little experiment is to be able to look at a scene and determine whether there are any absolute paths in it. As in, paths starting with either a drive letter and a colon (Windows), or a forward slash (Unix).

I’d like to look at a scene with these paths:

$ROOT/myfile.mb
$OTHERROOT/myotherfile.mb
/server/mythirdfile.mb

And go “aha, that’s 1 absolute path, and it belongs to node.myAttr”.

​

[Roy Nieterau]

For completeness sake. This is how to list unresolved filenames with the file command:

files = cmds.file(query=True, list=True, unresolvedName=True)

[Marcus Ottosson]

cmds.file(query=True, list=True, unresolvedName=True)

I think that’s it!

​

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/232e4654-4676-4c5e-9ca6-35feb1d63bd3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Roy Nieterau]

And sorry for the spamming, but it seems that V-Ray 3.0 actually does register the types for its file attributes.
In Maya 2015 it doesn't list the Alembic attributes, but I assume it will with coming version (maybe even Maya 2016)?
Autodesk will likely push developers to register it for their custom nodes correctly, this way archive scene also supports it correctly.

Anyway, I just noticed that it's important to perform a refresh before you query!
Without doing that I just had it list nothing at all.

Also the filePathEditor command actually does list references!
Here's some better explaining code:

import os

# Refresh the file list cache (note that this is relatively slow for large scenes)
cmds.filePathEditor(refresh=True)

# Get the file directories
directories = cmds.filePathEditor(query=True, listDirectories='', relativeNames=False)

# Get the files in use
for directory in directories:
    files = cmds.filePathEditor(query=True, listFiles=directory)
   
    # With the file attributes (also see earlier reply with `grouper()` function to iterate over it)
    # file_attr = cmds.filePathEditor(query=True, listFiles=directory, relativeNames=True, withAttribute=True)
   
    # To get the full path of the files in this directory
    full_paths = [os.path.join(directory, file) for file in files]
    print full_paths

By setting relativeNames to True it actually returns the unresolved paths. Even if a fileName is in the long format in a file attribute and is within the current workspace then this command will not resolve it into a relative path. It will just read the value as 'raw' from the attribute. Do note that sometimes it adds an additional "../" in front of it.

Hope this helps!

So you should be able to just register missing types temporarily (use the tmp flag) and remove them afterwards if you want to include additional nodeTypes in solely your query without messing with other settings in the current (or next) Maya session.

-Roy

[Fredrik Averpil]

Hehe, it seems to me that you are trying to achieve something we do here; validate all filepaths before publishing.
Am I right? ;)
We want all our filepaths to be relative to the Maya project at all times and there's no silver bullet for this, I'm afraid. You're going to have to combine several methods. I wouldn't mind sharing our dictionary but it's far from covering all filepaths from all imaginable plugins etc and obviously it won't cover Maya refs. It just consists of the node attributes which are used on a regular basis in our studio.

About a a-Ray 3.0, I believe that was me asking them to make their file attributes detectable by the filepaths editor. However, in the end I have need up ditching the filepaths editor and instead made my own solution

// Fredrik

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/78356830-dce9-46ce-b5f9-bda2ae2689f9%40googlegroups.com.

[Fredrik Averpil]

Typing on the iPad really sucks... And with the line breaks bug in Google Inbox it's just... bad.

[Marcus Ottosson]

validate all filepaths before publishing.

You’re absolutely right.

As I’m not alone in solving it, here’s some more research.

I’ve found that if I temporarily export an .ma of nodes, excluding those that I know won’t have one and may be heavy to export (such as mesh), I can parse the resulting ASCII file for /. The file ends up light, regardless of how “heavy” a scene is, and takes under a few milliseconds to export.

The / in the file is used for either:

1. A file path, even on Windows
2. A comment

Excluding comments, the rest results to all paths present in the scene.

Though I haven’t tested it thoroughly, I’d expect to run into the need for additional parsing for:

1. Relative paths, without /
2. Expressions or the like that also use /

​

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAD%3DwhWMxNTmWZO7zMw%3DBkgumcf8C5cKrJv1Vg-0Ky3VaVW2mKQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Fredrik Averpil]

Yes, I used to parse .ma too. So that could be one way of dealing with it. I don't believe there is such a thing called a relative path inside of the .ma though. If I remember correctly, all paths are always absolute when you peak inside an .ma file.

Some old notes which may or may not be completely up to date regarding all of this, but perhaps may be of some use for you: http://fredrik.averpil.com/post/88067886541

// Fredrik

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOBR%3DmaE_bcj2OLZffjfzEO-7FtUYU5VZ2%3DDAQV1SzTQGw%40mail.gmail.com.

[Roy Nieterau]

You could end up 'reading' custom attributes made by the user which you might not want to change/validate (eg. metadata/pipeline data or even unrelated data).
Setting up a dictionary/set of attributes for those missing is a better and more lightweight solution.

Also note that you could still rely on reading the attributes yourself by checking the registered types of the filePathEditor, but since it allows to query the unresolved names anyway... why bother (outside of optimization)?
Or was the latest blob of code I posted still not functionally what you were looking for?

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/78356830-dce9-46ce-b5f9-bda2ae2689f9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAD%3DwhWMxNTmWZO7zMw%3DBkgumcf8C5cKrJv1Vg-0Ky3VaVW2mKQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

[Sivanny Selvakumar]

Hi Marcus,

Just a quick note about the fileNames attribute. It's actually obsolete and hasn't been used since Maya 6.0. It was only kept around for backwards compatibility. After Maya 6.5, reference paths are only accessible via the referenceQuery and the file command. Other attributes that are not used anymore include connectionList, setAttrList, addAttrList, deleteAttrList, brokenConnectionList, parentList, edits and multiParentList. placeHolderList is still used, though. It looks like the node docs don't really state this, but they totally should!

Just FYI,

Sivanny

[Marcus Ottosson]

Oh, that's an interesting note, thanks for sharing Sivanny.

So, how would one access the path from a reference node? Is the information not stored in the node at all? Where is it stored?

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/471b115e-6e9e-4a3b-bdf1-4ca0dc2d44b0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Sivanny Selvakumar]

It's stored in internal data structures. So the only way to access/change it is via the referenceQuery command, the file command or MFnReference.

On Wednesday, August 12, 2015 at 9:45:22 AM UTC-7, Marcus Ottosson wrote:

Oh, that's an interesting note, thanks for sharing Sivanny.

So, how would one access the path from a reference node? Is the information not stored in the node at all? Where is it stored?

On 12 August 2015 at 17:27, Sivanny Selvakumar <sivanny.s...@gmail.com> wrote:

Hi Marcus,

Just a quick note about the fileNames attribute. It's actually obsolete and hasn't been used since Maya 6.0. It was only kept around for backwards compatibility. After Maya 6.5, reference paths are only accessible via the referenceQuery and the file command. Other attributes that are not used anymore include connectionList, setAttrList, addAttrList, deleteAttrList, brokenConnectionList, parentList, edits and multiParentList. placeHolderList is still used, though. It looks like the node docs don't really state this, but they totally should!

Just FYI,

Sivanny


On Wednesday, August 12, 2015 at 12:12:17 AM UTC-7, Marcus Ottosson wrote:

What’s more, it doesn’t look like reference nodes contain their unresolved filename, maybe this is resolved by the editor itself?

# Always returns a resolved path?
cmds.getAttr("ben02_RN.fileNames[0]")

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/471b115e-6e9e-4a3b-bdf1-4ca0dc2d44b0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com