Tagging MB files with metadata

[AK Eric]

It would be of great use if I could (somehow) tag mb files with metadata that could be read at system level, never having to open the mb itself in Maya.

I already have a solution that will save out a json/xml next to the file whenever the users saves, with queryable info.  But this is lossy, can decouple from the file if it's ever moved, etc.

Being able to tag an actual mb with data would be great (in the same way you can say, check exif data on an image).

I've tried some examples doing this in Python with pickle, on both ma & mb files, but... it corrupts the files.  Ironically, I can store and retrieve metadata, it just wrecks everything else in the file :P

Maybe not possible.  But I thought I'd see if someone actually had a more elegant solution for this.

thanks

[fruit...@gmail.com]

I'd be curious to find something as well ! I had a quick look at it ages ago, and couldn't find anything robust. If you're on Unix, you can attach infos against a file (including a .mb), but that doesn't seem to exist on windows. And I'd like to find something cross-platform
You can also write notes in maya, but then, as you said, you need to open the file, and even in standalone mode, it can take some time..
However, I think the safest option would be to deal with separated config files. As you say, one can 'break' the pipeline by moving the config file away from its scene file, but I'd say it is acceptable... I mean if an animator removes everything in the outliner, he can't complain the rig is broken ; if someone starts messing around with files he doesn't know, pipeline can't be responsible for that.
Still, by curiosity, I'd be curious to know if there is a way of storing infos against an .mb file

[Marcus Ottosson]

If .ma is an option, then it supports comment blocks.

// Comment here
...
addAttr ...
connectAttr..

A perhaps more common approach might be to store metadata relative an absolute path in a database.

{
  "c:\path\to\some\file.mb": {
    "any": "data",
    "here": True
  }
}

Then you can query files and metadata somewhat similarly.

fname = r"c:\path\to\some\file.mb"

with open(fname) as f:
  f.read()

with metadata(fname) as f:
  f.read()

Where metadata() is a function of your own design, for example..

import contextlib
import mongo

@contextlib.contextmanager
def metadata(fname):
  return mongo....find_one({"fname": fname})

Leaving out the details of how to communicate with the database for brevity.
​

​

[Juan Cristóbal Quesada]

following your attempts at using cPickle in mb files which seems
interesting to me... Cant you just append a binary datablock with your
data at the end of the .mb file and just preprocess the file reading and
deleting that appended block of data before opening the file in Maya? If
you do it right you should leave de postprocessed file as the original
one exactly with same size. Using a fixed sized datablock big enough to
hold your metadata should be easier. As a drawback you should code your
own "file open.."

[Juan Cristóbal Quesada]

following your attempts at using cPickle in mb files which seems
interesting to me... Cant you just append a binary datablock with your
data at the end of the .mb file and just preprocess the file reading and
deleting that appended block of data before opening the file in Maya? If
you do it right you should leave de postprocessed file as the original
one exactly with same size. Using a fixed sized datablock big enough to
hold your metadata should be easier. As a drawback you should code your
own "file open.."


El 11/02/2018 a las 15:38, fruit...@gmail.com escribió:

[AK Eric]

Marcus:  MA not an option, so no comment blocks, but thanks, good idea.  However, I like your idea of just writing data to somewhere on the server that corresponds to the Maya file in question.  That could be a legit answer.  not tagging the file with metadata itself, but the data isn't living next to the Maya file itself, which I find scary :P

Juan:  Your thought has merit, and technically I could do it with a pre-scene-open callback.  Would be worth a test to prove out, but it's a little scary that unless that callback fires, your scene is garbage.  However, it does make for an interesting way to copy-protect in-house data....

[Ian Jones]

Well if you like your MB and your current json/xml file solution but just want them tied together so they don't get disconnected you may want to consider just using a .zip to bind them. Uncompressed they can be very fast and you can access files directly (to query your metadata etc) without having to extract the zip with python. You'll just need a bit of code to enable opening/saving them back together.

https://docs.python.org/2/library/zipfile.html

Ian

On Sun, Feb 11, 2018, 8:34 PM AK Eric <war...@sbcglobal.net> wrote:

Marcus:  MA not an option, so no comment blocks, but thanks, good idea.  However, I like your idea of just writing data to somewhere on the server that corresponds to the Maya file in question.  That could be a legit answer.  not tagging the file with metadata itself, but the data isn't living next to the Maya file itself, which I find scary :P

Juan:  Your thought has merit, and technically I could do it with a pre-scene-open callback.  Would be worth a test to prove out, but it's a little scary that unless that callback fires, your scene is garbage.  However, it does make for an interesting way to copy-protect in-house data....

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/4485d4ba-a768-4116-a1da-87445e01da69%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alok Gandhi]

Why not use the fileInfo() to store the data?

Can be read outside maya as well by parsing the ma/mb

Sent from my iPhone

> --
> You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/065048f8-05b6-beea-10d1-2302693168d2%40gmail.com.

[Marcus Ottosson]

By golly, that does actually work!

from maya_scenefile_parser import MayaBinaryParser

fname =  "C:/Users/marcus/Desktop/temp.mb"

class Parser(MayaBinaryParser):
    def on_file_info(self, key, value):
        print("%s = %s" % (key, value))

with open(fname, "rb") as f:
    parser = Parser(f)
    parser.parse()

Output:

mykey = myvalue
application = maya
product = Maya 2015
version = 2015
cutIdentifier = 201503261530-955654
osv = Microsoft Windows 8 Enterprise Edition, 64-bit  (Build 9200)\n

Where mykey = myvalue was added via cmds.fileInfo("myvalue", "mykey") before saving the mb file. Tested on a file saved with Maya 2015.

• https://github.com/mottosso/maya-scenefile-parser

Nice one, Alok!

​

> To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/065048f8-05b6-beea-10d1-2302693168d2%40gmail.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/64C17806-7F48-456D-B05B-01DA327C8106%40gmail.com.

[AK Eric]

Thanks Alok & Marcus:  I tried that code and it does indeed work.  Nice job on the scenefile parser Marcus!

[Marcus Ottosson]

Glad it works, however I can't take much credit for the parser; I believe most of the work is originally from cgkit, and the rest from the parent repository of my fork, https://github.com/agibli/sansapp

On 12 February 2018 at 17:50, AK Eric <war...@sbcglobal.net> wrote:

Thanks Alok & Marcus:  I tried that code and it does indeed work.  Nice job on the scenefile parser Marcus!

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/6c44d89c-d1c5-4c93-9ffa-d2466b6b9617%40googlegroups.com.

[AK Eric]

Right on, credit :)

FYI, I figured out you can use cpickle.dumps to embed arbitrary python data straight into the value of fileInfo, allowing you to query it outside of Maya, powerful.

However, on large files (200+megs) it can still take a good 20 seconds to parse.  I may end up just storing all this data to a folder on a server for easy/fast lookup, but it was still a fun exercise.  Thanks for the help everyone.

[Mark Jackson]

Damn this is a nice idea, never thought of the fileInfo command, I've got to dig into this thanks guys!

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/563ef7ef-a5a4-4007-aa2d-f978cf569aff%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

-------------------------------------
Mark Jackson
CEO / Technical Director
red9consultancy.com

[aj...@eightvfx.com]

What file system are you using?
You can directly use NTFS ADS(Alternate Data Streams) if you're on Windows.

Basically you can write anything to your "c:/project/mayascene.mb:trololo", ":attr1", ":whatever" even with the shell commands, or your file manager.
You can even keep Maya scene versions inside of a single file, tagging all other copies with attribute like "mayascene.mb:v001"

[Marcus Ottosson]

Basically you can write anything to your “c:/project/mayascene.mb:trololo”, “:attr1”, “:whatever” even with the shell commands, or your file manager.

Sounds interesting! Would you be able to post an example?​

​

[Kthulhu Fhtagn]

On Thursday, May 3, 2018 at 3:26:33 PM UTC-7, Marcus Ottosson wrote:

Basically you can write anything to your “c:/project/mayascene.mb:trololo”, “:attr1”, “:whatever” even with the shell commands, or your file manager.

Sounds interesting! Would you be able to post an example?​

Simple windows cmd script, create empty directory somewhere and run it:

---

@echo off
copy c:\windows\explorer.exe test_subject.txt
dir /r
pause


type c:\windows\win.ini > test_subject.txt:attr1
dir /r
pause


more < test_subject.txt:attr1

At the end you will have a single 'test_subject.txt' file with the contents of explorer.exe in the main stream and 'win.ini' attached as 'test_subject.txt:attr1

---

 

​

[Joss13]

Some more examples here: http://www.flexhex.com/docs/articles/alternate-streams.phtml

[Joss13]

And it will work in python as well:

import sys, os, shutil, subprocess, time


curdir = os.path.dirname(__file__)
target = os.path.join(curdir, 'test_subject.txt')
target2 = os.path.join(curdir, 'test_subject.txt:attr1')


shutil.copy2('c:/windows/explorer.exe', target)
os.system('dir /r')
time.sleep(3)


shutil.copy2('c:/windows/win.ini', target2)
os.system('dir /r')
time.sleep(3)

[AK Eric]

Another great solution, thanks!