Version control in production

[郭金锋]

Hey guys,

Wonder if version control is a big issue in your pipeline, if so, could you please share some tips on this? 

Most of the production application files are binary, and popular version control solutions/ideas are code/ascii-oriected, which make them less appliable for production(am I correct?). How do you figure this out? I know lots of big studios have Shotgun or equivalent production management system, but i don't think my supervisor would be willing to pay that much. I heard Tactic is also a good one(and a python one), have you guys ever used Tactic? How good is it? 

Thanks,

Jerry 

[Marcus Ottosson]

This is an incredibly broad question, you might have to get a bit more specific. For example, how are you doing it currently, how many are you and do you have access to people who can code?

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/636fc5a2-263b-4527-a481-e82753dc2eae%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Marcus Ottosson
konstr...@gmail.com

[Justin Israel]

Most of the production application files are binary, and popular version control solutions/ideas are code/ascii-oriected,

git tracks ascii or binary files by hashing their content. If the file is ascii, then it can provide the extra tools for diff, merge, changes, ...

 

which make them less appliable for production(am I correct?). How do you figure this out? I know lots of big studios have Shotgun or equivalent production management system, but i don't think my supervisor would be willing to pay that much. I heard Tactic is also a good one(and a python one), have you guys ever used Tactic? How good is it? 

 

I haven't had the opportunity to use either Shotgun or Tactic yet, given the studios I have worked for thus far. But here are the kinds of combinations I have seen:

• Version/Publishing solution is its own service ; Production Tracking / Assignments is its own service that sources from VnP
• Pure manual file system conventions for storing in a versioned structure; completely detached production tracking
• Pure manual file system ... ; Separate tools update metadata + Production tracking sources that metadata  

The one thing I haven't worked with yet is the off the shelf solutions that set you up with the entire pipeline in one box: Asset and Production Tracking, Version and Publishing, data management

Like Marcus was saying, it comes down to:

1. how robust of a system you want, for integration
2. how many areas of the the pipeline it needs to cover
3. how many people it needs to serve
   
4. how much you are willing to spend
5. do you have on-site developers

"Rolling your own" can be an endeavor, but then again if your needs are very small then maybe you only need it to do a few simple things. 

[Nils Lerin]

If what you are after is a robust and user-friendly version control system then I would suggest perforce. Its used extensively in the games industry.

But like the others say, if you are interested in a combined tool with production tracking, reviews, metadata, messaging and what not it becomes a much larger and complicated topic.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA3x9rMPPDpV5ifeduMQCCSpzfjzH4zYTos%2B_v%2BAXmbbUg%40mail.gmail.com.

[Eduardo Grana]

Hello Jerry!

regarding

I heard Tactic is also a good one(and a python one), have you guys ever used Tactic? How good is it?

I had the experience to work with TACTIC and is quite good, you have it managing the file system and handling some relational dbs tracking most of the stuff.

It takes a little time to set up, but some more to understand it because documentation is not always easy to find , and you find yourself digging into the code

to find out whats happening.

It works through a web browser, it tracks assets and tasks and can make reports. You can make a review system too.

You can also have it running on a cloud, like aws.com.
Here are some videos to checkout
http://community.southpawtech.com/tactic-content/quickstart

Most of it is written in pyhton, and you get the source code, but also there are some javascript parts specially in the web browser side.

Regarding the versioning, it handles a naming convention and it writes one file for each version. Not the most efficient way, i know,
but you can backup old version and restore them whenever you want.

The only serious problem i found in TACTIC was that the support is really expensive if you have few seats in your studio,
since its not based on users, but in the topic you want to ask about (like end user, setup, developer, etc).
http://www.southpawtech.com/what-we-offer/support/

In the other hand, you have some free forums and you get some help from time to time, mostly from the

TACTIC developers team.

Other thing to be aware of, is that if you work with tactic, you will have to have someone taking care of it,

since it will stop working in some ocasions.

What i really liked about it, it was the client api. With it you can get all the data about your project using pyhton,

(even from maya doing some one line tweak in the code). Also you have an expression language that is awesome

to navigate through the different dbs that are related.
http://community.southpawtech.com/content/tech-series-expression-language

You also have it in github too if you want to contribute...
https://github.com/Southpaw-TACTIC/TACTIC

In my opinion, maybe the best option to hit the ground running, but you'll have to adapt your production stuff to it.

Hope this is usefull!

Eduardo

--
Eduardo Graña
www.eduardograna.com.ar

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAGkBUFfoykmV9VK6_DXNjUG7n43Ya0beC52JMSog9i%2BK6_TCVg%40mail.gmail.com.

[郭金锋]

Hey Justin,

Thanks for your reply, we have about 200 computers for production and around 50 projects on-going at the same time. Not like big studios, most of our projects are part of the full production pipeline, which means project A maybe animation while project B might be only rendering, which confused me a lot since each project has its own naming and file structuring conventions, and most of our projects require us to map a network url as a drive. By mapping the shared storage everyone has the right to get there and work on it, which makes tracking files difficult. 

I do realize that Git has the ability to track the whole structure and hash each file it doesn't ignore, but my question is, will the reset function that Git offers violate the binary file? 

And on three approaches you listed on the last email, do you mean using svn or git to track files on a pure file system(with...)? Or add a database to track them?

Thanks!

Jerry

在 2014年2月17日星期一UTC+8上午3时39分01秒，Justin Israel写道：

[郭金锋]

Hello Eduardo!

Thank you for such detailed reply! Awesome!

You know what, I did try Tactic today. After installing PostgreSQL, TACTIC and Apache, I logged onto the Tactic server. I tried to demo the system to my coworkers, even though I found it exciting, their response were not as good.

The first problem for them is language, most of them don't speak English as I do, nor could they understand the complex process to do a commit in English interface. I will try to translate most of it to see what will turn out. Well the second thing is the pipeline, TACTIC brings a professional pipeline concept to us, which, to be honest, is a little bit above us. But i do think that concept is what we should work for, and should not be a problem.

Maybe version control itself is not the key, but tracking file is? Everyday my coworkers could have tons of problems with wrong files, maybe keep file metadata into a DB could help...

I think I will try translating TACTIC, cause it is really cool!

Thanks for your advice! Cheers!

Jerry 

[郭金锋]

Well, we have about 200 computers for production, and it's mainly digital asset creation not code itself. We are currently doing it manually by specifying naming conventions. However, this approach always turns out not as good as we wish since "s01c03_v03.ma" will not remind us of anything... 

[Marcus Ottosson]

You've got 200 artists running 50 projects simultaneously and you haven't got naming conventions sorted out? :O Wow, never heard of such quantities, less so without a very robust pipeline.

What are the inputs and outputs of your projects? Are you producing full 3d film from storyboards? What's the target? Tv, cinema?

Where are you based?

On 17 February 2014 14:47, 郭金锋 <guojinf...@gmail.com> wrote:

Well, we have about 200 computers for production, and it's mainly digital asset creation not code itself. We are currently doing it manually by specifying naming conventions. However, this approach always turns out not as good as we wish since "s01c03_v03.ma" will not remind us of anything... 

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/46a741b3-468b-4027-b2f9-b391d980edc5%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Marcus Ottosson
konstr...@gmail.com

[Ævar Guðmundsson]

 I know lots of big studios have Shotgun or equivalent production management system, but i don't think my supervisor would be willing to pay that much

TACTIC, the open source version is free:

  http://community.southpawtech.com/get-started

fTrack, $39 per 3 seats

  https://www.ftrack.com/#pricing

Shotgun, $49 per seat

  https://www.shotgunsoftware.com/pricing

 

  These are not high numbers compared to the savings you’ll get by removing the complications of getting to the data as half an hour spent on that per person, only requires 8 instances and you will have blown through a full workday’s worth of expenses on a relatively needless operation,i.e. digging around for where things are.

  I’m not sure how the others solve it but you can for example utilize Tank (called shotgun pipeline toolkit today but Tank is a more fun name in relation to a Shotgun) to handle the file system mount you describe so it’s a lot more manageable that way, only difference would be the users would need to “sign-in” to a task in order to get the project mounted.

http://www.youtube.com/watch?v=BY4zZrBwCjk

 

  most of our projects are part of the full production pipeline, which means project A maybe animation while project B might be only rendering

  This is not as different from other standards as you might think.  This workflow exists at larger studios as well but the difference is just in the definition of the word “project”.  Sounds like all you need is a “master container” then all your “stages” i.e. animation,rendering,simulation become constituants of the master container.

  This means to adopt a full blown production pipeline your users potentially don’t need to change anything in the way they work but you or your supervisor tie it together by redefining project to stages.  Give this about 6 months and the artists will rather want to work on projects than stages and the concept will catch on leaving your workflow migrated with ease.

  Ultimately you will need something like this, if not only just to account for the lost time digging for data otherwise.

 

  most of our projects require us to map a network url as a drive

  Excellent!  There are too many benefits to doing that to go into really but make sure the drives are mounted accurately, a difference in a samba mount or an nfs mount can mean a difference of weeks in total accumulated lost waiting time on annual basis.  Just to make a case in general by mounting drives rather than accessing the server structure remotely you get full removal of directory structure complications.

 

track files on a pure file system(with...)? 

  With either embedded metadata inside your binary or ascii, then using grep or strings to extract it, or in a .info stub file either next to your data or stored on a separate metadata server.

 

not as good as we wish since "s01c03_v03.ma" will not remind us of anything

  This is actually a more directory structure issue than a naming convention issue.  Keep in mind that file names are not the last readable stub you get displayed, a full filename would be:

C:\this\project\with\that\shot\s01c03_v03.ma

  You can see the naming convention part is not the identifiable part, where the file is kept identifies the element being worked on.

  Keeping placement of files in order is not easy though, so you’ll need a publisher, something that ensures when an artist wants to pass on files to the next artist he/she does not have to worry about where to put the file, just click “publish” and let the distribution code put it where it should go.  This means the fixed location will be a known location where anyone picks up any data, always in the same directory depending on what project is mounted on a drive P:\maya\renders for example would always include the Maya renders for the project in question.

/////

  With such a high number of stages and people I’m going to venture a guess that you are working on a gaming pipeline rather than a visual effects one, if that is the case then just make sure whatever system you adopt to transfer the “shots and sequences” terminology you find in the documentation into a “scene and element” terminology and the pipeline approach is then near identical ( with a notable missing feature in all the mainstream production managing software being a sandbox area in parallel with shot review turntable videos where you upload assets and can view them in an in-game environment, basically a video stream on your website coming from your engine, this would let your supervisors and directors do quick reviews even on their phones. )

 As for the non-english speaking part, Shotgun for example is a web based approach, as an example I'm using the target language as Spanish here below, in which case you get half away with just using things like auto-translate in Chrome.  Translating the API itself is a hazardous sport, means you need to update your translation for every iteration and addition that comes from the developers so it may turn into a full time job.  Again here Shotgun has the strength of you being able to work in english but you rename only your “entities”.  Some people rename their “shot/sequence” category to “shots/sequences” so it can just as well be renamed to “disparo/secuencia”.  That means a one-off translation which is performed as a definition at project startup anyway will ripple through to all projects.

  Hope this helps, it’s a fun subject, just make sure not to point that Shotgun at anyone since too much discipline/enforcement can definitely take away from the creative side the system is meant to support and enhance, try introducing the time saving benefits of doing this in a more organized fashion and surely your supervisor will love the free time he gets to work on the project data rather than search for the project data.

 

 

[Eduardo Grana]

Hello Everyone!

I just love this topic, i would be great to come up with a 'python inside maya' workflow and pipeline guidelines. Or even Pipeline Standards!

Maybe we can make the industry a better place to work!

Thanks everyone for sharing
.

Cheers!

Eduardo

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/1f238a86-751c-4745-9122-76d6dd6fd582%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Eduardo Graña
www.eduardograna.com.ar

[Asi Sudai]

Interesting question, would be happy to add my 2 cents.

Currently and in the past I found that a combination of publishing framework together with a database can work great.

Each publish is store in the database with at lest those basic field:

context ( Shot/Asset ) 

type/group ( rig, geo, maya-scene-file ) 

status ( approved, rejected... pending review )

user ( who publish )

location ( of files/folder )

date ( publish date )

With that you could do useful queries like this:

findPublishes( assetID=1, type='rig', status='approved' )

The publishing framework would handle the file-system transactions and permission:

it should do something like this:

- receive data/files/folder to publish

- pre-register the transaction ( create and lock version 01 )

- creates place for the data in a user-read-only location

- copy the published files/folder into

- lock permissions for read only

- post-register the transaction ( finish publishing version 01 )

* in case the publish failed for any reason, the manager will revert back the operation and delete the transaction records.

The database ( could be Shotgun using publishFile entity or some custom entity or a local mysql  ) will handle the following:

- keep track of published versions per asset/shot

- keep track if an asset/shot got checked out

- keep track of the version status ( approve, latest... ) 

this approach can support multi users&shows and even multi OS working together.

hopes that helps a little,

Asi

 

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/1f238a86-751c-4745-9122-76d6dd6fd582%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Eduardo Graña
www.eduardograna.com.ar

[co...@grid-vfx.com]

Hi,
I worked with both tactic and shotgun/tank.
Tactic is free but the code is really old, it doesn't scale well and it's full of bug that even southpaw tech
don't know how to fix.
It also require some work to be integrated with the CG app.

Shotgun/tank is really great for small and mid size projet and it come with some integration
with a lot of softwares (maya, nuke, ...).
But you need to be ok to follow "the tank way" of managing your pipeline.

If you are only interested by a asset tracking part (no production management)
i really think one of the best option it to develop your own tools on top of some python library (like sqlAlchemy or Django).

Greets,
Colas