Initial Commit ;)

[hooly]

Thanks for this Szabolcs, Great idea - ' still don't quiet understand
how the H.E. maya dev list fizzled out.

I'm curious to know if anyone uses these tools for maya development?
Rake as a build tool.
Git and Github for version control.

My github account is here:
http://github.com/hoolymama
Lotsa stuff, not much is documented yet - just these:
http://hoolymama.github.com/build_mama/
http://hoolymama.github.com/doc_mama/

[Ren Zhi Zhang]

I'm using Boost Build (which is also used to build the popular Boost C++
libraries from www.boost.org) to build my Maya plug-ins on Windows and
Linux (sorry, don't have Macs but should also work). However, I must say
the learning curve is steep and it took me three weeks just to figure
out how to assemble a correct build file with all the features I need.
So unless you are already using the Boost libraries and you are sick of
maintaining separate make files for Visual Studio and GCC then it's
probably *not* worth the effort. On the other hand, once you have the
build file, the same build file can be used to build plug-ins on
different platforms with different compilers for different Maya
versions, and each new plug-in is just one additional line of code.
Very, very slick :-)

I'm also using Git, though I've only started just recently.

Cheers!

Ren Zhi

P.S. There was some talk about reviving the Render Theory list just
before Highend shut down. Do anyone know what happened? Alternatively
does anyone know any Renderman or mentral ray related lists?

[Carsten Kolve]

Nice one Julian - there is loads of realy useful stuff in there! -
We've been using bendyBones on a nr of projects for "wobbly things"
and it's nice and stable and fast :-)

Ok, back to your question - we've been using scons for building for a
while now but are switching over to cmake as we speak (much nicer for
cross-platform / cross architecture building) - no experience with
rake, sorry. For version control we are using svn (actually, a custom
python wrapper to svn to simplify common operations), but our pipeline
dudes are pushing hard for git (mainly cause the whole branching
business is much better).
Don't really have an opinion on how git makes you're life easier as a
Maya - developer, I guess it mostly depends on the amount and
complexity of your projects and the nr of people you have to share
your code with.

Again, thanks for sharing!
Carsten

[Horvátth Szabolcs]

Hi Julian,

> Thanks for this Szabolcs, Great idea - ' still don't quiet understand how the H.E. maya dev list fizzled out.
>

You're welcome. Hope the list continues working as if nothing happened,
because it had the very good signal to noise ratio.
I guess none knows why it was discontinued so fast. It would have been
much easier setting up a mailing list if the old list was still active
your a few days.
Probably it is one of the reasons it *was not* active. :)

> I'm curious to know if anyone uses these tools for maya development?
> Rake as a build tool.
> Git and Github for version control.
>

For building we were checking our Scons but it was not really used just
tested. For version control we used Subversion but since we would like
to avoid using a central repository server
we're taking a close look at Git.

> My github account is here:
> http://github.com/hoolymama
> Lotsa stuff, not much is documented yet - just these:
>

Loads of interesting stuff, thanks! Even taking a look will take a
week... :)
By the way, just our of interest, is it the fruit of your own
development or were the guys in suits at DNeg or MPC generous enough to
release them publicly?

Cheers,
Szabolcs

[Horvátth Szabolcs]

> P.S. There was some talk about reviving the Render Theory list just
> before Highend shut down

Yep, a few people replied but the Highend list was shut down before
anything happened really.

> Do anyone know what happened?

Yep, nothing. Or to be precise I created a similar google group called
Render_he3d but did not have the time to send the address to the
rendering list.
I'll probably send an invitation to the posters but since its we pretty
much dead lately (don't know why, it was super-cool in the beginning,
with Larry Gritz posting alongside
Thomas Driemeyer) I don't know if it will work.

> Alternatively does anyone know any Renderman or mentral ray related
> lists?

For renderman you have the comp.graphics.rendering.renderman group but I
don't know if that has any traffic these days. And you get the Pixar
Renderman support forums
if you're a customer. For mental ray you can subscribe to the `mental
ray` and the `mental ray shader` mailing lists from the mental images
site or use their forum that
used to be the LAMRUG.org site's forum. (The mental images mailing lists
are really good with ancient users subscribed but there are quite few
new posts these days.)

Cheers,
Szabolcs

[julian mann]

 

For building we were checking our Scons but it was not really used just tested. For version control we used Subversion but since we would like to avoid using a central repository server
we're taking a close look at Git.

I'm finding git to be quite solid - many times I got into a mess with svn where I ended up trashing the repo and starting again. Git has take a while to sink in, new jargon etc, - but I love the way that when you switch to a different branch, the text in your text editor changes straight away. I get the feeling it will be much easier to collaborate on OS projects with git.

There's a cool, and characteristically arrogant talk about git by Linus here: http://www.youtube.com/watch?v=4XpnKHJAok8

Loads of interesting stuff, thanks! Even taking a look will take a week... :)

By the way, just our of interest, is it the fruit of your own development or were the guys in suits...

No - their stuff stays with them.  - though I wouldn't be surprised if there were some more open source initiatives by big shops soon. 

If you are at the stage of testing build systems, I'd recommend taking a look at Rake too. http://martinfowler.com/articles/rake.html


Cheers

Julian

[Horvátth Szabolcs]

>
> I'm finding git to be quite solid - many times I got into a mess with
> svn where I ended up trashing the repo and starting again. Git has
> take a while to sink in, new jargon etc, - but I love the way that
> when you switch to a different branch, the text in your text editor
> changes straight away. I get the feeling it will be much easier to
> collaborate on OS projects with git.

One thing we've had some problems with was renaming files. Git is
supposed to track stuff based on content but it never found renamed
files and they always appeared as new files without any previous logs.

> There's a cool, and characteristically arrogant talk about git by
> Linus here: http://www.youtube.com/watch?v=4XpnKHJAok8

Yes, I saw that one, its quite funny actually. It was a big plus for
choosing source control software, I alway preferred software developers
with a decent sense humor. ;)

> If you are at the stage of testing build systems, I'd recommend taking
> a look at Rake too. http://martinfowler.com/articles/rake.html

Rake is Ruby and most of our scripts and moving towards Python, so if it
has to be a script language based system than I'd prefer Python.

Cheers,
Szabolcs

[Hradec]

This Linus talk is hilarious!!! Thanks for this "pearl" of enjoyment, Julian... I'm laughing my pants off!!! LOL!!

-H

--
Hradec

[Hai Nguyen]

I think I might have chimed in a bit late for this thread. Never
touched Rake....I've been using CMake
for the past 2 years with a fair bit of success on cross platform
building of Maya plug-ins. There
are some whacky pitfalls, but overall - it's been pretty awesome.

[julian mann]

Hi all,

Thanks for the feedback about build tools and version control.  -- sorry about being so slack with a reply.
 Carsten, I'm really glad bendyBones is getting some milage ;) cheers! 

Re: Building
Seems CMake is gaining popularity. Guess I should check it out. I googled CMake vs <others> but didn't see much to persuade me away from Rake at the moment. In one thread the CMake evangelist says, "CMake doesn't have functions, but it has a pretty good macro language"..then examples.. ADD_CUSTOM_COMMAND() or EXECUTE_PROGRAM().
I stopped there, because a.) thats just ugly, and b.) I feel that writing build scripts is programming, so you need the most powerful programming language you can get your hands on. Rake and SCons are DSLs built on powerful programming languages,  they just add the few ingredients that you need to do builds - that is: a dependency mechanism , modification time checking, lazy file globbing.

For example, my maya module build script also builds all documentation from templates, converts jpgs to xpms in different formats, performs GIT checkins, checkouts, and branch changes etc. It handles cross platform builds pretty easily. Nothing is built unless it's out of date. Ruby's meta-programing power makes it all fun.

Re: GIT
Seems to me GIT has great potential to go beyond source code control. It has good compression and delta handling. Branching and merging is easy. The repo is local. So why not start a maya project, make a scene, call it final and commit the project. As you progress, commit your changes -. When you want to experiment, make a new branch. You have to annotate your commits:  e.g. "testing with particles half the size - rendered as tube type". When you do git diff, you would get
--- a/scenes/final.ma
+++ b/scenes/final.ma

-      setAttr "particleRenderType" 7
+     setAttr "particleRenderType" 9

It would be cool to wrap all the git commands in a submenu of the FileMenu maybe.
I haven't thought this through, so flame me if you like ;)

cheers

julian

2009/9/3 Hai Nguyen <sayi...@gmail.com>

[Horvátth Szabolcs]

Seems to me GIT has great potential to go beyond source code control. It has good compression and delta handling. Branching and merging is easy. The repo is local. So why not start a maya project, make a scene, call it final and commit the project. As you progress, commit your changes -. When you want to experiment, make a new branch. You have to annotate your commits:  e.g. "testing with particles half the size - rendered as tube type". When you do git diff, you would get
--- a/scenes/final.ma
+++ b/scenes/final.ma

-      setAttr "particleRenderType" 7
+     setAttr "particleRenderType" 9

It would be cool to wrap all the git commands in a submenu of the FileMenu maybe.
I haven't thought this through, so flame me if you like ;)

No flame really, just some tiny sparks... :)
The question for me about using source control for Maya files is: what extra do you get doing this? If you use ascii files you can already do a diff on two files and get the same difference data.
File sizes would be much smaller using Git but you'd get extra load / save delays (checking in and out of the repository, the most annoying when you're looking for a setup that you remember
using in one of the earlier scenes) and most of the time user time worths more than disk space.
Doing annotations is a nice addition too but adding source control into the maya GUI is way more work than implementing a simple log window that pops up when you do a `save as` and lets you log your changes.
So whats that extra stuff that does worth using this method in you opinion? The elegance of the solution is tempting (Its very cool and TDish) but I suppose it raises more issues than it solves.

But I'm all ears for some extra pros and cons!

Cheers,
Szabolcs

[Hai Nguyen]

2009/9/5 Horvátth Szabolcs <szab...@impresszio.hu>

No flame really, just some tiny sparks... :)
The question for me about using source control for Maya files is: what extra do you get doing this? If you use ascii files you can already do a diff on two files and get the same difference data.
File sizes would be much smaller using Git but you'd get extra load / save delays (checking in and out of the repository, the most annoying when you're looking for a setup that you remember
using in one of the earlier scenes) and most of the time user time worths more than disk space.
Doing annotations is a nice addition too but adding source control into the maya GUI is way more work than implementing a simple log window that pops up when you do a `save as` and lets you log your changes.
So whats that extra stuff that does worth using this method in you opinion? The elegance of the solution is tempting (Its very cool and TDish) but I suppose it raises more issues than it solves.

But I'm all ears for some extra pros and cons!

Horvátth....one of the characters in a show i'm working is named that. :) 

The last part of your email brings up a good point and one that people keep debating endlessly. I have two things to share about it...and in my mind, I feel like they're on the exact opposite ends of the spectrum of the debate:

1) I was working for a company a few years ago and we were approached by some dudes that worked for the software arm of def2shoot about a product that they were developing that effectively managed your entire pipeline for you. The name escapes me right now. Anyway, it was advertised that it would automatically version files from X packages. At some points it seem like it was even obfuscating the file system from you. The end result was having this cool thing that you could watch clips and access files from anywhere in the company without too much pain. I don't know how far they got with it, we ended up not using it. 

2) Worked for another company not too soon after that where they wanted to do things like replace the Maya save menu with their own stuff and use SVN on the back end to version the files. One of the first things that came up for debate was how well the binary deltafication of files was against the ASCII counter part and compressibility for disk space, etc, etc.. This open up all sorts of cans to all sorts of worms. How fast did the file server shave to be - how many do we need? How do we tag/name files so the tools can know about them? Does this work for XSI and After Effects? It turned out to be a huge design problem with seemingly very minimal gains. And there's always that one guy says,  "Well...if we removed X it would work." I'm not sure if that one guy realizes it's still a lot of work. 

I think from a "grander view" the automatic versioning/checking/backup/whatever is really tempting - much like you said. But I feel that a lot of people who really want that haven't considered the possible catastrophic results while in production. One practical example that an artist brought up when I was talking to him about it was, it's easier for him to say save a file with _v05 than to make a comment in a versioned entry. That sounds silly I know, because versioning seems to accomplish the same thing for him. But that's the way that a lot of people work and it's hard to counteract it with "training" - because to be frank, they don't care. It's weird, but I've noticed that even on short shows that have pressing deadlines - it's hard to remember the day or date that you made a change. But it's easier to recall that it was around version X. 

I flip flop between artist and TD pretty often and to be honest, I'd rather suffer from my own mistakes in my files and tools than have to deal with a system that, at times, seems to work against me. 

I think if you had a few TD dudes that very well understand the problem, they could definitely put something together in some amount of time...but it would take a few iterations to get something that just operates smoothly.

Just my $0.02.

- Hai

[julian mann]

2009/9/5 Hai Nguyen <sayi...@gmail.com>

I'll chip back in. I totally agree;) People only use software and processes they like, and they only like stuff that doesn't make them think. I'm working alone at the moment - I'm modeler, rigger, animator, developer, systems guy , so all the stuff I'm developing is for shots and I'm constantly thinking "How can I eliminate the need to do or think about X?"

If a developer, in isolation from production, took up the git-maya-versioning project, like you say - it would be catastrophic. "Dont talk to me about deltas and head-refs - just gimme the file I saved last night".  -- And if there had been a huge investment in that isolated development, the facility might start policing it "you have to use this system - we put a lot of money into it". (shudders).

But if the tool was used in production - while it was being built, from day 1, then every "feature" would be useful - or changed to be useful - or  canned. I think there are very few bad ideas, but a lot of crap implementations.

I think it would be a mistake to think that git and svn are the same thing. btw, Github is a distraction here - its more like a backup and a place to share. I had a problem getting my head around that at first because I was still in SVN-central mode.


But I did a few tests...

After 3 commits  (of 1 100Mb file and 1 30Mb file =130Mb) the total disk usage was 154Mb

I guess it would depend on workflow whether it was useful or not. I have rotating autosaves - and I save a scene every playblast (so there's always a movie that represents the scene). As a first attempt at this. I would .gitignore the autosaves - then commit - the scene file, same filename. Plenty more to think about - what else to ignore - sysadmins always have this problem anyway and end up writing huge exclusion lists. 

Is it worth it? - i have no idea- maybe not. I love the idea of filenames becoming irrelevant ( like Jeff Raskin proposed.) - and branch names gaining importance ;) -

Re: performance:
Branching is instant.
Commits took about twice as long as a maya ascii save on 100Mb file. 50sec commit / for 23sec maya save . Not great, but it would happen in the background and you wouldn't be waiting for it.

Checkouts and switching to an existing branch were significantly faster than commits. ~5sec for 130Mb.

The big benefit I believe is that the version you want to checkout will have a detailed, and instantly accessible description in the commit message - not an arcane coded filename with a number on the end. You will have a record of everything you wanted to say about each commit at the time it was commited. The commit is not just a file, but everything in the repo that worked together to make up that revision.

What about other artists forking your repo - another thread maybe ;)

I think one of the biggest negative points about using git for this, is a problem of perception inherent in the way we work, and not in any SCM. Git models a directed graph -  like this http://github.com/danielharan/integrity/network .  Its complicated but it is what it is and its how people work and collaborate - sometimes you just have to branch off and experiment. Sometimes you want to go back to before the branch. The problem is that artists and producers often don't like looking at DAGs or even admitting that that's what they are working with. People like linear single tracks. I'm pretty sure that's a contributing factor to the endlessness of the debate. If I get time later in the week I'll experiment some more.

cheers


julian