Software Design Questions

[David Martinez]

Hello,

I'm working on a new script and I'm looking for some advice in relation to Software Design.

The tool that I'm working on allows to import files into the current scene and make changes to the hierarchy. The tool is going to be used by a few different departments and the way files are imported depends both on the department using the tool and the options selected from the interface. Here is a list of the things that I want to accomplish:

Here are some things to keep in mind

Creating classes for Project

Different projects might have different folder structures and the data is not necessarily in the same place for each different project. Because of that, I thought that the best thing to do would be to create a base class in which I define the basic functionality and then. for those projects that have a different structure, I can subclass from the base class and overwrite the methods that need to be tweaked. By doing so, I won't be duplicating code but just only changing the one that I need to change.

Question 1:
Does that sound like the best way to go?

Isolate Functionality

I understand that the best thing to do is isolate functionality and avoid having functions/methods that try and tackle with a bunch of unrelated stuff. Having said that, I'm not sure when to stop breaking things down and I often end up with functions/methods that have simple calls to built-in or third party libraries. When that happens, it is crear to me that I've gone too far.

Question 2:
How do you decide what needs to be broken down and what doesn't?

Allow different UIs

I want to be able to control the same functionality using different UIs (only one would be running at the same time). Because of that, it is clear to me that none of the core functionality should be part of the interface but isolated (see previous point)

This would allow me to have different versions of the UI and or different ways to display the data. This could also be used to load one or the other based on the existence of some libraries on the system. Ideally, I would like to be able to switch between different UIs at runtime.

Again, it looks to me like OOP might be helpful here. The way I see it, I could create a base class that communicates with the main script and then a series of subclasses that, not only display a different interface but overwrite the functionality of those methods when necessary.

Question 3:
Do you think the use of classes is justified here? Would do rather use another approach?

Linking different elements together

I have to admit that I'm having trouble on seeing how to link the different elements together. At the moment I have different classes for the UI, the project and also for the importer itself (given the fact that depending on the department, the import process should be different).

At the moment, I have the UI create an instance of the importer class, since it the department is defined in the interface and the importer will relate on some of the instance attributes of the UI. Having said that, I have the feeling that the instance of the importer should be created somewhere else and only change when the department is changed instead of the UI itself. If that's the case, I'm not sure how I would communicate with the instance variables of the UI.

Question 4:
What do you think it would be the best way to structure the code in this case? If I create an instance of the importer outside of the UI,
should I pass the instance of the UI as an attribute in order to access its class attributes? I want to make sure I'm structuring the code in a way that makes sense.

Thanks a lot in advance,

Cheers,

[Justin Israel]

Hey David....

On Fri, May 9, 2014 at 8:24 PM, David Martinez <david.mar...@gmail.com> wrote:

Hello,

I'm working on a new script and I'm looking for some advice in relation to Software Design.

The tool that I'm working on allows to import files into the current scene and make changes to the hierarchy. The tool is going to be used by a few different departments and the way files are imported depends both on the department using the tool and the options selected from the interface. Here is a list of the things that I want to accomplish:

Here are some things to keep in mind

Creating classes for Project

Different projects might have different folder structures and the data is not necessarily in the same place for each different project. Because of that, I thought that the best thing to do would be to create a base class in which I define the basic functionality and then. for those projects that have a different structure, I can subclass from the base class and overwrite the methods that need to be tweaked. By doing so, I won't be duplicating code but just only changing the one that I need to change.

Question 1:
Does that sound like the best way to go?

It may not be the best approach to try and describe each new project in terms of a subclass. Unless I misunderstood you and your subclasses are just "variations" of a "type" of project. What might be more flexbile is to create a concrete project class that has a common interface, but can derive its locations and properties from a template. Something that allows you to whip up a new version of a textual template when there is a new project, and nothing in your codebase has to be changed. This template might indicate where logs go, or where image outputs go, or maya scenes, etc.  

But it sounds like the right idea in terms of your base class, to have a consistent interface being provided. That base class might end up just being the concrete class. 

 

Isolate Functionality

I understand that the best thing to do is isolate functionality and avoid having functions/methods that try and tackle with a bunch of unrelated stuff. Having said that, I'm not sure when to stop breaking things down and I often end up with functions/methods that have simple calls to built-in or third party libraries. When that happens, it is crear to me that I've gone too far.

Question 2:
How do you decide what needs to be broken down and what doesn't?

I read a book on refactoring once (I don't remember the name, sorry. I am also a film major, so I have only ready *one* book on refactoring).

It mentioned something about refactoring after you begin to repeat yourself 3 or more times. There is generally a basic concept you want to follow of separating concerns, and having a class serve one and only one purpose.  But that doesn't mean you always have to worry so much about how much to separate upfront. That is the point of refactoring. When a single class or function starts looking like it is doing more than its targeted purpose... you can split it up. 

 

Allow different UIs

I want to be able to control the same functionality using different UIs (only one would be running at the same time). Because of that, it is clear to me that none of the core functionality should be part of the interface but isolated (see previous point)

This would allow me to have different versions of the UI and or different ways to display the data. This could also be used to load one or the other based on the existence of some libraries on the system. Ideally, I would like to be able to switch between different UIs at runtime.

Again, it looks to me like OOP might be helpful here. The way I see it, I could create a base class that communicates with the main script and then a series of subclasses that, not only display a different interface but overwrite the functionality of those methods when necessary.

Question 3:
Do you think the use of classes is justified here? Would do rather use another approach?

OOP does not always == best. It is just a design concept. OOP aside, you definitely have the right idea that it is beneficial to separate your business logic from your display logic. One library should solely focus on providing that layer of representing your core concepts like loading, parsing, opening, saving,... Then you can have any number of views that wrap around this logic. This means you can also have command line tools that build on the same core library. When you want to then change the way something is loaded, you only have one place to change it. Your display code shouldn't care at all how the business logic works. It should only worry about how to communicate between the business logic and the user inputs, and how to display the available controls and data.  

 

Linking different elements together

I have to admit that I'm having trouble on seeing how to link the different elements together. At the moment I have different classes for the UI, the project and also for the importer itself (given the fact that depending on the department, the import process should be different).

At the moment, I have the UI create an instance of the importer class, since it the department is defined in the interface and the importer will relate on some of the instance attributes of the UI. Having said that, I have the feeling that the instance of the importer should be created somewhere else and only change when the department is changed instead of the UI itself. If that's the case, I'm not sure how I would communicate with the instance variables of the UI.

Question 4:
What do you think it would be the best way to structure the code in this case? If I create an instance of the importer outside of the UI,
should I pass the instance of the UI as an attribute in order to access its class attributes? I want to make sure I'm structuring the code in a way that makes sense.

I would think the importer should have no knowledge of UI concepts. It might help (and this might sound related to the previous question) if you think of them in completely different pieces. How can the importer stand alone? How can it function without your UI, given only parameters. Also, it may be beneficial to avoid defining department stuff in code unless it is a modular type system and not something part of the core. That way you don't have to update core logic when department details change or you need to create the concept of a new department. This may go back to templates or configuration files that are loaded into an interface. Forgive me though if my answer sounds vague. It is possible you could need to create a few interfaces which you are referring to as departments. 

I'm not sure how to fully answer this one, but ultimately I know it involves thinking about them as completely separate pieces. 

 

Thanks a lot in advance,

Cheers,

Hope this helps a little bit. 

 

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAMLeNpz%2BWStu%2B8yp6s2LWaY1JtJJ4_fjrU1agGruUFY18xv_Zw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Marcus Ottosson]

Software design is a big topic, is it ok if I refer you to some books instead of answering them here?

Based on your questions, here is what I'd suggest, in this order:

1. Code Complete
   
2. Domain-driven design
   
3. API Design C++
   
4. Game Engine Architecture
   
5. Physically Based Rendering

Code Complete

Give this book a quick glance (its rather large), it should prepare you for what lies ahead.

Domain-driven design

Consider adopting this thinking to your object models.

API Design for C++

Not specifically for Python, but provides some very good methodologies on dividing interface from implementation.

Game Engine Architecture

To wet your appetite, this one might open your eyes as to how *big* a piece of software can get.

Physically Based Rendering

Whereas this one might open your eyes as to how a *complex* piece of software can look.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA34h7C7h7ccOTgt0bB04Mad7wTEAPyzGBxp_gNqF0X%2Baw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Erkan Özgür Yılmaz]

You might be interested in:

https://pythonhosted.org/stalker/tutorial.html

At least it will help you to start from some where. And I should start ptomoting it as it is pretty mature right now.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOBxKr5Wfiqg8iMWNHESEgg7KVu7e2BWnqz%3DjQbTz%2BcCrQ%40mail.gmail.com.

[Marcus Ottosson]

Ah! I knew I recognised your name. Good work with Stalker, got it recommended to me by a friend of mine a while back. I may just have another look. :)

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAGNmyx7kLeZr%2B0M%3Dgv%2BoQ6%2BbSb%2BEmfynOQxJPcEvH2Zahe29MA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Justin Israel]

I'm surprised Stalker didn't come up in the other thread about path structures and parsing (regex/schema/pyparsing/...). Seems pretty relevant! I also just recognized your name as having been associated with Stalker, when I came across it a few years back. 

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOD8AqQjExcJrwdnnysnPh5w6_AXLTyybWztF0Hx%2BRRPQg%40mail.gmail.com.

[Erkan Özgür Yılmaz]

Hey Justin,

I should start promoting Stalker more, it was my little baby for the last 4-5 years or in a much meaner perspective it was all my brain farts structured and ordered in a way that may be helpful to others, and I think it is time for letting it to fly and start its own journey.

E.Ozgur Yilmaz
eoyilmaz.blogspot.com

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA0BqEUdhT81%2BRSguRQ6WEnkFmYZdb%2BW%3Dr6%3D8s7sF5vbxQ%40mail.gmail.com.

[LIJU kunnummal]

Hi David,

    Creating classes for Project

       

     Maintaining different classes for different project is not good idea as justin said. I would recommend creating config files(eg:xml) for each projects, these config describes all the information about the project like project name, different paths, mount drive, artist working on this projects and even folder structure etc. You can write generic classes which will use this config to define your project and create the folder structures too. Even you can create a tool to create this config (and related files) which can be handled by any user(producers, supervisors etc) to setup a project. This way there no need to rewrite codes for every project.

Cheers

[Justin Israel]

Well maybe not the artist names in there since that is something that usually changes and is handled in the project tracking system.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/961aae65-8b60-4248-9c15-41864e466c17%40googlegroups.com.

[Nils Lerin]

That's incredible Erkan! I've been planning on developing something very similar to stalker for some time now but seeing how stalker seems to do exactly what I wanted I might continue from there. You don't happen to have a video or screenshot demonstrating the Anima interfaces?

/Nils

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAGNmyx7twouZ%2BU33NXXnKvoosjrUPrHHfE8X%2BNXDVWSOfo8Nrg%40mail.gmail.com.

[Nils Lerin]

Oops, just noticed there was a new thread dedicated to stalker

st

[David Martinez]

Hey,

@Marcus: Thanks for the book recommendations! Ironically enough, I'm currently reading Code Complete.   :-)
@Erkan: I have to say that Stalker looks fantastic and worth looing into at some point!

@Justin / @LIJU: Thanks a lot for sharing your thoughts.

Even though that the usage of something like Stalker sounds like the best way to go, I'd like to get some feedback to see the places where I'm going wrong and why. Here are some additional thoughts about the original idea:

Projects

The only reason why I suggested OPP is that in this case, they are indeed 'types' of the same kind of project. The differences between them are pretty small and the rest of the functionality (methods) and data (variables) should remain the same for all projects/types. All what I intend to do is cater for those small exceptions.

In addition to that, most of the tools we have are used in all of those 'types of projects'. At the moment they rely on hard coded paths which is a problem because every time a path changes in a project, we need to add an exception within the code and it's not ideal.

The current base class for Project looks something like this. Keep in mind that this is only an example, The real class has way more functionality and data. And it does more than returning instance variables. (get files, performs checks, reads data etc):

https://gist.github.com/davidmartinezanim/6709ff6af1e60cb71759

After reading your emails, I can see the advantage of using a files to store most of the data for the projects. As you say, that would free the code from any changes and the only thing that would need changing would be text file. I have a couple of questions though:

Question 1:

@Justin: In your email, you mention the use of 'textual templates'. I'm afraid that I haven't heard that term before. Could you explain what it means?

Question 2:
In the past, I've been using XML to save some basic preferences files but since I discovered JSON, that's all I use. Which one would you use in this case? XML or JSON? Are there any reasons to pick one or the other?

Different UIs

I'd like to clarify something in regards of the UI.

Different departments don't have a different interface for the the loader. The interface is pretty much the same but some departments have additional options (check boxes, buttons, etc). Instead an re-writing the new interface with just a couple of additions, I wanted to use OOP to get the shared structure and only add those elements where needed. (and have all of them share the same interface)

Question 3: Is this still a bad idea?

Linking different elements together

Looks like the best thing to do is to create the loader as a separate element and pass the parameters across. Since some departments require more info that others, I suppose that the loader should set some default values and have the ability to apply any changes to the variables through the parameters, right?

Thanks again

--
David Martinez - Technical Animator

 

Email: david.mar...@gmail.com

Website: http://www.elusiveideas.com

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAGkBUFfj1sKc%3DEKVYUJeGVwc0m3ex9ATrJRor5mhtg3v3KEezw%40mail.gmail.com.

[Justin Israel]

On Sun, May 11, 2014 at 7:38 PM, David Martinez <david.mar...@gmail.com> wrote:

Hey,

@Marcus: Thanks for the book recommendations! Ironically enough, I'm currently reading Code Complete.   :-)
@Erkan: I have to say that Stalker looks fantastic and worth looing into at some point!

@Justin / @LIJU: Thanks a lot for sharing your thoughts.

Even though that the usage of something like Stalker sounds like the best way to go, I'd like to get some feedback to see the places where I'm going wrong and why. Here are some additional thoughts about the original idea:

Projects

The only reason why I suggested OPP is that in this case, they are indeed 'types' of the same kind of project. The differences between them are pretty small and the rest of the functionality (methods) and data (variables) should remain the same for all projects/types. All what I intend to do is cater for those small exceptions.

In addition to that, most of the tools we have are used in all of those 'types of projects'. At the moment they rely on hard coded paths which is a problem because every time a path changes in a project, we need to add an exception within the code and it's not ideal.

The current base class for Project looks something like this. Keep in mind that this is only an example, The real class has way more functionality and data. And it does more than returning instance variables. (get files, performs checks, reads data etc):

https://gist.github.com/davidmartinezanim/6709ff6af1e60cb71759

After reading your emails, I can see the advantage of using a files to store most of the data for the projects. As you say, that would free the code from any changes and the only thing that would need changing would be text file. I have a couple of questions though:

Question 1:

@Justin: In your email, you mention the use of 'textual templates'. I'm afraid that I haven't heard that term before. Could you explain what it means?

I did actually mean it would be fine to create subclasses for types of projects, like "Feature", "Commercial", "...". These would be interfaces. But the concrete details for a given feature project, or a given commercial might be better described from a config file. Then the class just loads in the settings to know where paths should go, file types, etc. I was throwing out the term "texual templates" kind of loosely. I meant it would be a human readable configuration template that is parsed, as opposed to describing your specific project in code.

 

Question 2:
In the past, I've been using XML to save some basic preferences files but since I discovered JSON, that's all I use. Which one would you use in this case? XML or JSON? Are there any reasons to pick one or the other?

XML is kind of a dated format for configuration files now. INI, JSON or even YAML are a lot more common and easier to read. INI isn't very flexible but is probably the most forgiving in terms of syntax. JSON has the best balance of performance between INI and YAML, and you just have to get the syntax valid, as one character can break the entire parsing. YAML is a superset of JSON and has a ton of fancy formatting and the ability to self reference. Like, you can define something once and then refer to it later in other parts of the file. 

 

Different UIs

I'd like to clarify something in regards of the UI.

Different departments don't have a different interface for the the loader. The interface is pretty much the same but some departments have additional options (check boxes, buttons, etc). Instead an re-writing the new interface with just a couple of additions, I wanted to use OOP to get the shared structure and only add those elements where needed. (and have all of them share the same interface)

Question 3: Is this still a bad idea?

Because departments don't really change often, this could be a candidate for defining your properties as department modules/classes that get loaded in and provide the specific options to be added. Or you could again go the route of config files, but that would mean you are taking the extra step of needing to have a "declarative" UI. So ya probably best to just keep it simple and have them dynamically pick the matching department class to provide the options. I've done stuff like this for defining properties that surround different types in my system. 

 

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAMLeNpxSm_ztAgBBS6rLKyJ%3DLj9jNWiN17cVhnK7q5WPhreYMA%40mail.gmail.com.

[David Martinez]

Thanks again Justin,

The only type of project that we have at the moment is 'Game'. That means that even though that we have several different  projects, all of them are of the same type,

If I understood what you meant, you are suggesting to create either a class or a method that contains the interface for projects that would need to be fed by the contents of a file that contains the specific details for each project so we can still change the details without messing the code. Is that correct?

--
David Martinez - Technical Animator

 

Email: david.mar...@gmail.com

Website: http://www.elusiveideas.com

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA3TrMAYij%3DvnsoG43f81Pdmt4%3DuDvNr4rXu%3D%3DMdHttBqA%40mail.gmail.com.

[Marcus Ottosson]

David, that is excellent, then it sounds like you're on the right path. :)

The only reason why I suggested OPP is that in this case, they are indeed ‘types’ of the same kind of project. The differences between them are pretty small and the rest of the functionality (methods) and data (variables) should remain the same for all projects/types. All what I intend to do is cater for those small exceptions.

Defining content as hierarchy of objects is called Data Modelling and is very similar to Class Modelling, but differ in some important ways and I’d suggest understanding them both in your case.

The difference is, while Class Modelling defines a hierarchy of behaviour and responsibility, Data Modelling defines a hierarchy of information, often in terms of simplified concepts.

E.g. a Map is a data model of complex terrain, a Line object may represent a road and a Square a building. Another data model is our use of schemas (can’t believe this term has showed up three days in a row now :) in which we define a “terrain” of film concepts - e.g. film/sequence/shot.

E.g. you could define a Url with Class Modeling as consisting of a Scheme, Host and Path; Path possibly being sub-classed based on domain - e.g. for Windows versus Posix paths.

With Data Modelling and OOP, I would suggest thinking more in terms of composition rather than inheritance - a “has-a” relationship between objects, as opposed to a “is-a”. You could then move on to defining behaviour as separate objects more fit for Class Modelling that operate on your data-modelled objects. (i.e. responsibility-driven design).

What you end up with is loosely-coupled data objects and loosely-coupled behavioural objects, orchestrated in each of your applications.