Realism vs KISS

[Alan Isaac]

Use this thread to discuss the trade-offs between realism and KISS (and related methodological issues).  This is also the right place to discuss the required Coen article.

[Natalie Chambers]

The tradeoffs between KISS and realism are important when looking at the output one wishes to generate. As discussed by Coen: The “simple” camp places it emphases closer to the work of formal theorizing. Its sparse, more mathematical simplicity is consistent with the “universal” goal of formal theorizing. Closer to field studies, the “realistic” camp explores more particularistic behavior systems. According to McGrath’s model, realistic models, like field studies, require more observation of behavior. 

This strategic and important differentiation helps us evaluate and better understand when to use simple vs realism models or when it is best to model somewhere between the two extremes. Realistic models use complexity to their advantage to demonstrate that they have not omitted any important variables that you would find in a real situation. This is important of models because a primary critique of any model is that it has omitted variable bias. By incorporating more inputs it is easier to prevent having biased results and avoiding this type of critique. However, just as the quote above says if the intent of the model is to create general theories then simplicity is often better. Guns and butter is the most simple economic model and is taught in almost every introductory economics course. Of course one can argue that it is too simple and lacks important inputs but the point of this model is not to model the behaviors of economies for everyday transactions so a simple model is actually better in this case. It makes sense that the models attempt to achieve different outcomes. 

A well-designed simple model claims to include all the essential features of a process in the world.  

This statement makes me think of the prisoner´s dilemma. A simple model that is used in countless examples across various disciplines to explain real world situations is a well designed simple model that also perceived well in the realism camp. The prisoner´s dilemma (most of you probably know it) but I think it illustrates the medium between the KISS and realism quite well. 

One concern I have with Coen´s discussion is her lack distinction between the models and outcomes both for realism and simple by doing so she automatically requires the reader to group outcomes and models within the same categories. I think the distinction in complexity between inputs and outcomes is important to understand how we present models to our audience. 

For economics it is important to simplify results for people to understand and interpret however it doesn´t mean that the model or evaluation has to be simplified just that the analysis needs to be simplified for people to understand. Like with any scientific discipline it is important to be able to explain one´s findings in a simple manner so that they are useful for a wide audience (this is particularly important if your audience is far removed from your field; ex. medical scientists and their patients, patients need to know and understand (to some degree) what they are prescribed and what that will do for them but they don´t need to understand all of the experiments and tests that went into creating it.)

[Kevin Carrig]

I would agree with Natalie and her perspective on the differences between KISS and realism based models. When creating a model, the decision between the two most often depends on the desired outcome of the simulation. If we hope to extrapolate the results of the models to a larger, macro-level dynamic, the simplicity of a KISS model lends itself to more general outcomes and representations of the real-world.However, I'm not sure that simplicity is completely analogous to a model with relatively few inputs. Moreover, I think simplicity is a function of the clarity of the results and how easily we can discern the interactions between the inputs. 

It seems as if the complexity of a model stems from the interactions of the inputs and the difficulty of assessing causality in the model. For me, the trade off between additional inputs (or a more refined model) and a more general models lies in ease of interpretation. To establish causality in a large, macro system, we can represent complex systems as a function of several, smaller and simpler models. Alternatively, we could utilize some forward-step method where we gradually introduce inputs and assess the effect of that input on the dependent outcome as well as the values of the other independent variables. 

[Amanda Saville]

I agree with both Natalie and Kevin's discussion on the differences between the two camps in the sense that the choice of approach depends on the desired outcome.  While Natalie raises a good point about the ability of a modeler to defend a realistic model after covering all (or most) possible variables, I think Kevin is right in his suggestion that it may be more valuable to gradually introduce inputs, or to create a combination of models, to evaluate the significance of each variable rather than present an all-encompassing model.  Even in the case of a field study, a realistic model may not have the same power as a simpler model--not only for the ease with which a researcher could explain the model to his/her audience, but also for a lack of parsimony.

While it is clear, and I think the general consensus in this thread thus far agrees with Coen, that for more general theories a simpler model is better, I am not convinced that even in situations of studies that are more case-specific that a realistic model is desirable.  Assuming that the researcher will also be producing a qualitative discussion of his/her research that will reach some kind of conclusion, it would seem to me that it would be more desirable to have a simpler model that is more robust than an over-complicated model that essentially leaves too many pots boiling on the stove.  If a researcher is attempting to answer a question that will call for some policy action on the part of a government, for example, a realistic model may still not support any significant conclusion.

Additionally, Kevin touched on an important point by saying, "It seems as if the complexity of a model stems from the interactions of the inputs and the difficulty of assessing causality in the model..."  Thus a model's complexity does not depend on the over-complication of inputs, but rather then interaction of those inputs and the significant impact of each input on the final outcome.

[Matthew Reardon]

Although Coen discusses realism vs KISS models for the "relatively new discipline of computational organization studies", this has a popular debate in economics.  Practically all models presented in your econ classes would fall under the "keep it simple" group.  One of the most popular critiques I hear in class (and I'm sure all of you have heard) is that a simplified version cannot possibly capture the complexity of the true relationship. I think Natalie's reasoning was correct; the reasoning and results have to be understood by people in a variety of fields to have a widespread impact.  As Kevin pointed out, the usefulness of additional inputs depend on the ease of interpretation.  To me, this is best explained by the quote from Natalie's post,  "A well-designed simple model claims to include all the essential features of a process..."

 I think the benefits between KISS and realism depend on the initial objective.  For me, simplified models are best used to present underlying frameworks or foundations.  These models are usually built with the purpose of future expansion; similar to open-source software.  Users can adapt or incorporate additional information "easily" for their own respective use.  Realism would therefore be appropriate for modeling specific real-world outcomes, not the underlying framework.  As Coen pointed out, simple rules can generate complex outcomes. 

Finally, I'd like to expand on the omitted variable bias mentioned by Natalie.  It is definitely true that the primary critique of a model is the omitted variable problem, it is important to distinguish the actual cause of bias.  Simply incorporating more variables is not guaranteed to reduce the bias.  OVB stems from leaving out important causal variables.  When using regressions, OVB occurs when the missing variable is a significant determinant AND correlated with at least one of the included variables (for linear regression, this indicates that there is currently a correlation between the error term and the included predictors).  This is an important distinction to make when presented with the OVB critique, since not all missing variables cause a bias.  If anybody has been to a conference where papers are presented, during the Q&A someone always suggest an alternative measurement or additional variable.  This is usually met with one of two general responses: "hhmm that's interesting" or "we tried but it had no impact on results."  This circles back to the quote mentioned earlier; the essential features are included.

[Matthew Reardon]

I came across a quote when studying for my advanced theory comp that falls in line with the benefits we've discussed of a simple model.  Hein and Stockhammer (A Modern Guide to Keynesian Macroeconomics and Economic Policies) present a simple post-keynesian macro model to model interactions between interest rates, demand, unemployment and inflation.  Their intro contains a line explaining why they chose a simple model.  

"We will offer a highly stylised model that does not claim realism in assumptions but rather serves to highlight key differences in simple PK and NCM models."  (pg 113)

The author's stated purpose is to present an alternative to NCM models and highlight the key differences in policy packages derived from both.  This supports the comments made by Natalie, Kevin, Amanda and myself that the tradeoffs between the model "class" depends on the objective.  Here, the authors want a straightforward comparison between models, which is why they chose to keep it simple. 

[Mia Raths]

I agree with the consensus of the definitions between the KISS and the realism methods of modelling. I think we covered the most important points of each model.

While I was reading this, I kept thinking of the two different methods in terms of processing and logic.  What I mean by this is that I consider the KISS method of modelling to be deductive, or top-down processing.  A large amount of information can be used to describe a smaller event or idea.  In the KISS method, a very general , simple model, can be expanded in order to describe a specific model that modellers want to look into.  By adding particular variables, or slightly changing the environment in several ways, one can model many different situations from one general model.

The realism camp of modellers take a bottom-up, inductive form of logic.  This means that a from a complex model, one can derive, induce, the actions of a more general model.  If several similar complex models bring about similar results, modellers will be led to believe that the same result will come from other similar cases.

This being said, the next question I pondered was which is the better way to come to a conclusion, or are they both useful.  I do believe that both methods have benefits, but I also believe that the realism model may have more costs.  A simple model is a great way to start.  Looking at the skeleton model, it will be easier to determine the basic agents, actions and results.  Tweaking these even slightly will create the exact model that the modeller is looking to recreate.  A database of simple models would be a terrific resource for all other modellers to have access to in order build off of.  The only negative is that the model, as it is, may need more work in order to clearly display the appropriate outcome of a more detailed model.

Models made by modeller from the realism camp are very useful because whatever the case trying to be recreated, all the components necessary for that particular model are there, leaving very little dispute of the outcome.  Unfortunately they cannot describe any other models the way they are designed.  It is also much more difficult to remove extra details and components of a realistic model and still maintain its integrity, than it is to add more variables to a model made by those in the KISS camp.  It easier to construct than to deconstrruct a model, so beginning with a  simple one and then expanding upon it, the KISS model, seems to be a better bet.

[Alan G Isaac]

**Database of Models**

On 5/24/2014 2:13 PM, Mia Raths via AU-ECON-ABS wrote:
> A database of simple models would be a terrific resource for all other modellers to have access to in order build off of.

This is quite right and widely recognized. It is why NetLogo ships with an extensive Models Library, which I hope you all are digging into. These
models have the advantage that they have been carefully simplified and vetted, so they have great pedagogical utility. In addition, you can find over
1000 NetLogo models in the Modeling Commons: http://modelingcommons.org; but be careful, these are not vetted and are of varying quality.

Cheers,
Alan Isaac