Nash-Sutcliffe or PBIAS?

[Justin Goldstein]

Colleagues,
   I am currently calibrating my model and am getting excellent monthly Nash-Sutcliffe (N-S) values (0.88) and decent/good seasonal N-S on flow (0.6-0.9)!  I have yet to run daily.  However, after initial parameter modifications based on sensitivity analysis and autocalibration, I'm getting even higher N-S values but poorer PBIAS ones.  It seems that N-S and PBIAS are inversely proportional on monthly and seasonal flow in my research.  For instance, my initial N-S for winter flow was 0.64 with a PBIAS of 29.26%.  After running autocalibration on the snowmelt parameters, my N-S shot up to 0.9 but my PBIAS also increased to 30.48%.  My initial N-S for spring was 0.76 with a PBIAS of 1.6%, the autocallibration on snowmelt increased N-S to 0.77 and my PBIAS to 2.2%.  My N for seasonal flow is 14 for winters and 15 for spring (it's lower for winter, defined as winter as Dec/Jan/Feb, because I'm beginning my simulation in January).  I see similar patterns on monthly flow.  My review of the literature indicates that N-S is most commonly used, but according to the following article, the minimization of PBIAS values is also really important:

"

Moriasi, D. N., J. G. Arnold, M. W. Van Liew, R. L. Bingner, R. D. Harmel, and T. L. Veith

(2007), Model evaluation guidelines for systematic quantification of accuracy in watershed simulations, Trans. ASABE, 50(3), 885-900.

  So, I have two questions.
   A) Has anyone seen a PBIAS for seasonal flow that is 30% or more?  This seems a bit high to me but Moriasi et al. (2007) indicates that PBIAS up to 25% on monthly flow is acceptable.
   B) Even more important: would I be correct in assuming that my primary goal should be maxmization of N-S, and then to just use PBIAS if the values fall within an acceptable range (and not work actively to calibrate based on N-S?)

Thanks very much for your insights.
-Justin

[Charles Ikenberry]

Yes, you should really try to balance good NSE and PBIAS statistics.  If you must "compromise" one vs. the other, consider what you'll be doing with the model output.  How would a poorer PBIAS affect the application vs. a poorer NSE?  If you're concerned primarily with runoff vs. low flow periods, which statistic is more important to "optimize."  Is there a range of flows that you're model is performing worse for than others, and how does this affect each statistic?  You might want to run the NSE and PBIAS calculations for certain flow regime (high, medium, low, etc) or for a certain time frame (spring, summer, fall, etc.) depending on how you'll use the predicted flows.  Try to obtain the best statistics for the range of data that may be used to draw conclusions or recommend watershed management practices.

--Charles

---

From: Justin Goldstein <gold...@gmail.com>
To: ArcSWAT <arc...@googlegroups.com>; SWAT-user <swat...@googlegroups.com>
Sent: Monday, October 10, 2011 8:54 AM
Subject: [SWAT-user:3017] Nash-Sutcliffe or PBIAS?

--
You received this message because you are subscribed to the Google Groups "SWAT-user" group.
To post to this group, send email to swat...@googlegroups.com.
To unsubscribe from this group, send email to swatuser+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/swatuser?hl=en.