Re: Baseflow calibration using SWAT CUP

[Karim Abbaspour]

Here is my reply. Please write to swatcup google group for more help.

Best, Karim

1. The files 'observed+.txt' and ''observed.txt' have the same data for the example projects, by this I mean that in observed.txt we input 'total flow' so similarly when we go ahead for dynamic baseflow separation do we input total flow or surface flow? ie. column 3 is total flow and column 4 is base flow or columns 3 and 4 are surface and baseflow respectively? 

- If you use the dynamic flow separation option, you need to use a separation program to separate the flow into base and non-base flows. These are then treated as two variables and with proper weighing, you can optimize both of them with equal weights and achieve a better estimate for both of them. The format of observed+.txt should then be used. That means your observed.txt file should have an extra column as shown in observed+.txt, column 4 is the baseflow column (but you should call this file observed.txt).

2.The user manual states that in Echo/Echo_goal.txt at the last line contribution of each variable to objective function is given but this is the value we need to input in 'observed.txt' before we start any iteration.i have attached a trial run i got, it does not seem to separate the flow into components using the NSE as objective function.

- There are formulas to calculate the weights (see manual). But if you want to have exactly the same contribution you need to run without weights once, and see the proportion of contrition in the echo file, then you can exactly calculate the weights as you want. For NS, it seems to work for me.

3. When performing dynamic flow separation "observed_rch.txt" does this also take the separated flow components from question 1. above or just the Total flow. 

- observed_rch.txt is only used to extract variable from the rch file. As swat does not output baseflow, this is unchanged. You could suggest to swat people that they should include flow separation in swat.

4.  In the manual it says when the var-threshold is set to a negative number and weights for the upper and lower threshold to 1  then multi-component assignment is zero. So the weights have to be such that the sum of the upper and lower equals 1? eg. 0.6 for upper and 0.4 for lower? the "observed+.txt" file shows 1 for both as an example.

- If flow separation is not considered, then weights are irrelevant. If considered, then weights can be anything you want. They don't have to add up to 1

5. On page 55 of the manual "var_Threshold= is a threshold where a signal is divided into two parts. We refer to this as a “multi-component” assignment (see Abbaspour et al., 2004). Values smaller than the threshold and values larger than the threshold are treated as two variables. This is to ensure that, for example, base flow has the same values as the peak flows. If you choose option 2 for objective function, i.e., mean square error, then base flow may not have much effect on the optimization, hence, peak flow will dominate the processes. With this option they can be given the same weight. This option is most effective for option 2 of objective function and is not defined for R2 and bR2."  

The bold sentences seem to contradict each other and thus i am confused as to the appropriate objective function that would include the effect of base flow in the optimization.  Could you please clarify?

- In the first highlight I mean if you don't choose weights then that is what happens. Weights are effective if you choose option 2,5,8,9,10

 Best, Karim

On Monday, March 30, 2015 6:44 PM, Eeshan Kumar <ish.a...@gmail.com> wrote:

Greetings Dr.Abbaspour,

Could you please provide answers to queries related to baseflow calibration as mentioned in the previous email?

Thank you for your time.

Sincerely,

Eeshan

On 19 March 2015 at 14:24, Eeshan Kumar <ish.a...@gmail.com> wrote:

Greetings Dr.Abbaspour,

Thank you for the clarification, however i still have some doubts in relation dynamic baseflow separation, could you please clarify the following questions?

1. The files 'observed+.txt' and ''observed.txt' have the same data for the example projects, by this I mean that in observed.txt we input 'total flow' so similarly when we go ahead for dynamic baseflow separation do we input total flow or surface flow? ie. column 3 is total flow and column 4 is base flow or columns 3 and 4 are surface and baseflow respectively? 

2.The user manual states that in Echo/Echo_goal.txt at the last line contribution of each variable to objective function is given but this is the value we need to input in 'observed.txt' before we start any iteration.i have attached a trial run i got, it does not seem to separate the flow into components using the NSE as objective function.

3. When performing dynamic flow separation "observed_rch.txt" does this also take the separated flow components from question 1. above or just the Total flow. 

4.  In the manual it says when the var-threshold is set to a negative number and weights for the upper and lower threshold to 1  then multi-component assignment is zero. So the weights have to be such that the sum of the upper and lower equals 1? eg. 0.6 for upper and 0.4 for lower? the "observed+.txt" file shows 1 for both as an example.

5. On page 55 of the manual "var_Threshold= is a threshold where a signal is divided into two parts. We refer to this as a “multi-component” assignment (see Abbaspour et al., 2004). Values smaller than the threshold and values larger than the threshold are treated as two variables. This is to ensure that, for example, base flow has the same values as the peak flows. If you choose option 2 for objective function, i.e., mean square error, then base flow may not have much effect on the optimization, hence, peak flow will dominate the processes. With this option they can be given the same weight. This option is most effective for option 2 of objective function and is not defined for R2 and bR2."  

The bold sentences seem to contradict each other and thus i am confused as to the appropriate objective function that would include the effect of base flow in the optimization.  Could you please clarify?

Thank you for your time in advance.

Sincerely,

Eeshan

On 2 March 2015 at 12:04, Karim <k_abb...@yahoo.com> wrote:

You need to give to the swatcup both components of flow and the optimization program will try to find a solution that best fits the two components. The file observed+.txt in SUFI2.IN directory shows the format.

Best,

Karim

On Feb 27, 2015, at 19:11, Eeshan Kumar <ish.a...@gmail.com> wrote:

Greetings Dr.Abbaspour,

I am a graduate student at the University of Arkansas, United States and I am trying to calibrate baseflow using SWAT CUP. I had read your earlier post in the SWAT CUP Google gro​up (https://groups.google.com/forum/#!searchin/swat-cup/gurdeep$20singh/swat-cup/wRMe8sRBmuQ/7P9Zga0wX2IJ​) where you had mentioned that a baseflow extraction program could be written to extract baseflow. I have my observed baseflow and a program (baseflow.exe) that calculates the baseflow from the output.sub file. Since it is SUFI2_run.bat that calls SUFI2_execute.exe that in turn calls SUFI2_makeinput.exe, SWAT_Edit.exe, SWAT.exe and SUFI2_extract_rch.exe based on inputs from file par_inf.txt, par_val.txt, SUFI2_swEdit.def (please correct if there is a mistake), now when I am trying to integrate my baseflow program (baseflow.exe) in the above algorithm I think that I am missing something as to where the baseflow program should be called so that after each simulation the baseflow is calculated from output.sub file. Currently, my baseflow program is only able to calculate for the last simulation number since it is not able to enter the loop of SUFI2_execute.exe program.

Could you please help on this?

Thank you for your time.

Sincerely,

Eeshan

--

Eeshan Kumar

Graduate Research Assistant

Dept. of Bio. & Agri. Engg.

<image001.png>

+1(479) 6846282 (M)

No Breath, No Life. Know Breath, Know Life.

~Sri Sri Ravi Shankar

--

Eeshan Kumar

Graduate Research Assistant

Dept. of Bio. & Agri. Engg.

+1(479) 684-6282 (M)

"Celebrate life. Care for others and share whatever you have with those less fortunate than you. Broaden your vision, for the whole world belongs to you." - Sri Sri Ravi Shankar

--

Eeshan Kumar

Graduate Research Assistant

Dept. of Bio. & Agri. Engg.

+1(479) 684-6282 (M)

"Celebrate life. Care for others and share whatever you have with those less fortunate than you. Broaden your vision, for the whole world belongs to you." - Sri Sri Ravi Shankar

[kardosmate]

Dear All and especially dear Mr Abbaspour,

the calibration for baseflow is not really clear for me, despite reading most of the manual and the refererenced papers. SWAT, as also mentioned in the cited mail, doesn't print baseflow and lateral flow in the output.rch file. In contrary, it prints them in the output.sub file. In order to get a baseflow time series, all the subbasins upstream to the gage have to screened from output.sub, their area-weighted mean has to be calculated (+ additional unit conversion from mm to m3/s). Does SWAT-CUP do this? If not, what does it consider as "calculated baseflow" when doing a calibration?

Anyway, why do I get the attached error after adding the 4th column to observed.txt and observed_rch.txt?

(Btw, the observed+.txt referred to in the manual can't be found. I suppose it's name changed to observed_dyn_flow_exple.txt.)

Thanks in advance for paying attention!

BR

 Máté Kardos from Budapest, Hungary