SWAT+ Toolbox (invitation to testing)

[Celray James CHAWANDA]

Dear all,

I have been working on new software to assist in SWAT+ modelling (run model, error checking, sensitivity analysis, calibration and evaluation, model).

If you would like to test it, follow these links with a description of the tool and instructions on how to install and use it: Part 1 and Part 2. Please remember to provide feedback (impressions, bugs, feature requests e.t.c.)

Celray James

[Dennis Trolle]

Hi James,

 

I just ran a MSc class that tested the SWAT+ Toolbox. They followed your two videos and were all able to run a successful calibration for their own case. Generally, it worked really well. Here is just a short report back:

 

All the students could make the software crash if they deleted the text/number at “Max Iterations” under the calibration tab. This would cause the software to crash and shut down. As a workaround, they found that if they highlighted the existing text/number, and typed their own value on top of this, the software would work and calibration could be run just fine.

 

Also, there seems to be some issues with the integration in the QSWAT+ plugin, as on some computers, the “Step 4. Visualize” would not work anymore, after installing SWAT+ Toolbox (a re-install of the QSWAT plugin fixed this problem).

 

We have been working with SWAT+ in-class for some weeks now, and we focus on both hydrology and eutrophication aspects. Therefore, if would be great if the various phosphorus species transported in channels/streams were also to be included as an option for an observed variable – and ideally that you could turn ON several different observations (e.g. with user-defined weights) when calculating a multi-objective function during calibration.

 

Really nice job – I am sure this will make a big difference to the SWAT+ community.

 

Best

Dennis    

[Celray James CHAWANDA]

Thank you so much for the feedback, Dennis.

I am very happy to hear that you were able to use SWAT+ Toolbox in class successfully.

I have fixed the “Max Iterations”bug for the next version. I am going to take a closer look at the integration because currently that only works if QSWAT+ is installed using recommended settings (not 'installed for everyone' on the PC). I think the best way is to have the integration done in the original QSWAT+ code. I will see how I can work with Chris George on this.

Phosphorus observation will be available in the next version (0.4.7) and multi-objective calibration is scheduled for v0.5.0.

With Kind Regards,

James

[Marcel Pereira de Andrade]

Hello James,

First of all congratulations for your work.

I'm testing  SWAT+ Toolbox for my research. I am getting this error message in the sensitivity analysis step: "Could not read the file par_sample.stb. Make sure it exists."

Can you help me?

Kind Regards,

Marcel

[Celray James]

Dear Marcel,

 

A few students have reported this error. However, when they tried again, it disapeared. Please change the number of samples for sensitivity analysis (make sure you have at least 40 samples for two parameters). In the meanwhile, I am investigating why this error is happening, as such I would appreciate if you gave me your txtinout and instructions on how I can reproduce the error so that I can fix it. Let me know how it goes and how I can help further. Email me link to your zipped txtinout using: swatplus_celrayjames[at]outlook.com

 

           With Kind Regards,

           Celray James CHAWANDA

--
You received this message because you are subscribed to the Google Groups "SWAT+" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swatplus+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/f989b636-76ef-444c-8008-78954a4e6555n%40googlegroups.com.

 

[Jim Han]

Hi, James,

Thank you for creating such a great tool. 

I was trying to test the new tool today. After installation, I cannot enter the tool. I followed the steps in the video by clicking Step 4 "SWAT+ Toolbox" and hit okay to create a project. Then I click on Step 4 again and nothing pops up.

I also tried to start from SWAT+ Toolbox directly, click the project name, hit YES to load a new SWAT+ Toolbox project, and then the interface closed/disappeared/crashed by itself. 

I then uninstalled and re-installed everything. Still not working. Has anybody reported similar issues? (P.s., I tried QGIS 3.4, 3.10 and 3.16, neither works).

Let me know if I can provide more details/files.  Thank you.

--Jim

[Celray James CHAWANDA]

Apologies for the delay.

I think the project file has a problem, try deleting the project file and creating a new SWAT+ Toolbox Project. I have recently noticed that this happens only if SWAT+ Toolbox cannot validate the project file and instead of alerting the user of any problem it finds, it just exits. I have put this on my to-do list and I will address it in the next release.

If, after recreating the project, SWAT+ Toolbox still exits, let me know so that we can schedule a zoom call to get information on how I can fix any bugs we discover.

With Kind Regards,

Celray James CHAWANDA

[Jim Han]

James,

Thanks for the response, but it still exists after re-creating the project. 

--Jim

[Sam H]

Hi James,

It looks like a great program. I'm testing in right now and at first I got this error:  "Could not read the file par_sample.stb. Make sure it exists."

I exited the program and tried the sensitivy analysis with fewer parameters and more iterations and now I get a new error:  "an error occured while running SWAT+ -Make sure output files are not open. Try running the model executable in TXTInOut.

No output files are open that I know of and I'm not sure what is meant by "the model executable in TxtInOut. I can't seem to find such a file in the TxtInOut folder.

Do you have any idea what I'm doing wrong?

Thanks in advance!!

Sam Hias

Op woensdag 25 november 2020 om 14:27:42 UTC+1 schreef Jim Han:

[Celray James CHAWANDA]

Hi, did you manage to solve the problem?

The problem is that in some models setups, some parameters make SWAT+ crash when added to calibration.cal and prevent sensitivity analysis from working correctly. When you add many parameters, make sure you can run SWAT with those parameters applied. If SWAT+ does not run in SWAT+ Toolbox, copy and paste the debug executable into the TxtInOut and run in command prompt to see the problem (You can also run with debug option in SWAT+ Editor).

Kind Regards,

James

[Sam H]

Hi,

Thanks for responding. Yes I did manage to get the models running, but the results are way too high (eg 25000 m³/s insead of 1,8 m³/s) so I suspect there is a problem with the number notation (. vs, or something). I tried changing the way numbers are notated in windows (. instead of , for decimals), but no change. Any ideas?

Thanks in advance

Sam Hias

Op di 8 dec. 2020 om 21:12 schreef Celray James CHAWANDA <celray....@gmail.com>:

You received this message because you are subscribed to a topic in the Google Groups "SWAT+" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/swatplus/NPPf9WVPXpA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to swatplus+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/013934ec-32d9-418c-8033-ce1af324c17bn%40googlegroups.com.

[Celray James CHAWANDA]

Hi Sam,

Is this happening if you run in SWAT+ Toolbox only or also if you run in SWAT+ Editor? Can you check flow values after running your model in SWAT+ Editor and see if it is a number format issue, a routing problem or something else?

Let me know how it goes.

Regards,

James

[Sam H]

Hi James,

I figured it out. Apparently my system keeps changing the number formatting to use "," as a decimal separator instead of "." You have to change it to the correct formatting ("." as a decimal separator) before running the model in SWAT+Toolbox. If you change it after you already ran the model once, it does not have any effect. I'm running the automatic calibration now. 

BTW: Maybe you could consider using other coëfficiënts in the automatic calibration process in addition to NSE? NSE has a tendency to have a bias towards the higher peakflows. If you use logNSE the lower flows are optimized instead of the peakflows, for some models that could be beneficial. 

Thanks James!

Sam Hias

Op ma 14 dec. 2020 om 14:52 schreef Celray James CHAWANDA <celray....@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/6a6b43f0-0461-46f8-85cd-16c780764ff5n%40googlegroups.com.

[rofi cahyono]

Hi James,

I faced the same problem with Sam, the result of the simulation is way too high (eg 500000 instead of 60), this applies to swat + editor and swat + toolbox,  NSE value negative hundred million, fyi I use swat + toolbox v.0.5.1, thanks in advance.

Regards,

Rofi

attached observation file and weather station file

[Sam H]

Rofi,

Go to regional settings in windows and change the number formatting. The decimal separator should be ".". Do this before you try to run the model.

Here you can find how to change the number formatting in windows 10: https://www.top-password.com/blog/change-number-format-in-windows-10/

Op do 17 dec. 2020 om 15:56 schreef rofi cahyono <rof...@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/233fc955-4829-42a5-9550-378c8f233c5dn%40googlegroups.com.

[rofi cahyono]

Hi Sam.

Thanks for your help, my NSE changed from negative million to negative thousands, after I tried several times the value of NSE did not change (negative thousands), it seemed to be going in the right direction, but the NSE value was still far from what it should be, is this something to do with changes in my settings, where It shows the difference with the example on the page you showed or I missed something important

[Celray James]

Can you show us the page that shows the graph in SWAT+ Toolbox?

 

           With Kind Regards,

           Celray James CHAWANDA

 

To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/8745d77f-a1dc-4661-93bb-b81b33a45ed0n%40googlegroups.com.

 

[rofi cahyono]

Hi James,

here is my result.

[Celray James]

It seems like your observations are constant and almost zero, can you send the observation file so that we see how it is formatted? Remember the correct formation is rows in your csv file of

date(dd/mm/yyyy),[value]

To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/9e32b2fc-5c2d-4fd1-ad64-a7e9de01bfa1n%40googlegroups.com.

 

[rofi cahyono]

Hi James, attached my observation file

[rofi cahyono]

Actually I have observation data starting from 2008-2019, only for the years 2011-2013 there is no data. I took observational data from 2011-2019 because it adjusted to existing precipitation data (2011-2019) .

[Sam H]

Hi Rofi,

Are you sure your observational and/or weather data are correct and in the right unit (mm for precipitation and m³/s for discharge)? The discharge seems so low compared to your precipitation. If you look at december 2014 around 400mm of precipitation fell that resulted in barely 0.5 m³/s. That doesn't seem right unless it's a very small channel. Are you sure you aren't missing a decimal separator in your precipitation file? 

Op vr 18 dec. 2020 om 07:34 schreef rofi cahyono <rof...@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/9c2e22f0-ed4f-4802-9ea3-4f43d339dbf2n%40googlegroups.com.

[Sam H]

Hi,

There is no easy way to change water level data to daily discharge, it very much depends on the characteristics of the specific river. 

Op vr 18 dec. 2020 om 10:05 schreef rofi cahyono <rof...@gmail.com>:

Hi Sam,

They used to say the unit is m3 / second for observation data, but I will try, if the data is not discharge data but water level data, what I want to ask is, is there an application that can change the daily water level data to in daily discharge?  

Regards,

To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/571a2d4a-54d0-4b6d-be8b-b7c64324f900n%40googlegroups.com.

[rofi cahyono]

Hi Sam,

Just now I was able to contact them again, indeed it was daily discharge data in m3 / second.

Regards 

[rofi cahyono]

Hi James,

For some reason, yesterday I took data for a different year range from the previous data. The result from swatplus toolbox v.06 for daily flow still gets NSE -0.2, what should I do to get a positive NSE value.

attached my observation file and weather station file.

Regards,

Rofi

[Celray James]

Firstly, can you verify that there was a spike in observations of about 1500 cubic metres in observations?

To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/6ddee134-3761-4891-b1b1-3ba2a956c705n%40googlegroups.com.

 

[rofi cahyono]

Hi James,

Based on information from the data, it is an extreme flow that has occurred up to 1996, since the recording in 1988.

Regard's

Rofi

[rofi cahyono]

when I change extreme flow with -99, the NSE getting worst in -0.7

[Celray James]

Dear Rofi,

 

It is important to note that not all model issues can be solved with calibration. The fact that you have and extreme event that is not showing up in simulations reveals that the quality of inputs is not great. Check if you can see a similar peak in precipitation in weather input (precipitation). If it is not present, I would not be surprised that calibration is a problem. If your weather data is from global databases, I recommend doing the calibration at monthly timestep because usually, the global databases have a low resolution in space and tend to capture less detail in time which in turn makes successful calibration at daily timestep virtually impossible (especially for small catchments).

To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/a8fd7aeb-b276-49ea-8b2c-725c8d664607n%40googlegroups.com.

 

[Sam H]

Hi Rofi,

Also the NSE coëfficiënt is very much biased towards errors in peakflows as it uses the square of the difference values between simulation and observation. Removing this one outlier will already increase your NSE I think.

Good luck,

Sam

Op di 12 jan. 2021 om 21:00 schreef Celray James <celray....@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/swatplus/1E791CD1-4779-49FB-9B68-A6B42E2ED542%40hxcore.ol.

[Kerrie Taylor-Jones]

Thanks Celray, looks interesting! 

Unfortunately I am having issues with the QSWAT+ aspect of the toolbox (the toolbox appears to be working fine for me when used independently from QGIS). 

When I try to load existing projects in QGIS, I now get an error message (attached). I also now get an error message when I try and create a new project, with a error message complaint about a DEM that has always worked fine previously. 

Any suggestions? 

With thanks,

Kerrie Ann

[Collins Owusu]

Thanks James for the SWAT+ Toolbox.

I am currently testing the v0.7.4. I realized the Obj.Fx.Value Evolution curve doesn't come out right when you try to minimize an objective function such as RMSE.

The plot should normally start from a high value and show you the decrease but the initial values keep vanishing from the plot.

I also have a question. During automatic calibration, does the tool save the parameter values and the model output for each iteration where you can go and get the values?

I realized that for my model, the best parameter values after the automatic calibration doesn't give a realistic hydrograph compared to the observed. Some events are not well simulated by these values.

While I was monitoring the calibration, one iteration gave a very good representation, and I was wondering if I can go back and retrieve the values for that iteration number.

Thank you,

Collins Owusu

[Herminio Sabino]

Hello, James

First of all, I want to congratulate you on the great work with the development of the software.

I'm participating in the SWAT + training offered by EMWER.

I'm using SWAT + Toolbox 0.7.4 to calibrate a basin in the semiarid region of Brazil.

Some questions arose.

1) Can I select the sub-basins upstream of the points to perform the calibration?

2) How can I maintain the calibrated values of the upstream sub-basins for the next point process?

3) is it possible to add the KGE function for analysis?

Thanks in advance for helping

[ana corrochano]

Hello, I would like to know if someone can give me a format example in case I want to add observation flow file in SWAT + toolbox with 15 min timestep? I've been trying but I'm getting error, and only being able to include daily timestep.

Many thanks,

Ana.

[Celray James CHAWANDA]

Please post SWAT+ Toobox related questions to https://groups.google.com/g/sptoolbox

C. James CHAWANDA