Future 95PPU in LSU scale

202 views
Skip to first unread message

Payam

unread,
Feb 7, 2024, 3:20:31 AMFeb 7
to R-SWAT
 Hi Dr. Nguyen and all,

I am trying to use HPCs to run my calibrated SWAT+ model for the future climate dataset and extract the 95 PPU ranges for all the grid cells that I have in my model (32,000 LSUs) and map them. Since I am running the model for the future, clearly I do not need to implement an objective function because there is literally no observed data to run the objective function against. All I need is to extract the LSU-scaled 95 PPUs for all the grid cells that I have through a 100-simulation iteration. Could you please give me some advice on how I can update the 'RSWAT_script_SUFI2' code to run on HPCs and achieve my goal?  
Message has been deleted

Tam Nguyen

unread,
Feb 7, 2024, 4:01:56 PMFeb 7
to R-SWAT
could  you  share with me  the RSWATproject.rds,  I can  make an example  code and share the  code here

Payam

unread,
Feb 8, 2024, 4:14:37 AMFeb 8
to R-SWAT
Hi Dr. Nguyen,

Thanks for getting back to me.

"RSWATproject.rds" could be found attached.

RSWATproject.rds

Tam Nguyen

unread,
Feb 8, 2024, 4:16:43 AMFeb 8
to R-SWAT
Now you could install RSWAT package (https://github.com/tamnva/R-SWAT/tree/master) and then you will see the script for doing this in RSWAT vignettes (look like the attached file)

(Must install RSWAT to run this script)
run_SWATplus_with_given_parametersets.html
Message has been deleted
Message has been deleted

Payam

unread,
Feb 9, 2024, 2:46:40 AMFeb 9
to R-SWAT
Thank you so much, Dr. Nguyen, for sending the script. Just a small correction that I found needs to be applied in the 'readAllOutVar' function. In line 48 of the code ("" for (ifolder in 1:length(coreFolder)){""), 'coreFolder' should actually be 'coreFolders', where the last 's' was omitted. I made the adjustment on my end, and it worked perfectly.

I have another question regarding the extraction of the 95PPU for the number of subbasins. Currently, the code only extracts the 95PPU for one subbasin (Reach = c("1")). How can we modify it to extract the 95PPU for all the units specified with year, month, day, and unit number in the output file?

Your assistance is greatly appreciated.

Tam Nguyen

unread,
Feb 9, 2024, 3:02:44 AMFeb 9
to R-SWAT
Thanks for spotting this mistake. I have fixed this issue and update the GitHub repo (reinstall RSWAT package to update this)

If you want to extract for 100 reaches (just example) you need to replace line 54 in this file https://github.com/tamnva/R-SWAT/blob/master/vignettes/run_SWATplus_user_parameters.R with

Reach = paste(as.character(c(1:100)), collapse  = ",")

For this question: "...for all the units specified with year, month, day, and unit number in the output file?" I am not clear what do you mean, could you give the example data file here

Tam

Payam

unread,
Feb 9, 2024, 5:46:07 PMFeb 9
to R-SWAT
I tried updating 'Reach = paste(as.character(c(1:100)), collapse = ",")' and extracted data from 'lsunit_wb_mon.txt' for the years 2003 to 2004 (basically 'numTimesteps <- c(24)'). However, I encountered an error in the 'output <- readAllOutVar(workingFolder, numTimesteps)' section of the code. Here's the error:

Error in matrix(data[, 1], nrow = numTimesteps[var] + 1) :
invalid 'nrow' value (too large or NA).

Regarding the example data file you mentioned, I included a sample with two variables (let's say ET and PET) and two units over the two years. This represents the format I'm aiming to extract. Please ignore the actual numbers; they're just for illustrative purposes."
95PPU_Sample_Output.txt

Tam Nguyen

unread,
Feb 10, 2024, 2:14:44 AMFeb 10
to R-SWAT
if you did for 100 reaches, your numTimesteps should be a vector with a length of 100, each element of this vector indicates the number of output timesteps of each variable. For examples incase you output timesteps are the same for all 100 reach, use

numTimesteps <- rep(24,100)

Payam

unread,
Feb 12, 2024, 1:48:48 PMFeb 12
to R-SWAT
I tried that too. But still doesn't like it (I am extracting from "lsunit_wb_mon.txt" file for the period of ("2002-01-01", "2003-12-31")) for 100 reaches:

numTimesteps <- rep(24,100)
output <- readAllOutVar(workingFolder, numTimesteps)

Error:  Error in output[[var]] : subscript out of bounds
run_SWATplus_user_parameters.Rmd

Tam Nguyen

unread,
Feb 14, 2024, 4:00:55 AMFeb 14
to R-SWAT
There is a bug in the readAllOutVar function, I have fixed this and updated the code, it should work now

Payam

unread,
Feb 15, 2024, 5:37:04 AMFeb 15
to R-SWAT
Hi Dr. Nguyen,

Thank you so much for solving the issue. it's working now.

I have another question about running your code in R instead of RStudio. you have mentioned in the R-SWAT github that "if you run in R, package vignettes cannot be built". I am wondering if this version of 95ppu code could be run in R as well since I need to run the code in HPCs. 
if yes, how "package vignettes" can be created?

Tam Nguyen

unread,
Feb 15, 2024, 5:40:50 AMFeb 15
to R-SWAT
package vignettes are a kind of documentation of the package. This DOES NOT affect the functionalities of the package. You can install using R and still use all functions of the package. I have update the README - please see detail there

Payam

unread,
Feb 19, 2024, 2:10:04 PMFeb 19
to R-SWAT
Hi Dr. Nguyen,

Thank you for your explanation. I could successfully install the ""RSWAT" library on the Linux cluster. However, the only issue that exists now is executing the SWAT+ executable file. Your script creates multiple copies of the TextInOut folder with the SWAT+ executable file included in the 'workingFolder'. However,  the following error is showed up when it comes to running them even though I have switched to the Linux environment version of the executable file :

Error : "sh: line 1: /home/R-SWAT-master/EXE_File/rev60.5.4_64rel.exe: Permission denied"

Do you happen to see such an error before?

Tam Nguyen

unread,
Feb 19, 2024, 3:24:05 PMFeb 19
to R-SWAT
actually I haven't tested with linux. In linux the executable SWAT+ does not have an extension .exe. I will check it next week. You can check this out this week, something relates to calling this file in RSWAT (lines 185-189) in this file https://github.com/tamnva/R-SWAT/blob/master/R/runSWATpar.R

Tam Nguyen

unread,
Feb 27, 2024, 11:11:10 AMFeb 27
to R-SWAT
it should work now on linux. I have updated the code

Payam

unread,
Feb 27, 2024, 11:51:47 AMFeb 27
to R-SWAT
Hi Dr. Nguyen,

I was able to use the previous version of the code using the Linux format of SWAT+ model (rev60.5.4_64rel_linux). Can you please elaborate on what is the main difference of the current version of the code with the previous one?

Tam Nguyen

unread,
Feb 28, 2024, 3:01:45 AMFeb 28
to R-SWAT
There is a very small changes. Example in window I have the SWAT executable file is "swat.exe" and linux the SWAT executable file is "rev60.5.4_64rel_linux ". Calling these executable file different in window and linux

in Window:    system("swat.exe")
in Linux:         system("./rev60.5.4_64rel_linux ")


In the code line 189-193 were change from:

previous version:

system(trimws(exeFile[length(exeFile)]))

to current version:

    if(.Platform$OS.type == "unix") {
      system(paste0("./", trimws(exeFile[length(exeFile)])))
    } else {
      system(trimws(exeFile[length(exeFile)]))
    }

you can see the updated file here https://github.com/tamnva/R-SWAT/blob/master/R/runSWATpar.R (L189-193)

Payam

unread,
Mar 7, 2024, 4:12:59 PMMar 7
to R-SWAT
Hi Dr. Nguyen,

I hope you are doing well!

I was able to run SWAT+ in the Linux cluster using the previous version of the R-SWAT-master depository which I had installed. Yesterday, I updated the R-SWAT library and the R-SWAT-master folder and this time the script that I was using can not execute "rev60.5.4_64rel_linux" file and I am hitting the following error. I am wondering if you have changed each of the codes recently since I was able to run SWAT+ using the previous version of the library.

> runSWATpar(workingFolder,TxtInOutFolder,outputExtraction,ncores,SWATexeFile,
+           parameterValue,paraSelection,caliParam,copyUnchangeFiles,fileCioInfo,
+           dateRangeCali,firstRun)
sh: line 1: .//home/s_jalalija/R-SWAT-master/data/rev60.5.4_64rel_linux: No such file or directory
sh: line 1: .//home/s_jalalija/R-SWAT-master/data/rev60.5.4_64rel_linux: No such file or directory
sh: line 1: .//home/s_jalalija/R-SWAT-master/data/rev60.5.4_64rel_linux: No such file or directory
sh: line 1: .//home/s_jalalija/R-SWAT-master/data/rev60.5.4_64rel_linux: No such file or directory
sh: line 1: .//home/s_jalalija/R-SWAT-master/data/rev60.5.4_64rel_linux: No such file or directory
sh: line 1: .//home/s_jalalija/R-SWAT-master/data/rev60.5.4_64rel_linux: No such file or directory
sh: line 1: .//home/s_jalalija/R-SWAT-master/data/rev60.5.4_64rel_linux: No such file or directory
sh: line 1: .//home/s_jalalija/R-SWAT-master/data/rev60.5.4_64rel_linux: No such file or directory
Error in { : task 1 failed - "cannot open the connection"
Calls: runSWATpar -> %dopar% -> <Anonymous>
Execution halted

Tam Nguyen

unread,
Mar 10, 2024, 10:26:20 AMMar 10
to R-SWAT
Hi Payam, it should work now, before there was an error because the file path in windows and unix are different which I didn't know, Now I have fixed that and test on unix, it runs without issue
Note, remember to set writting right to your whole folder so SWAT+ can run and write output files there
Reply all
Reply to author
Forward
0 new messages