Save the results of several calculations into the single output file.
24 views
Skip to first unread message
Эрвин Визард (Earvin Wizard)
Nov 29, 2024, 11:12:33 AM11/29/24
to Morpheus users
Dear colleagues, dear Morpheus maintainers,
During modeling process we could use Param Sweep to account stochastic nature of tissue processes – running several calculations with the same parameter values.
We can use Logger —> Output –> TextOutput to save the evolution of the variable of interest.
However the results will be saved into separate .xls files.
Do we have possibility to save output variable from different runs into one file to accelerate data processing? For example, in the form of different columns of one table.
--
Sincerely yours,
During modeling process we could use Param Sweep to account stochastic nature of tissue processes – running several calculations with the same parameter values.
We can use Logger —> Output –> TextOutput to save the evolution of the variable of interest.
However the results will be saved into separate .xls files.
Do we have possibility to save output variable from different runs into one file to accelerate data processing? For example, in the form of different columns of one table.
--
Sincerely yours,
Ivan Azarov
Lutz Brusch
Nov 29, 2024, 12:52:32 PM11/29/24
to Morpheus users
Hi Ivan,
thank you for this good idea! We often need statistics over multiple runs but this is not covered within Morpheus.
Morpheus does provide the Analysis/External plugin which you can use to run a shell command (or shell script or Python script or other) after each simulation (or more often if needed) and let this copy&append data from a logger file of that simulation to one common file for all simulations (of a sweep). For an example that uses external scripts for post-hoc statistics (here over cells within that single run) please see https://identifiers.org/morpheus/M0007 but your own copy&append script should be much simpler.
Best,
Lutz
PS. As noted before, a related need for qualitative comparison of many PLOTs from a ParamSweep is met within Morpheus by the ImageTable button (and following dialog) in the top right of the sweep's results overview page.
thank you for this good idea! We often need statistics over multiple runs but this is not covered within Morpheus.
Morpheus does provide the Analysis/External plugin which you can use to run a shell command (or shell script or Python script or other) after each simulation (or more often if needed) and let this copy&append data from a logger file of that simulation to one common file for all simulations (of a sweep). For an example that uses external scripts for post-hoc statistics (here over cells within that single run) please see https://identifiers.org/morpheus/M0007 but your own copy&append script should be much simpler.
Lutz
PS. As noted before, a related need for qualitative comparison of many PLOTs from a ParamSweep is met within Morpheus by the ImageTable button (and following dialog) in the top right of the sweep's results overview page.
Эрвин Визард (Earvin Wizard)
Nov 30, 2024, 12:53:37 PM11/30/24
to Morpheus users
Thank you, Lutz!
I have to test these possibilities..
I have to test these possibilities..
пятница, 29 ноября 2024 г. в 20:52:32 UTC+3, Lutz Brusch:
Эрвин Визард (Earvin Wizard)
Dec 5, 2024, 8:59:40 AM12/5/24
to Morpheus users
Yes, It is really easy in R Statistics:
library(tidyverse)
#################################################################
source_path <- dirname(rstudioapi::getSourceEditorContext()$path)
setwd(source_path )
getwd()
#################################################################
file.list <- list.files(pattern='*.csv', recursive = TRUE)
pure_list <- grep("logger.csv$", file.list, value = TRUE)
###
df.list <- lapply(pure_list, read.csv) # Some kind of function to perform reading csv
df <- bind_rows(df.list, .id = "id") %>% select(-time.1)
###
Nsub <- length(unique(df$id)) # To derive SE
df_sum <- df %>% group_by(time) %>%
dplyr::summarize(Mean = mean(num_T, na.rm=TRUE), SD = sd(num_T, na.rm=TRUE), SE = sd(num_T, na.rm=TRUE)/sqrt(Nsub)) %>%
mutate(UPPER = Mean + SE, LOWER = Mean - SE) %>%
filter(row_number() %% 5 == 1) # To decrease sampling
#################################################################
source_path <- dirname(rstudioapi::getSourceEditorContext()$path)
setwd(source_path )
getwd()
#################################################################
file.list <- list.files(pattern='*.csv', recursive = TRUE)
pure_list <- grep("logger.csv$", file.list, value = TRUE)
###
df.list <- lapply(pure_list, read.csv) # Some kind of function to perform reading csv
df <- bind_rows(df.list, .id = "id") %>% select(-time.1)
###
Nsub <- length(unique(df$id)) # To derive SE
df_sum <- df %>% group_by(time) %>%
dplyr::summarize(Mean = mean(num_T, na.rm=TRUE), SD = sd(num_T, na.rm=TRUE), SE = sd(num_T, na.rm=TRUE)/sqrt(Nsub)) %>%
mutate(UPPER = Mean + SE, LOWER = Mean - SE) %>%
filter(row_number() %% 5 == 1) # To decrease sampling
суббота, 30 ноября 2024 г. в 20:53:37 UTC+3, Эрвин Визард (Earvin Wizard):
Reply all
Reply to author
Forward
0 new messages