multicore processing ?

[Stephane Ransac]

Hi,

I use COPASI to adjust the parameters of a small metabolic network. I have 56 experimental conditions, each giving me 2 results that I try to adjust with 90 parameters of my metabolic network. The adjustments are very long (1 to 4 weeks depending on the chosen method). So I have 2 questions:
The first, would it be possible to regularly save the parameters and the model updated during execution (once a day?). Indeed, if for any reason (power outage, etc.) my computer stops, I lose several days, even weeks of work.
The second, the program only uses one core of my computer, would it be possible to use several cores in order to reduce the execution time of my optimization.
Thank you for your answers
Best regards
Stéphane

[Frank Bergmann]

Hello Stéphane,

 

for such longrunning computations I’d recommend that you prepare the COPASI file to be executed by CopasiSE. Here you’d essentially ensure that the parameter estimation task is marked as executable and a report file is defined (the default report is fine, you just need to specify a file-name), and the ‘update model’ flag is set on the parameter estimation task. You’d then run the task from the command line ([1]) like so:

./CopasiSE input_file.cps -s output_file.cps or

./CopasiSE input_file.cps -s output_file.cps --report-file report.txt

 If a non-deterministic optimization algorithm was chosen (or randomize start values), you could then run multiple of these tasks in parallel (using different output files, perhaps different report files).

Should your computer shut off in the meantime, the report file corresponding to the input file used, can be used to restore the last found parameter values. For that we have an option in the CopasiUI in the Tools menu as ‘load Parameter Estimation protocol’.

At the end you would load the parameters corresponding to the best objective values found to inspect them.

I hope this helps,

Best

Frank



[1]: https://copasi.org/Support/User_Manual/Model_Creation/Commandline_Version_and_Commandline_Options/

[Stephane Ransac]

thanks for your answer.
I tried to use CopasiSE for my parameter estimation, the problem is that I am always forced to stop the adjustment otherwise, it would still take much longer than 4 weeks. I stop it when the graph shows me that there is no more progress. So I check the parameters that are blocking before restarting a parameter estimation with possibly another method. With CopasiSE, I do not see the progress and I cannot stop it smoothly in order to obtain the report backup and the data update.
So that is why I would be interested in CopasiUI being able to use several processor cores to speed up the work or use the processors in graphics cards.

In addition, I have another question. Due to my many data and my many parameters, when I ask to stop the parameter estimation by clicking on the "stop" button (white square on blue circle) it takes about 4 hours before it actually stops in order to allow me to save my model and interact again with Copasi.
Is there a way to reduce this waiting time (without of course clicking on the white cross on red circle because that cancels all the estimation made for days)?

Kind regards
Stéphane

[Stephane Ransac]

thanks for your answer.
I tried to use CopasiSE for my parameter estimation, the problem is that I am always forced to stop the adjustment otherwise, it would still take much longer than 4 weeks. I stop it when the graph shows me that there is no more progress. So I check the parameters that are blocking before restarting a parameter estimation with possibly another method. With CopasiSE, I do not see the progress and I cannot stop it smoothly in order to obtain the report backup and the data update.
So that is why I would be interested in CopasiUI being able to use several processor cores to speed up the work or use the processors in graphics cards.

In addition, I have another question. Due to my many data and my many parameters, when I ask to stop the parameter estimation by clicking on the "stop" button (white square on blue circle) it takes about 4 hours before it actually stops in order to allow me to save my model and interact again with Copasi.
Is there a way to reduce this waiting time (without of course clicking on the white cross on red circle because that cancels all the estimation made for days)?

Kind regards
Stéphane

Le mercredi 26 février 2025 à 10:58:20 UTC+1, Frank Bergmann a écrit :

[Frank Bergmann]

I probably would still use CopasiSE, just while preparing the parameter estimation setup in the graphical user interface i would enable 'Enable additional optimization parameters' in the preferences menu, and then on the optimization method choose 'Stop after # Stalled Generations' / Iterations for those methods that support it and set it sufficiently high. That parameter helps to stop early in case of no progress. Using this together with multiple CopasiSE processes could help.

As for the time it takes after stopping. It might make sense, to disable 'calculate statistics' for most of your runs until you have a run you are happy with, and for that one select current solution statistics with 'calculate statistics' enabled. Otherwise after a click on stop the statistics will be calculated which in your case seems to take a lot of time. 

Out of interest, which optimization method are you using? Just so i can verify, that that algorithm stops as soon as possible when requested. 

If you are writing the report file for the parameter estimation, you could terminate Copasi, and restore the last parameters found from the report file using the protocol option described in my last mail. 

Since you ask specifically about versions of COPASI with parallel processing, we wont be able to use graphics cards (since we allow for arbitrary kinetics which dont lend themselves for use in CUDA), however there is an experimental build option for Copasi with OpenMP enabled. On that one OpenMP can be used for the particle swarm algorithm. 

best

Frank

[Pedro Mendes]

*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Stephane,

we are actively working on a version of COPASI that uses all the cores
in your CPU. We are still in the debugging phase but the benchmarks we
have carried out already show somewhat mixed results. On most laptop
CPUs (i7 or i5) you get a 2.5x speedup when using 4 cores. We get better
results on desktop/server CPUs (xeon) where you may get 4x using 6 cores.

The current prototype only has parallelized particle swarm, but we will
be working on some of the other algorithms soon. Also, I think we only
have the current prototype in the CopasiSE version (but I may be wrong
on that)

In summary, we may have a multi-core version soon, but don't expect
miracles with it... if you want to try out the prototype let us know and
we can share it with you (no guarantees, of course, as it is still alpha
code)

Pedro

> Commandline_Version_and_Commandline_Options/ <https://
> copasi.org/Support/User_Manual/Model_Creation/
> Commandline_Version_and_Commandline_Options/>

>
>
>
> On Wednesday, February 26, 2025 at 10:43:00 AM UTC+1
> sra...@gmail.com wrote:
>
> Hi,
> I use COPASI to adjust the parameters of a small metabolic
> network. I have 56 experimental conditions, each giving me 2
> results that I try to adjust with 90 parameters of my
> metabolic network. The adjustments are very long (1 to 4
> weeks depending on the chosen method). So I have 2 questions:
> The first, would it be possible to regularly save the
> parameters and the model updated during execution (once a
> day?). Indeed, if for any reason (power outage, etc.) my
> computer stops, I lose several days, even weeks of work.
> The second, the program only uses one core of my computer,
> would it be possible to use several cores in order to reduce
> the execution time of my optimization.
> Thank you for your answers
> Best regards
> Stéphane
>

> --
> You received this message because you are subscribed to the Google
> Groups "COPASI User Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to copasi-user-fo...@googlegroups.com
> <mailto:copasi-user-fo...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/copasi-
> user-forum/c87d5cf0-26c8-4a73-af40-88ccc75dfc7cn%40googlegroups.com
> <https://groups.google.com/d/msgid/copasi-user-forum/c87d5cf0-26c8-4a73-
> af40-88ccc75dfc7cn%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Pedro Mendes, PhD
Professor and Director
Richard D. Berlin Center for Cell Analysis and Modeling
University of Connecticut School of Medicine
group website: http://www.comp-sys-bio.org

[Hoops, Stefan (sh9cq)]

Hello Stephane,

There is another method to stop CopasiSE the option --MaxTime will
cause the optimization to stop gracefully and complete the report.

Thanks,
Stefan

--
Stefan Hoops, Ph.D.
Research Associate Professor
Biocomplexity Institute & Initiative
University of Virginia
995 Research Park Boulevard
Charlottesville, VA 22911

Phone: +1 540 570 1301
Email: sho...@virginia.edu

[Stephane Ransac]

thanks for your answers.

@Frank
I will try to disable statistics, I see that they are indeed checked in my session, it may be the reason for this long time before stopping.

The methods I use most often are
Simulated Annealing (most of the time)
Hooke & Jeeves (sometimes)
the others even less often.

The methods I use do not allow 'Stop after # Stalled Generations' but I can try other methods allowing it with CopasiSE

@Stefan
I can't find the "MaxTime" option in the parameter estimation methods, for which methods this option should be available. Otherwise, there is "Iteration Limit", but not with the "Simulated Annealing" method that I use most often

@Pedro
This is good news that should reduce my waiting times.
I would like to test your new version if you have it in CopasiUI version because with CopasiSE there is no graph that allows me to see that my estimate is no longer progressing, allowing me to stop it before the end.

Kind regards
Stéphane

[Stephane Ransac]

Hello Pedro,
I found a Linux server on which I can run CopasiSE. So I should be able to test the alpha version of the multicore CopasiSE.
Kind regards
Stéphane