How to run a complex model

[Samuel Dotson]

Hi all, 

I'm running into an issue where I'll try running a complex model on my machine (12 cores, 32gb RAM, 500 GB disk) and the temoa model will build and the solver will start working then it gets to the reporting variables step and the process is killed. This happens with the model I'm working with and with the models maintained by the TemoaProject team (e.g. TemoaProject/data/US_Regional.sql). I've attached the error message, but I'm not sure what I'm doing wrong. Is my machine simply not powerful enough to run the model? Do I need to use a better solver than coincbc?

I have tried running the model on the TemoaCloud user interface and it doesn't seem to work there either.

Best, 

Sam Dotson

[Temoa Project]

Hi Sam,

We've only tested the larger databases (the size of US_Regional) with CPLEX and Gurobi, and have run into memory issues in a few instances. I suspect we will see similar issues with coincbc. In case you aren't already aware: if you are at an academic institution, both CPLEX and Gurobi provide free licenses. 

Our approach so far has been to use improved computing infrastructure when possible (we have access to several computing clusters), and solve the models in myopic mode. In addition, when using CPLEX and Gurobi, we were able to solve some instances when limiting the number of threads, as specified to the solvers, to well below the total number of cores available. This also seemed to help with memory issues (we suspect this is because some of the solvers try to create multiple instances of the model when solving with the barrier algorithm). To do this, you'll need to make some changes to the temoa_run.py file (see example here).

Let us know if this helps!

Best,

Aranya

[Samuel Dotson]

Thanks Aranya, 

I was afraid that would be your answer. Yes I'm aware of the academic licenses (I have a CPLEX license), my issue with commercial software is that it reduces the "openness" of the research/science. The open source nature of Temoa is one of its key benefits! That said, I'll try it with CPLEX and see if it works. 

I haven't tried using the myopic functionality yet, I'm not quite sure how it works. Does it allow you to solve the model "piecewise?" Do you think coin could solve a complex model using this strategy?

Best, 

Sam Dotson

--
You received this message because you are subscribed to a topic in the Google Groups "Temoa Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/temoa-project/mExg7zCdoAI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to temoa-projec...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/temoa-project/5f7a0985-ac6c-40f4-83e7-6e53b70ee876n%40googlegroups.com.

--

Samuel G. Dotson

Master of Science in Nuclear Engineering 2021

Bachelor of Science in Engineering Physics

University of Illinois at Urbana-Champaign

Pronouns: he/him/his

sg...@illinois.edu

[Temoa Project]

Hi Sam,

We have had similar discussions about commercial software within our team as well. 

To run the model in myopic mode: you'll just need to specify this in the config file. There's a section called "Solve Myopically" where the --myopic option is commented out. You will need to uncomment that option to have Temoa solve the database in myopic mode. Once you run the model at the command line, it will offer you a few options: you can have the model solve several years at a time, but only keep the results from the first year and then proceed to the following year (a rolling horizon or limited foresight approach). Or you could solve the model in a mode with no foresight, where you only solve one year at a time (so the model has no information about future demands, technologies, etc). A caveat - make sure that the config file is only named config_sample, or you may run into issues (we're working to update this). 

Let us know if this approach works for you. We haven't tried this approach with coincbc, so this would be helpful for us to know what you find. 

Best,

Aranya

[Samuel Dotson]

Hi Aranya,

I will try the myopic approach first, it's requesting a table "MyopicBaseYear." I'm not sure what to put here, or how many years to add. I would like to try the limited foresight approach, first. Do you have any guidance on this? (the years in my simulation are 2025,2030,...,2045,2050)

Best, 

Sam

[Temoa Project]

Hi Sam,

temoa_schema should have information on the MyopicBaseYear table - you don't need to enter this information in advance, you just need to make sure the empty table exists in the database at least. It should auto-populate it to the first year of your simulation (2025) when you first run it in myopic mode. 

Best,

Aranya