Concrete vs Abstract model assign data

[Hongwei Jin]

Hi there,

Actually, I have two questions:

1. what's the performance of concretemodel v.s. abstractmodel. In my perspective, the concretemodel reads data inside the python code, so I would consider it as in-memory data-loading. While the abstractmodel will load data from .dat file, so I would consider it as on-disk data-loading. So if we consider the pre-optimization process, the concretemodel will be better?

2. Since the ideal of using abstractmodel is coming from other modeling language, such as AMPL. And I also notice that there are two ways to assign data into abstractmodel, either using command line or using ModelData object. While all the ModelData will read data from .dat file or load data from csv formatted file, which is not a better way in python code. In real applications, it will read more data from DB, and the data flow will cause big issues when dump them into formatted specific data file and read them again in pyomo. So is there another to assign data of an AbstractModel object taking the advantage of python, such as pandas dataframe or numpy arrays?

Thanks in advance.

HJ

[Gabriel Hackebeil]

1. what's the performance of concretemodel v.s. abstractmodel. In my perspective, the concretemodel reads data inside the python code, so I would consider it as in-memory data-loading. While the abstractmodel will load data from .dat file, so I would consider it as on-disk data-loading. So if we consider the pre-optimization process, the concretemodel will be better?

The key difference is when components are constructed. In a ConcreteModel, construction takes place as soon as the component is assigned to the model and, thus, any data required to initialize that component must be available (e.g., by calling a user supplied rule). For an AbstractModel, construction is delayed until create(...) is called, at which point components are constructed in the order in which they are declared (so the same dependency orders apply).

However, you can always use a ConcreteModel in an “abstract” setting by placing its definition inside a function and passing some sort of data object in as an argument that is subsequently used to initialize components. Similarly, you can use an AbstractModel in a “concrete” setting by loading data from whatever source you prefer above the model definition and initializing components with it (or using it directly in constraint expressions).

2. Since the ideal of using abstractmodel is coming from other modeling language, such as AMPL. And I also notice that there are two ways to assign data into abstractmodel, either using command line or using ModelData object. While all the ModelData will read data from .dat file or load data from csv formatted file, which is not a better way in python code. In real applications, it will read more data from DB, and the data flow will cause big issues when dump them into formatted specific data file and read them again in pyomo. So is there another to assign data of an AbstractModel object taking the advantage of python, such as pandas dataframe or numpy arrays?

There is no need to reformat your data into cvs or dat files if you already have other ways of accessing it directly in your Python process. The .dat format is more of a convenience for users that are moving models over from AMPL. In 90% of cases, you can use the same .dat files. However, our .dat file parser is written in Python, whereas the AMPL version is written in C, so the performance hit can be large in our case.

Python gives you access to a vast toolset for managing data on your own, take advantage of it when you can. If load the data yourself (using some database API), you can use it directly in constraint expressions or to initialize model parameters. E.g.,

model.index = Set(initialize=data[’s’])

model.p = Param(model.index, initialize=data[‘p’])

This can type of declaration can be used whether or not ‘model' is Abstract or Concrete.

Gabe

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.