Using multi-dimensional numpy masked arrays as data input

[herge...@gmail.com]

Dear Community,

I am new to Pyomo and am trying to figure out if this optimization modeling language is able to deal with the problem I am trying to solve.

After reading the Data Input section of the online documentation it is not clear to me if a multi-dimensional numpy array (e.g. stored in a .npz file) can be used as a data source to initialize the model. I am working in the field of Climate Science and we store our data in NetCDF files. Python is able to read NetCDF files and store the data as masked numpy arrays. The input data I need is 3 dimensional (312x144x72, which is time x longitude x latitude) and masked (e.g. no values over the ocean). Will Pyomo be able to deal with large, multi-dimensional masked numpy arrays (containing floats) as parameters?

I am happy to provide more information on the exact problem I am trying to solve if that helps =).

Thanks for your help!

Kind regards,

Nadja

[Gabriel Hackebeil]

Pyomo operates at a scalar level. We have the concept of indexed components, which can be multi-dimensional, but expressions still need to be generated using scalar operations.

When NumPy objects are used in this way, they actually perform worse than their respective built-in Python types. So while NumPy is still a generally useful tool to use alongside Pyomo (for importing / generating / analyzing data), it is better to convert NumPy array objects to Python lists (or lists of lists, etc.) before using them to build a Pyomo model. I think there is a method on NumPy arrays called “tolist” that does this for you.

If you are using a ConcreteModel, and you import the data prior to building the model (e.g., by passing the data into a function that creates the model), then you do not need to convert the data to Param objects. You can simply using it directly to build objective / constraint expressions and to define index sets.

Gabe

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nadja Herger]

Hi Gabe,

Thanks a lot for your swift response and the advice to convert NumPy arrays into Python lists before building my ConcreteModel. I implemented a very simplified model of the problem I am trying to solve and Pyomo seems to be able to deal with my multidimensional arrays =).

Thanks!

Nadja

[SK]

Hi Gabriel, 

I have a similar problem where my parameters are big 2-D numpy arrays. After looking at your recommendation, I tried initialzing the parameters using lists (converting arrays to list), which gives me an error. After some research, I realized it can only be initialized using dicts. I am not sure what am I missing here, because I cannot do it with lists?

SK