Get duals with PySP

[eedua...@gmail.com]

How can I retrieve the duals after using runef or runph?

I saw in a previous post that someone recommended the use of --solver-suffixes=[rc,dual] if the solver is cplex, but how can I

specify it when calling runef or runph and where will the duals be after?.  Additionally, I m currently using cplex though

I would prefer a more flexible solution that works with other solvers too if possible.

Regards,

Eduardo

[Watson, Jean-Paul]

This is possible, but the mechanics (and the possibility, for that matter) depend on exactly what you are trying to do. Are you trying to access the duals on non-anticipativity constraints, or scenario constraints? The latter is easier. The former is fraught with issues, which we can explain. 

jpw

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[eedua...@gmail.com]

I need to get the scenario constraints duals.

[Gabriel Hackebeil]

Eduardo,

If you place a Suffix component named “dual” on your reference model it will have the desired effect. E.g.,

model.dual = Suffix(direction=Suffix.IMPORT)

You can find online documentation about the Suffix component here: https://software.sandia.gov/downloads/pub/pyomo/PyomoOnlineDocs.html#_suffixes

You will likely need to implement a solution writer plugin in order to query these suffix values because neither the runph or runef commands will make any attempt to output these. You might also run into another issue with runef in that the suffix values get loaded into a suffix component on the master extensive-form model (leaving the instance suffix containers empty). If you get this far, or run into any other issues let us know.

Regards,

Gabe

[eedua...@gmail.com]

I am using runph and therefore the problem with runef is not a concern. I aim to solve reasonably quickly huge problems and for that to take advantage

of the parallelization potential of ph. I found out that to retrieve the duals I need an instance (already solved). For that purpose PYOMO provides callbacks.

I defined in the ReferenceModel.py a few callbacks for testing:

________________

def pyomo_print_results(options, instance, results):

print ("pyomo_print_results")

def pyomo _preprocess(options=None):

   print "CALLBACK"

________________

When I run runph they where never called. Do they have a different name in PySP (already tried prefixing pysp_ instead of pyomo_)?

Am I missing something?

About the duals

To be precise, the duals I am looking for are the ones of the constraint of the

global problem. It would be even better if I can get the in the same data structure

as the results returned when you run the following code:

solv = solver.Provider.solver( ... )

results = solv.solve( ..., suffixes=['dual'] )

[Gabriel Hackebeil]

________________

def pyomo_print_results(options, instance, results):

print ("pyomo_print_results")

def pyomo _preprocess(options=None):

   print "CALLBACK"

________________

When I run runph they where never called. Do they have a different name in PySP (already tried prefixing pysp_ instead of pyomo_)?

Am I missing something?

No, pyomo callbacks only apply to the ‘pyomo’ command-line executable. PySP takes a slightly different approach.

If you want to inject code into the PH algorithm you can do so by implementing an extension plugin. The way this works is you copy the file “<install_dir>/pyomo/pyomo/pysp/plugins/examplephextension.py” to the location/name of your choice. Inside this file you will find two classes (which you can rename as well) with methods having names like ‘post_iteration_k_solve’ and descriptors of when they get called during the PH algorithm. You can fill in these methods with whatever you want (you can even break the algorithm if you try hard enough).

The second class is for when you use the phpyro solver manager. I’ll skip explaining just in case you already know about it, and just say that it is a more-parallelized version of the pyro solver manager (instead of pyro_mip_server, you launch one or more phsolverservers). With phpyro, the scenario instances are not constructed on the master process, and so only the minimum amount of solution info is sent back to the master to do the PH parameter updates. If you plan on using phpyro, then you will have to go extract duals on the solver servers because that information won’t make it back to the master. Note that the standard pyro solver manager builds all instances on the master process and just spawns threads to do the solves.

I’m out of time for now, but see where you can get with this and shoot back any questions you have. There’s plenty of example code in the plugins directory to help figure out how to navigate around the ph objects as well.

Gabe

P.S. Here is an example command-line that uses the two extension classes with phpyro as the solver manager (extension classes are in junk.py)

mpirun -np 1 pyomo_ns -k -r -n localhost : 

            -np 1 dispatch_srvr localhost : 

            -np 3 phsolverserver localhost --user-defined-extension=junk.py : 

            -np 1 runph -m model.py -i ScenarioStructure.dat -r 1.0 --user-defined-extension=junk.py --solver-manager=phpyro --phpyro-required-workers=3 --pyro-hostname=localhost

[eedua...@gmail.com]

After some tests I have decided to change to phpyro solver manager.  To get the duals I ll need to extend the second class methods: def post_iteration_0_solves(self, ph) or def post_iteration_k_solves(self, ph). At this point how can I retrieve the restriction duals? How can I run this plugin for each phsolverserver? What is the type of the ph argument of post_iteration_k_solves() and what data does it contain?

Regards,

Eduardo

[eedua...@gmail.com]

Instead of implementing a post_iteration_k_solves() could I access the duals when running a solutionwriter extension? If possible, how and where are they stored?

Eduardo

[Gabriel Hackebeil]

The duals are stored on the Pyomo scenario instances. If you are using phpyro, these instance only live on the phsolverservers, so they won’t be available in a solution writer extension.

It is possible using a PHExtension (in say post_iteration_k_solves) to transmit a custom function invocation to each of the phsolverservers that collects the dual information and returns a dictionary of the results you want. It’s pretty simple in practice and there are tons of example in the source. For instance, searching for “*collect_cut_data” in benders.py would be a place to start. The utility you would likely need to familiarize yourself with is called “transmit_external_function_invocation” and it’s defined in phsolverserverutils.py

Gabe