How to use more than one processor with Pyomo

[Ricardo Trincado]

Hello!

I'm currently running my model in a cluster, but I'm only using one processor of all the ones i have required. I'm using gurobi as my solver and I've been reading the documentation. It says that I need to give gurobi the "thread"option. How can I do that in order to run my code?

My other question is how to run the solver manager option. I understand that i need to run the pyomo_ns, then the dispatch server and then multiple pyro_mip_server. The problem is that i need to run it in a queue system with a .pbs file.

Thank you very much for your help,

Ricardo Trincado

[Watson, Jean-Paul]

It looks like you are running with pysp, and maybe ph? If so, then you just need to use the following command line option on runph: —scenario-solver-options=“threads=20”.

If you instead just using pyomo, the option is just: —solver-options

As far as running on the cluster, you just need to form the command line for mpiexec, e.g., as follows:

export PYRO_BROKEN_MSGWAITALL=1

export PYRO_NS_PORT=3200

export PYRO_NS_BC_PORT=3200

mpiexec --verbose --report-bindings -npernode 8 \

-x PATH -x PYRO_NS_PORT -x PYRO_NS_BC_PORT -n 1 ~/sp/pyomo/bin/python ~/sp/pyomo/bin/pyomo_ns : \

-x PATH -x PYRO_NS_PORT -x PYRO_NS_BC_PORT -n 1 ~/sp/pyomo/bin/python ~/sp/pyomo/bin/dispatch_srvr : \

-x PATH -x PYRO_NS_PORT -x PYRO_NS_BC_PORT -n 81 ~/sp/pyomo/bin/python ~/sp/pyomo/bin/phsolverserver : \

-x PATH -x PYRO_NS_PORT -x PYRO_NS_BC_PORT -n 1 ~/sp/pyomo/bin/python ~/sp/pyomo/bin/runph \

--solver-manager=phpyro --shutdown-pyro … >& results.out

There will undoubtedly be a lot of system-specific things at the top of the PBS file, but this will get you started.

Also, you should run “phsolverserver” instead of “pyro_mip_server” - it’s significantly faster, by orders of magnitude.

jpw

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ricardo Trincado]

Thank you for the quick response. I try this code but it shows me the next error:

/var/spool/torque/mom_priv/jobs/17993.bigz.mazinger.SC: line 9: mpiexec: command not found

Traceback (most recent call last):

  File "/usr/lib/python2.6/site-packages/pyomo/pysp/phinit.py", line 1251, in main

    ans = exec_ph(options)

  File "/usr/lib/python2.6/site-packages/pyomo/pysp/phinit.py", line 1150, in exec_ph

    with PHFromScratchManagedContext(options) as ph:

  File "/usr/lib64/python2.6/contextlib.py", line 16, in __enter__

    return self.gen.next()

  File "/usr/lib/python2.6/site-packages/pyomo/pysp/phinit.py", line 927, in PHFromScratchManagedContext

    ph = PHFromScratch(options)

  File "/usr/lib/python2.6/site-packages/pyomo/pysp/phinit.py", line 896, in PHFromScratch

    ph = PHAlgorithmBuilder(options, scenario_tree)

  File "/usr/lib/python2.6/site-packages/pyomo/pysp/phinit.py", line 812, in PHAlgorithmBuilder

    host=options.pyro_hostname)

  File "/usr/lib/python2.6/site-packages/pyutilib/component/core/core.py", line 934, in __call__

    return PluginFactory(_interface._factory_cls[_name], args, **kwds)

  File "/usr/lib/python2.6/site-packages/pyutilib/component/core/core.py", line 964, in PluginFactory

    obj = cls(*args, **kwds)

  File "/usr/lib/python2.6/site-packages/pyomo/solvers/plugins/smanager/phpyro.py", line 48, in __init__

    AsynchronousActionManager.__init__(self)

  File "/usr/lib/python2.6/site-packages/pyomo/opt/parallel/manager.py", line 76, in __init__

    self.clear()

  File "/usr/lib/python2.6/site-packages/pyomo/solvers/plugins/smanager/phpyro.py", line 58, in clear

    self.client = pyutilib.pyro.Client(host=self.host)

  File "/usr/lib/python2.6/site-packages/pyutilib/pyro/client.py", line 37, in __init__

    self.ns = get_nameserver(host)

  File "/usr/lib/python2.6/site-packages/pyutilib/pyro/util.py", line 79, in get_nameserver

    raise SystemExit

SystemExit

[Watson, Jean-Paul]

Whoops – I’m losing track of e-mails. First, try “mpiexec” on the linux command line. With no arguments. If that yields a command-not-found, then you’ll have to work with your system administrators to figure out how to add the correct directory to your PATH variable.