NEOS Server

[Michael Negus]

Hello,

I'm having some trouble using the NEOS server in Pyomo.

I'm calling a NEOS solver similar to how it's done both on the forum and on the NEOS guide here, http://neos-guide.org/content/third-party-tools, but my output is saying NEOS is not available.

Here's how I'm calling the solver:

instance = model.create_instance('FirstSys.dat')
opt = SolverFactory("ipopt")
solver_manager = SolverManagerFactory('neos')
results = solver_manager.solve(instance, opt=opt)
results.write()

And here's the output Pyomo is giving me:

WARNING: "[base]/site-packages/pyomo/neos/kestrel.py", 67, __init__
    NEOS is temporarily unavailable.
Traceback (most recent call last):
  File "/home/thy68636/DAWN_stable/Spin Systems/FirstSys.py", line 66, in <module>
    results = solver_manager.solve(instance, opt=opt)
  File "/dls_sw/apps/python/anaconda/1.7.0/64/lib/python2.7/site-packages/pyomo/opt/parallel/async_solver.py", line 33, in solve
    return self.execute(*args, **kwds)
  File "/dls_sw/apps/python/anaconda/1.7.0/64/lib/python2.7/site-packages/pyomo/opt/parallel/manager.py", line 106, in execute
    ah = self.queue(*args, **kwds)
  File "/dls_sw/apps/python/anaconda/1.7.0/64/lib/python2.7/site-packages/pyomo/opt/parallel/manager.py", line 121, in queue
    return self._perform_queue(ah, *args, **kwds)
  File "/dls_sw/apps/python/anaconda/1.7.0/64/lib/python2.7/site-packages/pyomo/neos/plugins/kestrel_plugin.py", line 93, in _perform_queue
    % (solver_name, str(sorted(self._solvers.keys()))))
pyomo.opt.parallel.manager.ActionManagerError: Solver 'asl' is not recognized by NEOS. Solver names recognized:
[]

Does anyone have any idea why this is happening?

Just to note, I have ipopt installed locally and running my code with that locally works fine, so there shouldn't be anything wrong with my actual Pyomo script hopefully.

Thanks.

[William Hart]

This looks like a possible bug.  What version of Pyomo are you using?  What operating system?

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gabriel Hackebeil]

I was able to reproduce this on Pyomo trunk. It is now fixed. Note that you can also just pass the solver name ‘ipopt’ rather than creating a Pyomo solver plugin. E.g.,

solver_manager.solve(instance, opt=‘ipopt’)

This is advantageous because it does not require Ipopt be available on your system. Currently Pyomo solver plugins can only be instantiated when the solver is available locally, which defeats the purpose of using the NEOS solver manager.

Gabe

[Michael Negus]

Unfortunately I'm still getting the same error. I've updated from trunk, it says I have version "Pyomo 4.4 (VOTD)", is this the right one? Sorry I'm quite new to this!

Also is there anything else you need to install as well as Pyomo to make the neos solver work? I may have missed something!

Thanks
Michael

[William Hart]

Did you update from the github "trunk"?  We recently moved pyomo to github, and I think that's what Gabe was referring to.

[Michael Negus]

I followed the instructions for installing from github that are listed here:
http://www.pyomo.org/blog/2015/12/9/installing-pyomo-trunk-using-pip
Is that from trunk?

[Siirola, John D]

That should put you on trunk, but if you did it correctly, you should see:

 

% pyomo --version

Pyomo 4.4 (trunk {git}) (CPython 2.7.12 on Linux 2.6.32-642.1.1.el6.x86_64)

 

Getting “VOTD” indicated that you have a snapshot from somewhere and not an actual git checkout.

 

John

 

 

From: pyomo...@googlegroups.com [mailto:pyomo...@googlegroups.com] On Behalf Of Michael Negus
Sent: Friday, July 15, 2016 7:57 AM
To: Pyomo Forum <pyomo...@googlegroups.com>
Subject: [EXTERNAL] Re: NEOS Server

 

I followed the instructions for installing from github that are listed here:
http://www.pyomo.org/blog/2015/12/9/installing-pyomo-trunk-using-pip
Is that from trunk?

http://www.pyomo.org/blog/2015/12/9/installing-pyomo-trunk-using-pip

[Michael Negus]

Ah okay, do you suggest I uninstall Pyomo and try to install it again? I tried doing the "pip install -U Pyomo" command but it still says I'm on VOTD.

[Gabe Hackebeil]

That process described on the Pyomo blog installs a snapshot of trunk. It can not be updated via pip. You need to uninstall Pyomo and Pyutilib (using pip) and then run the entire process again.

If you want to install from source and be able to update the repositories, then you need to manually clone both Pyomo and Pyutilib from GitHub and the run "python setup.py develop" inside the two directories.

Gabe

[Michael Negus]

Right, I'll get on that!

I've checked my source code though, and I have the changes that you made to the NEOS plugin on there, so I've definitely got an updated enough version for it to work, so i don't know.

[Gabriel Hackebeil]

So you are saying you get the same error as before when you run the following?

opt = SolverFactory("ipopt")
solver_manager = SolverManagerFactory('neos')

results = solver_manager.solve(model, opt=opt)

If so, I believe that can only mean that you have Pyomo installed in more than one location.

Gabe

[Michael Negus]

Hi! Sorry for the delayed reply.
I'm getting a similar error, but not the same one i.e. it doesn't say 'asl' is not recognised, now it just says ipopt isn't recognised.
I've also double checked everywhere on my system and I'm pretty certain that the Pyomo it's calling is the only time it appears on the system.

I'm running this code:

instance = model.create_instance('FirstSys.dat')

solver_manager = SolverManagerFactory('neos')

results = solver_manager.solve(instance, opt="ipopt")
results.write()

And getting this output:

WARNING: "[base]/site-packages/pyomo/neos/kestrel.py", 67, __init__
    NEOS is temporarily unavailable.
Traceback (most recent call last):

  File "/home/thy68636/DAWN_stable/Spin Systems/FirstSys.py", line 70, in <module>
    results = solver_manager.solve(instance, opt="ipopt")
  File "/home/thy68636/.local/lib/python2.7/site-packages/pyomo/opt/parallel/async_solver.py", line 33, in solve
    return self.execute(*args, **kwds)
  File "/home/thy68636/.local/lib/python2.7/site-packages/pyomo/opt/parallel/manager.py", line 106, in execute
    ah = self.queue(*args, **kwds)
  File "/home/thy68636/.local/lib/python2.7/site-packages/pyomo/opt/parallel/manager.py", line 121, in queue
    return self._perform_queue(ah, *args, **kwds)
  File "/home/thy68636/.local/lib/python2.7/site-packages/pyomo/neos/plugins/kestrel_plugin.py", line 96, in _perform_queue
    % (solver_name, str(sorted(self._solvers.keys()))))
pyomo.opt.parallel.manager.ActionManagerError: Solver 'ipopt' is not recognized by NEOS. Solver names recognized:
[]

Thank you,
Michael

[Siirola, John D]

Are you behind a proxy server? 

 

I just verified that trunk works with a direct connection to the internet, but when I am behind a proxy server, I can reproduce the error you report below.

 

john

[Michael Negus]

Yes, I'm currently on a remote connection to a different Linux machine via NoMachine, would that cause it?
Thanks,
Michael

[Gabriel Hackebeil]

Comparing your situation to what John said in his email, yes, it sounds like that is the source of the problem. I’m not sure how to help with this issue, however.

Gabe

[Michael Negus]

Hmm, well I've now managed to get a Linux machine to try it on and it's still giving the same error. It's a work computer so part of a server but the actual machine is a Linux machine, so I don't see why it wouldn't work.
Any suggestions? Sorry to be a pain

[Siirola, John D]

I don't think the problem is the machine (or the OS). My guess is that you are sitting on a network that is behind a proxy server: that is, your machine is not directly connected to the Internet, but instead is sitting behind a firewall that intercepts outgoing network traffic. This is very common on corporate networks. If you had to set environment variables like http_proxy on Linux before pip would install Pyomo, then you are definitely behind a proxy server. 

At this point, I am hoping that Bill will weigh in with suggestions for getting the Neos interface working behind a proxy...

John

[Michael Negus]

Ah right, that makes sense. Thanks for the explanation! I'm quite new to a lot of this.
But yeh, unfortunately that does put me in a bit of a predicament. My task is to test multiple different solvers, specifically non-linear ones, using Pyomo. The NEOS server seems to be the best way at accessing a lot of these solvers, as many of them require you to buy them if you're not using them via NEOS. I guess if the NEOS interface can't work behind a proxy I'll have to try and sort some sort something out with the IT support at my work, but we'll see.

Thanks for all that you've done so far anyway and the advice!

[Michael Negus]

I've spoken to the IT guys where I work. They said that the reason it doesn't work is because of the port it accesses NEOS through, which is 3332.
Apparently this is quite an unusual port and is blocked by the firewall at my work due to threat from viruses.
Is the choice of port with Pyomo or NEOS? As in, do you have to connect to NEOS through this port?

[Siirola, John D]

The port is set by the NEOS XML-RPC server.

[William Hart]

The port is specified by the NEOS server, so this isn't a Pyomo thing.  If your IT guys won't make an exception, then you're stuck.

[Bill Hart]

I've updated the Pyomo master branch to connect to the NEOS server using HTTPS on port 3333.  You can try this out and see if that works from within your firewall.

--Bill

[Michael Negus]

Hi Bill. Unfortunately that's still blocked by the firewall at my work, but I've followed other routes now anyway so it's okay. Thanks very much for your help though!

Michael