Using Pyomo with Django

[Ryan Skene]

I am a rookie programmer (at it for about 2 years) and I am working on a school project that uses linear programming. I'd like to use a web portal built through django that takes in certain data and then runs that data through a pyomo lp and glpk solver.

I have hit a material snag in this approach, in that it appears that this is not functionality pyomo is currently capable of supporting. 

When I try to run Pyomo through Django, I get the following error: "signal only works in main thread"

There was a bit of a discussion on this in an earlier post, related to signaling/threading in cherrypy. Use with Django must suffer from the same problem.

I was just hoping for some high-level guidance on how to address this problem. I have no experience in signals/threading. In the post above, one user suggested commenting out signal code from pyutilib.

I also am not well-versed in having separate python processes communicating to each other and sharing data with actually importing them as modules, but i am guessing this code be a possibility.

Hoping someone can point in a fruitful direction.

[Ryan Skene]

anyone who reads this

Two options:    

1) use Django's --noreload --nothreading options on the runserver command. This eliminates the problem, but Django no longer conveniently reloads when you make code changes.

2) use Celery. setup in worker, pyomo has no issues. 

[Timo Korvola]

On 17.03.2018 21:37, Ryan Skene wrote:
> When I try to run Pyomo through Django, I get the following
> error: "signal only works in main thread"

This appears to be caused by Pyutilib attempting something that I don't
think should be necessary, at least not on Unix. When the user hits ^C
on a terminal, SIGINT is sent to the foreground process group. Parents
normally don't need to pass SIGINT to children, which get it directly as
members of the same process group. No idea how this works on Windows
though.

--
Timo Korvola

[Cristian Cocheci]

You're right about the cause, the question still remains how one can overcome this. One post I found suggested to "comment out the signal handling code in pyutilib". I am not knowledgeable enough in python to do that, so I am stuck for now.

--
        Timo Korvola

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Siirola, John D]

There is a PR for PyUtilib (https://github.com/PyUtilib/pyutilib/pull/32) that adds a global flag that sets the default value for whether or not PyUtilib’s subprocess run_command will attempt to set up the signal handlers.  Because Pyomo never overrides that default, changing the default in multi-threaded environments should work for you.

 

I expect that Bill will review / merge it in soon (tonight??) and then everyone will be good to go.  We are planning on cutting a Pyomo 5.4.4 release very soon (this week), and this fix should be available as part of that release.

 

john

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.

[Cristian Cocheci]

Got it, thank you! That indeed fixes this problem.

On Mon, Apr 9, 2018 at 4:20 PM, Siirola, John D <jds...@sandia.gov> wrote:

There is a PR for PyUtilib (https://github.com/PyUtilib/pyutilib/pull/32) that adds a global flag that sets the default value for whether or not PyUtilib’s subprocess run_command will attempt to set up the signal handlers.  Because Pyomo never overrides that default, changing the default in multi-threaded environments should work for you.

 

I expect that Bill will review / merge it in soon (tonight??) and then everyone will be good to go.  We are planning on cutting a Pyomo 5.4.4 release very soon (this week), and this fix should be available as part of that release.

 

john

 

From: pyomo...@googlegroups.com [mailto:pyomo-forum@googlegroups.com] On Behalf Of Cristian Cocheci
Sent: Monday, April 9, 2018 7:32 AM
To: pyomo...@googlegroups.com
Subject: [EXTERNAL] Re: Using Pyomo with Django

 

 

You're right about the cause, the question still remains how one can overcome this. One post I found suggested to "comment out the signal handling code in pyutilib". I am not knowledgeable enough in python to do that, so I am stuck for now.

On Mon, Apr 9, 2018 at 6:32 AM, Timo Korvola <Timo.K...@vtt.fi> wrote:

On 17.03.2018 21:37, Ryan Skene wrote:
> When I try to run Pyomo through Django, I get the following
> error: "signal only works in main thread"

This appears to be caused by Pyutilib attempting something that I don't
think should be necessary, at least not on Unix.  When the user hits ^C
on a terminal, SIGINT is sent to the foreground process group.  Parents
normally don't need to pass SIGINT to children, which get it directly as
members of the same process group.  No idea how this works on Windows
though.

--
        Timo Korvola

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum+unsubscribe@googlegroups.com.

[William Hart]

I just approved this PR.  I expect that we'll cut a new release in the next day or so.

[Lukas Hump]

Hi Ryan,

I am trying to do the same, but when i try to use the glpk solver i get the following error:

No executable found for solver 'glpk'

Then when i try: pip install glpk the following error occurs:

error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools

I think the last error occurs because i use django in a virtual environment since when i start to deploy the django project on a website, there is a clean environmen aswell. 

Could you tell me how you implemented the glpk solver in the django app?

[Watson, Jean-Paul]

That means GLPK is not in your PATH environment variable. You need to be able to get something other than “command not found” when you type “glpsol” from a command line prompt.

 

jpw

 

From: <pyomo...@googlegroups.com> on behalf of Lukas Hump <hum...@gmail.com>
Reply-To: "pyomo...@googlegroups.com" <pyomo...@googlegroups.com>
Date: Thursday, April 12, 2018 at 7:38 AM
To: Pyomo Forum <pyomo...@googlegroups.com>
Subject: [EXTERNAL] Re: Using Pyomo with Django

 

Hi Ryan,

--

You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lukas Hump]

Yes you were right i solved that problem by explicitly defining the path to the glpsol.exe 

    opt = SolverFactory("glpk", executable="Path to glpsol.exe/glpsol.exe")

Am Donnerstag, 12. April 2018 15:40:01 UTC+2 schrieb JPW:

That means GLPK is not in your PATH environment variable. You need to be able to get something other than “command not found” when you type “glpsol” from a command line prompt.

 

Thanks for your quick answer 

From: <pyomo...@googlegroups.com> on behalf of Lukas Hump <hum...@gmail.com>
Reply-To: "pyomo...@googlegroups.com" <pyomo...@googlegroups.com>
Date: Thursday, April 12, 2018 at 7:38 AM
To: Pyomo Forum <pyomo...@googlegroups.com>
Subject: [EXTERNAL] Re: Using Pyomo with Django

 

Hi Ryan,

 

I am trying to do the same, but when i try to use the glpk solver i get the following error:

No executable found for solver 'glpk'

Then when i try: pip install glpk the following error occurs:

 

 

error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools

 

I think the last error occurs because i use django in a virtual environment since when i start to deploy the django project on a website, there is a clean environmen aswell. 

 

Could you tell me how you implemented the glpk solver in the django app?

 

Am Samstag, 17. März 2018 21:45:51 UTC+1 schrieb Ryan Skene:

I am a rookie programmer (at it for about 2 years) and I am working on a school project that uses linear programming. I'd like to use a web portal built through django that takes in certain data and then runs that data through a pyomo lp and glpk solver.

 

I have hit a material snag in this approach, in that it appears that this is not functionality pyomo is currently capable of supporting. 

 

When I try to run Pyomo through Django, I get the following error: "signal only works in main thread"

 

There was a bit of a discussion on this in an earlier post, related to signaling/threading in cherrypy. Use with Django must suffer from the same problem.

 

I was just hoping for some high-level guidance on how to address this problem. I have no experience in signals/threading. In the post above, one user suggested commenting out signal code from pyutilib.

 

I also am not well-versed in having separate python processes communicating to each other and sharing data with actually importing them as modules, but i am guessing this code be a possibility.

 

Hoping someone can point in a fruitful direction.

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lukas Hump]

Hello, 

after I got the solver running with Django locally with these settings:

opt = SolverFactory("glpk")

opt.set_executable("/app/administration/solver/GLPK/w64/glpsol.exe")

pyutilib.subprocess.GlobalData.DEFINE_SIGNAL_HANDLERS_DEFAULT = False

results = opt.solve(model)

I tried to deploy my Django project on Heroku but when i run the solver(executes the model building and command on a request) i get the following Error:

ValueError: Failed to set executable for solver glpk. File with name=/app/administration/solver/GLPK/w64/glpsol.exe either does not exist or it is not executable. To skip this validation, call set_executable with validate=False.

Then when I set validate = False (opt.set_executable("/app/administration/solver/GLPK/w64/glpsol.exe",validate=False)) I run into the following Error:

ERROR: Solver (glpk) returned non-zero return code (-1)

Solver (glpk) did not exit normally

I am not sure where this error is coming from. I think it could be one of the following two issues:

1. The executable I am pointing at is a windows executable and Heroku does not support these.

2. I only got 512mb ram so the solver does terminate the process. 

I searched the internet for about half a day now and I have now idea how to tackle this problem.

Am Donnerstag, 12. April 2018 15:40:01 UTC+2 schrieb JPW:

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Siirola, John D]

You should almost never need to set “validate=False”.  Can you run at the command prompt and not within Pyomo?  For reference, I would expect running glpsol to produce the following output:

 

GLPSOL: GLPK LP/MIP Solver, v4.65

No input problem file specified; try glpsol --help

 

At a more fundamental level, when you “deploy” your application, are you also copying over the glpsol executable?  Is the path the same on the deployed image (I suspect it is not)?  For that matter, is the OS even the same?

 

john

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.