Permission Error [Errno 13] with results.yml with more complex abstract model, no issues with knapsack concrete example

[Aml8992]

Hi all,

I have a permission error with results.yml with a large abstract model that includes a piecewise linear expression. I have two questions:

1) I verified that I can write to the location where I am printing results.yml by running knapsack-concrete.py successfully. What are other potential reasons why permission would be denied to results.yml?

2) The results.yml error appeared after introducing the piecewise function into the model. It is currently coded as such: 

model.FloodPw = Piecewise(model.T, model.CFlood, model.QSwRes, pw_pts=bpts, pw_repn='BIGM_BIN', pw_constr_type='EQ', f_rule=fcost, unbounded_domain_var=True)

Using glpk as my solver, what would be some recommendations for pw_repn as it does not support sos2 constraints? Is there an issue with the piecewise expression itself?

Traceback for the run is below:

ERROR: Unexpected exception (PermissionError) while running model H:/ERD/ANCHOR/Assignments/07_WMOSTEnhancement/02_WMOSTcoding/WMOSTv3/v3_PyomoCoding/Round2-codingv3afromv2.1/Wmodel.py:

[Errno 13] Permission denied: 'results.yml'

Traceback (most recent call last):

  File "c:\anaconda3\lib\runpy.py", line 184, in _run_module_as_main

    "__main__", mod_spec)

  File "c:\anaconda3\lib\runpy.py", line 85, in _run_code

    exec(code, run_globals)

  File "C:\Anaconda3\Scripts\pyomo.exe\__main__.py", line 9, in <module>

    sys.exit(main())

  File "c:\anaconda3\lib\site-packages\pyomo\scripting\pyomo_main.py", line 83, in main

    retval = _options.func(_options, _unparsed)

  File "c:\anaconda3\lib\site-packages\pyomo\scripting\plugins\solve.py", line 214, in solve_exec

    name='pyomo solve')

  File "c:\anaconda3\lib\site-packages\pyutilib\workflow\task.py", line 109, in __call__

    self.execute()

  File "c:\anaconda3\lib\site-packages\pyomo\util\_task.py", line 160, in execute

    retval = self._fn(**self._kwds)

  File "c:\anaconda3\lib\site-packages\pyomo\scripting\util.py", line 931, in run_command

    retval = command(options=options, parser=parser)

  File "c:\anaconda3\lib\site-packages\pyomo\scripting\pyomo_command.py", line 354, in run_pyomo

    opt=opt_data.opt)

  File "c:\anaconda3\lib\site-packages\pyutilib\workflow\task.py", line 109, in __call__

    self.execute()

  File "c:\anaconda3\lib\site-packages\pyomo\util\_task.py", line 163, in execute

    retval = self._fn(data, **self._kwds)

  File "c:\anaconda3\lib\site-packages\pyomo\scripting\util.py", line 661, in process_results

    format=data.options.postsolve.results_format)

  File "c:\anaconda3\lib\site-packages\pyomo\opt\results\results_.py", line 83, in write

    OUTPUT = open(kwds['filename'],"w")

Thanks,

Alyssa

[Gabriel Hackebeil]

I can’t think of a scenario where this would be caused by the Piecewise component. What is the output from GLPK when this happens? You might have include "--stream-solver” in the set of command-line options in order to see the solver output.

Gabe

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Aml8992]

Hi Gabe,

Unfortunately (or rather fortunately), after running the model this morning, I am no longer getting the same error. I would provide the GLPK log, but the temporary files no longer exist. The outputs from the cdmoutput.txt file are below.

[    0.00] Setting up Pyomo environment

[    0.00] Applying Pyomo preprocessing actions

[    0.71] Creating model

**WARNING: Piecewise component 'FloodPw[(1,)]' has detected slopes of consecutive piecewise segments to be within 0.1 of one another. Refer to the Piecewise help documentation for information on how to disable this warning.

[   28.25] Applying solver

Solver log file: 'C:\Users\LeA\AppData\Local\Temp\tmpwnhep713.glpk.log'

Solver solution file: 'C:\Users\LeA\AppData\Local\Temp\tmp7r_ogvv_.glpk.raw'

Solver problem files: ('C:\\Users\\LeA\\AppData\\Local\\Temp\\tmpuhwyli1q.pyomo.lp',)

GLPSOL: GLPK LP/MIP Solver, v4.57

Parameter(s) specified in the command line:

 --write C:\Users\LeA\AppData\Local\Temp\tmp7r_ogvv_.glpk.raw --wglp C:\Users\LeA\AppData\Local\Temp\tmpak40rv6o.glpk.glp

 --cpxlp C:\Users\LeA\AppData\Local\Temp\tmpuhwyli1q.pyomo.lp

Reading problem data from 'C:\Users\LeA\AppData\Local\Temp\tmpuhwyli1q.pyomo.lp'...

C:\Users\LeA\AppData\Local\Temp\tmpuhwyli1q.pyomo.lp:998986: warning: lower bound of variable 'FloodPw(1)_bin_y(1)' redefined

C:\Users\LeA\AppData\Local\Temp\tmpuhwyli1q.pyomo.lp:998986: warning: upper bound of variable 'FloodPw(1)_bin_y(1)' redefined

113331 rows, 100582 columns, 558379 non-zeros

9130 integer variables, all of which are binary

1008116 lines were read

Writing problem data to 'C:\Users\LeA\AppData\Local\Temp\tmpak40rv6o.glpk.glp'...

927677 lines were written

GLPK Integer Optimizer, v4.57

113331 rows, 100582 columns, 558379 non-zeros

9130 integer variables, all of which are binary

Preprocessing...

3707 constraint coefficient(s) were reduced

35386 rows, 31626 columns, 195677 non-zeros

9125 integer variables, all of which are binary

Scaling...

 A: min|aij| =  2.997e-18  max|aij| =  8.791e+11  ratio =  2.933e+29

GM: min|aij| =  5.366e-08  max|aij| =  1.864e+07  ratio =  3.473e+14

EQ: min|aij| =  3.011e-15  max|aij| =  1.000e+00  ratio =  3.321e+14

2N: min|aij| =  3.069e-15  max|aij| =  1.613e+00  ratio =  5.257e+14

Constructing initial basis...

Size of triangular part is 31734

Solving LP relaxation...

GLPK Simplex Optimizer, v4.57

35386 rows, 31626 columns, 195677 non-zeros

      0: obj =   4.798808564e+05 inf =   1.837e+06 (7006)

    500: obj =   5.570902605e+05 inf =   1.610e+06 (6756) 3

   1000: obj =   5.696416711e+05 inf =   1.515e+06 (6473) 4

   1500: obj =   5.684372329e+05 inf =   1.468e+06 (6417) 3

   2000: obj =   6.620873486e+05 inf =   1.335e+06 (5986) 4

   2500: obj =   6.617553431e+05 inf =   1.269e+06 (5642) 5

   3000: obj =   6.613274762e+05 inf =   1.209e+06 (5356) 3

   3500: obj =   6.602259273e+05 inf =   1.190e+06 (4903) 5

   4000: obj =   6.628755177e+05 inf =   1.139e+06 (4589) 5

   4500: obj =   6.555426995e+05 inf =   1.137e+06 (4485) 4

   5000: obj =   6.435705042e+05 inf =   1.107e+06 (4222) 2

   5500: obj =   5.958739903e+05 inf =   1.066e+06 (3995) 3

   6000: obj =   7.650060384e+05 inf =   9.359e+05 (3654) 3

   6500: obj =   2.336245725e+06 inf =   8.967e+05 (3455) 4

   7000: obj =   6.582684580e+06 inf =   8.685e+05 (3285) 3

   7500: obj =   1.088010499e+07 inf =   8.459e+05 (3176) 4

   8000: obj =   1.572699965e+07 inf =   8.452e+05 (3155) 4

   8500: obj =   1.958781452e+07 inf =   8.259e+05 (3063) 4

   9000: obj =   1.644184533e+08 inf =   7.162e+05 (2610) 1

   9500: obj =   1.986653725e+08 inf =   5.129e+05 (2351) 3

  10000: obj =   2.305488212e+08 inf =   4.910e+05 (2260) 3

  10500: obj =   2.547986415e+08 inf =   7.523e+04 (2140) 3

  11000: obj =   2.469463593e+08 inf =   6.259e+04 (2037) 4

  11500: obj =   2.103713668e+08 inf =   3.524e+04 (1906) 3

  12000: obj =   1.558976274e+08 inf =   5.156e+03 (1694) 4

  12500: obj =   1.397855052e+08 inf =   3.062e+03 (1277) 1

  13000: obj =   1.359285713e+08 inf =   2.333e+03 (804)

  13500: obj =   1.360216288e+08 inf =   1.761e+03 (306)

Warning: numerical instability (primal simplex, phase I)

Warning: numerical instability (primal simplex, phase I)

  13980: obj =   1.441027862e+08 inf =   1.419e-11 (0) 6

* 14000: obj =   1.430578456e+08 inf =   1.424e-11 (8152)

* 14500: obj =   1.321115807e+08 inf =   1.419e-11 (7790) 5

* 15000: obj =   1.235937224e+08 inf =   1.418e-11 (7276) 4

* 15500: obj =   1.235937224e+08 inf =   1.418e-11 (6776)

* 16000: obj =   1.232965664e+08 inf =   1.420e-11 (6335) 1

* 16500: obj =   1.158591255e+08 inf =   1.415e-11 (5931) 5

* 17000: obj =   1.113043948e+08 inf =   1.415e-11 (5425) 3

* 17500: obj =   1.113043948e+08 inf =   1.415e-11 (4925)

* 18000: obj =   7.464367863e+07 inf =   1.412e-11 (4018) 3

* 18500: obj =   2.708906008e+07 inf =   1.387e-11 (2385) 3

* 19000: obj =   4.854647007e+05 inf =   1.387e-11 (289) 3

* 19500: obj =   4.664505466e+05 inf =   1.408e-11 (0) 1

OPTIMAL LP SOLUTION FOUND

Integer optimization begins...

+ 19500: mip =     not found yet >=              -inf        (1; 0)

+ 19506: >>>>>   4.664505466e+05 >=   4.664505466e+05   0.0% (2; 0)

+ 19506: mip =   4.664505466e+05 >=     tree is empty   0.0% (0; 3)

INTEGER OPTIMAL SOLUTION FOUND

If you have any additional thoughts as to why the error occurred in the first place (it was not isolated, but occurred consistently for multiple model runs), it'd be much appreciated.

Thanks,

Alyssa

[Gabriel Hackebeil]

My guess is that it had to do with multiple processes trying to use the same file. Windows likely denies permission for a new process to use a file if the one that created that file has not released it’s lock on it.

Now that might be indicative of us not properly closing files in the pyomo command, but we also often come across cases where Windows is slow to release a file lock (e.g., when trying to clean up temporary files). I’ll do a quick scan of the code to see if this is possibly our fault.

Gabe

[Aml8992]

Thanks for the quick reply and I'll keep that in mind as we do further testing.