TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

[Izzy]

Hi all,

I am quite new at this stuff...

Just trying to run a PySP model (you may see the ReferenceModel.py codes below). However, I got this error message, stating that:

Importing module=/Users/<myusername>/Desktop/model/ReferenceModel.py

Module successfully loaded

ERROR: Rule failed when generating expression for constraint Cons2:

TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

ERROR: Constructing component 'Cons2' from data=None failed:

TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

ERROR: "[base]/site-packages/pyomo/pysp/scenariotree/instance_factory.py", 629, construct_scenario_instance

Failed to create model instance for scenario=Scenario1

Exception encountered. Scenario tree manager attempting to shut down.

runef: TYPE ERROR:

unsupported operand type(s) for *: 'int' and 'NoneType'

To obtain further information regarding the source of the exception, use the --traceback option

I've used the command "runef --model-directory=model --instance-directory=scenarios --solve --solver=cplex" to run the model.

Can anyone help me with this? What is wrong with the 'Cons2' ? Or, what is the crux of the problem?

Thanks, Izzy

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

ReferenceModel.py:

from pyomo.core import *

model = AbstractModel()

model.A = Param(within=NonNegativeReals)

model.B = Param(within=NonNegativeReals)

model.G = Param(within=NonNegativeReals)

model.b = Param(within=Binary)

def max_rule(model):

    return max(model.A, model.B)

def min_rule(model):

    return min(model.A, model.B)

model.Mx = Param(rule=max_rule)

model.Mn = Param(rule=min_rule)

model.P = Var(bounds=(0.0, 10.0))

model.d = Var(within=Reals)

def d_rule(model):

    return model.d == (model.P - model.G)

model.Cons0 = Constraint(rule=d_rule)

def binary_cons_rule1(model):

    return model.b * model.d >= 0

model.Cons1 = Constraint(rule=binary_cons_rule1)

def binary_cons_rule2(model):

    return (1 - model.b) * (-1 * model.d) >= 0

model.Cons2 = Constraint(rule=binary_cons_rule2)

def FirstStage_rule(model):

    return 0

model.FirstStageCost = Expression(rule=FirstStage_rule)

def SecondStage_rule(model):

    return model.b * (model.d * (model.Mx * 1.03 - model.A)) + (1 - model.b) * ((-1) * model.d * (model.A - model.Mn * 0.97))

model.SecondStageCost = Expression(rule=SecondStage_rule)

def objective_rule(model):

    return model.FirstStageCost + model.SecondStageCost

model.Objective_Func = Objective(rule=objective_rule, sense=minimize)

 

[Gabriel Hackebeil]

Judging from the constraint rule for Cons2, it is very likely that the parameter “b” has not been assigned a value in the data file. Although, I would have expected that error to show up earlier, like during the construction of Cons1.

Would you mind attaching the data file for Scenario1?

Gabe

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Izzy]

Hi Gabriel, thanks for the prompt reply.

I was thinking of in the same vein; why I am not having a problem with 'Cons1' as well. That's weird. 

And yes, I did not assign any value to "b" as it has already been defined as a binary variable. Apparently, it is supposed to take values 0 or 1.

Please find my ScenarioStructure.dat, Scenario1.dat, Scenario2.dat and Scenario3.dat files below. Values of parameters in scenarios are arbitrary; just to check whether the model works.

Thanks, Izzy.

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

_________ScenarioStructure.dat_________

set Stages := FirstStage SecondStage ;

set Nodes := RootNode

             Scenario1Node

             Scenario2Node

             Scenario3Node ;

param NodeStage := RootNode         FirstStage

                   Scenario1Node    SecondStage

                   Scenario2Node    SecondStage

                   Scenario3Node    SecondStage ;

set Children[RootNode] := Scenario1Node

                                           Scenario2Node

                                           Scenario3Node ;

param ConditionalProbability := RootNode          1.0

                                Scenario1Node     0.33333333

                                Scenario2Node     0.33333334

                                Scenario3Node     0.33333333 ;

set Scenarios := Scenario1

                           Scenario2

                           Scenario3 ;

param ScenarioLeafNode :=

                    Scenario1      Scenario1Node

                    Scenario2      Scenario2Node

                    Scenario3      Scenario3Node ;

set StageVariables[FirstStage] :=  P [*];

set StageVariables[SecondStage] := d [*];

param StageCost := FirstStage  FirstStageCost

                   SecondStage SecondStageCost ;

_________Scenario1.dat_________

# scenario-1

param A := 150 ;

param B := 160 ;

param G := 7 ;

_________Scenario2.dat_________

# scenario-2

param A := 150 ;

param B := 140 ;

param G := 8 ;

_________Scenario3.dat_________

# scenario-3

param A := 150 ;

param B := 100 ;

param G := 5 ;

[Gabriel Hackebeil]

When I run things on the developer version of Pyomo, I get an error during Cons1 construction (which makes sense). So you might want to consider updating to the most recent version of Pyomo just in case you are running into some kind of bug (pip install -U Pyomo).

Another thing to note is that, depending on the value of b, either Cons1 or Cons2 will result in another exception due to returning a trivial value (because you will have 0*lhs >= 0). To get around this error you would need to rewrite them as:

def binary_cons_rule1(model):

    if value(model.b) == 0:

         return Constraint.Skip

    return model.b * model.d >= 0
model.Cons1 = Constraint(rule=binary_cons_rule1)

def binary_cons_rule2(model):

    if value(model.b) == 1:

        return Constraint.Skip

    return (1 - model.b) * (-1 * model.d) >= 0
model.Cons2 = Constraint(rule=binary_cons_rule2)

Gabe

[Izzy]

Gabe Hi,

I've got the version update done. "Pyomo 4.4.1 (VOTD) (CPython 2.7.10 on Darwin 14.5.0)"

Also modified the code just like you suggested. 

Additionally, I've changed the way I defined "b"; it is now being considered as a binary variable rather than a parameter within binary subset ({0 ,1}). Accordingly, it takes part in the set of 'Second-Stage Variables' along with "d". As "b" and "d" take their values after the model determines the values of "A", "B" and "G", which are stochastic parameters, with regard to scenario data.

You may see the modified ReferenceModel.py codes below and the scenario stuff are attached.

Now, the model comes up with this new problem:

$ runef --model-location=model --instance-directory=scenario_data --solve --solver=cplex

Importing module=/Users/<myusername>/Desktop/model/ReferenceModel.py

Module successfully loaded

ERROR: Rule failed when generating expression for constraint Cons1:

ValueError: No value for uninitialized NumericValue object b

ERROR: Constructing component 'Cons1' from data=None failed:

ValueError: No value for uninitialized NumericValue object b

ERROR: "[base]/site-packages/pyomo/pysp/scenariotree/instance_factory.py", 629, construct_scenario_instance

Failed to create model instance for scenario=Scenario1

Exception encountered. Scenario tree manager attempting to shut down.

runef: VALUE ERROR:

No value for uninitialized NumericValue object b

To obtain further information regarding the source of the exception, use the --traceback option

*** WARNING: The following options were explicitly set but never accessed during execution of this command:

 - solve: True

 - solver: cplex

*** If you believe this is a bug, please report it to the PySP developers.

It seems that it is not enough to define "b" as a binary variable; we are asked to initialize a value for it.

What shall I do next? Any further suggestions...

Thanks, Izzy!

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

ReferenceModel.py______

from pyomo.core import *

#

# Model

#

model = AbstractModel()

#

# Parameters

#

model.A = Param(within=NonNegativeReals)

model.B = Param(within=NonNegativeReals)

model.G = Param(within=NonNegativeReals)

def max_rule(model):

    return max(model.A, model.B)

def min_rule(model):

    return min(model.A, model.B)

model.Mx = Param(rule=max_rule)

model.Mn = Param(rule=min_rule)

#

# Variables

#

model.b = Var(within=Binary)

model.P = Var(bounds=(0.0, 10.0))

model.d = Var(within=Reals)

#

# Constraints

#

def d_rule(model):

    return model.d == (model.P - model.G)

model.Cons0 = Constraint(rule=d_rule)

def binary_cons_rule1(model):

    if value(model.b) == 0:

        return Constraint.Skip

    return model.b * model.d >= 0

model.Cons1 = Constraint(rule=binary_cons_rule1)

def binary_cons_rule2(model):

    if value(model.b) == 1:

        return Constraint.Skip

    return (1 - model.b) * (-1 * model.d) >= 0

model.Cons2 = Constraint(rule=binary_cons_rule2)

#

# Stage-specific Computations

#

def FirstStage_rule(model):

    return 0

model.FirstStageCost = Expression(rule=FirstStage_rule)

def SecondStage_rule(model):

    return model.b * (model.d * (model.Mx - model.A)) + (1 - model.b) * ((-1) * model.d * (model.A - model.Mn))

model.BecondStageCost = Expression(rule=SecondStage_rule)

#

# Objective Function

#

def objective_rule(model):

    return model.FirstStageCost + model.BecondStageCost

model.Objective_Func = Objective(rule=objective_rule, sense=minimize)

[Gabriel Hackebeil]

Now that b is a variable, you no longer need the "if value(b) == …” clauses in the constraint rules for Cons1 and Cons1.

Gabe

<Scenario1.dat><Scenario2.dat><Scenario3.dat><ScenarioStructure.dat>

[Izzy]

Great, now the model seems able to overcome all previous problems.

But. Now, I encounter that message:

$ runef --model-directory=models --instance-directory=scenarios --solve --solver=cplex

WARNING: "[base]/site-packages/pyomo/pysp/util/config.py", 1867, __call__

DEPRECATED: The '--model-directory' command-line option has been deprecated and will be removed in the future. Please use --model-location instead.

WARNING: "[base]/site-packages/pyomo/pysp/util/config.py", 1867, __call__

DEPRECATED: The '--model-directory' command-line option has been deprecated and will be removed in the future. Please use --model-location instead.

Importing module=/Users/<myusername>/Desktop/models/ReferenceModel.py

Module successfully loaded

Initializing extensive form algorithm for stochastic programming problems.

Exception encountered. Scenario tree manager attempting to shut down.

runef: RUN-TIME ERROR:

EF solve failed solution status check:

Solver Status: error

Termination Condition: unknown

Solution Status: <undefined>

To obtain further information regarding the source of the exception, use the --traceback option

CPLEX is on the path. I've double-checked; simply typing 'cplex' command results in the launch of CPLEX.

I've tried GLPK. Unfortunately, it did not help me so much, as GLPK is unable to handle objective func.s with quadratic terms.

What might be the matter this time?

Thanks a lot ! Izzy.

P.S. I know, we are now so beyond the scope of the subject that I've started initially. But I preferred to stay in the same subject for the sake of traceability and to avoid too many new subjects.

[Watson, Jean-Paul]

Can you add the command line options “--verbose” and “--output-solver-log” and re-post the results? I suspect an issue with invoking the solver, and this will provide more diagnostic information.

 

jpw

[Izzy]

Here is the output:

$ runef --model-location=model --instance-directory=scenario_data --solve --solver=cplex --verbose --output-solver-log

Importing model and scenario tree files

Importing module=/Users/<myusername>/Desktop/model/ReferenceModel.py

Module successfully loaded

Time to import model and scenario tree structure files=0.07 seconds

Scenario Tree Detail

----------------------------------------------------

Tree Nodes:

Name=RootNode

Stage=FirstStage

Parent=None

Conditional probability=1.0000

Children:

Scenario1Node

Scenario2Node

Scenario3Node

Scenarios:

Scenario1

Scenario2

Scenario3

Name=Scenario1Node

Stage=SecondStage

Parent=RootNode

Conditional probability=0.3333

Children:

None

Scenarios:

Scenario1

Name=Scenario2Node

Stage=SecondStage

Parent=RootNode

Conditional probability=0.3333

Children:

None

Scenarios:

Scenario2

Name=Scenario3Node

Stage=SecondStage

Parent=RootNode

Conditional probability=0.3333

Children:

None

Scenarios:

Scenario3

----------------------------------------------------

Stages:

Name=FirstStage

Tree Nodes: 

RootNode

Variables: 

P : [*] 

Cost Variable: 

FirstStageCost

Name=SecondStage

Tree Nodes: 

Scenario1Node

Scenario2Node

Scenario3Node

Variables: 

b : [*] 

d : [*] 

Cost Variable: 

SecondStageCost

----------------------------------------------------

Scenarios:

Name=Scenario1

Probability=0.3333

Leaf node=Scenario1Node

Tree node sequence:

RootNode

Scenario1Node

Name=Scenario2

Probability=0.3333

Leaf node=Scenario2Node

Tree node sequence:

RootNode

Scenario2Node

Name=Scenario3

Probability=0.3333

Leaf node=Scenario3Node

Tree node sequence:

RootNode

Scenario3Node

----------------------------------------------------

Scenario tree is valid!

Initializing ScenarioTreeManagerClientSerial with options:

 *                           verbose: True

 -                        disable_gc: False

 -                           profile: 0

 -                         traceback: False

 -     output_scenario_tree_solution: False

 -          solution_saver_extension: ()

 -         solution_loader_extension: ()

 -                   solution_writer: ()

 -                       output_file: efout

 *                             solve: True

 -             output_scenario_costs: None

 - output_instance_construction_time: False

 -        compile_scenario_instances: False

 -                      output_times: False

 *                    model_location: model

 -                   model_directory: None

 *            scenario_tree_location: scenario_data

 -                instance_directory: None

 -       objective_sense_stage_based: None

 -        postinit_callback_location: ()

 -                    bounds_cfgfile: None

 - aggregategetter_callback_location: ()

 -                 aggregate_cfgfile: None

 -         scenario_tree_random_seed: None

 -                scenario_tree_seed: None

 - scenario_tree_downsample_fraction: 1.0

 -     scenario_bundle_specification: None

 -             create_random_bundles: 0

 -                    profile_memory: 0

 -                       cvar_weight: 1.0

 -            generate_weighted_cvar: False

 -                        risk_alpha: 0.95

 -                          cc_alpha: 0.0

 -                  cc_indicator_var: None

 -                            mipgap: None

 *                            solver: cplex

 -                         solver_io: None

 -                    solver_manager: serial

 -                    solver_options: ()

 -                 disable_warmstart: False

 -                         pyro_host: None

 -                         pyro_port: None

 -                     pyro_shutdown: False

 -             pyro_shutdown_workers: False

 -            symbolic_solver_labels: False

 *                 output_solver_log: True

 -                 keep_solver_files: False

 -             output_solver_results: False

 -                     shutdown_pyro: None

 -             shutdown_pyro_workers: None

 -    activate_jsonio_solution_saver: None

Constructing scenario tree instances

Scenario-based instance initialization enabled

Creating instance for scenario=Scenario1

Data for scenario=Scenario1 loads from file=/Users/<myusername>/Desktop/scenario_data/Scenario1.dat

Creating instance for scenario=Scenario2

Data for scenario=Scenario2 loads from file=/Users/<myusername>/Desktop/scenario_data/Scenario2.dat

Creating instance for scenario=Scenario3

Data for scenario=Scenario3 loads from file=/Users/<myusername>/Desktop/scenario_data/Scenario3.dat

Time to construct scenario instances=0.01 seconds

Linking instances into scenario tree

Time link scenario tree with instances=0.00 seconds

ScenarioTreeManagerClientSerial is successfully initialized

Overall initialization time=0.01 seconds

Initializing extensive form algorithm for stochastic programming problems.

Creating extensive form instance

Creating variables for master binding instance

Time to construct extensive form instance=0.00 seconds

Queuing extensive form solve

Welcome to IBM(R) ILOG(R) CPLEX(R) Interactive Optimizer 12.6.3.0

  with Simplex, Mixed Integer & Barrier Optimizers

5725-A06 5725-A29 5724-Y48 5724-Y49 5724-Y54 5724-Y55 5655-Y21

Copyright IBM Corp. 1988, 2015.  All Rights Reserved.

Type 'help' for a list of available commands.

Type 'help' followed by a command name for more

information on commands.

CPLEX> Logfile 'cplex.log' closed.

Logfile '/var/folders/89/4w6rn1p140d5gl5dqsx1d60r0000gn/T/tmpcwRYyp.cplex.log' open.

CPLEX> Problem '/var/folders/89/4w6rn1p140d5gl5dqsx1d60r0000gn/T/tmpn6VIUP.pyomo.lp' read.

Read time = 0.00 sec. (0.00 ticks)

CPLEX> Warning: no MIP start values read, no MIP start loaded.

MIP start file '/var/folders/89/4w6rn1p140d5gl5dqsx1d60r0000gn/T/tmpsKdIpc.cplex.mst' read.

CPLEX> Problem name         : /var/folders/89/4w6rn1p140d5gl5dqsx1d60r0000gn/T/tmpn6VIUP.pyomo.lp

Objective sense      : Minimize

Variables            :      11  [Nneg: 1,  Box: 3,  Free: 4,  Binary: 3,

                                 Qobj: 6]

Objective nonzeros   :       2

Objective Q nonzeros :       6

Linear constraints   :       7  [Equal: 7]

  Nonzeros           :      13

  RHS nonzeros       :       4

Quadratic constraints:       6  [Greater: 6]

  Linear terms       :       3

  Quadratic terms    :       6

  RHS nonzeros       :       0

Variables            : Min LB: 0.000000         Max UB: 10.00000       

Objective nonzeros   : Min   : 3.333333         Max   : 16.66667       

Objective Q nonzeros : Min   : 3.333333         Max   : 16.66667       

Linear constraints   :

  Nonzeros           : Min   : 1.000000         Max   : 1.000000       

  RHS nonzeros       : Min   : 1.000000         Max   : 8.000000       

Quadratic constraints:

  Linear terms       : Min   : 1.000000         Max   : 1.000000       

  Quadratic terms    : Min   : 1.000000         Max   : 1.000000       

  RHS nonzeros       : Min   : all zero         Max   : all zero       

CPLEX> Warning:  No solution found from 1 MIP starts.

Retaining values of one MIP start for possible repair.

CPLEX Error  5002: 'c_l_x16_' is not convex.

Presolve time = 0.00 sec. (0.00 ticks)

Root node processing (before b&c):

  Real time             =    0.00 sec. (0.02 ticks)

Parallel b&c, 4 threads:

  Real time             =    0.00 sec. (0.00 ticks)

  Sync time (average)   =    0.00 sec.

  Wait time (average)   =    0.00 sec.

                          ------------

Total (root+branch&cut) =    0.00 sec. (0.02 ticks)

Error termination, CPLEX Error  5002.

Solution time =    0.00 sec.

Deterministic time = 0.02 ticks  (10.19 ticks/sec)

CPLEX> CPLEX Error  1217: No solution exists.

No file written.

CPLEX> Waiting for extensive form solve

Done with extensive form solve - loading results

EF solve failed solution status check:

Solver Status: error

Termination Condition: unknown

Solution Status: <undefined>

Exception encountered. Scenario tree manager attempting to shut down.

Closing ScenarioTreeManagerClientSerial

runef: RUN-TIME ERROR:

EF solve failed solution status check:

Solver Status: error

Termination Condition: unknown

Solution Status: <undefined>

To obtain further information regarding the source of the exception, use the --traceback option

[Watson, Jean-Paul]

Well – CPLEX is installed just fine. The issue is that your model can’t be solved by CPLEX. It reports a particular constraint that is non-convex, and CPLEX can’t handle general non-convex constraints. If you add the option –symbolic-solver-labels and re-run, it will provide a more human-readable name (that relates to your model component names) that should allow you to identify which constraint is “at fault”.

[Izzy]

It is Cons1.

Okay. I will go through the model fundamentals to find out if I can somehow modify the problematic constraint.

But, on the other hand, isn't there a way to by-pass this non-convexity problem? Using another solver (gurobi, lindo, etc.) or any other way that I've not even heard of.

Izzy.

[Watson, Jean-Paul]

You could use ipopt – it’s a local solver, and can’t guarantee global optimality. There are other options, but I would start there.

[Izzy]

Alright then, worths a try.

Thanks JPW and Gabe.

I will probably be posting again soon, regarding the "other" options.

Izzy.