Thanks Victor. This is very helpful. I am also interested in the binary variable method. How does using binary variables solve the problem? It might help if I give a little background on the problem.
I am trying to set up a relatively simple dynamic programming problem in AMPL.
I have two controls, q and z, plus a stochastic element y. z is discrete.
The state space is (x,y) where
x = z_t-1
The Bellman for this problem is:
V(x,y) = max_q,z (payoff(x,y,z,q) + beta* E[V(z,y')])
Where the expectation is taken over y'.
At the optimal q=q* and z=z*, it should be the case that:
V(x,y) = (payoff(x,y,z*,q*) + beta* E[V(z*,y')])
I setting this up by letting the solver choose q and z to minimize the difference between the left and right hand side of the equation subject to some constraints on q and z.
The payoff function has an analytical form in q and z. The problem comes with integrating over the expected future value. Since V(z,y) is an unknown function, I have to index it based on the value of chosen for z. I don't see another way around it. How might I use binary variables to allow for this type of indexing?