Hybrid Model

[Atefeh Fakourrad]

Dear Prof. Bierlaire,

I am trying to estimate an ordered probit regression in PandasBioegme. I have attached the code and have two questions about it:

1- I have identified two latent variables using exploratory factor analysis. Therefore, some of the indicators are associated with one latent variable and others are connected with the other latent variable in the measurement equations. When I define the function of "loglike", should I sum all the probabilities given that I have two separate latent variables? 

loglike = (

log(P_Negative_Impact)+log(P_Accept_More)+log(P_Care)+log(P_No_Change)+log(P_Receive_More)+log(B_Preventive)

)

2- There is a running issue in the last piece of the code where I want to estimate the parameters. Can you please let me know what this error (please see below) is about and what I should do?  

[12:51:52] < General > *** Initial values of the parameters are obtained from the file __LatentOrdered.iter [12:51:52] < Warning > Cannot read file __LatentOrdered.iter. Statement is ignored. [12:51:53] < General > Log likelihood (N = 3456): -1.797693e+308 Gradient norm: inf Hessian norm: inf

--------------------------------------------------------------------------- biogemeError Traceback (most recent call last) <ipython-input-101-31b7a149babd> in <module>() 1 # Estimate the parameters ----> 2 results = biogeme.estimate() 3 4 print(f'Estimated betas: {len(results.data.betaValues)}') 5 print(f'final log likelihood: {results.data.logLike:.3f}')

5 frames

/usr/local/lib/python3.7/dist-packages/biogeme/biogeme.py in estimate(self, bootstrap, algorithm, algoParameters) 872 # yep.stop() 873 --> 874 output = self.optimize(self.betaInitValues) 875 xstar, optimizationMessages = output 876 # Running time of the optimization algorithm 

/usr/local/lib/python3.7/dist-packages/biogeme/biogeme.py in optimize(self, startingValues) 1098 1099 results = self.algorithm( -> 1100 theFunction, startingValues, self.bounds, self.algoParameters 1101 ) 1102 return results /usr/local/lib/python3.7/dist-packages/biogeme/optimization.py in simpleBoundsNewtonAlgorithmForBiogeme(fct, initBetas, bounds, parameters) 496 eta1=eta1, 497 eta2=eta2, --> 498 enlargingFactor=enlargingFactor, 499 ) 500 /usr/local/lib/python3.7/dist-packages/biogeme/algorithms.py in simpleBoundsNewtonAlgorithm(fct, bounds, x0, proportionTrueHessian, infeasibleConjugateGradient, delta0, tol, steptol, cgtol, maxiter, eta1, eta2, enlargingFactor) 1898 nhev = 0 1899 if proportionTrueHessian > 0: -> 1900 f, g, H = fct.f_g_h() 1901 nfev += 1 1902 ngev += 1

/usr/local/lib/python3.7/dist-packages/biogeme/biogeme.py in f_g_h(self, batch) 1410 if self.recalculate: 1411 self.fv, self.gv, self.hv, _ = self.like_deriv( -> 1412 self.x, self.scaled, hessian=True, bhhh=False, batch=batch 1413 ) 1414 self.bhhhv = None /usr/local/lib/python3.7/dist-packages/biogeme/biogeme.py in calculateLikelihoodAndDerivatives(self, x, scaled, hessian, bhhh, batch) 648 if not np.isfinite(gradnorm): 649 error_msg = f'The norm of the gradient is {gradnorm}: g={g}' --> 650 raise excep.biogemeError(error_msg) 651 652 if self.saveIterations: 

biogemeError: The norm of the gradient is inf: g=[-1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308]

KInd regards,

Atefeh.

[Bierlaire Michel]

On 21 Aug 2021, at 15:50, Atefeh Fakourrad <a.fak...@gmail.com> wrote:

Dear Prof. Bierlaire,

I am trying to estimate an ordered probit regression in PandasBioegme. I have attached the code and have two questions about it:

1- I have identified two latent variables using exploratory factor analysis. Therefore, some of the indicators are associated with one latent variable and others are connected with the other latent variable in the measurement equations. When I define the function of "loglike", should I sum all the probabilities given that I have two separate latent variables? 

I don’t understand your question. 

For an explanation about latent variables in Biogeme, see http://transp-or.epfl.ch/documents/technicalReports/Bier18b.pdf

In any case, I would suggest to start estimating a model without latent variables, and integrate them later on, one at a time.

loglike = (

log(P_Negative_Impact)+log(P_Accept_More)+log(P_Care)+log(P_No_Change)+log(P_Receive_More)+log(B_Preventive)

)

2- There is a running issue in the last piece of the code where I want to estimate the parameters. Can you please let me know what this error (please see below) is about and what I should do?  

As mentioned by the error message "The norm of the gradient is inf”. There is a clear numerical issue. 

You have a specification problem. 

Did you look at the FAQ? The init loglikelihood is -1.797693e+308 and no iteration is performed. What should I do?

[12:51:52] < General > *** Initial values of the parameters are obtained from the file __LatentOrdered.iter [12:51:52] < Warning > Cannot read file __LatentOrdered.iter. Statement is ignored. [12:51:53] < General > Log likelihood (N = 3456): -1.797693e+308 Gradient norm: inf Hessian norm: inf

--------------------------------------------------------------------------- biogemeError Traceback (most recent call last) <ipython-input-101-31b7a149babd> in <module>() 1 # Estimate the parameters ----> 2 results = biogeme.estimate() 3 4 print(f'Estimated betas: {len(results.data.betaValues)}') 5 print(f'final log likelihood: {results.data.logLike:.3f}')

5 frames

/usr/local/lib/python3.7/dist-packages/biogeme/biogeme.py in estimate(self, bootstrap, algorithm, algoParameters) 872 # yep.stop() 873 --> 874 output = self.optimize(self.betaInitValues) 875 xstar, optimizationMessages = output 876 # Running time of the optimization algorithm 

/usr/local/lib/python3.7/dist-packages/biogeme/biogeme.py in optimize(self, startingValues) 1098 1099 results = self.algorithm( -> 1100 theFunction, startingValues, self.bounds, self.algoParameters 1101 ) 1102 return results /usr/local/lib/python3.7/dist-packages/biogeme/optimization.py in simpleBoundsNewtonAlgorithmForBiogeme(fct, initBetas, bounds, parameters) 496 eta1=eta1, 497 eta2=eta2, --> 498 enlargingFactor=enlargingFactor, 499 ) 500 /usr/local/lib/python3.7/dist-packages/biogeme/algorithms.py in simpleBoundsNewtonAlgorithm(fct, bounds, x0, proportionTrueHessian, infeasibleConjugateGradient, delta0, tol, steptol, cgtol, maxiter, eta1, eta2, enlargingFactor) 1898 nhev = 0 1899 if proportionTrueHessian > 0: -> 1900 f, g, H = fct.f_g_h() 1901 nfev += 1 1902 ngev += 1

/usr/local/lib/python3.7/dist-packages/biogeme/biogeme.py in f_g_h(self, batch) 1410 if self.recalculate: 1411 self.fv, self.gv, self.hv, _ = self.like_deriv( -> 1412 self.x, self.scaled, hessian=True, bhhh=False, batch=batch 1413 ) 1414 self.bhhhv = None /usr/local/lib/python3.7/dist-packages/biogeme/biogeme.py in calculateLikelihoodAndDerivatives(self, x, scaled, hessian, bhhh, batch) 648 if not np.isfinite(gradnorm): 649 error_msg = f'The norm of the gradient is {gradnorm}: g={g}' --> 650 raise excep.biogemeError(error_msg) 651 652 if self.saveIterations: 

biogemeError: The norm of the gradient is inf: g=[-1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308]

KInd regards,

Atefeh.

--
You received this message because you are subscribed to the Google Groups "Biogeme" group.
To unsubscribe from this group and stop receiving emails from it, send an email to biogeme+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/biogeme/c5fe0815-ac75-4747-a240-eaa4d6987d01n%40googlegroups.com.
<Latent_Variables2.ipynb - Colaboratory.pdf>

[Atefeh Fakourrad]

Dear Prof. Bierlaire,

Thanks for your reply. Let me clarify my questions using an example: 

First, I start with a basic question about one of the notations used in the document explaining the estimation of choice models with latent variables. I need to ensure that I have grasped it correctly. What does sigma-star exactly mean in the measurement equations and how can we interpret it (N0.18 in the document)?

Ii=B0i + Bi*x-bar_s+sigma_i-star*epsilon_i 

My second question is about the specifications of the structural models. Let's assume that I am estimating an ICLV model with two latent variables: Car Lovers and Mobile Users. I need to have one sigma_s for each latent variable (in total, two) but only one omega for numerical integration, as follows, right? 

# Define a random parameter, normally distributed, designed to be used

# for numerical integration

omega = RandomVariable('omega')

density = dist.normalpdf(omega)

sigma_s1 = Beta('sigma_s1', 1, None, None, 0)

sigma_s2 = Beta('sigma_s2', 1, None, None, 0)

CARLOVERS = coef_intercept1     + coef_age_65_more * age_65_more + ... + sigma_s1 * omega

Mobile Users = coef_intercept2     + coef_mobile_usage * mobile_usage + ... + sigma_s2 * omega

Would you please let me know if this formulation is correct?

My last question is about the formulation of the "condlike" function. Let's assume Envir01 is an indicator associated with Car Lovers and Internet2 is an indicator related to Mobile Users. Should I multiply the probabilities of these indicators regardless of their association with either of the latent variables (please see below)

  condlike = ( P_Envir01 * P_Internet2 * condprob )

Many thanks for your help.

Best regards,

Ati. 

[Bierlaire Michel]

On 28 Aug 2021, at 13:49, Atefeh Fakourrad <a.fak...@gmail.com> wrote:

Dear Prof. Bierlaire,

Thanks for your reply. Let me clarify my questions using an example: 

First, I start with a basic question about one of the notations used in the document explaining the estimation of choice models with latent variables. I need to ensure that I have grasped it correctly. What does sigma-star exactly mean in the measurement equations and how can we interpret it (N0.18 in the document)?

Ii=B0i + Bi*x-bar_s+sigma_i-star*epsilon_i 

It is the scale of the error term of the measurement equation. Interpretation: the square of this parameter is the variance of the error term. 

My second question is about the specifications of the structural models. Let's assume that I am estimating an ICLV model with two latent variables: Car Lovers and Mobile Users. I need to have one sigma_s for each latent variable (in total, two) but only one omega for numerical integration, as follows, right? 

No. If you have two structural equations, you need two different error terms. As a consequence, I’d suggest to use Monte-Carlo integration instead of numerical integration. 

# Define a random parameter, normally distributed, designed to be used

# for numerical integration

omega = RandomVariable('omega')

density = dist.normalpdf(omega)

sigma_s1 = Beta('sigma_s1', 1, None, None, 0)

sigma_s2 = Beta('sigma_s2', 1, None, None, 0)

CARLOVERS = coef_intercept1     + coef_age_65_more * age_65_more + ... + sigma_s1 * omega

Mobile Users = coef_intercept2     + coef_mobile_usage * mobile_usage + ... + sigma_s2 * omega

Would you please let me know if this formulation is correct?

My last question is about the formulation of the "condlike" function. Let's assume Envir01 is an indicator associated with Car Lovers and Internet2 is an indicator related to Mobile Users. Should I multiply the probabilities of these indicators regardless of their association with either of the latent variables (please see below)

  condlike = ( P_Envir01 * P_Internet2 * condprob )

Yes. The likelihood of an observation is the product of the likelihood of each individual measurement equation, including the choice. 

To view this discussion on the web visit https://groups.google.com/d/msgid/biogeme/ecaf6f45-9713-40fa-a94a-a57834ad04bcn%40googlegroups.com.

[Peyman]

Dear Prof. Bierlaire,

Thanks for your reply. Following your guidance, I used Monte-Carlo integration to estimate the scale of the error terms of the structural equations (I have two latent variables) but the model cannot be estimated (there is an error: The second derivatives matrix is close to singularity). Following is the code related to the structural equations. Would you please let me know if I have missed something? 

# Define a random parameter, normally distributed, designed to be used

sigma_s1 = Beta('sigma_s1', 1, None, None, 0)

sigma_s2 = Beta('sigma_s2', 1, None, None, 0)

CARLOVERS = coef_intercept1  + coef_age_65_more * age_65_more + ... + sigma_s1 *  bioDraws('EC', 'NORMAL_MLHS')

Mobile Users = coef_intercept2 + coef_mobile_usage * mobile_usage + ... + sigma_s2 * bioDraws('EC', 'NORMAL_MLHS')

Kind regards,

Ati.

[Bierlaire Michel]

Did you try sequential estimation first, as explained in the document? 

You probably need to normalize one of the parameters. 

Also, in your implementation, you provide the exact same error term for car_lovers an mobile_users. I would suggest to use two different ones. 

bioDraws(‘EC_CL', 'NORMAL_MLHS’) and bioDraws(‘EC_MU', 'NORMAL_MLHS’)

To view this discussion on the web visit https://groups.google.com/d/msgid/biogeme/e7f249d2-54f3-4a4a-af1b-6eb6945e7db6n%40googlegroups.com.