Null log likelihood in Pandas Biogeme

161 views
Skip to first unread message

Maryna Öztürker

unread,
Sep 8, 2020, 2:30:45 AM9/8/20
to Biogeme
Dear Prof. Bierlaire,

I am trying to find the model which better describes the dataset. I have estimated several models using Pandas Biogeme, namely MNL, NL, ML (panel structure) with error component, taste heterogeneity in ASC and attribute Travel Time.
I have used Python Biogeme in the past and noticed that the output file in the new version doesn't contain null log-likelihood for any of the models and the model fit is now calculated using initial log-likelihood. And the latter one is different for each of ML models, but it was equal to null log-likelihood and was staying constant in all models estimated in  Python Biogeme 
Is there a reason for this change?

Here is table with the output of all the models I estimated:
Model-page-001.jpg
The last ML model with taste heterogeneity in travel time shows the best model fit, but if to look at sigmas they have very small values and opposite to it is the ML model with taste heterogeneity in ASC (see the output files). I have recalculated manually the models fit (in yellow in the table) using null log-likelihood like in previous versions of Biogeme, but I am not sure if this is a correct approach.
How can I compare the models in this case when looking at Rho-square? Is LRS valid for comparison of MNL, NL and ML models?

And another question is about the convergence of the models with a different number of draws: What can be the reason that the same model reaches and doesn't reach convergence with a different number of draws? In my case, ML (taste heterogeneity in ASC) didn't reach convergence at 200 draws but could reach at 400 and 1000. And with Travel time heterogeneity reverse, at 200 reached and not at 1000 draws.

Looking forward to your response. 

Kind regards,
Maryna Ozturker

BASE MODEL_ML_taste_TIME_non-traders excl-200.html
BASE MODEL_ML_taste_ASC_non-traders excl-1000.html

Bierlaire Michel

unread,
Sep 8, 2020, 2:38:34 AM9/8/20
to maryna....@gmail.com, Bierlaire Michel, Biogeme
The null log likelihood will be available in the next version.
If you start the estimation with all coefficients equal to zero, the init log likelihood is the null log likelihood. 
You can also calculate it from your data file:  - sum_n log(J_n), where J_n is the number of alternatives available for individual n. 



On 7 Sep 2020, at 16:09, Maryna Öztürker <maryna....@gmail.com> wrote:

Dear Prof. Bierlaire,

I am trying to find the model which better describes the dataset. I have estimated several models using Pandas Biogeme, namely MNL, NL, ML (panel structure) with error component, taste heterogeneity in ASC and attribute Travel Time.
I have used Python Biogeme in the past and noticed that the output file in the new version doesn't contain null log-likelihood for any of the models and the model fit is now calculated using initial log-likelihood. And the latter one is different for each of ML models, but it was equal to null log-likelihood and was staying constant in all models estimated in  Python Biogeme 
Is there a reason for this change?

Here is table with the output of all the models I estimated:
<Model-page-001.jpg>
The last ML model with taste heterogeneity in travel time shows the best model fit, but if to look at sigmas they have very small values and opposite to it is the ML model with taste heterogeneity in ASC (see the output files). I have recalculated manually the models fit (in yellow in the table) using null log-likelihood

Null log likelihood is a constant. It has nothing to do with the fit. It is used only as a reference point. 

ike in previous versions of Biogeme, but I am not sure if this is a correct approach.
How can I compare the models in this case when looking at Rho-square? Is LRS valid for comparison of MNL, NL and ML models?

I would suggest that you use AIC and BIC to compare models. 


And another question is about the convergence of the models with a different number of draws: What can be the reason that the same model reaches and doesn't reach convergence with a different number of draws? In my case, ML (taste heterogeneity in ASC) didn't reach convergence at 200 draws but could reach at 400 and 1000. And with Travel time heterogeneity reverse, at 200 reached and not at 1000 draws.

The (simulated) likelihood function may vary a lot depending on the number of draws. 200 draws is likely to be insufficient. And it may be the case that 1000 also. 



Looking forward to your response. 

Kind regards,
Maryna Ozturker


--
You received this message because you are subscribed to the Google Groups "Biogeme" group.
To unsubscribe from this group and stop receiving emails from it, send an email to biogeme+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/biogeme/0d3b0b55-7979-4606-9333-13e00726ddfen%40googlegroups.com.
<Model-page-001.jpg><BASE MODEL_ML_taste_TIME_non-traders excl-200.html><BASE MODEL_ML_taste_ASC_non-traders excl-1000.html>

Reply all
Reply to author
Forward
0 new messages