Apollo vs Biogeme Comparison Mixed Logit

[Mohamed Khalifa]

Hello Prof. Bierlaire,

I hope you are doing well. I am translating models from Apollo to BIOGEME for educational purposes. I've successfully translated models with taste heterogeneity (random coefficients), and the parameter magnitudes match well between Apollo and BIOGEME.However, I'm encountering discrepancies when translating models with Alternative-Specific Variances (ASV) and Error Components. The parameter magnitudes differ significantly, even though the model specifications appear equivalent.

I've attached both the Apollo  and BIOGEME implementations for your review. I would greatly appreciate any insights on whether this is a parameterization/scaling difference or if there's an issue with my BIOGEME specification.

Thank you very much for your time and best regards,
Mohamed

[Mohamed Khalifa]

Note:

In my comparison between Mixed Logit the results of the random coefficients (the parameters as well as variance) are in the same magnitude. This is not the case for the models I created for random error components as well as alternative specific variance. Only the parameters ( betas) are in the same magnitude while the variance could be quite different. My assumptions for the difference lie either on one of the following:

• Due to the drawing process, we get different draw. Yet some magnitude differences are too big.
• Apollo and R use different parametrization.
• It has to do with the distribution.

If it is possible of you to take the time, I would appreciate the insights.

[Michel Bierlaire]

Based on what you describe, this does not appear to be a software issue, but rather a model specification and identification issue.

In particular:
    • Error component and ASV models require a proper normalization to be identified. This is discussed in detail in Joan Walker’s PhD thesis. If the model is not properly identified, different parameterizations or different sets of simulation draws can lead to very different numerical values of the parameters, even though the underlying model behavior is equivalent. In such cases, the log-likelihood value is the meaningful quantity to compare, and it should be very similar across implementations.
    • Using only 500 draws is likely insufficient, especially for error component models. With too few draws, simulation noise can mask identification problems, making the estimates appear unstable or inconsistent across software. Increasing the number of draws is often necessary not only for accuracy, but also to reveal whether the model is truly identified.

As a next step, I would strongly recommend trying Bayesian estimation, which is available in BIOGEME 3.3.2. Bayesian estimation is often very helpful to diagnose identification issues, as lack of identification typically manifests itself through poorly behaved or non-informative posterior distributions.

> --
> You received this message because you are subscribed to the Google Groups "Biogeme" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to biogeme+u...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/biogeme/a2b0df02-b28c-4a76-bc3c-8f755beaed23n%40googlegroups.com.

Michel Bierlaire
Transport and Mobility Laboratory
School of Architecture, Civil and Environmental Engineering
EPFL - Ecole Polytechnique Fédérale de Lausanne
http://transp-or.epfl.ch
http://people.epfl.ch/michel.bierlaire

[Mohamed Khalifa]

Hello Prof Bierlaire,

thank you for your suggestion. I will try working on it and post the results here as it might be helpful for someone in the future. I had another question: CNL models in Biogeme have a runtime of almost 500 seconds more compared to Apollo. For Mixed Logit and MNL models, the runtime is close to each other. Is there a reason for that? How can I investigate reasons for it further? ( resources , articles)

Best,

Mohamed Khalifa