MNL model with frequencies

[Fani Hatziioannidu]

Dear Group and dear Professor,

I am working on this MNL model for the past couple of months.

I have a very big dataset of revealed preferences. I want to find out if large datasets from mobile phone data can be used for calibrating mode choice logit functions.

I either get error messages like this

biogemeError: The norm of the gradient is inf: g=[-1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308 -1.79769313e+308]

or a perfect model rho square 1 or the opposite with rho square 0.

data csv file example where a1,a2,a3 frequency of choice

CHOICE1	CHOICE2	CHOICE3	av1	av2	av3	Time1	Time2	Time3	a1	a2	a3
1	2	3	1	1	1	10	20	25	10	5	5
1	2	3	1	1	0	12	18	9999	9	9	0
1	2	3	1	1	1	8	18	18	9	5	4

but I have also tried another structure for the choice column

CHOICE	av1	av2	av3	Time1	Time2	Time3	a1	a2	a3
1	1	1	1	10	20	25	10	5	5
2	1	1	0	12	18	9999	9	9	0
3	1	1	1	8	18	18	9	5	4

Is my data structure ok ?

panda python model

# Associate utility functions with the numbering of alternatives
V = {1: V1, 2: V2, 3: V3}

# Associate the availability conditions with the alternatives
av = {1: av1, 2: av2, 3: av3}

# Associate the choice with the alternatives
CHOICE = {1: CHOICE1, 2: CHOICE2, 3: CHOICE3} (is this line useful ?)

# The choice model is a logit, with availability conditions
logprob = a1 * _bioLogLogit (V,av,1) + a2 * _bioLogLogit (V,av,2) + a3 * _bioLogLogit (V,av,3)

How can I correct my model and get reasonable results?

Looking forward to your response and thank you for your time.

ps. I remember Laurie Garrow in a lecture in 2005 arguing that in similar cases with big datasets from airlines of revealed preferences we have to randomly select instead of using the whole sample. Is that so?

Or this had to do with computing limitations of the past?

kind regards,

--

Fani Hatziioannidu
Dipl.Eng.