Time-varying transition matrix

[Nomi Tilap]

Hi there,

I was hoping I could verify I have specified a custom SSM correctly in statsmodels.

The state space representation is:

 Main question: time-varying transition matrix 

1.       The transition matrix is time-varying; it contains 2 exogenous data series, iAT_t and AT_t . I tried to adapt the logic for a time-varying design matrix from Model 3 of https://www.statsmodels.org/stable/examples/notebooks/generated/statespace_custom_models.html , namely to specify matrices as three-dimensional with time in the third dimension, but would really appreciate anyone can verify I have specified the model correctly in statsmodels  The relevant code is below (and also attached). 

Other questions 

It is possible that these questions will be resolved  to 1, if my code is incorrect. 

2.       Constants in the state vector: The state space representation should make gamma and delta constants even though they are in the state vector. After fitting, when I check the state vector’s fitted or smoothed estimates using model.filtered_state or model.smoothed_state, I see that their values are not constant, although they converge to constants by the end of the data. Is this normal? 

 3.       Convergence: to check convergence of fitting, is model.mle_retvals['converged'] the correct value? There is also model.filter_results.converged and model.smoother_results.converged which I have ignored, and they sometimes are false when the mle_retvals value is true. 

Thanks!

[Chad Fulton]

Hello,

Your main question:

(1) Yes, it looks to me like you've set up the time-varying transition matrix correctly.

Two notes:

- You do not need to make the other matrices (design, selection) time-varying, since they are actually time-invariant. (But it doesn't hurt to do it the way you've done it either, so it is up to you).

- Right now, your `update` method is indented too far. It should be a class method at the same level as __init__.

Your other questions:

(2) Yes, gamma and delta should be constants in the smoothed_state vector (but not in the filtered_state vector, since the system is still learning about their value over time).  Can you double-check that they are actually varying in the smoothed_state vector?

(3) Yes, model.mle_retvals['converged'] will tell you if the numerical optimizer has converged to parameters that meet its definition for an optimum. model.filter_results.converged tells you if the Kalman filter has converged, and this property does not tell you anything about how good your model fits (in fact, since you have a time-varying model, in general it will not converge. But again, this is not problematic.).

Best,

Chad

--
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/ff9e9375-11fa-4fdd-9b01-55140491c75bn%40googlegroups.com.

[Nomi Tilap]

Hello Chad,

 Many thanks for your help. Yes the update method above was indented too far, it is not in our program. We have run both versions above and ones with time-invariant design, selection matrices. Still our smoothed-state parameters are not constant at the start (see below). Could there be any other reason for this?

 

Thanks!