Slow model fitting with a Type III spatiotemporal interaction

[Insang Song]

Dear R-INLA community:

Hello, I am working on models with spatiotemporal interaction terms. 

I found that fitting models with a type-III (unstructured temporal * structured spatial) model took a much longer time than fitting models with the other three types of spatiotemporal interaction terms. I guessed this is because the rank is a bit higher ((n-1)T; following Blangiardo and Cameletti 2015:241) than that of other counterparts, but this does not hold as the type-I interaction has the higher rank (nT).

I think this is counterintuitive as well as against the assumption suggested in Vincente et al. (2020:1429; https://doi.org/10.1007/s00477-020-01808-x), quote:

"Regarding computing times for the models presented in Sect. 2, models with Type II and Type IV interactions are the slowest regardless of the fitting technique ... One reason for this may be that the number of constraints is much higher for Type II and Type IV than for the Type I and Type III counterparts."

If I understood correctly, the matrix for Type-III would be more sparse than that of Type-IV, because Type-III takes a diagonal matrix for the temporal term. A reproducible example taken from Chapter 7.1.2 in Blangiardo and Cameletti (2015) (attached) yielded running time in my laptop as below (*please note that no 'experimental' feature in the up-to-date version of INLA package was applied here):

Type I: 10.1 seconds

Type II: 53.1 seconds

Type III: 327 seconds

Type IV: 63.6 seconds

I am suffering from this issue right now since I aim to compare four types of spatiotemporal interactions with much larger data. The model with Type III interaction term with my data (3108*14) fits terribly slow and consumes 60+GB of memory from time to time (even with PARDISO and experimental feature of the testing version of INLA). I hope I can understand why Type III interaction takes a longer time than other interactions from your insight.

Thank you very much.

Best,

[Esmail Abdul Fattah]

Hello, 

It should not be the case: type 3 is slower than type 4. The speed depends on the number of constraints each type has.  The precision matrix is composed of at least 5 blocks: RW (size n) (of order L and L usually equals to 1 or 2), iid (size n, time), besag (size m), iid (size m, space) and rgeneric (type 1,2,3 or 4: which is the Kronecker product of two of the previous models).

The number of constraints in each type:

1. type 1: L + 1

2. type 2: Lm + L + 1

3. type 3: n + L + 1

4. type 4: Lm + n - 1

Make sure you use the right model for the interaction model using "generic0", and the right rank for the matrix. 

 f(id, 

    model="generic0", 

    Cmatrix = ,

    constr= , 

    extraconstr = ,

    rankdef= )

You need to use "extraconstr" to list the constraints you have in each case, which I didn't see in the code. You can use eigen decomposition to find these constraints. "constr" does only sum to zero constraint. However, it can be implemented in different ways, but this is what do I recommend. 

Best Regards,

Esmail 

[Insang Song]

Thank you for the response, Esmail.

I became interested in how INLA::f function works. I have thought that the code below would result in--

f(timeid, model='iid', group='spaceid', control.group=list(model='besag', graph=graph))

kronecker(I_t, D-W), where I_t is a diagonal matrix with the dimension of the number of unique timeids, D is Diagonal(rowSums(W)), and W is a spatial contiguity matrix. But it seems that it works exactly opposite, such that it would give kronecker(D-W, I_t). Please correct me if I am wrong.

The code I attached in the last thread is with minimal edits (for reproducibility) from the example of Blangiardo and Cameletti (2015)'s book, which is one of the most frequently cited references on spatial and spatio-temporal modeling in R-INLA (657 times cited as of April 4, 2022 in Google Scholar). I think their applications using INLA::f function seem not to have the same number of constraints in spatiotemporal interaction models with the number of constraints you explained. My speculation is still very sketchy because I only checked the number of constraints of models with spatiotemporal interaction terms. If my speculation would be found to be (unfortunately) correct, studies that cited their book, including mine, will be severely impacted.

Put it short, I hope I could get any references for the details of how INLA::f works in the context of multilevel models with spatiotemporal interaction terms.

Thank you.

Best,

Insang

2021년 12월 2일 목요일 오후 11시 20분 2초 UTC-8에 esm...@r-inla.org님이 작성:

[Finn Lindgren]

Hi,
the f-models are built from three parts:
f(A, ..., group = B, ..., replicate = C)
the individual Q-matrices are Q_A, Q_B, and Q_C, where Q_C is an
identity matrix (i.e. like "iid").

The joint precision is
Q_C x Q_B x Q_A
and the A-part indices vary fastest in the indexing in the internal
latent variable storage, like this:

> expand.grid(a=1:3,b=1:2,c=1:4)
a b c
1 1 1 1
2 2 1 1
3 3 1 1
4 1 2 1
5 2 2 1
6 3 2 1
7 1 1 2
8 2 1 2
9 3 1 2
...

(note that kronecker ordering conventions aren't universal, so in some
literature it may be written the other way around, but the indexing
above is what determines the ordering in inla)
Typically, I recommend having the most complicated part of the model
as part "A", use replicate for any "iid" kronecker construction, and
group for models less complex than the main "A" part.

But for these "Type I,II,III,IV interaction" models, it becomes rather
complex to specify the models correctly, due to the constraints, and
Esmail's suggestion to construct the constraints numerically from the
actual precision matrices seems like a sensible suggestion to avoid
accidentally missing a constraint, or constructing the wrong
constraints.
It does look like the Blangiardo and Cameletti example (if you
included all constraints set in the book) partly (or even mostly)
ignores the constraints for these models, as it doesn't use
extraconstr at all. I would suggest contacting the book authors about
that.
There is an Errata on the book webpage at
https://sites.google.com/a/r-inla.org/stbook/errata but this issue
isn't listed there.

Finn

> --
> You received this message because you are subscribed to the Google Groups "R-inla discussion group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to r-inla-discussion...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/r-inla-discussion-group/fb1b7f50-d603-4373-ad1f-0c4e16e387e7n%40googlegroups.com.



--
Finn Lindgren
email: finn.l...@gmail.com

[Helpdesk]

also make sure to use the most recent testing version for these models
as they have been improvements in the code dealing with many
constraints.