sdreport error "problem too large"

[Rujia Bi]

Dear TMB experts,

I developed a complicated spatiotemporal model using data from 156 time steps. The model ran successfully and converged. However, I received an error at the final step when executing rep <- sdreport(obj):

Error in .local(a, b, ...) : CHOLMOD error 'problem too large' at file 'Utility/t_cholmod_allocate_dense.c', line 61 Calls: sdreport ... solve -> solve -> solve -> solve -> .local -> .Call.

I only ADREPORTED a 156×18 matrix, which is necessary for my post-analyses. I am wondering if this error is due to the large size of the spatial and spatiotemporal random effects. Any guidance would be greatly appreciated.

Thanks!

[Rujia Bi]

Update: I don't need the covariance matrix for the ADREPORTed matrix (all I need are est and se). Would it be helpful if I set rep <- sdreport(obj, getReportCovariance = FALSE)?

[Kasper Kristensen]

Without more info - at the very least a 'traceback()' - I can't say for sure, but here is a guess of what might be going wrong:

At some point sdreport calculates the derivative of all ADREPORTed quantities (2808 in your case) wrt all random effects (nr=156*nspace ?) and performs sparse solve using this derivative as RHS.

Because R uses the 32 bit CHOLMOD routines, it can't do sparse solve with RHS longer than ca 2^31.

So, if the number of spatial grid cells for your application is greater that 2^31 / 2808 / 156 = 4902 you might have this problem.

[Rujia Bi]

Dear Kasper,

Thank you for your response.  I finally solved this problem by setting: rep <- sdreport(obj, getJointPrecision = FALSE, getReportCovariance = FALSE). 

I then changed the previous model, which treated the year effect as a random effect, to one that treats the year effect as fixed. Since the dimension of the year effect is 156×18, the sdreport function becomes very time-consuming. Is there any way to speed it up?

Thanks,

Rujia