TFCE and bootstrap

[Benjamin Kay]

I apologize for asking an extremely basic question!  The Smith et al 2009 paper on TFCE describes a method for using permutation under the null hypothesis to select a TFCE threshold that controls for family-wise error rate.  The procedure is to create an empirical distribution of the maximum TFCE value under the null hypothesis using permuted data.  Then pick the 95th percentile from distribution as the TFCE threshold to control for FWE.

The NISOx toolbox lets us do TFCE using a wild bootstrap.  I couldn't find this modification to TFCE published‡, so I want to check my understanding.  To get an appropriate null distribution of TFCE values, do we bootstrap under then null hypothesis like this?

Suppose Y are the data, X is the predictor of interest, and Z are nuisance covariates. Solve full model for B and G.

Y = XB + ZG + ϵ

Resample the residuals as ϵ*.

Resample Y under the null hypothesis of B = 0 as:

Ynull* = ZG + ϵ*

Solve the full model, and use SWE to get corresponding t-values for Bnull.

Ynull* = XBnull* + ZG + ϵ'

Rinse and repeat.

Is this basically correct?  (And if resampling the residuals using the Rademacher distribution, is this similar to Freedman-Lane permutation with sign flipping?)

‡ Except here, where the basic concept of how they did a bootstrap under the null hypothesis was also hard to follow: https://doi.org/10.1016%2Fj.jneumeth.2014.08.003

[Thomas Nichols]

Dear Benjamin,

I apologize for asking an extremely basic question!  The Smith et al 2009 paper on TFCE describes a method for using permutation under the null hypothesis to select a TFCE threshold that controls for family-wise error rate.  The procedure is to create an empirical distribution of the maximum TFCE value under the null hypothesis using permuted data.  Then pick the 95th percentile from distribution as the TFCE threshold to control for FWE.

This a good summary, yes.

The NISOx toolbox lets us do TFCE using a wild bootstrap.  I couldn't find this modification to TFCE published‡, so I want to check my understanding. 

Sorry we haven't published anything on the specific combination of TFCE+WB, but there is ample literature on the wild bootstrap.  More details on the WB for SwE (but not on TFCE) can be found in Bryan Guillaume's thesis (chapter 4)

Guillaume, B. (2015). Accurate Non-Iterative Modelling and Inference of Longitudinal Neuroimaging Data. Maastricht University / University of Liège. Retrieved from https://orbi.uliege.be/handle/2268/186284

*However* the thesis concludes that a restricted wild bootstrap test is best (where you construct a bootstrap sample with the null hypothesis imposed, e.g. dropping a tested covariate from the model), referred to as "R-SwE". However, after seeing that it really made no difference in practice, we instead set unrestricted WB "U-SwE" as the default (while you can try R-SwE by changing "Type of SwE" in the model configuration, it is incompatible with cluster inference and so TFCE, for reasons I can't summon now).

In the U-SwE you have the original fit betahat, and then you create new samples Y* = X betahat + e* where e* are the residuals (Y - X betahat) multiplied by 1 or -1 (in blocks by subject), and then we get a sample of (alternative, non-null, "unrestricted") betahat*'s.  In essence, a null distribution is constructed by centering the betahat*'s with betahat.  Bryan's chapter 4 goes through this in detail.

To get an appropriate null distribution of TFCE values, do we bootstrap under then null hypothesis like this?

So in summary, the WB is under the alternative, but we construct a null distribution that removes the signal from the observed data.

Suppose Y are the data, X is the predictor of interest, and Z are nuisance covariates. Solve full model for B and G.

Y = XB + ZG + ϵ

Resample the residuals as ϵ*.

Resample Y under the null hypothesis of B = 0 as:

Ynull* = ZG + ϵ*

Solve the full model, and use SWE to get corresponding t-values for Bnull.

Ynull* = XBnull* + ZG + ϵ'

Rinse and repeat.

You have here nicely described the R-SwE approach, yes.  

-Tom

Is this basically correct?  (And if resampling the residuals using the Rademacher distribution, is this similar to Freedman-Lane permutation with sign flipping?)

‡ Except here, where the basic concept of how they did a bootstrap under the null hypothesis was also hard to follow: https://doi.org/10.1016%2Fj.jneumeth.2014.08.003

--
You received this message because you are subscribed to the Google Groups "SwE-Toolbox Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swe-toolbox...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/swe-toolbox/16af7c6f-cc0c-4072-9b6e-ce7833e4fe20n%40googlegroups.com.

__________________________________________________________
Thomas Nichols, PhD
Professor of Neuroimaging Statistics
Nuffield Department of Population Health | University of Oxford
Big Data Institute | Li Ka Shing Centre for Health Information and Discovery
Old Road Campus | Headington | Oxford | OX3 7LF | United Kingdom
T: +44 1865 743590 | E: thomas....@bdi.ox.ac.uk
W: http://nisox.org | http://www.bdi.ox.ac.uk

[Benjamin Kay]

Thank you for the succinct answers!  You are right, Chapter 4.2.1-4.2.3 of Bryan's thesis  4.2.5 of Bryan's thesis describe R-SwE vs U-SwE, and 4.2.5 describes their use in cluster inference (without TFCE). Interestingly, 4.2.5 makes no comparison of U-SwE vs R-SwE for cluster inference.

(while you can try R-SwE by changing "Type of SwE" in the model configuration, it is incompatible with cluster inference and so TFCE, for reasons I can't summon now).

This is surprising!  Could this simply be an issue software/implementation compatibility?

[Thomas Nichols]

Dear Benjamin,

On 18 May 2023, at 19:39, Benjamin Kay <benj...@benkay.net> wrote:

Thank you for the succinct answers!  You are right, Chapter 4.2.1-4.2.3 of Bryan's thesis  4.2.5 of Bryan's thesis describe R-SwE vs U-SwE, and 4.2.5 describes their use in cluster inference (without TFCE). Interestingly, 4.2.5 makes no comparison of U-SwE vs R-SwE for cluster inference.

(while you can try R-SwE by changing "Type of SwE" in the model configuration, it is incompatible with cluster inference and so TFCE, for reasons I can't summon now).

This is surprising!  Could this simply be an issue software/implementation compatibility?

See line 267 of swe_cp_WB.m... it is simply enforced: "If clusterWise inference, force the U-SwE to be used".  

If this is so for cluster, I would think it would have to be for TFCE but we haven't coded it that way.

-Tom

To view this discussion on the web visit https://groups.google.com/d/msgid/swe-toolbox/3384c71e-79fe-4b73-90c4-deba9ec6c32cn%40googlegroups.com.