SnPM pipeline and variance smoothing

[Marco Caviezel]

Dear SnPM experts,

after reading the paper: "Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates" (http://www.pnas.org/content/113/28/7900.full), I decided that I want to apply a permutation approach to avoid the critic of false positive results.

Because I am new to SnPM, I first would like to double-check my processing pipeline.

In my experiment, I have 22 subjects undergoing a learning experiment.

I want to apply a one sample T-test, so I use the "MultiSub: One Sample T test on diffs/contrasts".

Am I right to choose the con-images of my 1st level analysis as "Images to analyze", which are generated by the "normal" processing pipeline of SPM?

In my 2nd level model in SnPM, I have to apply four nuisance parameter as covariates, resulting in a df of 17.

I calculated the model with and without "variance smooting (8 8 8)" and found better results with "variance smoothing" turned on.

In the description, it says that variance smoothing increases power if df<20, so I think it is reasonable to use it.

Now my question is if I somehow make myself vulnerable to reviewers if I use variance smoothing?

I am very thankful for any support.

Best regards,

Marco

[Thomas Nichols]

Dear Marco,

after reading the paper: "Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates" (http://www.pnas.org/content/113/28/7900.full), I decided that I want to apply a permutation approach to avoid the critic of false positive results.

Because I am new to SnPM, I first would like to double-check my processing pipeline.

In my experiment, I have 22 subjects undergoing a learning experiment.

I want to apply a one sample T-test, so I use the "MultiSub: One Sample T test on diffs/contrasts".

Am I right to choose the con-images of my 1st level analysis as "Images to analyze", which are generated by the "normal" processing pipeline of SPM?

That's right; you specify as input the first level con images, just like you do for a regular SPM 2nd level analysis.

In my 2nd level model in SnPM, I have to apply four nuisance parameter as covariates, resulting in a df of 17.

I calculated the model with and without "variance smooting (8 8 8)" and found better results with "variance smoothing" turned on.

In my experience DF=20 is the threshold... for DF>20 variance smoothing doesn't make any difference, for DF<10 or so it can have a profound impact.  For DF=17 I'm not surprised it helps, but I assume it's not a huge difference.

 

In the description, it says that variance smoothing increases power if df<20, so I think it is reasonable to use it.

Now my question is if I somehow make myself vulnerable to reviewers if I use variance smoothing?

I've never had any reports of reviewer skepticism.  In general, nonparametric method are regarded as more 'trustworthy' as they make fewer assumptions, so reviewers don't generally have a problem.  By using variance smoothing you are basically expressing a prior believe that the true variance map (e.g if you had 1000's of subjects) is smooth, and thus smoothing your DF=17 variance map simply improves it without biasing it much.

Let me know if this all makes sense.

-Tom

--
You received this message because you are subscribed to the Google Groups "Statistical Nonparametric Mapping" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snpm-support+unsubscribe@googlegroups.com.
To post to this group, send email to snpm-s...@googlegroups.com.
Visit this group at https://groups.google.com/group/snpm-support.
To view this discussion on the web, visit https://groups.google.com/d/msgid/snpm-support/c9310238-2c0e-4e5a-b8ed-4f85a64e279c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

__________________________________________________________

Thomas Nichols, PhD

Professor of Neuroimaging Statistics

Oxford Big Data Institute

Li Ka Shing Centre for Health Information and Discovery

Nuffield Department of Population Health

University of Oxford

Email: thomas....@bdi.ox.ac.uk

Web: http://nisox.org

[Marco Caviezel]

Dear Tom,

thank you very much for your quick reply!

This totally makes sense and I am happy, that I can use variance smooting in my analysis.

I first was a little concerned that reviewers could accuse me of p-hacking, but as you explained this should not be the case.

Interestingly, in SPM I found only cluster FWE (p=0.05 corrected (CDT 0.001) results, whereas using SnPM I got FWE (p=0.05) results on peak level.

Do you know if this often is the case?

Best regards,

Marco

Marco Caviezel

PhD Student

University of Basel

Switzerland

Am Freitag, 8. September 2017 10:56:45 UTC+2 schrieb Thomas Nichols:

Dear Marco,

after reading the paper: "Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates" (http://www.pnas.org/content/113/28/7900.full), I decided that I want to apply a permutation approach to avoid the critic of false positive results.

Because I am new to SnPM, I first would like to double-check my processing pipeline.

In my experiment, I have 22 subjects undergoing a learning experiment.

I want to apply a one sample T-test, so I use the "MultiSub: One Sample T test on diffs/contrasts".

Am I right to choose the con-images of my 1st level analysis as "Images to analyze", which are generated by the "normal" processing pipeline of SPM?

That's right; you specify as input the first level con images, just like you do for a regular SPM 2nd level analysis.

In my 2nd level model in SnPM, I have to apply four nuisance parameter as covariates, resulting in a df of 17.

I calculated the model with and without "variance smooting (8 8 8)" and found better results with "variance smoothing" turned on.

In my experience DF=20 is the threshold... for DF>20 variance smoothing doesn't make any difference, for DF<10 or so it can have a profound impact.  For DF=17 I'm not surprised it helps, but I assume it's not a huge difference.

 

In the description, it says that variance smoothing increases power if df<20, so I think it is reasonable to use it.

Now my question is if I somehow make myself vulnerable to reviewers if I use variance smoothing?

I've never had any reports of reviewer skepticism.  In general, nonparametric method are regarded as more 'trustworthy' as they make fewer assumptions, so reviewers don't generally have a problem.  By using variance smoothing you are basically expressing a prior believe that the true variance map (e.g if you had 1000's of subjects) is smooth, and thus smoothing your DF=17 variance map simply improves it without biasing it much.

Let me know if this all makes sense.

-Tom

--
You received this message because you are subscribed to the Google Groups "Statistical Nonparametric Mapping" group.

To unsubscribe from this group and stop receiving emails from it, send an email to snpm-support...@googlegroups.com.

To post to this group, send email to snpm-s...@googlegroups.com.
Visit this group at https://groups.google.com/group/snpm-support.
To view this discussion on the web, visit https://groups.google.com/d/msgid/snpm-support/c9310238-2c0e-4e5a-b8ed-4f85a64e279c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Nichols]

Dear Marco,

This totally makes sense and I am happy, that I can use variance smooting in my analysis.

I first was a little concerned that reviewers could accuse me of p-hacking, but as you explained this should not be the case.

Interestingly, in SPM I found only cluster FWE (p=0.05 corrected (CDT 0.001) results, whereas using SnPM I got FWE (p=0.05) results on peak level.

Do you know if this often is the case?

Well, from the Eklund work (and lots of previous work) we know that parametric/RFT/SPM peak/voxel-level results are very conservative, so it's not surprising that SnPM is more powerful.

For cluster size, I presume you are implying that SnPM did not find any cluster-wise significance; if this is the case, note that the Eklund found that even CDT 0.001 can be a touch liberal, so it's possible that the SPM result reflects a slightly inflated significance.  

-Tom

 

To unsubscribe from this group and stop receiving emails from it, send an email to snpm-support+unsubscribe@googlegroups.com.

To post to this group, send email to snpm-s...@googlegroups.com.
Visit this group at https://groups.google.com/group/snpm-support.

To view this discussion on the web, visit https://groups.google.com/d/msgid/snpm-support/c654e0cf-bc9a-4314-99ed-560e8b3b7499%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.